Aspiration Windows: Rubbish!

Discussion of chess software programming and technical issues.

Moderators: hgm, Rebel, chrisw

Gian-Carlo Pascutto
Posts: 1243
Joined: Sat Dec 13, 2008 7:00 pm

Re: Aspiration Windows: Rubbish!

Post by Gian-Carlo Pascutto »

michiguel wrote:
lucasart wrote:
diep wrote:
Gian-Carlo Pascutto wrote:
ZirconiumX wrote: I know this sounds like a search bug
Right.
Worth nomination for post of the year isn't it?
Also for nomination
http://talkchess.com/forum/viewtopic.php?t=44306
The technical forum is the proper place for newbies to request technical help with the chance of being helped (or ignored in the worst case). Ganging up on them is not really the purpose.
My advice was serious. There is no point in discussing or testing aspiration windows if you have clear and obvious bugs in the more basic aspects of your search.

Get your search working first and then we can discuss which enhancements work or not.

If the original poster doesn't want to accept this he's going to have a hard time.
User avatar
michiguel
Posts: 6401
Joined: Thu Mar 09, 2006 8:30 pm
Location: Chicago, Illinois, USA

Re: Aspiration Windows: Rubbish!

Post by michiguel »

Gian-Carlo Pascutto wrote:
michiguel wrote:
lucasart wrote:
diep wrote:
Gian-Carlo Pascutto wrote:
ZirconiumX wrote: I know this sounds like a search bug
Right.
Worth nomination for post of the year isn't it?
Also for nomination
http://talkchess.com/forum/viewtopic.php?t=44306
The technical forum is the proper place for newbies to request technical help with the chance of being helped (or ignored in the worst case). Ganging up on them is not really the purpose.
My advice was serious. There is no point in discussing or testing aspiration windows if you have clear and obvious bugs in the more basic aspects of your search.

Get your search working first and then we can discuss which enhancements work or not.

If the original poster doesn't want to accept this he's going to have a hard time.
I was not referring to your message, but the following two, which did not contribute. One thing is to point there is a bug, another is to laugh in group with no useful feedback.

Miguel
User avatar
Don
Posts: 5106
Joined: Tue Apr 29, 2008 4:27 pm

Re: Aspiration Windows: Rubbish!

Post by Don »

Gian-Carlo Pascutto wrote:
michiguel wrote:
lucasart wrote:
diep wrote:
Gian-Carlo Pascutto wrote:
ZirconiumX wrote: I know this sounds like a search bug
Right.
Worth nomination for post of the year isn't it?
Also for nomination
http://talkchess.com/forum/viewtopic.php?t=44306
The technical forum is the proper place for newbies to request technical help with the chance of being helped (or ignored in the worst case). Ganging up on them is not really the purpose.
My advice was serious. There is no point in discussing or testing aspiration windows if you have clear and obvious bugs in the more basic aspects of your search.

Get your search working first and then we can discuss which enhancements work or not.

If the original poster doesn't want to accept this he's going to have a hard time.
There was no problem with your post Gina-Carlo.

Don
Capital punishment would be more effective as a preventive measure if it were administered prior to the crime.
petero2
Posts: 690
Joined: Mon Apr 19, 2010 7:07 pm
Location: Sweden
Full name: Peter Osterlund

Re: Aspiration Windows: Rubbish!

Post by petero2 »

tpetzke wrote:Hi Matthew,

1 one ply search is easy to debug. Note the position where it happens and then for every move in the list look which one causes the trouble and why it is going so deep.

You can do this step by step, you will probably note something wrong with the code that lets you bang your head on the wall a few times. (Like one move captures the king or so)

This happened to me quite a few times.

Thomas...
In Texel I have written code that dumps the whole search tree to a file. Things like alhpa, beta, depth, ply, score, score type (upper/lower bound or exact), evaluation score and hash key are stored. I can then run the program in a special mode that lets me navigate in the search tree stored in the file. This makes it relatively easy to understand why the program is doing something stupid even in deep searches and without disabling parts of the search algorithms.

The tree logging/analysis code is in the files treeLogger.hpp and treeLogger.cpp in this zip file: http://web.comhem.se/petero2home/texel101.zip
User avatar
Don
Posts: 5106
Joined: Tue Apr 29, 2008 4:27 pm

Re: Aspiration Windows: Rubbish!

Post by Don »

petero2 wrote:
tpetzke wrote:Hi Matthew,

1 one ply search is easy to debug. Note the position where it happens and then for every move in the list look which one causes the trouble and why it is going so deep.

You can do this step by step, you will probably note something wrong with the code that lets you bang your head on the wall a few times. (Like one move captures the king or so)

This happened to me quite a few times.

Thomas...
In Texel I have written code that dumps the whole search tree to a file. Things like alhpa, beta, depth, ply, score, score type (upper/lower bound or exact), evaluation score and hash key are stored. I can then run the program in a special mode that lets me navigate in the search tree stored in the file. This makes it relatively easy to understand why the program is doing something stupid even in deep searches and without disabling parts of the search algorithms.

The tree logging/analysis code is in the files treeLogger.hpp and treeLogger.cpp in this zip file: http://web.comhem.se/petero2home/texel101.zip
That is an excellent idea. I have done this myself but only in an ad-hoc manner to solve a specific bug.
Capital punishment would be more effective as a preventive measure if it were administered prior to the crime.
wgarvin
Posts: 838
Joined: Thu Jul 05, 2007 5:03 pm
Location: British Columbia, Canada

Re: Aspiration Windows: Rubbish!

Post by wgarvin »

petero2 wrote: In Texel I have written code that dumps the whole search tree to a file. Things like alhpa, beta, depth, ply, score, score type (upper/lower bound or exact), evaluation score and hash key are stored. I can then run the program in a special mode that lets me navigate in the search tree stored in the file. This makes it relatively easy to understand why the program is doing something stupid even in deep searches and without disabling parts of the search algorithms.

The tree logging/analysis code is in the files treeLogger.hpp and treeLogger.cpp in this zip file: http://web.comhem.se/petero2home/texel101.zip
For Windows users: This TabView tool might also be useful for viewing dumps of trees. I've never used it myself, but you ought to be able to dump a whole tree into a text file and use TabView to drill down to the part you are interested in.
diep
Posts: 1822
Joined: Thu Mar 09, 2006 11:54 pm
Location: The Netherlands

Re: Aspiration Windows: Rubbish!

Post by diep »

Don wrote:
petero2 wrote:
tpetzke wrote:Hi Matthew,

1 one ply search is easy to debug. Note the position where it happens and then for every move in the list look which one causes the trouble and why it is going so deep.

You can do this step by step, you will probably note something wrong with the code that lets you bang your head on the wall a few times. (Like one move captures the king or so)

This happened to me quite a few times.

Thomas...
In Texel I have written code that dumps the whole search tree to a file. Things like alhpa, beta, depth, ply, score, score type (upper/lower bound or exact), evaluation score and hash key are stored. I can then run the program in a special mode that lets me navigate in the search tree stored in the file. This makes it relatively easy to understand why the program is doing something stupid even in deep searches and without disabling parts of the search algorithms.

The tree logging/analysis code is in the files treeLogger.hpp and treeLogger.cpp in this zip file: http://web.comhem.se/petero2home/texel101.zip
That is an excellent idea. I have done this myself but only in an ad-hoc manner to solve a specific bug.
that's what i do to debug parallel search since 1999. dumping to a textfile had however 2 disadvantages:

a) the files would get too big to download back then (i had a 56k6 modem at home and Bob's quad Xeon is obviously in USA).

b) writing to a textfile takes a lot of system time, so the odds a parallel bug occurs dramatically reduces.

So i store it in shared memory (which means you're got a limited time for the error to occur) then dump that to a binary file. That binary file compressed i can download and then with a parser parse it to a textfile with all information.

It's really useful for all sorts of things, as long as you know where to look at inside those huge files. The expanded textfiles usually were gigabytes huge.

Try to find a parallel bug then!

In 2002 i changed the habit for the parallel search - i wrote on a paper the parallel functionality and tried to prove it correct on paper. That removed a LOT of the pain :)
Antonio Torrecillas
Posts: 90
Joined: Sun Nov 02, 2008 4:43 pm
Location: Barcelona

Re: Aspiration Windows: Rubbish!

Post by Antonio Torrecillas »

Based on an issue I've had some time ago, here my two cents.
On a normal search, alpha beta full window, when we prune it's because somewhere else we find a PV with value equal to the value of beta. So we can safely prune.
Now with aspiration window, this assumption is no longer valid. If we don't fail in the first move we have a PV node/path valid with his score, and the assumption is valid again. We prune value >= beta because somewhere else we have found beta. But we can now fail in the first/PV node so the PV is not a real PV.
To complete the scenario we add null move to the sauce. Suppose that, for some strange situation, you return in the false PV a null move. It's quite easy that the score will be a Mate score.(your phantom).
Now you wide the window and research, you get a true PV just you don't benefit from the quick Nullmove in the PV and you search to the nominal depth which is 3 ply deeper than your previous aspiration search.
I suggest that when a mate score is returned you inspect the pv to see if there is a null move. Or disable null move to see if the phantom disappear.

I don't think your base search is wrong.

Dissident regards, Antonio.
asanjuan
Posts: 214
Joined: Thu Sep 01, 2011 5:38 pm
Location: Seville, Spain

Re: Aspiration Windows: Rubbish!

Post by asanjuan »

ZirconiumX wrote:I have tested and tested, and I have one conclusion. They simply don't work for me.

I've watched many a test game where it searches normally, and is doing quite well, then suddenly gets a 'persistent phantom mate score' which pushes the window size to the point where it takes 5 seconds to perform a *one ply* search. This costs the program valuable time, and it loses.

I know this sounds like a search bug, but running with [-ValueInf, +ValueInf] works perfectly. It loses a ply of search, but never has a phantom mate score.

Matthew:out
¿Are you using Futility pruning on PV nodes?
With futility, in a 1 ply search, when you have an infinite window in a PV node, futility doesn't affect. But if you use a small window, it can hurt and return invalid mate values, because you can be pruning all the moves.

hope this helps.
Regards.
Still learning how to play chess...
knigths move in "L" shape ¿right?
User avatar
hgm
Posts: 27829
Joined: Fri Mar 10, 2006 10:06 am
Location: Amsterdam
Full name: H G Muller

Re: Aspiration Windows: Rubbish!

Post by hgm »

You can futility-prune in a PV node after alpha is raised, right? The window is only {-INF, INF} on the search of the first move. After that alpha will be raised to some finite value, which very well might be above the current eval. E.g. suppose the first move captured a hanging Queen. After that, all moves that do not capture at least a Hawk are pretty futile.