hgm wrote:The problem that scores of a variation get lost when you go deeper in a variation, and then return to (before) a previously analyzed one, is not related to persistent hash. The latter only makes it possible to transfer analysis between different sessions with the engine.
What you complain about (and also bugged me, when I started to do thorough analysis) is caused by a bad replacement strategy for hash entries. Most engines number the searches they do, and remember in the hash table which search was responsible for creating a stored entry, or remember what was the last search that actually used the stored information. If they then stumble on entries that are not used in any recent search (sometimes even just the current search), they assume the positions they refer to no longer are reachable from the current root position, and thus useless. They are treated as empty, and immediately overwritten.
This works very well in games, where no take-backs are done. Such as in the games used to determine the rating of the engine. For interactive analysis this is fatal, however: as soon as you go into an alterative variation, and play an irreversible move, all your previous analysis will be purged from the hash table, as the positions are no longer reachable. But of course they remain reachable, through the take-back you will sooner or later do to back-poropagate the score of the newly analyzed variation.
The solution is quite simple: engines should realize that an interactive analysis session is in fact a single search, where the low plies are governed by the user, and the root positions of the various analysis searches are not root positions at all, but just internal nodes of the tree the user is constructing. As such, they should not increment the search number during analysis. They are just not different searches, but part of the larger one.
In my engines incrementing the search number only before searches intended to play in a game ('thinking'), and leaving it as it was in an analysis search, did solve the problem.
I always thought it, as ours peoples are a bit related. I live near Alghero, in Sardinia island, and it was a Catalan town. Its old name was "Alguier".
I always thought it, as ours peoples are a bit related. I live near Alghero, in Sardinia island, and it was a Catalan town. Its old name was "Alguier".
I have done also a version of the current Stockfish (only Windows x64 popcnt) that have this change:
Dann Corbit wrote:There was a project some time ago that did something like this.
The pv nodes were stored in a database as it played.
These pv nodes were reloaded into hash when the engine was started for the next game.
It was also possible to preload the hash with a huge database of EPD records, if you so desired.
It was called Stockfish PA GTB.
It was too much effort for the author to maintain.
The same thing happens to many other such projects (CFish is a good example).
I reviewed the code and seem that is only able to use .bin books and to save a log file, but no special hash management stuff.
Another possibility to improve the analysis would be to add the ability to help the engine from outside during a running search. If the human user has some extra knowledge about some positions from the search tree (e.g. it recognizes a position as a fortress, or if he has checked the position in the Lomonosov tablebases) wouldn't be nice to communicate that to the engine?
The engine should accept a value attached to a fen string during the analysis and it has to save it as a permanent entry in the hash for example by a UCI command like this:
seed "fen string" "value"
Or even the engine itself could ask for help in some positions! Not only from the user but from another engine or a special software which does tablebase probes.
WB protocol has the setscore SCORE DEPTH command for that. With that you can instruct the engine to use the given score for the current position for all search requests up to the given depth. If higher depth is requested, it should do a normal search. You would have to specify the position in a preceding setboard FEN command, or by moving to it.
Unfortunately there is no way yet to have XBoard send this command. I could make it use the move type-in dialog for tgat, I guess.
cdani wrote:I reviewed the code and seem that is only able to use .bin books and to save a log file, but no special hash management stuff.
Thanks a lot for SF with less hash- aging, could it be called like that too?
I guess it works similar to UCI- options like "Persistent Hash" in Rybka or "Keep Hash Tables" in Shredder?
PA- GTB- SF had a "learning file" to which certain hash- entries were stored autmatically according to boundaries of eval- differences, user could adopt as well as the largeness of the stored file.
Yet it never got nearly as big as the hash tables themselves, so the princple seemed to be the one of learning files earlier versions of Shredder and Houdini had and Hiarcs still has.
cdani wrote:I reviewed the code and seem that is only able to use .bin books and to save a log file, but no special hash management stuff.
Thanks a lot for SF with less hash- aging, could it be called like that too?
I guess it works similar to UCI- options like "Persistent Hash" in Rybka or "Keep Hash Tables" in Shredder?
Not really. As told by HGM, It just treats any hash entry as if it was from the current search during the whole infinite analysis session, to avoid discard that easy hash entries with newer but shallower depth.
