My "official" request to top engine programmers

[Rodolfo Leoni]

Of course I'm nobody, just someone who was a chess player some years ago.

I retired from tournaments, but I still run my analyses with the help of engines. I don't like too much automatic analyses, or engine tourneys to make statistics for best moves. Statistics never say how many incorrect moves were played with a position. I like to make moves, trying to understand the position, take them back to check new variations, and so on.

I sometimes use asmFish, Stockfish, Andscacs and McBrain. I think these engines are great. But they haven't a persistent hash system, and they can't propagate scores while running the variations. I also tried Komodo 9, but its "persistent hashes" don't persist very much. The reason is that when making moves and taking them back to go to a new variation newest entries overwrite oldest ones. To be really persistent, they should be an additional hash which stores and automatically saves positions, scores and depths. Of course, highly selected moves as the PV ones.

I wrote an e-mail to Daniel Jose about 1 month ago. As a gentleman like he is, he replied he found the whole description of a correct score propagarion "really interesting", specifying his priority for making his engine stronger. I understand and I accept this answer.

I'm asking if someone is interested in developing this feature. It'd be greatly interesting if (e.g.) McBrain or another Fish fork has such a system. That'd become much more user-oriented than it is.

Well, I said it. If a top engine programmer wants a more detailed description of what I consider to be the optimal PH system, I'll post the full description as it was in the mail I sent to Daniel.

Regards,
Rodolfo

[Ferdy]

Of course I'm nobody, just someone who was a chess player some years ago.

I retired from tournaments, but I still run my analyses with the help of engines. I don't like too much automatic analyses, or engine tourneys to make statistics for best moves. Statistics never say how many incorrect moves were played with a position. I like to make moves, trying to understand the position, take them back to check new variations, and so on.

I sometimes use asmFish, Stockfish, Andscacs and McBrain. I think these engines are great. But they haven't a persistent hash system, and they can't propagate scores while running the variations. I also tried Komodo 9, but its "persistent hashes" don't persist very much. The reason is that when making moves and taking them back to go to a new variation newest entries overwrite oldest ones. To be really persistent, they should be an additional hash which stores and automatically saves positions, scores and depths. Of course, highly selected moves as the PV ones.

I wrote an e-mail to Daniel Jose about 1 month ago. As a gentleman like he is, he replied he found the whole description of a correct score propagarion "really interesting", specifying his priority for making his engine stronger. I understand and I accept this answer.

I'm asking if someone is interested in developing this feature. It'd be greatly interesting if (e.g.) McBrain or another Fish fork has such a system. That'd become much more user-oriented than it is.

Well, I said it. If a top engine programmer wants a more detailed description of what I consider to be the optimal PH system, I'll post the full description as it was in the mail I sent to Daniel.

Regards,
Rodolfo

Looks to me IDEA in Aquarium GUI is better for you.

[hgm]

The problem that scores of a variation get lost when you go deeper in a variation, and then return to (before) a previously analyzed one, is not related to persistent hash. The latter only makes it possible to transfer analysis between different sessions with the engine.

What you complain about (and also bugged me, when I started to do thorough analysis) is caused by a bad replacement strategy for hash entries. Most engines number the searches they do, and remember in the hash table which search was responsible for creating a stored entry, or remember what was the last search that actually used the stored information. If they then stumble on entries that are not used in any recent search (sometimes even just the current search), they assume the positions they refer to no longer are reachable from the current root position, and thus useless. They are treated as empty, and immediately overwritten.

This works very well in games, where no take-backs are done. Such as in the games used to determine the rating of the engine. For interactive analysis this is fatal, however: as soon as you go into an alterative variation, and play an irreversible move, all your previous analysis will be purged from the hash table, as the positions are no longer reachable. But of course they remain reachable, through the take-back you will sooner or later do to back-poropagate the score of the newly analyzed variation.

The solution is quite simple: engines should realize that an interactive analysis session is in fact a single search, where the low plies are governed by the user, and the root positions of the various analysis searches are not root positions at all, but just internal nodes of the tree the user is constructing. As such, they should not increment the search number during analysis. They are just not different searches, but part of the larger one.

In my engines incrementing the search number only before searches intended to play in a game ('thinking'), and leaving it as it was in an analysis search, did solve the problem.

[Rodolfo Leoni]

Looks to me IDEA in Aquarium GUI is better for you.

Hi Ferdinand,

a friend of mine has an old version of Rybka Aquarium. As we were used to make common analyses I had the opportunity to study how it works. It could have changed in these years, but I believe most things are exactly as they were.

I don't like automated analyses. I know one can interact with his own moves, but this system tends to encourage a player to go sleeping and to check the results the morning after. And I'm not sure score propagation is correct, as it should be depth-related. One cannot correctly propagate a score which is got at depth 25 to a previous position at (e.g.) depth 30. When one changes TCs in IDeA he actually can get wrong propagations.

Rodolfo

[Rodolfo Leoni]

The problem that scores of a variation get lost when you go deeper in a variation, and then return to (before) a previously analyzed one, is not related to persistent hash. The latter only makes it possible to transfer analysis between different sessions with the engine.

What you complain about (and also bugged me, when I started to do thorough analysis) is caused by a bad replacement strategy for hash entries. Most engines number the searches they do, and remember in the hash table which search was responsible for creating a stored entry, or remember what was the last search that actually used the stored information. If they then stumble on entries that are not used in any recent search (sometimes even just the current search), they assume the positions they refer to no longer are reachable from the current root position, and thus useless. They are treated as empty, and immediately overwritten.

This works very well in games, where no take-backs are done. Such as in the games used to determine the rating of the engine. For interactive analysis this is fatal, however: as soon as you go into an alterative variation, and play an irreversible move, all your previous analysis will be purged from the hash table, as the positions are no longer reachable. But of course they remain reachable, through the take-back you will sooner or later do to back-poropagate the score of the newly analyzed variation.

The solution is quite simple: engines should realize that an interactive analysis session is in fact a single search, where the low plies are governed by the user, and the root positions of the various analysis searches are not root positions at all, but just internal nodes of the tree the user is constructing. As such, they should not increment the search number during analysis. They are just not different searches, but part of the larger one.

In my engines incrementing the search number only before searches intended to play in a game ('thinking'), and leaving it as it was in an analysis search, did solve the problem.

Well, you say what I said before, you just say it better.

BTW, are you still developing Joker? I've been away these last years and I need some updates about what works are still in progress.

About technical solution, I have no other reference than Critter example. Richard Vida built Critter session file as a separate, sizeable hash. With this solution he avoided complicated hash handlings and, most of all, file is updated every time a move is made on the board. That means one can resume an analysis when he wants, as all of his wokr was saved. It uses both forward and backward (2 plies) propagation when a score drop is detected. Not a perfect system, but the best I've ever seen.

The problem is, Critter could be 200 ELOs weaker than Komodo and fishes... That's why I'm asking if some top engine programmer is interested in developing this feature.

[Ferdy]

Looks to me IDEA in Aquarium GUI is better for you.

Hi Ferdinand,

a friend of mine has an old version of Rybka Aquarium. As we were used to make common analyses I had the opportunity to study how it works. It could have changed in these years, but I believe most things are exactly as they were.

It is more powerful now, and remember as the engine gets stronger, the IDEA analysis gets stronger too.

I don't like automated analyses. I know one can interact with his own moves, but this system tends to encourage a player to go sleeping and to check the results the morning after.

The good thing about it is that when you wake up in the morning you will be presented with analysis. Your questions the night before will be possibly answered, like what happen if I follow this line? What is wrong with this move? and others. The engine expansion algorithms are now more controllable by the user too. But you can disable the engine expansion if you really want to, it is just an option.

And I'm not sure score propagation is correct, as it should be depth-related. One cannot correctly propagate a score which is got at depth 25 to a previous position at (e.g.) depth 30. When one changes TCs in IDeA he actually can get wrong propagations.

Did you mean mini-max? or backsolving in bookup? Mini-max in Aquarium is based on score. If not what do you mean exactly by score propagation?

[hgm]

BTW, are you still developing Joker? I've been away these last years and I need some updates about what works are still in progress.

No, I stopped developing Joker in 2008. Then I started to write a new engine (Spartacus), based on an incrementally updated attack map, and a larger board, which could also handle 12x8 and 10x10 boards. But I never got to finish that, and by now my ideas on how to do the attack map have evolved, so it probably would need a rewrite again. (If I can still find the source code...) Nevertheless it is already a bit stronger than Joker, Elo wise, even though I have not gotten to writing any evaluation for it beyond piece-square tables.

The problem is that IMO writing engines for orthodox Chess is a complete waste of time. There are already thousands of those, and no one needs another one (if it is not significantly stronger than Stockfish). So I prefer to spend my time on engines for games where there is a real demand.

[carldaman]

The problem that scores of a variation get lost when you go deeper in a variation, and then return to (before) a previously analyzed one, is not related to persistent hash. The latter only makes it possible to transfer analysis between different sessions with the engine.

What you complain about (and also bugged me, when I started to do thorough analysis) is caused by a bad replacement strategy for hash entries. Most engines number the searches they do, and remember in the hash table which search was responsible for creating a stored entry, or remember what was the last search that actually used the stored information. If they then stumble on entries that are not used in any recent search (sometimes even just the current search), they assume the positions they refer to no longer are reachable from the current root position, and thus useless. They are treated as empty, and immediately overwritten.

This works very well in games, where no take-backs are done. Such as in the games used to determine the rating of the engine. For interactive analysis this is fatal, however: as soon as you go into an alterative variation, and play an irreversible move, all your previous analysis will be purged from the hash table, as the positions are no longer reachable. But of course they remain reachable, through the take-back you will sooner or later do to back-poropagate the score of the newly analyzed variation.

The solution is quite simple: engines should realize that an interactive analysis session is in fact a single search, where the low plies are governed by the user, and the root positions of the various analysis searches are not root positions at all, but just internal nodes of the tree the user is constructing. As such, they should not increment the search number during analysis. They are just not different searches, but part of the larger one.

In my engines incrementing the search number only before searches intended to play in a game ('thinking'), and leaving it as it was in an analysis search, did solve the problem.

Well, you say what I said before, you just say it better.

BTW, are you still developing Joker? I've been away these last years and I need some updates about what works are still in progress.

About technical solution, I have no other reference than Critter example. Richard Vida built Critter session file as a separate, sizeable hash. With this solution he avoided complicated hash handlings and, most of all, file is updated every time a move is made on the board. That means one can resume an analysis when he wants, as all of his wokr was saved. It uses both forward and backward (2 plies) propagation when a score drop is detected. Not a perfect system, but the best I've ever seen.

The problem is, Critter could be 200 ELOs weaker than Komodo and fishes... That's why I'm asking if some top engine programmer is interested in developing this feature.

You'd think this [correct score propagation in the hash] was a top priority for commercial engine developers. I mean, so many people buy an engine for analysis purposes, so why wouldn't they provide an analysis-friendly engine? It makes no sense, other than to surmise that game-playing Elo is mostly everything to them. No wonder we're still in dark ages when it comes to good analysis features. [I do like IDEA, but it's a resource intensive hog of a program, and I prefer to interact with the analysis as much as possible]

As seen above, HGM is aware of the problem, and he even proposed an elegant solution; Richard Vida offered a good solution with his *free* (but now old) engine. If freeware programmers can offer improved analysis 'tools', why won't the top engine developers do the same or better, especially when they charge for their engines and they should 'owe' it to their customers?
Just wondering 'out loud'...

Regards,
CL

[Rodolfo Leoni]

The problem that scores of a variation get lost when you go deeper in a variation, and then return to (before) a previously analyzed one, is not related to persistent hash. The latter only makes it possible to transfer analysis between different sessions with the engine.

What you complain about (and also bugged me, when I started to do thorough analysis) is caused by a bad replacement strategy for hash entries. Most engines number the searches they do, and remember in the hash table which search was responsible for creating a stored entry, or remember what was the last search that actually used the stored information. If they then stumble on entries that are not used in any recent search (sometimes even just the current search), they assume the positions they refer to no longer are reachable from the current root position, and thus useless. They are treated as empty, and immediately overwritten.

This works very well in games, where no take-backs are done. Such as in the games used to determine the rating of the engine. For interactive analysis this is fatal, however: as soon as you go into an alterative variation, and play an irreversible move, all your previous analysis will be purged from the hash table, as the positions are no longer reachable. But of course they remain reachable, through the take-back you will sooner or later do to back-poropagate the score of the newly analyzed variation.

The solution is quite simple: engines should realize that an interactive analysis session is in fact a single search, where the low plies are governed by the user, and the root positions of the various analysis searches are not root positions at all, but just internal nodes of the tree the user is constructing. As such, they should not increment the search number during analysis. They are just not different searches, but part of the larger one.

In my engines incrementing the search number only before searches intended to play in a game ('thinking'), and leaving it as it was in an analysis search, did solve the problem.

Well, you say what I said before, you just say it better.

BTW, are you still developing Joker? I've been away these last years and I need some updates about what works are still in progress.

About technical solution, I have no other reference than Critter example. Richard Vida built Critter session file as a separate, sizeable hash. With this solution he avoided complicated hash handlings and, most of all, file is updated every time a move is made on the board. That means one can resume an analysis when he wants, as all of his wokr was saved. It uses both forward and backward (2 plies) propagation when a score drop is detected. Not a perfect system, but the best I've ever seen.

The problem is, Critter could be 200 ELOs weaker than Komodo and fishes... That's why I'm asking if some top engine programmer is interested in developing this feature.

You'd think this [correct score propagation in the hash] was a top priority for commercial engine developers. I mean, so many people buy an engine for analysis purposes, so why wouldn't they provide an analysis-friendly engine? It makes no sense, other than to surmise that game-playing Elo is mostly everything to them. No wonder we're still in dark ages when it comes to good analysis features. [I do like IDEA, but it's a resource intensive hog of a program, and I prefer to interact with the analysis as much as possible]

As seen above, HGM is aware of the problem, and he even proposed an elegant solution; Richard Vida offered a good solution with his *free* (but now old) engine. If freeware programmers can offer improved analysis 'tools', why won't the top engine developers do the same or better, especially when they charge for their engines and they should 'owe' it to their customers?
Just wondering 'out loud'...

Regards,
CL

Hi Carl,

I agree 100% with your clear and direct opinion. The choice is between self-oriented engines and user-oriented ones. It seems the direction everybody took is the fight for gaining few ELOs per year, and that's absolutely self-oriented. But if we thing an human being will use the engine, we should consider what's most important for him.

To become user-oriented could be next step of engines evolution...

[Rodolfo Leoni]

The good thing about it is that when you wake up in the morning you will be presented with analysis. Your questions the night before will be possibly answered, like what happen if I follow this line? What is wrong with this move? and others. The engine expansion algorithms are now more controllable by the user too. But you can disable the engine expansion if you really want to, it is just an option.

Maybe I'm a bit old-fashioned, but... what's the taste of IDeA doing all the work? There's a lot of fun when one makes own analyses, even with engines help.

Did you mean mini-max? or backsolving in bookup? Mini-max in Aquarium is based on score. If not what do you mean exactly by score propagation?

I remember IDeA performed steps in sequence: variations deepening, alternative moves, and so on. At end of each step score was propagate. Yes, it could be the Min-Max process, the process that propagates scores down to the root position.