My "official" request to top engine programmers

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Harvey Williamson, bob

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Post Reply
User avatar
hgm
Posts: 22500
Joined: Fri Mar 10, 2006 9:06 am
Location: Amsterdam
Full name: H G Muller
Contact:

Re: My "official" request to top engine programmer

Post by hgm » Wed Jul 05, 2017 5:26 am

The problem that scores of a variation get lost when you go deeper in a variation, and then return to (before) a previously analyzed one, is not related to persistent hash. The latter only makes it possible to transfer analysis between different sessions with the engine.

What you complain about (and also bugged me, when I started to do thorough analysis) is caused by a bad replacement strategy for hash entries. Most engines number the searches they do, and remember in the hash table which search was responsible for creating a stored entry, or remember what was the last search that actually used the stored information. If they then stumble on entries that are not used in any recent search (sometimes even just the current search), they assume the positions they refer to no longer are reachable from the current root position, and thus useless. They are treated as empty, and immediately overwritten.

This works very well in games, where no take-backs are done. Such as in the games used to determine the rating of the engine. For interactive analysis this is fatal, however: as soon as you go into an alterative variation, and play an irreversible move, all your previous analysis will be purged from the hash table, as the positions are no longer reachable. But of course they remain reachable, through the take-back you will sooner or later do to back-poropagate the score of the newly analyzed variation.

The solution is quite simple: engines should realize that an interactive analysis session is in fact a single search, where the low plies are governed by the user, and the root positions of the various analysis searches are not root positions at all, but just internal nodes of the tree the user is constructing. As such, they should not increment the search number during analysis. They are just not different searches, but part of the larger one.

In my engines incrementing the search number only before searches intended to play in a game ('thinking'), and leaving it as it was in an analysis search, did solve the problem.

Rodolfo Leoni
Posts: 544
Joined: Tue Jun 06, 2017 2:49 pm
Location: Italy

Re: My "official" request to top engine programmer

Post by Rodolfo Leoni » Wed Jul 05, 2017 8:39 am

Ferdy wrote: Looks to me IDEA in Aquarium GUI is better for you.
Hi Ferdinand,

a friend of mine has an old version of Rybka Aquarium. As we were used to make common analyses I had the opportunity to study how it works. It could have changed in these years, but I believe most things are exactly as they were.

I don't like automated analyses. I know one can interact with his own moves, but this system tends to encourage a player to go sleeping and to check the results the morning after. And I'm not sure score propagation is correct, as it should be depth-related. One cannot correctly propagate a score which is got at depth 25 to a previous position at (e.g.) depth 30. When one changes TCs in IDeA he actually can get wrong propagations.

Rodolfo
F.S.I. Chess Teacher

Rodolfo Leoni
Posts: 544
Joined: Tue Jun 06, 2017 2:49 pm
Location: Italy

Re: My "official" request to top engine programmer

Post by Rodolfo Leoni » Wed Jul 05, 2017 8:52 am

hgm wrote:The problem that scores of a variation get lost when you go deeper in a variation, and then return to (before) a previously analyzed one, is not related to persistent hash. The latter only makes it possible to transfer analysis between different sessions with the engine.

What you complain about (and also bugged me, when I started to do thorough analysis) is caused by a bad replacement strategy for hash entries. Most engines number the searches they do, and remember in the hash table which search was responsible for creating a stored entry, or remember what was the last search that actually used the stored information. If they then stumble on entries that are not used in any recent search (sometimes even just the current search), they assume the positions they refer to no longer are reachable from the current root position, and thus useless. They are treated as empty, and immediately overwritten.

This works very well in games, where no take-backs are done. Such as in the games used to determine the rating of the engine. For interactive analysis this is fatal, however: as soon as you go into an alterative variation, and play an irreversible move, all your previous analysis will be purged from the hash table, as the positions are no longer reachable. But of course they remain reachable, through the take-back you will sooner or later do to back-poropagate the score of the newly analyzed variation.

The solution is quite simple: engines should realize that an interactive analysis session is in fact a single search, where the low plies are governed by the user, and the root positions of the various analysis searches are not root positions at all, but just internal nodes of the tree the user is constructing. As such, they should not increment the search number during analysis. They are just not different searches, but part of the larger one.

In my engines incrementing the search number only before searches intended to play in a game ('thinking'), and leaving it as it was in an analysis search, did solve the problem.
Well, you say what I said before, you just say it better. :)

BTW, are you still developing Joker? I've been away these last years and I need some updates about what works are still in progress.

About technical solution, I have no other reference than Critter example. Richard Vida built Critter session file as a separate, sizeable hash. With this solution he avoided complicated hash handlings and, most of all, file is updated every time a move is made on the board. That means one can resume an analysis when he wants, as all of his wokr was saved. It uses both forward and backward (2 plies) propagation when a score drop is detected. Not a perfect system, but the best I've ever seen.

The problem is, Critter could be 200 ELOs weaker than Komodo and fishes... That's why I'm asking if some top engine programmer is interested in developing this feature.
F.S.I. Chess Teacher

Ferdy
Posts: 3620
Joined: Sun Aug 10, 2008 1:15 pm
Location: Philippines

Re: My "official" request to top engine programmer

Post by Ferdy » Wed Jul 05, 2017 9:25 am

Rodolfo Leoni wrote:
Ferdy wrote: Looks to me IDEA in Aquarium GUI is better for you.
Hi Ferdinand,

a friend of mine has an old version of Rybka Aquarium. As we were used to make common analyses I had the opportunity to study how it works. It could have changed in these years, but I believe most things are exactly as they were.
It is more powerful now, and remember as the engine gets stronger, the IDEA analysis gets stronger too.
Rodolfo Leoni wrote: I don't like automated analyses. I know one can interact with his own moves, but this system tends to encourage a player to go sleeping and to check the results the morning after.
The good thing about it is that when you wake up in the morning you will be presented with analysis. Your questions the night before will be possibly answered, like what happen if I follow this line? What is wrong with this move? and others. The engine expansion algorithms are now more controllable by the user too. But you can disable the engine expansion if you really want to, it is just an option.
Rodolfo Leoni wrote:And I'm not sure score propagation is correct, as it should be depth-related. One cannot correctly propagate a score which is got at depth 25 to a previous position at (e.g.) depth 30. When one changes TCs in IDeA he actually can get wrong propagations.
Did you mean mini-max? or backsolving in bookup? Mini-max in Aquarium is based on score. If not what do you mean exactly by score propagation?

User avatar
hgm
Posts: 22500
Joined: Fri Mar 10, 2006 9:06 am
Location: Amsterdam
Full name: H G Muller
Contact:

Re: My "official" request to top engine programmer

Post by hgm » Wed Jul 05, 2017 9:30 am

Rodolfo Leoni wrote:BTW, are you still developing Joker? I've been away these last years and I need some updates about what works are still in progress.
No, I stopped developing Joker in 2008. Then I started to write a new engine (Spartacus), based on an incrementally updated attack map, and a larger board, which could also handle 12x8 and 10x10 boards. But I never got to finish that, and by now my ideas on how to do the attack map have evolved, so it probably would need a rewrite again. (If I can still find the source code...) Nevertheless it is already a bit stronger than Joker, Elo wise, even though I have not gotten to writing any evaluation for it beyond piece-square tables.

The problem is that IMO writing engines for orthodox Chess is a complete waste of time. There are already thousands of those, and no one needs another one (if it is not significantly stronger than Stockfish). So I prefer to spend my time on engines for games where there is a real demand.

carldaman
Posts: 1556
Joined: Sat Jun 02, 2012 12:13 am

Re: My "official" request to top engine programmer

Post by carldaman » Wed Jul 05, 2017 10:43 am

Rodolfo Leoni wrote:
hgm wrote:The problem that scores of a variation get lost when you go deeper in a variation, and then return to (before) a previously analyzed one, is not related to persistent hash. The latter only makes it possible to transfer analysis between different sessions with the engine.

What you complain about (and also bugged me, when I started to do thorough analysis) is caused by a bad replacement strategy for hash entries. Most engines number the searches they do, and remember in the hash table which search was responsible for creating a stored entry, or remember what was the last search that actually used the stored information. If they then stumble on entries that are not used in any recent search (sometimes even just the current search), they assume the positions they refer to no longer are reachable from the current root position, and thus useless. They are treated as empty, and immediately overwritten.

This works very well in games, where no take-backs are done. Such as in the games used to determine the rating of the engine. For interactive analysis this is fatal, however: as soon as you go into an alterative variation, and play an irreversible move, all your previous analysis will be purged from the hash table, as the positions are no longer reachable. But of course they remain reachable, through the take-back you will sooner or later do to back-poropagate the score of the newly analyzed variation.

The solution is quite simple: engines should realize that an interactive analysis session is in fact a single search, where the low plies are governed by the user, and the root positions of the various analysis searches are not root positions at all, but just internal nodes of the tree the user is constructing. As such, they should not increment the search number during analysis. They are just not different searches, but part of the larger one.

In my engines incrementing the search number only before searches intended to play in a game ('thinking'), and leaving it as it was in an analysis search, did solve the problem.
Well, you say what I said before, you just say it better. :)

BTW, are you still developing Joker? I've been away these last years and I need some updates about what works are still in progress.

About technical solution, I have no other reference than Critter example. Richard Vida built Critter session file as a separate, sizeable hash. With this solution he avoided complicated hash handlings and, most of all, file is updated every time a move is made on the board. That means one can resume an analysis when he wants, as all of his wokr was saved. It uses both forward and backward (2 plies) propagation when a score drop is detected. Not a perfect system, but the best I've ever seen.

The problem is, Critter could be 200 ELOs weaker than Komodo and fishes... That's why I'm asking if some top engine programmer is interested in developing this feature.
You'd think this [correct score propagation in the hash] was a top priority for commercial engine developers. I mean, so many people buy an engine for analysis purposes, so why wouldn't they provide an analysis-friendly engine? It makes no sense, other than to surmise that game-playing Elo is mostly everything to them. No wonder we're still in dark ages when it comes to good analysis features. [I do like IDEA, but it's a resource intensive hog of a program, and I prefer to interact with the analysis as much as possible]

As seen above, HGM is aware of the problem, and he even proposed an elegant solution; Richard Vida offered a good solution with his *free* (but now old) engine. If freeware programmers can offer improved analysis 'tools', why won't the top engine developers do the same or better, especially when they charge for their engines and they should 'owe' it to their customers?
Just wondering 'out loud'...

Regards,
CL

Rodolfo Leoni
Posts: 544
Joined: Tue Jun 06, 2017 2:49 pm
Location: Italy

Re: My "official" request to top engine programmer

Post by Rodolfo Leoni » Wed Jul 05, 2017 11:08 am

carldaman wrote:
Rodolfo Leoni wrote:
hgm wrote:The problem that scores of a variation get lost when you go deeper in a variation, and then return to (before) a previously analyzed one, is not related to persistent hash. The latter only makes it possible to transfer analysis between different sessions with the engine.

What you complain about (and also bugged me, when I started to do thorough analysis) is caused by a bad replacement strategy for hash entries. Most engines number the searches they do, and remember in the hash table which search was responsible for creating a stored entry, or remember what was the last search that actually used the stored information. If they then stumble on entries that are not used in any recent search (sometimes even just the current search), they assume the positions they refer to no longer are reachable from the current root position, and thus useless. They are treated as empty, and immediately overwritten.

This works very well in games, where no take-backs are done. Such as in the games used to determine the rating of the engine. For interactive analysis this is fatal, however: as soon as you go into an alterative variation, and play an irreversible move, all your previous analysis will be purged from the hash table, as the positions are no longer reachable. But of course they remain reachable, through the take-back you will sooner or later do to back-poropagate the score of the newly analyzed variation.

The solution is quite simple: engines should realize that an interactive analysis session is in fact a single search, where the low plies are governed by the user, and the root positions of the various analysis searches are not root positions at all, but just internal nodes of the tree the user is constructing. As such, they should not increment the search number during analysis. They are just not different searches, but part of the larger one.

In my engines incrementing the search number only before searches intended to play in a game ('thinking'), and leaving it as it was in an analysis search, did solve the problem.
Well, you say what I said before, you just say it better. :)

BTW, are you still developing Joker? I've been away these last years and I need some updates about what works are still in progress.

About technical solution, I have no other reference than Critter example. Richard Vida built Critter session file as a separate, sizeable hash. With this solution he avoided complicated hash handlings and, most of all, file is updated every time a move is made on the board. That means one can resume an analysis when he wants, as all of his wokr was saved. It uses both forward and backward (2 plies) propagation when a score drop is detected. Not a perfect system, but the best I've ever seen.

The problem is, Critter could be 200 ELOs weaker than Komodo and fishes... That's why I'm asking if some top engine programmer is interested in developing this feature.
You'd think this [correct score propagation in the hash] was a top priority for commercial engine developers. I mean, so many people buy an engine for analysis purposes, so why wouldn't they provide an analysis-friendly engine? It makes no sense, other than to surmise that game-playing Elo is mostly everything to them. No wonder we're still in dark ages when it comes to good analysis features. [I do like IDEA, but it's a resource intensive hog of a program, and I prefer to interact with the analysis as much as possible]

As seen above, HGM is aware of the problem, and he even proposed an elegant solution; Richard Vida offered a good solution with his *free* (but now old) engine. If freeware programmers can offer improved analysis 'tools', why won't the top engine developers do the same or better, especially when they charge for their engines and they should 'owe' it to their customers?
Just wondering 'out loud'...

Regards,
CL
Hi Carl,

I agree 100% with your clear and direct opinion. The choice is between self-oriented engines and user-oriented ones. It seems the direction everybody took is the fight for gaining few ELOs per year, and that's absolutely self-oriented. But if we thing an human being will use the engine, we should consider what's most important for him.

To become user-oriented could be next step of engines evolution...
F.S.I. Chess Teacher

Rodolfo Leoni
Posts: 544
Joined: Tue Jun 06, 2017 2:49 pm
Location: Italy

Re: My "official" request to top engine programmer

Post by Rodolfo Leoni » Wed Jul 05, 2017 11:15 am

Ferdy wrote: The good thing about it is that when you wake up in the morning you will be presented with analysis. Your questions the night before will be possibly answered, like what happen if I follow this line? What is wrong with this move? and others. The engine expansion algorithms are now more controllable by the user too. But you can disable the engine expansion if you really want to, it is just an option.
Maybe I'm a bit old-fashioned, but... what's the taste of IDeA doing all the work? There's a lot of fun when one makes own analyses, even with engines help.
Ferdy wrote:Did you mean mini-max? or backsolving in bookup? Mini-max in Aquarium is based on score. If not what do you mean exactly by score propagation?
I remember IDeA performed steps in sequence: variations deepening, alternative moves, and so on. At end of each step score was propagate. Yes, it could be the Min-Max process, the process that propagates scores down to the root position.
F.S.I. Chess Teacher

Ferdy
Posts: 3620
Joined: Sun Aug 10, 2008 1:15 pm
Location: Philippines

Re: My "official" request to top engine programmer

Post by Ferdy » Wed Jul 05, 2017 1:09 pm

Rodolfo Leoni wrote:Maybe I'm a bit old-fashioned, but... what's the taste of IDeA doing all the work? There's a lot of fun when one makes own analyses, even with engines help.
When you use idea to do the work by itself, you will taste later its analysis.

Example you setup e4 Nf6, then let idea do the work by itself. Later you will just browse the tree produced from idea. It will show you the best possible moves from a given position, it would also show you the 2nd best moves and others. Browsing the trees is also fun. But then again if you don't like it, just disable the automatic engine expansion.

It seems like you are a fun of infinite analysis, you can also send those positions from infinite analysis automatically to idea, so your infinite analysis sessions will not be lost, it is kept there in idea tree. You don't have to repeat your analysis again and again, you can simply load a particular idea session that you have done in the past, it can be alekhine idea or sicilian idea or nimzo-indian idea. Then start from there if you continue to work on it.

Rodolfo Leoni
Posts: 544
Joined: Tue Jun 06, 2017 2:49 pm
Location: Italy

Re: My "official" request to top engine programmer

Post by Rodolfo Leoni » Wed Jul 05, 2017 5:53 pm

Ferdy wrote: It seems like you are a fun of infinite analysis, you can also send those positions from infinite analysis automatically to idea, so your infinite analysis sessions will not be lost, it is kept there in idea tree. You don't have to repeat your analysis again and again, you can simply load a particular idea session that you have done in the past, it can be alekhine idea or sicilian idea or nimzo-indian idea. Then start from there if you continue to work on it.
I just think infinite analysis is a better tool, but an engine with PHs should be necessary to keep track ov variation scores.

I see IDeA evolved a lot in these years. With a PH engine and multipv mode it could be useful. Does it show engine PV(s) output while on analysis, or just its tree?
F.S.I. Chess Teacher

Post Reply