User – Engine Communication
Posted: Sun May 04, 2014 4:13 pm
In my opinion the latest art of communication between human and engine is still very primitive and leaves ample room for improvement. I am specifically talking about analyzing a certain chess position and refer to communication as the interchange of information with the aim to learn more about the characteristics of the specific position.
I did some brainstorming on the way the engine most effectively communicates its knowledge to the user and would really like to hear your thoughts on the topic. (How the user communicates his knowledge to the engine and guides its search is a different topic altogether)
I classify two use cases of the engine in position analysis: (1) explain the strongest plans and (2) anticipate perceived strong but flawed plans and explain refutations. Very closely related with (2) is also the case of explaining hidden threats, in which case the plan would be a null-move.
(1) is currently mainly implemented through showing one single PV or in multipv mode several PV for a series of moves of the current root position. It is then up to the user to play out the various moves to find interesting variations.
(2) is currently not implemented in engines.
The engine searches several million nodes per second – way too much as to process by a human being, so what are the nodes it actually ought to display to the user? What is the information the user actually wants to see? It depends. There are obviously several tradeoffs.
• Variation count vs. Variation depth: let’s say an engine deems a certain variation as interesting (this definition of an interesting variation should become clearer in the following points), how many moves of that variation should it display? There exist two objective measures: (a) only display the additional move, if the probability of it being part of the best line of play is greater than some threshold (or with the maximum number of moves to be displayed given, display the best n moves); (b) display moves, till a stand-pat is reached; note that currently engines are designed to minimize count and maximize depth.
• Width vs. depth: which variation should the engine rather show to the user – one at that root position that is an alternative to the PV or a variation to one of the moves of the PV? The neutral approach would again compare the probability of the particular variation being part of the “true” PV. E.g. if at the root position I have the choice between two moves, one, which we call the first move of the PV, is strongest with a probability of 70%, the alternative with 30%. After the first PV move, we have the choice between two equally good moves with 50% probability of being best each. In this case the alternative to the second PV move has a probability of being in the best line of 35% (70% * 50%) and should thus objectively be more relevant to the user than the alternative to the first PV move. That being said, it is perfectly understandable, that in certain situations the user prefers to see more variations closer to the root position and thus wants the engine to bias its choice of variations presented.
• Alpha-Beta: put less weight on displaying additional variations that counter non principle variations. That said, the engine should give the user the possibility to select multiple lines as PV.
As can be seen above, a lot of the assumptions depend on the estimation of probability of a move ending up as best move. Transpositions obviously complicate the issue significantly, however ignoring for that fact one can make some simple predictions just comparing evaluations of moves. For more elaborate measures one would have to conduct zero-window searches, to establish the score-potential of certain moves.
Regarding the “interesting refutation search” in (2), the engine should inform the user about evaluation drops in certain lines after a certain number of plies deep search. This depth should be (a) deep enough for the user not to be trivial and (b) the score jump should be large enough for the user to be relevant. The depth (a) should be settable by the user depending on skill-level; however I would count depth by trickiness for the user to spot. In that sense I would count checking moves, recaptures, repeated moving of the same piece, “local moves” and undefended interpositions as smaller than one ply and quiet moves as larger than one ply. The score difference (b) should also be settable depending on skill-level.
I did some brainstorming on the way the engine most effectively communicates its knowledge to the user and would really like to hear your thoughts on the topic. (How the user communicates his knowledge to the engine and guides its search is a different topic altogether)
I classify two use cases of the engine in position analysis: (1) explain the strongest plans and (2) anticipate perceived strong but flawed plans and explain refutations. Very closely related with (2) is also the case of explaining hidden threats, in which case the plan would be a null-move.
(1) is currently mainly implemented through showing one single PV or in multipv mode several PV for a series of moves of the current root position. It is then up to the user to play out the various moves to find interesting variations.
(2) is currently not implemented in engines.
The engine searches several million nodes per second – way too much as to process by a human being, so what are the nodes it actually ought to display to the user? What is the information the user actually wants to see? It depends. There are obviously several tradeoffs.
• Variation count vs. Variation depth: let’s say an engine deems a certain variation as interesting (this definition of an interesting variation should become clearer in the following points), how many moves of that variation should it display? There exist two objective measures: (a) only display the additional move, if the probability of it being part of the best line of play is greater than some threshold (or with the maximum number of moves to be displayed given, display the best n moves); (b) display moves, till a stand-pat is reached; note that currently engines are designed to minimize count and maximize depth.
• Width vs. depth: which variation should the engine rather show to the user – one at that root position that is an alternative to the PV or a variation to one of the moves of the PV? The neutral approach would again compare the probability of the particular variation being part of the “true” PV. E.g. if at the root position I have the choice between two moves, one, which we call the first move of the PV, is strongest with a probability of 70%, the alternative with 30%. After the first PV move, we have the choice between two equally good moves with 50% probability of being best each. In this case the alternative to the second PV move has a probability of being in the best line of 35% (70% * 50%) and should thus objectively be more relevant to the user than the alternative to the first PV move. That being said, it is perfectly understandable, that in certain situations the user prefers to see more variations closer to the root position and thus wants the engine to bias its choice of variations presented.
• Alpha-Beta: put less weight on displaying additional variations that counter non principle variations. That said, the engine should give the user the possibility to select multiple lines as PV.
As can be seen above, a lot of the assumptions depend on the estimation of probability of a move ending up as best move. Transpositions obviously complicate the issue significantly, however ignoring for that fact one can make some simple predictions just comparing evaluations of moves. For more elaborate measures one would have to conduct zero-window searches, to establish the score-potential of certain moves.
Regarding the “interesting refutation search” in (2), the engine should inform the user about evaluation drops in certain lines after a certain number of plies deep search. This depth should be (a) deep enough for the user not to be trivial and (b) the score jump should be large enough for the user to be relevant. The depth (a) should be settable by the user depending on skill-level; however I would count depth by trickiness for the user to spot. In that sense I would count checking moves, recaptures, repeated moving of the same piece, “local moves” and undefended interpositions as smaller than one ply and quiet moves as larger than one ply. The score difference (b) should also be settable depending on skill-level.