5 first choices

Discussion of anything and everything relating to chess playing software and machines.

Moderator: Ras

Lyudmil Tsvetkov
Posts: 6052
Joined: Tue Jun 12, 2012 12:41 pm

5 first choices

Post by Lyudmil Tsvetkov »

Here my Christmas puzzle to you.

Within a hypothesis an engine can play 2 main moves, A and B.

First 5 choice scores for move A in multi-PV after the line is deeply searched:

+30cps
0.0
0.0
-100cps
-200cps

First 5 choice scores for move B:

+20cps
+10cps
0.0
0.0
0.0

Which move should the engine play, A or B, and which move is stronger?

Obviously, the engine will like to play move A, as the 30cps score is best there, however, if you consider also the cumulative scores of the 5 first choices, one might conclude that actually move B is much better, especially bearing in mind the very close difference between the best scores for A and B.

Should only the first choice be important when considering what move to play, or maybe the engine can derive useful info also from its next-best choices, so that sometimes a first choice might become second, etc.?

The logic behind this is that the engine might go wrong with its eval assessment and search of a single move, but much more difficult so when it assesses and searches 5 or more moves.
Top
Ferdy
Posts: 4853
Joined: Sun Aug 10, 2008 3:15 pm
Location: Philippines

Re: 5 first choices

Post by Ferdy »

Lyudmil Tsvetkov wrote:Here my Christmas puzzle to you.

Within a hypothesis an engine can play 2 main moves, A and B.

First 5 choice scores for move A in multi-PV after the line is deeply searched:

+30cps
0.0
0.0
-100cps
-200cps

First 5 choice scores for move B:

+20cps
+10cps
0.0
0.0
0.0

Which move should the engine play, A or B, and which move is stronger?

Obviously, the engine will like to play move A, as the 30cps score is best there, however, if you consider also the cumulative scores of the 5 first choices, one might conclude that actually move B is much better, especially bearing in mind the very close difference between the best scores for A and B.

Should only the first choice be important when considering what move to play, or maybe the engine can derive useful info also from its next-best choices, so that sometimes a first choice might become second, etc.?

The logic behind this is that the engine might go wrong with its eval assessment and search of a single move, but much more difficult so when it assesses and searches 5 or more moves.
Interesting, I will try this in my engine, matching them using score difference between best and next best and the other always using the bestmove with the best score.

But I have a question to you, I am searching for techniques to determine the complixity of a given position using an engine. What I have done so far is experiment with the method used by Guid and Bratko, that is get the difference between best move and the next best move score, whenever an engine changes its best move at root as it increases its iteration depth. Example.

Code: Select all

depth 4 multipv 1 bestmove e2e4 score 10cp
depth 4 multipv 2 bestmove d2d4 score 8cp

depth 5 multipv 1 bestmove e2e4 score 9cp
depth 5 multipv 2 bestmove b1c3 score 6cp

depth 6 multipv 1 bestmove c2c4 score 12cp
depth 6 multipv 2 bestmove d2d4 score 8cp

depth 7 multipv 1 bestmove g1f3 score 11cp
depth 7 multipv 2 bestmove e2e4 score 7cp
Complexity = 12-8 + 11-7 = 4 + 4 = 8cp
Best move changes at depth 6 and 7.
The higher the value of complexity, the more it is complicated or difficult position.
What do you think of this idea? Is there any other method you may suggest to detect complexity of the position?

Here are some sample from my experiment so far.
Get pgn, convert to fen file, analyze each with sf at multipv 2, then monitor moves and score as it increases its depth until time expired.
The result in section B below showed that overall the game between Carlsen and Anand generates the highest average complexity value at 62.2, with a total complexity of 1929 from 32 position being evaluated. Note in that game (and all other games) positions below move 9 are not considered, also when engine bestscore is already above 300 cp or below -300cp. The most complex position according to this method and sf is between So and Vallejo.
[d]r1k2b1r/2p2pp1/1pn4p/pNp1Pb2/8/1P3N1P/PBPR1PP1/3R2K1 w - - 2 18
Wesley So - Francisco Vallejo-Pons; 49th Capablanca Memorial; 2014.05.09; r2.3; 1-0

[d]8/8/7p/pkb1P3/1p2K1P1/7P/8/4R3 w - - 0 40
Magnus Carlsen - Viswanathan Anand; Carlsen-Anand World Championship; 2014.11.23; r11; 1-0

[d]r3b2r/pk6/1pn3p1/2p2P2/2P2Bp1/2N4P/PP3K2/R2R4 w - - 2 24
Anish Giri - Hikaru Nakamura; FIDE Grand Prix Paris; 2013.10.04; r11; 1/2-1/2

Code: Select all

A. Test settings:
Analysis engine : id name Stockfish 201214 64 POPCNT
Threads         : 2
MultiPV         : 2
Hash            : 128 mb
Time/pos        : 2.0 sec
Positions from  : Ruy Lopez, Berlin defense C67

A.1. Legend:
RMC = root move change starting at iteration 4
MDE = maximum search depth reached by analysis engine
COM = complexity in centipawn (cp)

RMC  9, MDE  19, COM +216, FEN r1k2b1r/2p2pp1/1pn4p/pNp1Pb2/8/1P3N1P/PBPR1PP1/3R2K1 w - - 2 18,       Wesley So - Francisco Vallejo-Pons; 49th Capablanca Memorial; 2014.05.09; r2.3; 1-0
RMC  3, MDE  22, COM +192, FEN 8/8/7p/pkb1P3/1p2K1P1/7P/8/4R3 w - - 0 40,                             Magnus Carlsen - Viswanathan Anand; Carlsen-Anand World Championship; 2014.11.23; r11; 1-0
RMC  6, MDE  19, COM +178, FEN r3b2r/pk6/1pn3p1/2p2P2/2P2Bp1/2N4P/PP3K2/R2R4 w - - 2 24,              Anish Giri - Hikaru Nakamura; FIDE Grand Prix Paris; 2013.10.04; r11; 1/2-1/2
RMC  9, MDE  21, COM +169, FEN 8/4b3/4P3/1k4p1/p6P/1p1K4/8/4R3 w - - 0 44,                            Magnus Carlsen - Viswanathan Anand; Carlsen-Anand World Championship; 2014.11.23; r11; 1-0
RMC  8, MDE  18, COM +139, FEN r1k1bb1r/ppp1npp1/2p5/4P1Np/5PP1/7P/PPP5/RNBR2K1 w - - 0 14,           Anish Giri - Hikaru Nakamura; FIDE Grand Prix Paris; 2013.10.04; r11; 1/2-1/2
RMC  8, MDE  20, COM +119, FEN r1k2b1r/p1p2p2/1p2b2p/2pNPnp1/8/1P3N1P/PBP2PP1/3RR1K1 w - - 0 16,      Magnus Carlsen - Viswanathan Anand; Carlsen-Anand World Championship; 2014.11.23; r11; 1-0
RMC  5, MDE  19, COM +118, FEN r4b2/1kp2p2/4b2p/p2NP3/PpP1KnP1/7P/8/3RR3 w - - 0 32,                  Magnus Carlsen - Viswanathan Anand; Carlsen-Anand World Championship; 2014.11.23; r11; 1-0
RMC  9, MDE  17, COM +114, FEN r2k1b1r/pppb1ppp/2p5/4Pn2/8/5N1P/PPP2PP1/RNB2RK1 w - - 1 10,           Magnus Carlsen - Viswanathan Anand; Carlsen-Anand World Championship; 2014.11.23; r11; 1-0
RMC  6, MDE  20, COM +113, FEN 2b1rk2/1pp1npp1/p1p4r/P3P3/2NNR2p/1P3P1P/2PR1KP1/8 w - - 7 29,         Vassily Ivanchuk - Levon Aronian; Tal Memorial; 2011.11.18; r3; 0-1
RMC  8, MDE  19, COM +110, FEN r3bb1r/pkp1npp1/1pp5/4PPNp/6P1/7P/PPP2K2/RNBR4 w - - 1 16,             Anish Giri - Hikaru Nakamura; FIDE Grand Prix Paris; 2013.10.04; r11; 1/2-1/2
RMC  9, MDE  19, COM +109, FEN r1k4r/2p1bp2/1pn1b1pp/pNp1P3/5PPN/1P5P/PBPR4/3R2K1 w - - 1 21,         Wesley So - Francisco Vallejo-Pons; 49th Capablanca Memorial; 2014.05.09; r2.3; 1-0
RMC  7, MDE  17, COM +105, FEN r2r1b2/2p2p2/2k1bNnp/p1pNP1p1/P1P3P1/2B3KP/5P2/3RR3 w - - 1 26,        Magnus Carlsen - Viswanathan Anand; Carlsen-Anand World Championship; 2014.11.23; r11; 1-0
RMC  4, MDE  21, COM +105, FEN 8/8/4P2p/pkb5/4K1P1/1p5P/8/4R3 w - - 0 41,                             Magnus Carlsen - Viswanathan Anand; Carlsen-Anand World Championship; 2014.11.23; r11; 1-0
RMC 10, MDE  21, COM +102, FEN 2br3r/ppp1kpp1/2p5/3nP3/2N4p/PP3P1P/2P1N1P1/3RR1K1 w - - 0 21,         Vassily Ivanchuk - Levon Aronian; Tal Memorial; 2011.11.18; r3; 0-1
RMC  3, MDE  19, COM  +99, FEN 8/2N5/2k4p/p1b1P3/Ppb1K1P1/7P/8/4R3 w - - 1 38,                        Magnus Carlsen - Viswanathan Anand; Carlsen-Anand World Championship; 2014.11.23; r11; 1-0
RMC  8, MDE  17, COM  +97, FEN r1k2b1r/p1pb1pp1/1p5p/2p1Pn2/8/1PN2N1P/PBP2PP1/3R1RK1 w - - 0 14,      Magnus Carlsen - Viswanathan Anand; Carlsen-Anand World Championship; 2014.11.23; r11; 1-0
RMC  5, MDE  17, COM  +94, FEN r2k1b1r/pppb1ppp/2p5/4Pn2/8/5N1P/PPP2PP1/RNB2RK1 w - - 1 10,           Anish Giri - Hikaru Nakamura; FIDE Grand Prix Paris; 2013.10.04; r11; 1/2-1/2
RMC  2, MDE  20, COM  +94, FEN 6r1/pk6/1p2n3/2p3P1/2P2p2/2K1B3/PP6/5R2 w - - 0 34,                    Anish Giri - Hikaru Nakamura; FIDE Grand Prix Paris; 2013.10.04; r11; 1/2-1/2
RMC  5, MDE  19, COM  +89, FEN 2b1rk2/1pp2pp1/p6r/P1p1Pn2/2N2N1p/1P3K1P/2PR2P1/4R3 w - - 0 35,        Vassily Ivanchuk - Levon Aronian; Tal Memorial; 2011.11.18; r3; 0-1
RMC  6, MDE  17, COM  +88, FEN r1k2b1r/pppb1ppp/2p5/4Pn2/8/5N1P/PPP2PP1/RNBR2K1 w - - 3 11,           Anish Giri - Hikaru Nakamura; FIDE Grand Prix Paris; 2013.10.04; r11; 1/2-1/2
RMC  4, MDE  20, COM  +85, FEN 6r1/2k5/1p2b3/p1p1Pp2/P1P4p/1P2N2P/3R1K2/8 w - - 0 35,                 Wesley So - Francisco Vallejo-Pons; 49th Capablanca Memorial; 2014.05.09; r2.3; 1-0
RMC  1, MDE  22, COM  +82, FEN 4rk2/6p1/pp3p2/2p1nr2/2P1R2p/2PN3P/6P1/4R1K1 w - - 0 45,               Vassily Ivanchuk - Levon Aronian; Tal Memorial; 2011.11.18; r3; 0-1

[...]

Code: Select all

B. Complexity in cp for a white player in a unique game:

Overall complexity mean: 55.3
Standard deviation (SD): 7.5

 1. Magnus Carlsen - Viswanathan Anand; Carlsen-Anand World Championship; 2014.11.23; r11; 1-0
    Game mean complexity  62.2 (1929/31 ) normal (within 1 SD)

 2. Anish Giri - Hikaru Nakamura; FIDE Grand Prix Paris; 2013.10.04; r11; 1/2-1/2
    Game mean complexity  58.7 (1408/24 ) normal (within 1 SD)

 3. Wesley So - Francisco Vallejo-Pons; 49th Capablanca Memorial; 2014.05.09; r2.3; 1-0
    Game mean complexity  55.4 (1495/27 ) normal (within 1 SD)

 4. Vassily Ivanchuk - Levon Aronian; Tal Memorial; 2011.11.18; r3; 0-1
    Game mean complexity  44.9 (1795/40 ) low    (below 1 SD)


C. Platform
System    : Windows
Release   : 7
Version   : 6.1.7601
Machine   : AMD64
processor : Intel64 Family 6 Model 42 Stepping 7, GenuineIntel

Python version:
3.4.1 (v3.4.1:c0e311e010fc, May 18 2014, 10:45:13) [MSC v.1600 64 bit (AMD64)]
Top
Lyudmil Tsvetkov
Posts: 6052
Joined: Tue Jun 12, 2012 12:41 pm

Re: 5 first choices

Post by Lyudmil Tsvetkov »

Ferdy wrote:
Lyudmil Tsvetkov wrote:Here my Christmas puzzle to you.

Within a hypothesis an engine can play 2 main moves, A and B.

First 5 choice scores for move A in multi-PV after the line is deeply searched:

+30cps
0.0
0.0
-100cps
-200cps

First 5 choice scores for move B:

+20cps
+10cps
0.0
0.0
0.0

Which move should the engine play, A or B, and which move is stronger?

Obviously, the engine will like to play move A, as the 30cps score is best there, however, if you consider also the cumulative scores of the 5 first choices, one might conclude that actually move B is much better, especially bearing in mind the very close difference between the best scores for A and B.

Should only the first choice be important when considering what move to play, or maybe the engine can derive useful info also from its next-best choices, so that sometimes a first choice might become second, etc.?

The logic behind this is that the engine might go wrong with its eval assessment and search of a single move, but much more difficult so when it assesses and searches 5 or more moves.
Interesting, I will try this in my engine, matching them using score difference between best and next best and the other always using the bestmove with the best score.

But I have a question to you, I am searching for techniques to determine the complixity of a given position using an engine. What I have done so far is experiment with the method used by Guid and Bratko, that is get the difference between best move and the next best move score, whenever an engine changes its best move at root as it increases its iteration depth. Example.

Code: Select all

depth 4 multipv 1 bestmove e2e4 score 10cp
depth 4 multipv 2 bestmove d2d4 score 8cp

depth 5 multipv 1 bestmove e2e4 score 9cp
depth 5 multipv 2 bestmove b1c3 score 6cp

depth 6 multipv 1 bestmove c2c4 score 12cp
depth 6 multipv 2 bestmove d2d4 score 8cp

depth 7 multipv 1 bestmove g1f3 score 11cp
depth 7 multipv 2 bestmove e2e4 score 7cp
Complexity = 12-8 + 11-7 = 4 + 4 = 8cp
Best move changes at depth 6 and 7.
The higher the value of complexity, the more it is complicated or difficult position.
What do you think of this idea? Is there any other method you may suggest to detect complexity of the position?

Here are some sample from my experiment so far.
Get pgn, convert to fen file, analyze each with sf at multipv 2, then monitor moves and score as it increases its depth until time expired.
The result in section B below showed that overall the game between Carlsen and Anand generates the highest average complexity value at 62.2, with a total complexity of 1929 from 32 position being evaluated. Note in that game (and all other games) positions below move 9 are not considered, also when engine bestscore is already above 300 cp or below -300cp. The most complex position according to this method and sf is between So and Vallejo.
[d]r1k2b1r/2p2pp1/1pn4p/pNp1Pb2/8/1P3N1P/PBPR1PP1/3R2K1 w - - 2 18
Wesley So - Francisco Vallejo-Pons; 49th Capablanca Memorial; 2014.05.09; r2.3; 1-0

[d]8/8/7p/pkb1P3/1p2K1P1/7P/8/4R3 w - - 0 40
Magnus Carlsen - Viswanathan Anand; Carlsen-Anand World Championship; 2014.11.23; r11; 1-0

[d]r3b2r/pk6/1pn3p1/2p2P2/2P2Bp1/2N4P/PP3K2/R2R4 w - - 2 24
Anish Giri - Hikaru Nakamura; FIDE Grand Prix Paris; 2013.10.04; r11; 1/2-1/2

Code: Select all

A. Test settings:
Analysis engine : id name Stockfish 201214 64 POPCNT
Threads         : 2
MultiPV         : 2
Hash            : 128 mb
Time/pos        : 2.0 sec
Positions from  : Ruy Lopez, Berlin defense C67

A.1. Legend:
RMC = root move change starting at iteration 4
MDE = maximum search depth reached by analysis engine
COM = complexity in centipawn (cp)

RMC  9, MDE  19, COM +216, FEN r1k2b1r/2p2pp1/1pn4p/pNp1Pb2/8/1P3N1P/PBPR1PP1/3R2K1 w - - 2 18,       Wesley So - Francisco Vallejo-Pons; 49th Capablanca Memorial; 2014.05.09; r2.3; 1-0
RMC  3, MDE  22, COM +192, FEN 8/8/7p/pkb1P3/1p2K1P1/7P/8/4R3 w - - 0 40,                             Magnus Carlsen - Viswanathan Anand; Carlsen-Anand World Championship; 2014.11.23; r11; 1-0
RMC  6, MDE  19, COM +178, FEN r3b2r/pk6/1pn3p1/2p2P2/2P2Bp1/2N4P/PP3K2/R2R4 w - - 2 24,              Anish Giri - Hikaru Nakamura; FIDE Grand Prix Paris; 2013.10.04; r11; 1/2-1/2
RMC  9, MDE  21, COM +169, FEN 8/4b3/4P3/1k4p1/p6P/1p1K4/8/4R3 w - - 0 44,                            Magnus Carlsen - Viswanathan Anand; Carlsen-Anand World Championship; 2014.11.23; r11; 1-0
RMC  8, MDE  18, COM +139, FEN r1k1bb1r/ppp1npp1/2p5/4P1Np/5PP1/7P/PPP5/RNBR2K1 w - - 0 14,           Anish Giri - Hikaru Nakamura; FIDE Grand Prix Paris; 2013.10.04; r11; 1/2-1/2
RMC  8, MDE  20, COM +119, FEN r1k2b1r/p1p2p2/1p2b2p/2pNPnp1/8/1P3N1P/PBP2PP1/3RR1K1 w - - 0 16,      Magnus Carlsen - Viswanathan Anand; Carlsen-Anand World Championship; 2014.11.23; r11; 1-0
RMC  5, MDE  19, COM +118, FEN r4b2/1kp2p2/4b2p/p2NP3/PpP1KnP1/7P/8/3RR3 w - - 0 32,                  Magnus Carlsen - Viswanathan Anand; Carlsen-Anand World Championship; 2014.11.23; r11; 1-0
RMC  9, MDE  17, COM +114, FEN r2k1b1r/pppb1ppp/2p5/4Pn2/8/5N1P/PPP2PP1/RNB2RK1 w - - 1 10,           Magnus Carlsen - Viswanathan Anand; Carlsen-Anand World Championship; 2014.11.23; r11; 1-0
RMC  6, MDE  20, COM +113, FEN 2b1rk2/1pp1npp1/p1p4r/P3P3/2NNR2p/1P3P1P/2PR1KP1/8 w - - 7 29,         Vassily Ivanchuk - Levon Aronian; Tal Memorial; 2011.11.18; r3; 0-1
RMC  8, MDE  19, COM +110, FEN r3bb1r/pkp1npp1/1pp5/4PPNp/6P1/7P/PPP2K2/RNBR4 w - - 1 16,             Anish Giri - Hikaru Nakamura; FIDE Grand Prix Paris; 2013.10.04; r11; 1/2-1/2
RMC  9, MDE  19, COM +109, FEN r1k4r/2p1bp2/1pn1b1pp/pNp1P3/5PPN/1P5P/PBPR4/3R2K1 w - - 1 21,         Wesley So - Francisco Vallejo-Pons; 49th Capablanca Memorial; 2014.05.09; r2.3; 1-0
RMC  7, MDE  17, COM +105, FEN r2r1b2/2p2p2/2k1bNnp/p1pNP1p1/P1P3P1/2B3KP/5P2/3RR3 w - - 1 26,        Magnus Carlsen - Viswanathan Anand; Carlsen-Anand World Championship; 2014.11.23; r11; 1-0
RMC  4, MDE  21, COM +105, FEN 8/8/4P2p/pkb5/4K1P1/1p5P/8/4R3 w - - 0 41,                             Magnus Carlsen - Viswanathan Anand; Carlsen-Anand World Championship; 2014.11.23; r11; 1-0
RMC 10, MDE  21, COM +102, FEN 2br3r/ppp1kpp1/2p5/3nP3/2N4p/PP3P1P/2P1N1P1/3RR1K1 w - - 0 21,         Vassily Ivanchuk - Levon Aronian; Tal Memorial; 2011.11.18; r3; 0-1
RMC  3, MDE  19, COM  +99, FEN 8/2N5/2k4p/p1b1P3/Ppb1K1P1/7P/8/4R3 w - - 1 38,                        Magnus Carlsen - Viswanathan Anand; Carlsen-Anand World Championship; 2014.11.23; r11; 1-0
RMC  8, MDE  17, COM  +97, FEN r1k2b1r/p1pb1pp1/1p5p/2p1Pn2/8/1PN2N1P/PBP2PP1/3R1RK1 w - - 0 14,      Magnus Carlsen - Viswanathan Anand; Carlsen-Anand World Championship; 2014.11.23; r11; 1-0
RMC  5, MDE  17, COM  +94, FEN r2k1b1r/pppb1ppp/2p5/4Pn2/8/5N1P/PPP2PP1/RNB2RK1 w - - 1 10,           Anish Giri - Hikaru Nakamura; FIDE Grand Prix Paris; 2013.10.04; r11; 1/2-1/2
RMC  2, MDE  20, COM  +94, FEN 6r1/pk6/1p2n3/2p3P1/2P2p2/2K1B3/PP6/5R2 w - - 0 34,                    Anish Giri - Hikaru Nakamura; FIDE Grand Prix Paris; 2013.10.04; r11; 1/2-1/2
RMC  5, MDE  19, COM  +89, FEN 2b1rk2/1pp2pp1/p6r/P1p1Pn2/2N2N1p/1P3K1P/2PR2P1/4R3 w - - 0 35,        Vassily Ivanchuk - Levon Aronian; Tal Memorial; 2011.11.18; r3; 0-1
RMC  6, MDE  17, COM  +88, FEN r1k2b1r/pppb1ppp/2p5/4Pn2/8/5N1P/PPP2PP1/RNBR2K1 w - - 3 11,           Anish Giri - Hikaru Nakamura; FIDE Grand Prix Paris; 2013.10.04; r11; 1/2-1/2
RMC  4, MDE  20, COM  +85, FEN 6r1/2k5/1p2b3/p1p1Pp2/P1P4p/1P2N2P/3R1K2/8 w - - 0 35,                 Wesley So - Francisco Vallejo-Pons; 49th Capablanca Memorial; 2014.05.09; r2.3; 1-0
RMC  1, MDE  22, COM  +82, FEN 4rk2/6p1/pp3p2/2p1nr2/2P1R2p/2PN3P/6P1/4R1K1 w - - 0 45,               Vassily Ivanchuk - Levon Aronian; Tal Memorial; 2011.11.18; r3; 0-1

[...]

Code: Select all

B. Complexity in cp for a white player in a unique game:

Overall complexity mean: 55.3
Standard deviation (SD): 7.5

 1. Magnus Carlsen - Viswanathan Anand; Carlsen-Anand World Championship; 2014.11.23; r11; 1-0
    Game mean complexity  62.2 (1929/31 ) normal (within 1 SD)

 2. Anish Giri - Hikaru Nakamura; FIDE Grand Prix Paris; 2013.10.04; r11; 1/2-1/2
    Game mean complexity  58.7 (1408/24 ) normal (within 1 SD)

 3. Wesley So - Francisco Vallejo-Pons; 49th Capablanca Memorial; 2014.05.09; r2.3; 1-0
    Game mean complexity  55.4 (1495/27 ) normal (within 1 SD)

 4. Vassily Ivanchuk - Levon Aronian; Tal Memorial; 2011.11.18; r3; 0-1
    Game mean complexity  44.9 (1795/40 ) low    (below 1 SD)


C. Platform
System    : Windows
Release   : 7
Version   : 6.1.7601
Machine   : AMD64
processor : Intel64 Family 6 Model 42 Stepping 7, GenuineIntel

Python version:
3.4.1 (v3.4.1:c0e311e010fc, May 18 2014, 10:45:13) [MSC v.1600 64 bit (AMD64)]
Sorry, Ferdinand,

never thought what the difference between the best and second best move might account for, or be a reflection of.

Absolutely never. I am sure you have much more data on this, however, if you would like my layman's judgement, I think comparing just 2 moves might involve a fair share of chance, randomness concerning the nature of the position.

I do not think you can characterise a position by just 2 moves, but 5 or more moves certainly should be able to do that. If you compare 5 moves, then maybe you have sufficient statistical data to know what the position looks like.

For example, if only 1 move out of 5 is positive, all other negative, this might indicate such a position is a bit dubious.

In the above case, something simply could have gone wrong with search or eval of the best move.

In another position, where you have couple of positive scores, nothing to worry about.

So I would prefer to pick up a line where all 5 moves have a slight positive score, rather than a position with a single best move positive score, even if that score is bigger than the best move of the first instance, and all other 4 moves being negative.

But sorry again, I do not really think this has been sufficiently investigated, what the scores of the first 5 or 10 moves could account for, especially when they differ significantly in value. I also have not thought about that, just some very random impressions.

That is why I asked you to share your opinion.
Top
bob
Posts: 20943
Joined: Mon Feb 27, 2006 7:30 pm
Location: Birmingham, AL

Re: 5 first choices

Post by bob »

Ferdy wrote:
Lyudmil Tsvetkov wrote:Here my Christmas puzzle to you.

Within a hypothesis an engine can play 2 main moves, A and B.

First 5 choice scores for move A in multi-PV after the line is deeply searched:

+30cps
0.0
0.0
-100cps
-200cps

First 5 choice scores for move B:

+20cps
+10cps
0.0
0.0
0.0

Which move should the engine play, A or B, and which move is stronger?

Obviously, the engine will like to play move A, as the 30cps score is best there, however, if you consider also the cumulative scores of the 5 first choices, one might conclude that actually move B is much better, especially bearing in mind the very close difference between the best scores for A and B.

Should only the first choice be important when considering what move to play, or maybe the engine can derive useful info also from its next-best choices, so that sometimes a first choice might become second, etc.?

The logic behind this is that the engine might go wrong with its eval assessment and search of a single move, but much more difficult so when it assesses and searches 5 or more moves.
Interesting, I will try this in my engine, matching them using score difference between best and next best and the other always using the bestmove with the best score.

But I have a question to you, I am searching for techniques to determine the complixity of a given position using an engine. What I have done so far is experiment with the method used by Guid and Bratko, that is get the difference between best move and the next best move score, whenever an engine changes its best move at root as it increases its iteration depth. Example.

Code: Select all

depth 4 multipv 1 bestmove e2e4 score 10cp
depth 4 multipv 2 bestmove d2d4 score 8cp

depth 5 multipv 1 bestmove e2e4 score 9cp
depth 5 multipv 2 bestmove b1c3 score 6cp

depth 6 multipv 1 bestmove c2c4 score 12cp
depth 6 multipv 2 bestmove d2d4 score 8cp

depth 7 multipv 1 bestmove g1f3 score 11cp
depth 7 multipv 2 bestmove e2e4 score 7cp
Complexity = 12-8 + 11-7 = 4 + 4 = 8cp
Best move changes at depth 6 and 7.
The higher the value of complexity, the more it is complicated or difficult position.
What do you think of this idea? Is there any other method you may suggest to detect complexity of the position?

Here are some sample from my experiment so far.
Get pgn, convert to fen file, analyze each with sf at multipv 2, then monitor moves and score as it increases its depth until time expired.
The result in section B below showed that overall the game between Carlsen and Anand generates the highest average complexity value at 62.2, with a total complexity of 1929 from 32 position being evaluated. Note in that game (and all other games) positions below move 9 are not considered, also when engine bestscore is already above 300 cp or below -300cp. The most complex position according to this method and sf is between So and Vallejo.
[d]r1k2b1r/2p2pp1/1pn4p/pNp1Pb2/8/1P3N1P/PBPR1PP1/3R2K1 w - - 2 18
Wesley So - Francisco Vallejo-Pons; 49th Capablanca Memorial; 2014.05.09; r2.3; 1-0

[d]8/8/7p/pkb1P3/1p2K1P1/7P/8/4R3 w - - 0 40
Magnus Carlsen - Viswanathan Anand; Carlsen-Anand World Championship; 2014.11.23; r11; 1-0

[d]r3b2r/pk6/1pn3p1/2p2P2/2P2Bp1/2N4P/PP3K2/R2R4 w - - 2 24
Anish Giri - Hikaru Nakamura; FIDE Grand Prix Paris; 2013.10.04; r11; 1/2-1/2

Code: Select all

A. Test settings:
Analysis engine : id name Stockfish 201214 64 POPCNT
Threads         : 2
MultiPV         : 2
Hash            : 128 mb
Time/pos        : 2.0 sec
Positions from  : Ruy Lopez, Berlin defense C67

A.1. Legend:
RMC = root move change starting at iteration 4
MDE = maximum search depth reached by analysis engine
COM = complexity in centipawn (cp)

RMC  9, MDE  19, COM +216, FEN r1k2b1r/2p2pp1/1pn4p/pNp1Pb2/8/1P3N1P/PBPR1PP1/3R2K1 w - - 2 18,       Wesley So - Francisco Vallejo-Pons; 49th Capablanca Memorial; 2014.05.09; r2.3; 1-0
RMC  3, MDE  22, COM +192, FEN 8/8/7p/pkb1P3/1p2K1P1/7P/8/4R3 w - - 0 40,                             Magnus Carlsen - Viswanathan Anand; Carlsen-Anand World Championship; 2014.11.23; r11; 1-0
RMC  6, MDE  19, COM +178, FEN r3b2r/pk6/1pn3p1/2p2P2/2P2Bp1/2N4P/PP3K2/R2R4 w - - 2 24,              Anish Giri - Hikaru Nakamura; FIDE Grand Prix Paris; 2013.10.04; r11; 1/2-1/2
RMC  9, MDE  21, COM +169, FEN 8/4b3/4P3/1k4p1/p6P/1p1K4/8/4R3 w - - 0 44,                            Magnus Carlsen - Viswanathan Anand; Carlsen-Anand World Championship; 2014.11.23; r11; 1-0
RMC  8, MDE  18, COM +139, FEN r1k1bb1r/ppp1npp1/2p5/4P1Np/5PP1/7P/PPP5/RNBR2K1 w - - 0 14,           Anish Giri - Hikaru Nakamura; FIDE Grand Prix Paris; 2013.10.04; r11; 1/2-1/2
RMC  8, MDE  20, COM +119, FEN r1k2b1r/p1p2p2/1p2b2p/2pNPnp1/8/1P3N1P/PBP2PP1/3RR1K1 w - - 0 16,      Magnus Carlsen - Viswanathan Anand; Carlsen-Anand World Championship; 2014.11.23; r11; 1-0
RMC  5, MDE  19, COM +118, FEN r4b2/1kp2p2/4b2p/p2NP3/PpP1KnP1/7P/8/3RR3 w - - 0 32,                  Magnus Carlsen - Viswanathan Anand; Carlsen-Anand World Championship; 2014.11.23; r11; 1-0
RMC  9, MDE  17, COM +114, FEN r2k1b1r/pppb1ppp/2p5/4Pn2/8/5N1P/PPP2PP1/RNB2RK1 w - - 1 10,           Magnus Carlsen - Viswanathan Anand; Carlsen-Anand World Championship; 2014.11.23; r11; 1-0
RMC  6, MDE  20, COM +113, FEN 2b1rk2/1pp1npp1/p1p4r/P3P3/2NNR2p/1P3P1P/2PR1KP1/8 w - - 7 29,         Vassily Ivanchuk - Levon Aronian; Tal Memorial; 2011.11.18; r3; 0-1
RMC  8, MDE  19, COM +110, FEN r3bb1r/pkp1npp1/1pp5/4PPNp/6P1/7P/PPP2K2/RNBR4 w - - 1 16,             Anish Giri - Hikaru Nakamura; FIDE Grand Prix Paris; 2013.10.04; r11; 1/2-1/2
RMC  9, MDE  19, COM +109, FEN r1k4r/2p1bp2/1pn1b1pp/pNp1P3/5PPN/1P5P/PBPR4/3R2K1 w - - 1 21,         Wesley So - Francisco Vallejo-Pons; 49th Capablanca Memorial; 2014.05.09; r2.3; 1-0
RMC  7, MDE  17, COM +105, FEN r2r1b2/2p2p2/2k1bNnp/p1pNP1p1/P1P3P1/2B3KP/5P2/3RR3 w - - 1 26,        Magnus Carlsen - Viswanathan Anand; Carlsen-Anand World Championship; 2014.11.23; r11; 1-0
RMC  4, MDE  21, COM +105, FEN 8/8/4P2p/pkb5/4K1P1/1p5P/8/4R3 w - - 0 41,                             Magnus Carlsen - Viswanathan Anand; Carlsen-Anand World Championship; 2014.11.23; r11; 1-0
RMC 10, MDE  21, COM +102, FEN 2br3r/ppp1kpp1/2p5/3nP3/2N4p/PP3P1P/2P1N1P1/3RR1K1 w - - 0 21,         Vassily Ivanchuk - Levon Aronian; Tal Memorial; 2011.11.18; r3; 0-1
RMC  3, MDE  19, COM  +99, FEN 8/2N5/2k4p/p1b1P3/Ppb1K1P1/7P/8/4R3 w - - 1 38,                        Magnus Carlsen - Viswanathan Anand; Carlsen-Anand World Championship; 2014.11.23; r11; 1-0
RMC  8, MDE  17, COM  +97, FEN r1k2b1r/p1pb1pp1/1p5p/2p1Pn2/8/1PN2N1P/PBP2PP1/3R1RK1 w - - 0 14,      Magnus Carlsen - Viswanathan Anand; Carlsen-Anand World Championship; 2014.11.23; r11; 1-0
RMC  5, MDE  17, COM  +94, FEN r2k1b1r/pppb1ppp/2p5/4Pn2/8/5N1P/PPP2PP1/RNB2RK1 w - - 1 10,           Anish Giri - Hikaru Nakamura; FIDE Grand Prix Paris; 2013.10.04; r11; 1/2-1/2
RMC  2, MDE  20, COM  +94, FEN 6r1/pk6/1p2n3/2p3P1/2P2p2/2K1B3/PP6/5R2 w - - 0 34,                    Anish Giri - Hikaru Nakamura; FIDE Grand Prix Paris; 2013.10.04; r11; 1/2-1/2
RMC  5, MDE  19, COM  +89, FEN 2b1rk2/1pp2pp1/p6r/P1p1Pn2/2N2N1p/1P3K1P/2PR2P1/4R3 w - - 0 35,        Vassily Ivanchuk - Levon Aronian; Tal Memorial; 2011.11.18; r3; 0-1
RMC  6, MDE  17, COM  +88, FEN r1k2b1r/pppb1ppp/2p5/4Pn2/8/5N1P/PPP2PP1/RNBR2K1 w - - 3 11,           Anish Giri - Hikaru Nakamura; FIDE Grand Prix Paris; 2013.10.04; r11; 1/2-1/2
RMC  4, MDE  20, COM  +85, FEN 6r1/2k5/1p2b3/p1p1Pp2/P1P4p/1P2N2P/3R1K2/8 w - - 0 35,                 Wesley So - Francisco Vallejo-Pons; 49th Capablanca Memorial; 2014.05.09; r2.3; 1-0
RMC  1, MDE  22, COM  +82, FEN 4rk2/6p1/pp3p2/2p1nr2/2P1R2p/2PN3P/6P1/4R1K1 w - - 0 45,               Vassily Ivanchuk - Levon Aronian; Tal Memorial; 2011.11.18; r3; 0-1

[...]

Code: Select all

B. Complexity in cp for a white player in a unique game:

Overall complexity mean: 55.3
Standard deviation (SD): 7.5

 1. Magnus Carlsen - Viswanathan Anand; Carlsen-Anand World Championship; 2014.11.23; r11; 1-0
    Game mean complexity  62.2 (1929/31 ) normal (within 1 SD)

 2. Anish Giri - Hikaru Nakamura; FIDE Grand Prix Paris; 2013.10.04; r11; 1/2-1/2
    Game mean complexity  58.7 (1408/24 ) normal (within 1 SD)

 3. Wesley So - Francisco Vallejo-Pons; 49th Capablanca Memorial; 2014.05.09; r2.3; 1-0
    Game mean complexity  55.4 (1495/27 ) normal (within 1 SD)

 4. Vassily Ivanchuk - Levon Aronian; Tal Memorial; 2011.11.18; r3; 0-1
    Game mean complexity  44.9 (1795/40 ) low    (below 1 SD)


C. Platform
System    : Windows
Release   : 7
Version   : 6.1.7601
Machine   : AMD64
processor : Intel64 Family 6 Model 42 Stepping 7, GenuineIntel

Python version:
3.4.1 (v3.4.1:c0e311e010fc, May 18 2014, 10:45:13) [MSC v.1600 64 bit (AMD64)]
I think the thing you are trying to reach is harder to compute, but here's a starting point.

What if you look at the best score for each move at the root. The more bad moves there are (or the more "singular" the root position is since there is only one or two good moves to play) the more complex the position is. But apply this recursively. In essence, measure the "true width" of the tree, where "true width" represents the number of playable moves at each point. The narrower this width, the more precise you have to calculate to avoid making a blunder.

There are alternatives that have been discussed. For example, at position P a shallow search suggests that move A or B is playable, and that the rest are poorer. But a much deeper search shows that A or B are both bad and there is a better move C but it is hard to find.

We've had this discussion for at least 30 years, using the term "discernibility"- how easy or hard is it to find the correct move (or avoid a bad move) at some point in the tree? It is an interesting question, with a large number of difficult-to-measure/implement ideas. HiTech had a very elementary version of this idea in use. The case where for N iterations move M is best, but at iteration N+1 the bottom falls out and a new move M' is found to be best. But this move also has a score that is significantly worse than the previous iteration. What to do? HiTech simply played the original best move. The reasoning was "this move looked best until a very deep search proved it to be inferior, my new best choice is still inferior, just not as inferior as the old best choice, therefore rather than playing a move I see is going to lead to trouble, I will play my original best move and see if my opponent can work out the tactical details and find out why it is not so good." Against humans, it works well. I've tried it in the past. Against computers, it only works if you are significantly out-searching your opponent, something that is dangerous to assume.
Top
Ferdy
Posts: 4853
Joined: Sun Aug 10, 2008 3:15 pm
Location: Philippines

Re: 5 first choices

Post by Ferdy »

Sorry, Ferdinand,

never thought what the difference between the best and second best move might account for, or be a reflection of.

Absolutely never. I am sure you have much more data on this, however, if you would like my layman's judgement, I think comparing just 2 moves might involve a fair share of chance, randomness concerning the nature of the position.
The idea in getting the score between 2 moves (in every depth) is that if the difference is bigger that would mean that a position has some sort of complexity as long as this current bestmove is different from the bestmove in previous depth. In opposite take for example a position where there is an easy capture move, the engine most likely will choose the bestmove starting at depth 1 till depth 12 or until allocated time is consumed. This is not a good candidate for complex position because there are no move changes starting at depth 1. An important indication that an engine also suffers the difficulty of the position is when it changes its best move from depth to depth the higher the count of these changes the more complex will be the position. In fact if engine is allowed to think long and if it changes its bestmove at higher depths complexity value can be increased as was done by Erik here.
http://talkchess.com/forum/viewtopic.php?t=54571
I do not think you can characterise a position by just 2 moves, but 5 or more moves certainly should be able to do that. If you compare 5 moves, then maybe you have sufficient statistical data to know what the position looks like.

For example, if only 1 move out of 5 is positive, all other negative, this might indicate such a position is a bit dubious.

In the above case, something simply could have gone wrong with search or eval of the best move.

In another position, where you have couple of positive scores, nothing to worry about.

So I would prefer to pick up a line where all 5 moves have a slight positive score, rather than a position with a single best move positive score, even if that score is bigger than the best move of the first instance, and all other 4 moves being negative.

But sorry again, I do not really think this has been sufficiently investigated, what the scores of the first 5 or 10 moves could account for, especially when they differ significantly in value. I also have not thought about that, just some very random impressions.

That is why I asked you to share your opinion.
Extending the analysis further down to top 5 bestmove is certainly
interesting and should be explored. But for now I am experimenting
this 2 move score difference with move changes at different depths.
I want to see the actual position with high complexity value with this
method. The diagram I posted were the top 3 complex position.

I am surprised that some of these positions are ending or close to ending.
I have not used the egtb, perhaps the complexity value will be reduced.
Here is one of the top complex positions.
[d]8/4b3/4P3/1k4p1/p6P/1p1K4/8/4R3 w - - 0 44
I will have to browse and see all these complex positions generated.
Probably publish it here so humans will be able to judge which positions
rated complex by the system but in fact it is not.
After that I will look at the top best 5, and see if those less complex
in 2-move can be rectified using the 5-move method.

In this experiment I am analyzing the games of the white players
for those Tata Steel 2015 participants on specific opening Ruy Lopez,
Berlin Defense open variation. The one I posted were only a sample of games.
I still have other games from other players, so there will be a lot
of these complex positions that will be generated.
Top
Ferdy
Posts: 4853
Joined: Sun Aug 10, 2008 3:15 pm
Location: Philippines

Re: 5 first choices

Post by Ferdy »

I think the thing you are trying to reach is harder to compute, but here's a starting point.
I am exploring the method used by Guid and Bratko. Generate such complex positions and see the
actual positions, are the generated positions with high complexity value really complex?
What if you look at the best score for each move at the root. The more bad moves there are (or the more "singular" the root position is since there is only one or two good moves to play) the more complex the position is. But apply this recursively. In essence, measure the "true width" of the tree, where "true width" represents the number of playable moves at each point. The narrower this width, the more precise you have to calculate to avoid making a blunder.
The 2-move method of Guid and Bratko has this properties, that is getting the diffence between the top 2 moves at a given depth. If the difference between
these two moves are higher, the complexity value is also high, but to eliminate the easy move, the difference is only counted if the current bestmove at this depth
is different from the bestmove of the previous depth. But this can be improved further by applying the 5-move method as suggested by Lyudmil. The difference between the
top 1 and top 5 will also tell the complexity of the position.
There are alternatives that have been discussed. For example, at position P a shallow search suggests that move A or B is playable, and that the rest are poorer. But a much deeper search shows that A or B are both bad and there is a better move C but it is hard to find.

We've had this discussion for at least 30 years, using the term "discernibility"- how easy or hard is it to find the correct move (or avoid a bad move) at some point in the tree? It is an interesting question, with a large number of difficult-to-measure/implement ideas. HiTech had a very elementary version of this idea in use. The case where for N iterations move M is best, but at iteration N+1 the bottom falls out and a new move M' is found to be best. But this move also has a score that is significantly worse than the previous iteration. What to do? HiTech simply played the original best move. The reasoning was "this move looked best until a very deep search proved it to be inferior, my new best choice is still inferior, just not as inferior as the old best choice, therefore rather than playing a move I see is going to lead to trouble, I will play my original best move and see if my opponent can work out the tactical details and find out why it is not so good." Against humans, it works well. I've tried it in the past. Against computers, it only works if you are significantly out-searching your opponent, something that is dangerous to assume.
This is intesting, and I will try to incorporate such method that is if the bestmove has changed at higher depth, the score of bestmove at depth n should be compared
to the score of the bestmove at depth n-1. If the score at depth n-1 is higher than the score at depth n (each move at given depth is different), we can say that it is more
complex than when the situation is reversed.
Top
Lyudmil Tsvetkov
Posts: 6052
Joined: Tue Jun 12, 2012 12:41 pm

Re: 5 first choices

Post by Lyudmil Tsvetkov »

Ferdy wrote:
Sorry, Ferdinand,

never thought what the difference between the best and second best move might account for, or be a reflection of.

Absolutely never. I am sure you have much more data on this, however, if you would like my layman's judgement, I think comparing just 2 moves might involve a fair share of chance, randomness concerning the nature of the position.
The idea in getting the score between 2 moves (in every depth) is that if the difference is bigger that would mean that a position has some sort of complexity as long as this current bestmove is different from the bestmove in previous depth. In opposite take for example a position where there is an easy capture move, the engine most likely will choose the bestmove starting at depth 1 till depth 12 or until allocated time is consumed. This is not a good candidate for complex position because there are no move changes starting at depth 1. An important indication that an engine also suffers the difficulty of the position is when it changes its best move from depth to depth the higher the count of these changes the more complex will be the position. In fact if engine is allowed to think long and if it changes its bestmove at higher depths complexity value can be increased as was done by Erik here.
http://talkchess.com/forum/viewtopic.php?t=54571
I do not think you can characterise a position by just 2 moves, but 5 or more moves certainly should be able to do that. If you compare 5 moves, then maybe you have sufficient statistical data to know what the position looks like.

For example, if only 1 move out of 5 is positive, all other negative, this might indicate such a position is a bit dubious.

In the above case, something simply could have gone wrong with search or eval of the best move.

In another position, where you have couple of positive scores, nothing to worry about.

So I would prefer to pick up a line where all 5 moves have a slight positive score, rather than a position with a single best move positive score, even if that score is bigger than the best move of the first instance, and all other 4 moves being negative.

But sorry again, I do not really think this has been sufficiently investigated, what the scores of the first 5 or 10 moves could account for, especially when they differ significantly in value. I also have not thought about that, just some very random impressions.

That is why I asked you to share your opinion.
Extending the analysis further down to top 5 bestmove is certainly
interesting and should be explored. But for now I am experimenting
this 2 move score difference with move changes at different depths.
I want to see the actual position with high complexity value with this
method. The diagram I posted were the top 3 complex position.

I am surprised that some of these positions are ending or close to ending.
I have not used the egtb, perhaps the complexity value will be reduced.
Here is one of the top complex positions.
[d]8/4b3/4P3/1k4p1/p6P/1p1K4/8/4R3 w - - 0 44
I will have to browse and see all these complex positions generated.
Probably publish it here so humans will be able to judge which positions
rated complex by the system but in fact it is not.
After that I will look at the top best 5, and see if those less complex
in 2-move can be rectified using the 5-move method.

In this experiment I am analyzing the games of the white players
for those Tata Steel 2015 participants on specific opening Ruy Lopez,
Berlin Defense open variation. The one I posted were only a sample of games.
I still have other games from other players, so there will be a lot
of these complex positions that will be generated.
All positions that show big differences between best moves and other ones are tactical or highly tactical ones.

In endgames you simply have fewer reasonable moves, and therefore often you might have a single winning move in such highly tactical positions. In midgame, you might have 2 positive score moves, as well as couple of negative score moves.

All the positions you posted are highly tactical endgame positions. With a single best move.
They involve the existence of many passers, usually connected, passer races, etc., where s ingle tempo counts and the probability for a single winning move increases.

Again, in the endgame, because of fewre reasonable moves, you might have a single winning move here. In the same highly tactical positions in midgame, you will have 2 positive scores, and 2 negative.

So that it might make sense to exclude highly tactical positions at all. But I do not know how to do that. I read about quiescence search, but I do not understand it, as in practically every position you have some attacks/captures involved, so I really do not see how you make the distinction.
Top
Ferdy
Posts: 4853
Joined: Sun Aug 10, 2008 3:15 pm
Location: Philippines

Re: 5 first choices

Post by Ferdy »

Lyudmil Tsvetkov wrote:
Ferdy wrote:
Sorry, Ferdinand,

never thought what the difference between the best and second best move might account for, or be a reflection of.

Absolutely never. I am sure you have much more data on this, however, if you would like my layman's judgement, I think comparing just 2 moves might involve a fair share of chance, randomness concerning the nature of the position.
The idea in getting the score between 2 moves (in every depth) is that if the difference is bigger that would mean that a position has some sort of complexity as long as this current bestmove is different from the bestmove in previous depth. In opposite take for example a position where there is an easy capture move, the engine most likely will choose the bestmove starting at depth 1 till depth 12 or until allocated time is consumed. This is not a good candidate for complex position because there are no move changes starting at depth 1. An important indication that an engine also suffers the difficulty of the position is when it changes its best move from depth to depth the higher the count of these changes the more complex will be the position. In fact if engine is allowed to think long and if it changes its bestmove at higher depths complexity value can be increased as was done by Erik here.
http://talkchess.com/forum/viewtopic.php?t=54571
I do not think you can characterise a position by just 2 moves, but 5 or more moves certainly should be able to do that. If you compare 5 moves, then maybe you have sufficient statistical data to know what the position looks like.

For example, if only 1 move out of 5 is positive, all other negative, this might indicate such a position is a bit dubious.

In the above case, something simply could have gone wrong with search or eval of the best move.

In another position, where you have couple of positive scores, nothing to worry about.

So I would prefer to pick up a line where all 5 moves have a slight positive score, rather than a position with a single best move positive score, even if that score is bigger than the best move of the first instance, and all other 4 moves being negative.

But sorry again, I do not really think this has been sufficiently investigated, what the scores of the first 5 or 10 moves could account for, especially when they differ significantly in value. I also have not thought about that, just some very random impressions.

That is why I asked you to share your opinion.
Extending the analysis further down to top 5 bestmove is certainly
interesting and should be explored. But for now I am experimenting
this 2 move score difference with move changes at different depths.
I want to see the actual position with high complexity value with this
method. The diagram I posted were the top 3 complex position.

I am surprised that some of these positions are ending or close to ending.
I have not used the egtb, perhaps the complexity value will be reduced.
Here is one of the top complex positions.
[d]8/4b3/4P3/1k4p1/p6P/1p1K4/8/4R3 w - - 0 44
I will have to browse and see all these complex positions generated.
Probably publish it here so humans will be able to judge which positions
rated complex by the system but in fact it is not.
After that I will look at the top best 5, and see if those less complex
in 2-move can be rectified using the 5-move method.

In this experiment I am analyzing the games of the white players
for those Tata Steel 2015 participants on specific opening Ruy Lopez,
Berlin Defense open variation. The one I posted were only a sample of games.
I still have other games from other players, so there will be a lot
of these complex positions that will be generated.
All positions that show big differences between best moves and other ones are tactical or highly tactical ones.

In endgames you simply have fewer reasonable moves, and therefore often you might have a single winning move in such highly tactical positions. In midgame, you might have 2 positive score moves, as well as couple of negative score moves.
This is a very good observation that I have not thought. In this case for endings, we may have a best move where the difference to 2nd best is big, and this could be a good candidate for a tactical position. The question now would be how to identify the easy and difficult positions, I think this can be answered by observing how many move changes does the engine experienced when doing iterations from say depth 6 and above which is already covered in Guid/Bratko method. In middle game there are indeed 3 or more bestmoves that are close in score. In this case the 5-move method could be of help to further magnify the complexity of the position. But of course the phase of the position does not matter because we are interested for complex or difficult positions to evaluate, be it tactical/strategic/mid-game/ending.
All the positions you posted are highly tactical endgame positions. With a single best move.
They involve the existence of many passers, usually connected, passer races, etc., where s ingle tempo counts and the probability for a single winning move increases.

Again, in the endgame, because of fewre reasonable moves, you might have a single winning move here. In the same highly tactical positions in midgame, you will have 2 positive scores, and 2 negative.
The actual score of the bestmove at different depths is also an interesting criteria to give points in complexity calculation, even the observation of the trend (decreasing or increasing) as the engine analysis depth increases is also a good source for complexity calculations.
So that it might make sense to exclude highly tactical positions at all. But I do not know how to do that. I read about quiescence search, but I do not understand it, as in practically every position you have some attacks/captures involved, so I really do not see how you make the distinction.
We will include this tactical position, if the solution is difficult to find. Now we are starting to observe that strong human players are very good at tactical positions, seems like they can see most of the solutions to the tactical positions but their opponent could only see few of them.
Regarding tactical positions in ending, look at my first post, there is RMC that is root move change, it is a count how may times the engine is changing its mind regarding the best move from depth to depth, then look at the fen, the last value is the game move number. if RMC is low say 1 to 3 and the position is close to ending, and it has a high COM value that could be a tactical position.
Top

Return to “Computer Chess Club: General Topics”