How to build a computer analysis, using either a CECP or a UCI engine

[Roland Chastain]

Hello everybody.

I would like to create a GUI displaying a computer analysis for a game, like the one provided by lichess (for example).



Which commands could I send to an engine, in order to get informations like relative advantage of each player?

Would be interested both by CECP and UCI solutions.

Regards.

Roland

[hgm]

In UCI you would send a comment 'go infinite' after setting up the position to analyse with the usual 'position ... moves ...' command. Terminate with 'stop' before sending any other commands.

In CECP the command is 'analyze', and you terminate that with 'exit' (which brings you back into force mode). In CECP it is allowed to send 'setup' commands, moves and 'undo' during the analysis, after which the engine starts searching the new position.

In both cases the 'thinking output' is as usual. Meaning that in CECP you might need to send a 'post' command to make sure it will be given.

[Roland Chastain]

@hgm

Thank you.

As a first step, I made the following Tcl script (using CECP):

Code: Select all

#!/usr/bin/expect

set ENGINE /home/roland/Documents/echecs/sources/moteurs/olithink/5_11_7/olithink

spawn $ENGINE

send "xboard\n"
send "protover 2\n"
expect "done=1"

send "new\n"
send "hard\n"
send "post\n"
#send "e2e4\n"
send "setboard rnbqkbnr/pppppppp/8/8/4P3/8/PPPP1PPP/RNBQKBNR b KQkq e3 0 1\n"
send "analyze\n"
after 1000

send "exit\n"
send "quit\n"

#interact
expect eof

Does it look correct?

I get the following output:

Code: Select all

 1   -31      0         5  a7a6 
 2   -42      0        24  a7a6 b2b3 
 3    31      0        78  d7d5 e4d5 d8d5 
 4   -24      0       612  e7e6 d2d4 d7d5 f2f3 
 5    41      0      1570  g8f6 d2d3 d7d5 e4d5 
 6   -12      0      4321  d7d5 e4e5 d8d7 h2h3 d5d4 d2d3 
 7   -25      1      8136  d7d5 e4e5 b8a6 d2d4 c8f5 g1f3 c7c6 
 8   -20      1     12050  d7d5 e4e5 g8h6 g1f3 c8f5 d2d3 d5d4 
 1    -3      0         6  d7d5 
 2    -8      0        18  d7d5 e4e5 
 3    12      0        37  d7d5 e4e5 g8h6 
 4   -24      0       132  d7d5 e4e5 g8h6 g1f3 
 5   -23      0       424  d7d5 d2d3 d5e4 d3e4 d8d6 
 6   -20      0       827  d7d5 d2d3 d5e4 b1c3 e4d3 f1d3 
 7     1      0      2611  d7d5 d2d3 b8c6 e4d5 d8d5 b1c3 d5d4 
 8   -23      1     10252  d7d5 e4e5 d5d4 f1c4 b8c6 g1f3 c8e6 d2d3 
 9   -19      1     18300  d7d5 e4e5 d5d4 f1c4 b8c6 g1f3 g7g6 d2d3 f8h6 
10   -19      4     47045  e7e5 g1f3 b8c6 f1c4 d7d6 d2d3 f8e7 b1c3 c8e6 e1g1 
11   -17      8     86619  e7e5 g1f3 g8f6 b1c3 d7d6 h2h3 c8e6 d2d3 b8c6 c1e3 
12   -18     16    174360  d7d5 e4d5 d8d5 b1c3 d5e6 d1e2 g8f6 d2d3 b8c6 c1e3 f6g4 e3f4 g4e5 e1c1 
13   -16     23    261553  d7d5 e4d5 d8d5 d2d3 g8f6 b1c3 d5d6 g1f3 b8c6 c1e3 d6b4 d1b1 e7e6 
14   -24     41    466464  e7e5 b1c3 b8c6 g1f3 g8f6 f1b5 f8b4 b5c6 b7c6 f3e5 b4c3 d2c3 f6e4 
15   -24     75    856621  e7e5 b1c3 f8c5 g1f3 d7d6 c3a4 c5b6 a4b6 a7b6 d2d4 g8f6 c1g5 b8c6 c2c3 

[hgm]

Looks OK. But it would probably be better to send the move(s) ("usermove e2e4" after "force") rather than the position (through 'setboard'), so that the engine has the complete game hsitory in addition to the current position. This could make a difference for repetition draws, which it otherwise would not recognize.

Also notice that you just assume here that the engine will understand 'setboard' (and I assumed that the plain move needs to be prefixed with 'usermove'). This might not be the case for all engines, and it should formally be deduced from the 'feature' replies to 'protover 2' whether you have to send positions with 'setboard' or 'edit', and whether the 'usermove' prefix is necessary for relaying moves.

[Roland Chastain]

@hgm

Thank you.

Here is the current version of my script.

Code: Select all

#!/usr/bin/expect

#set ENGINE ~/Documents/echecs/sources/moteurs/olithink/5_11_7/olithink
#set ENGINE ~/Documents/echecs/sources/moteurs/cheng4/40/cheng4_linux_x64
#set ENGINE ~/Documents/echecs/sources/moteurs/hgm/fairymax/de2556f/fmax

if { $argc == 0 } {
  puts "Usage :\n  expect test.tcl ENGINE"
  exit 1
}
set ENGINE [lindex $argv 0]

spawn $ENGINE

send "xboard\n"
send "protover 2\n"

expect {
  timeout      { set prefix "" }
  "usermove=0" { set prefix "" }
  "usermove=1" { set prefix "usermove " }
}

expect "done=1"

send "new\n"
send "hard\n"
send "post\n"
send "force\n"

send "${prefix}e2e4\n"
send "analyze\n"
after 1000

send "${prefix}e7e5\n"
send "analyze\n"
after 1000

send "exit\n"
send "quit\n"

expect eof

Result with Cheng4:

Code: Select all

spawn /home/roland/Documents/echecs/sources/moteurs/cheng4/40/cheng4_linux_x64
xboard
protover 2
feature name=0 san=0 usermove=0 time=1 sigint=0 sigterm=0 pause=0 reuse=1 analyze=1 colors=0 setboard=1 nps=1 smp=1 debug=0 draw=0 playother=1 variants="normal,fischerandom" ics=0 memory=1 ping=0 option="Clear Hash -button" option="Hash -spin 32 1 16384" option="Threads -spin 1 1 64" option="OwnBook -check 1" option="LimitStrength -check 0" option="Elo -spin 2500 800 2500" option="MultiPV -spin 1 1 256" option="NullMove -check 1" option="Contempt -spin 0 -100 100" myname="Cheng 4.40 dev" done=1
new
hard
post
force
e2e4
analyze
  1    -39        0            2 h5
  1     -5        0            6 e5
  1     16        0           17 e6
  2     -3        0           50 e6
  2    -20        0           70 e6
  2    -20        0          104 e6 d4
  3      3        0          254 e6
  3      3        0          316 e6 d4 Nc6
  4    -14        0          512 e6
  4    -22        0          651 e6 d4 Nc6 d5 exd5
  5     -7        0         1043 e6
  5     -5        0         1334 e6 d4 Nc6 Nf3 b6
  6    -20        0         1735 e6
  6    -17        0         1984 e6 d4 Nc6 Nf3 Bb4+ c3 Bd6
  6    -14        0         3587 c5 Nc3 e6 d3 Nc6 Nf3
  7      1        0         6847 c5
  7      5        1         8901 c5 c3 e6 d4 cxd4 cxd4 Bb4+ Nc3
  8     -3        1        12277 c5 Nc3 e6 b3 Nc6 Bb2 b6 Nf3
  9     -5        2        20173 c5 d3 e6 Be2 Nc6 Nf3 b6 O-O Bb7
 10    -16        3        36076 c5 d3 e6 Be2 Nc6 Nf3 b6 O-O Bb7 Ng5
 10     -4        7        63247 d5 exd5 Qxd5 Nc3 Qe6+ Qe2 Nf6 Nb5 Qc6 Nd4 Qb6
 11    -14        9        85906 d5 exd5 Qxd5 Nc3 Qe6+ Be2 Nf6 d3 Nc6 Be3 Ng4 Nb5 Nxe3
 12    -16       25       227225 d5 exd5 Nf6 Nc3 Nxd5 d4 Nxc3 bxc3 g6 Nf3 Bg7 h3
 12    -15       35       316059 c5 d3 e6 Be2 Nc6 Nf3 b5 O-O a5 d4 cxd4 Nxd4
 13    -19       47       424572 c5 Nf3 e6 Be2 Nc6 O-O b6 d4 cxd4 Nxd4 Bb7 Bf4 Nf6
 14    -26       96       849993 c5 Nf3 Nc6 Be2 Nf6 Nc3 d6 O-O e5 d3 Nd4 Nd5 Nxd5 Nxd4
e7e5
analyze
  1     19        0            4 Nc3 Nc6
  1     19        0            3 Nc3
  2     19        0           63 Nc3 Nc6
  3     19        0          185 Nc3 Nc6 Nf3
  4     19        0          389 Nc3 Nc6 Nf3 Nf6
  5     19        0          648 Nc3 Nc6 Nf3 Nf6 Bb5
  6     19        0         1443 Nc3 Nc6 Nf3 h6 Be2 a6 O-O
  7     20        0         4169 Nc3 Nf6 Nf3 Bb4 Nd5 Nxd5 exd5
  8     18        0         7456 Nc3 Nf6 Nf3 Bb4 a3 Bxc3 dxc3 d6
  9     23        2        19495 Nc3 Nf6 Nf3 Nc6 Bb5 a6 Bxc6 dxc6 O-O
 10     16        3        32251 Nc3 Nf6 Nf3 Nc6 Bb5 a6 Bxc6 dxc6 Nxe5 Bb4
 11     13       11       103684 Nc3 Nf6 Nf3 Bb4 Nd5 Nxd5 c3 Nf6 cxb4 d5 exd5
 11     22       18       158976 Nf3 Nc6 Nc3 Nf6 Bb5 Bd6 d3 O-O O-O a6 Bc4
 12     19       25       221452 Nf3 Nc6 Nc3 Nf6 a3 a6 d4 exd4 Nxd4 Nxd4 Qxd4 b5
 13     18       52       454406 Nf3 Nc6 Bc4 Bc5 d3 Nf6 c3 O-O b4 Be7 b5 d5 Bxd5 Nxd5
 14     24       77       675130 Nf3 Nc6 Bc4 Bc5 d3 Nf6 c3 O-O b4 Be7 b5 d5 exd5 Na5 Nxe5
exit
quit

With Fairy-Max:

Code: Select all

spawn /home/roland/Documents/echecs/sources/moteurs/hgm/fairymax/de2556f/fmax
xboard
protover 2
tellics say     Fairy-Max 5.0b
tellics say     by H.G. Muller
feature myname="Fairy-Max 5.0b"
feature memory=1 exclude=1
feature setboard=0 xedit=1 ping=1 done=0
feature variants="normal,nocastle,shatranj,asean,makruk,cambodian,ai-wok,courier,knightmate,capablanca,gothic,janus,falcon,cylinder,berolina,super,seirawan,spartan,great,light-brigade,king-of-the-hill,bifurcator,team-mate,los-alamos,ciccolini,mexican,grande-acedrex,roman,almost-wildebeest,fairy"
feature option="Resign -check 0"
feature option="Resign Threshold -spin 800 200 1200"
feature option="Claim draw after -spin 50 0 200"
feature option="Ini File -file ./fmax.ini"
feature option="Multi-PV Margin -spin 0 0 1000"
feature option="Variant fairy selects -combo FIDE-Clobberers /// Clobberers-FIDE /// FIDE-Nutters /// Nutters-FIDE /// Clobberers-Nutters /// Nutters-Clobberers /// FIDE-Rookies /// Rookies-FIDE /// Clobberers-Rookies /// Rookies-Clobberers /// Nutters-Rookies /// Rookies-Nutters"
feature option="Makruk rules -combo makruk /// Cambodian /// Ai-wok"
feature option="Dummy Slider Example -slider 20 0 100"
feature option="Dummy String Example -string happy birthday!"
feature option="Dummy Path Example -path ."
feature option="Automatic persistent-hash dialog -check 0"
feature option="Info -button"
feature option="Save in hash file -button"
feature option="Clear Hash -button"
feature done=1
new
hard
post
force
e2e4
analyze
 1     15        0          3 c7c5
 2      0        0         17 c7c5 d2d4
 3     13        0        810 c7c5 d2d4 g8f6
 4     -1        0       1787 c7c5 d2d4 g8f6 b1c3
 4      0        0       4483 d7d5 b1c3 g8f6 f2f3
 5     10        4      25034 d7d5 e4d5 d8d5 b1c3 d5e5
 5     12        6      37293 d7d6 d2d4 b8c6 b1c3 c8e6
 5     13        8      47873 b8c6 b1c3 g8f6 d2d4 d7d5
 6      0       11      63914 b8c6 b1c3 g8f6 d2d3 d7d5 c1f4
 6      3       18     104514 g8f6 b1c3 d7d5 f1d3 b8c6 f2f4
 7     12       55     316335 g8f6 e4e5 f6d5 c2c4 d5b4 d2d4 b8c6
 7     13       84     485785 c7c6 d2d3 d7d5 b1c3 c8e6 c1f4 b8d7
e7e5
analyze
 1    -15        0          3 a2a4
 1     -6        0          3 d2d4
 1     15        0          3 b1c3
 2      0        0         23 b1c3 b8c6
 3     13        0        264 b1c3 b8c6 d2d3
 4      0        0        440 b1c3 b8c6 d2d3 d7d6
 5     13        5      28614 b1c3 b8c6 d2d3 d7d6 c1e3
 6      1        9      53558 b1c3 b8c6 g1f3 d7d6 d2d4 c8e6
 6      5       40     221177 g1f3 g8f6 b1c3 b8c6 d2d4 f8d6
exit
quit

With Olithink:

Code: Select all

spawn /home/roland/Documents/echecs/sources/moteurs/olithink/5_11_7/olithink
xboard
protover 2
feature setboard=1 myname="OliThink 5.11.7" variants="normal" colors=0 ping=1 sigint=0 sigterm=0 done=1
new
hard
post
force
e2e4
analyze
 1   -31      0         5  a7a6 
 2   -42      0        24  a7a6 b2b3 
 3    31      0        78  d7d5 e4d5 d8d5 
 4   -24      0       612  e7e6 d2d4 d7d5 f2f3 
 5    41      0      1570  g8f6 d2d3 d7d5 e4d5 
 6   -12      0      4321  d7d5 e4e5 d8d7 h2h3 d5d4 d2d3 
 7   -25      1      8136  d7d5 e4e5 b8a6 d2d4 c8f5 g1f3 c7c6 
 8   -20      1     12050  d7d5 e4e5 g8h6 g1f3 c8f5 d2d3 d5d4 
 9   -23      3     30263  e7e5 b1c3 b8c6 g1f3 d7d6 d2d4 c8e6 c1e3 
10   -14      4     48155  e7e5 b1c3 g8f6 g1f3 d7d6 d2d4 e5d4 d1d4 b8c6 
11   -17      7     79927  e7e5 b1c3 g8f6 g1f3 d7d6 h2h3 b8c6 d2d3 c8e6 c1e3 
12   -18     15    167154  d7d5 e4e5 c7c5 d2d4 c5d4 d1d4 b8c6 d4f4 e7e6 f4g3 d5d4 
13   -15     24    279581  d7d5 e4e5 c7c5 d2d4 c5d4 d1d4 b8c6 d4f4 e7e6 g1f3 f8e7 c1e3 c8d7 
14   -39     66    754934  e7e5 g1f3 b8c6 b1c3 a7a6 f1c4 d7d6 d2d3 f8e7 e1g1 c8e6 c4d5 g8f6 d5c6 b7c6 c1e3 
e7e5
analyze
 1    28      0         4  g1f3 
 2   -16      0        17  g1f3 b8c6 
 3    28      0        39  g1f3 b8c6 b1c3 
 4     5      0       175  g1f3 b8c6 d2d4 e5d4 
 5     6      0       545  g1f3 b8c6 d2d3 d7d6 c2c3 
 6    11      0       913  g1f3 b8c6 b1c3 d7d6 d2d4 g8f6 
 7    21      0      1341  g1f3 b8c6 b1c3 d7d6 d2d4 g8f6 h2h3 
 8    11      0      2449  g1f3 b8c6 b1c3 d7d6 d2d4 e5d4 f3d4 g8f6 
 9    23      0      3987  g1f3 b8c6 b1c3 g8f6 d2d4 e5d4 f3d4 f8c5 c1e3 
10    40      1     17785  g1f3 g8f6 b1c3 h7h6 f1c4 d7d6 d2d3 c8e6 e1g1 e6c4 
 1    28      0         4  g1f3 
 2     0      0        19  g1f3 g8f6 
 3    30      0        54  g1f3 d7d6 b1c3 
 4    11      0       124  g1f3 d7d6 d2d4 g8f6 
 5    32      0       385  g1f3 g8f6 d2d4 f6e4 f3e5 
 6    11      0      1029  g1f3 b8c6 b1c3 g8f6 d2d4 d7d6 
 7    21      0      1336  g1f3 b8c6 b1c3 g8f6 d2d4 e5d4 f3d4 
 8    15      0      4969  b1c3 b8c6 f1c4 d7d6 g1f3 c8e6 d2d3 e6c4 
 9    20      0      6922  b1c3 b8c6 f1c4 d7d6 g1f3 f8e7 d2d4 c6a5 d1d3 
10     8      1     12704  b1c3 b8c6 g1f3 g8f6 f1b5 f8c5 d2d3 d7d6 c1e3 e8g8 
11    44      6     81899  g1f3 b8c6 b1c3 f8c5 f3e5 c6e5 d2d4 c5d6 d4e5 d6e5 c3d5 d7d6 
12    44      8    101751  g1f3 b8c6 b1c3 f8c5 f3e5 c6e5 d2d4 c5d6 d4e5 d6e5 c3d5 d7d6 
13    49     14    173615  g1f3 b8c6 b1c3 g8f6 d2d4 e5d4 f3d4 f8b4 f2f3 e8g8 f1b5 b4c5 c1e3 
14    48     36    451114  g1f3 b8c6 b1c3 f8c5 f3e5 c6e5 d2d4 c5d6 d4e5 d6e5 d1d2 d7d6 f1e2 g8f6 f2f3 c8d7 e1g1 
15    32     83   1019356  g1f3 g8f6 f3e5 d8e7 e5f3 e7e4 f1e2 b8c6 e1g1 d7d5 b1c3 e4b4 c3b5 b4e7 d2d4 c8f5 c1f4 
exit
quit

[ovenel]

In regards to the analysis that Lichess does, they are open-sourced, so you would be able to review their code to try to understand a bit more of how they have implemented this. In particular, it looks like they have this implemented in the Fishnet repository (which is an application that people can install on their computer to donate compute resources to Lichess).

In particular, it looks like Lichess will break down an analysis request into the list of positions that occurred in the game. It will then iterate over these positions in reverse order, using Stockfish to analyze the position. Prior to sending the go command, it will ensure that the UCI_AnalyseMode option is set to true on the engine and that the skill level is set to 20. It also appears that the search is restricted for each position by node count, with a different limit depending on whether the HCE or NNUE version of Stockfish is being used. It looks like the Lichess server dictates these limits that are used, but according to the Protocol documentation, it appears that Stockfish 15 is limited to 1,500,000 nodes, Stockfish 14 to 2,100,000 nodes, and classical (or HCE) to 4,050,000. But it also indicates 7,000ms per ply (though I'm not sure that this is accurate to the current code).

In regards to the analysis of each move that was played compared to what the engine likes, Lichess also has this article explaining their accuracy calculation.

I'm not wholly familiar with how the Fishnet code works, but it seems that the basic idea is to split out the game into each position, and then iterate over them from the end to the beginning. For each of these positions, conduct a limited search to identify the best move and the evaluation. Then you can collect this information against what was actually played and identify the moves that had the most significant deviation from the engine's best play.

[ovenel]

I've been trying to find where the node limits are defined in the Lichess code so that I can see what they use for the current version of Stockfish. From what I can tell, they are defined in the Work.scala file with different limits for the different categories of analysis. For official broadcasts, the limit is 5,000,000 nodes/move, for manually requested analyses on a game or study, the limit is 1,000,000 nodes, and then there are some further limits for some automated analyses.

These limits are the basis for the actual limits which are returned to the Fishnet client's request for work in the JsonAPI.scala file. For Stockfish 16 & 17.1, the limits are used as defined above. However, for classical Stockfish (i.e. the version with a handcrafted evaluation function), the limit is 3 times what is defined above.

The timeout for analyzing each move is defined in the Cleaner.scala file and is 7 seconds regardless of what type of analysis is being performed. It also appears that Lichess has a maximum game length of 300 ply for their analysis, but I can't remember which files I saw that in.

It's probably wasn't super necessary for me to dig through this to find these specific details, but I've been wanting to implement something like this myself as well, and seeing what Lichess is using for their limits seems useful for getting an idea for where to start when running the engine analysis.