WDL

[MikeB]

As many of know, NN engines have been reporting WDL data with Scoring Percentage for some time. A while back, I took their formula to convert to a centipawn and applied it the centipawn evaluation to come up with LC0 like output for the Honey family of engines.

Today, the developers of SF committed a patch that reports out the WDL stats based on millions of games in the Fishcooking database. If enabled,
this is how the output looks like:

Position: 32/47
info depth 1 seldepth 1 multipv 1 score cp 304 wdl 982 18 0 nodes 218 nps 218000 tbhits 0 time 1 pv h1h2 g4f3 g1f3
info depth 2 seldepth 2 multipv 1 score cp 304 wdl 982 18 0 nodes 571 nps 571000 tbhits 0 time 1 pv h1h2 g4f3
info depth 3 seldepth 3 multipv 1 score cp 304 wdl 982 18 0 nodes 734 nps 734000 tbhits 0 time 1 pv h1h2 g4f3 g1f3
info depth 4 seldepth 4 multipv 1 score cp 114 wdl 486 507 7 nodes 2780 nps 1390000 tbhits 0 time 2 pv f3b7 g3h1 b7h1 c5d4
info depth 5 seldepth 6 multipv 1 score cp 44 wdl 176 793 31 nodes 4117 nps 2058500 tbhits 0 time 2 pv f3d5 g8g7 h1h2 c5d4 d3d4 d7e5
info depth 6 seldepth 7 multipv 1 score cp 55 wdl 211 764 25 nodes 4454 nps 1484666 tbhits 0 time 3 pv f3d5 g8g7 h1h2 c5d4 d3d4 d7e5
info depth 7 seldepth 11 multipv 1 score cp 9 wdl 91 846 63 nodes 10091 nps 2018200 tbhits 0 time 5 pv f3g4 h5g4 h1h2 c5d4 g1e2 e7e5
info depth 8 seldepth 11 multipv 1 score cp 101 wdl 417 574 9 nodes 11512 nps 1918666 tbhits 0 time 6 pv f3d5 g8g7 g1h3 g4h3 h1h3 c5d4 d3d4 d7e5
info depth 9 seldepth 13 multipv 1 score cp 60 wdl 229 749 22 nodes 21955 nps 1995909 tbhits 0 time 11 pv h1h2 c5d4 d3g6 e7g7 f3d5 g8h8 g6g5 g7g5 d2g5 f8f1
info depth 10 seldepth 19 multipv 1 score cp 0 wdl 75 848 77 nodes 45368 nps 2062181 tbhits 0 time 22 pv h1h2 c5d4 d2b4 e7b4 d3g6 g8h8 g6h6 h8g8
info depth 11 seldepth 20 multipv 1 score cp 0 wdl 77 848 75 nodes 52928 nps 2117120 tbhits 0 time 25 pv h1h2 c5d4 d2b4 e7b4 d3g6 g8h8 g6h6 h8g8 h6g6
info depth 12 seldepth 15 multipv 1 score cp 0 wdl 75 848 77 nodes 69474 nps 2105272 tbhits 0 time 33 pv h1h2 c5d4 d2b4 e7b4 d3g6 g8h8 g6h6 h8g8
info depth 13 seldepth 28 multipv 1 score cp 19 wdl 111 837 52 nodes 80480 nps 2117894 tbhits 0 time 38 pv h1h2 c5d4 d2b4 e7b4 d3g6 g8h8 g6h6 h8g8 f3d5 f8f7 g1f3 g4f3
bestmove h1h2 ponder c5d4

Below is a Stockfish-XIr3 ( SF functional equaliavent) with WDL enabled, and output using a Score % formula based om Centipawn Evaluation (cp below) as well as calculating the scoring % from the WDL data provided by SF ( sp below)

info depth 1 seldepth 1 multipv 1 score cp 861 wdl 982 18 0 sp 991 nodes 218 nps 109000 tbhits 0 time 2 pv h1h2 g4f3 g1f3
info depth 2 seldepth 2 multipv 1 score cp 861 wdl 982 18 0 sp 991 nodes 571 nps 285500 tbhits 0 time 2 pv h1h2 g4f3
info depth 3 seldepth 3 multipv 1 score cp 861 wdl 982 18 0 sp 991 nodes 734 nps 367000 tbhits 0 time 2 pv h1h2 g4f3 g1f3
info depth 4 seldepth 4 multipv 1 score cp 665 wdl 486 507 7 sp 739 nodes 2780 nps 1390000 tbhits 0 time 2 pv f3b7 g3h1 b7h1 c5d4
info depth 5 seldepth 6 multipv 1 score cp 566 wdl 176 793 31 sp 572 nodes 4117 nps 1372333 tbhits 0 time 3 pv f3d5 g8g7 h1h2 c5d4 d3d4 d7e5
info depth 6 seldepth 7 multipv 1 score cp 582 wdl 211 764 25 sp 593 nodes 4454 nps 1484666 tbhits 0 time 3 pv f3d5 g8g7 h1h2 c5d4 d3d4 d7e5
info depth 7 seldepth 11 multipv 1 score cp 513 wdl 91 846 63 sp 514 nodes 10091 nps 1681833 tbhits 0 time 6 pv f3g4 h5g4 h1h2 c5d4 g1e2 e7e5
info depth 8 seldepth 11 multipv 1 score cp 647 wdl 417 574 9 sp 704 nodes 11512 nps 1918666 tbhits 0 time 6 pv f3d5 g8g7 g1h3 g4h3 h1h3 c5d4 d3d4 d7e5
info depth 9 seldepth 13 multipv 1 score cp 589 wdl 229 749 22 sp 603 nodes 21955 nps 1995909 tbhits 0 time 11 pv h1h2 c5d4 d3g6 e7g7 f3d5 g8h8 g6g5 g7g5 d2g5 f8f1
info depth 10 seldepth 19 multipv 1 score cp 499 wdl 75 848 77 sp 499 nodes 45368 nps 2062181 tbhits 0 time 22 pv h1h2 c5d4 d2b4 e7b4 d3g6 g8h8 g6h6 h8g8
info depth 11 seldepth 20 multipv 1 score cp 500 wdl 77 848 75 sp 501 nodes 52928 nps 2117120 tbhits 0 time 25 pv h1h2 c5d4 d2b4 e7b4 d3g6 g8h8 g6h6 h8g8 h6g6
info depth 12 seldepth 15 multipv 1 score cp 499 wdl 75 848 77 sp 499 nodes 69474 nps 2105272 tbhits 0 time 33 pv h1h2 c5d4 d2b4 e7b4 d3g6 g8h8 g6h6 h8g8
info depth 13 seldepth 28 multipv 1 score cp 529 wdl 111 837 52 sp 529 nodes 80480 nps 2117894 tbhits 0 time 38 pv h1h2 c5d4 d2b4 e7b4 d3g6 g8h8 g6h6 h8g8 f3d5 f8f7 g1f3 g4f3
bestmove h1h2 ponder c5d4
Nodes/Second: 2117894

amazingly close....even the nps ;>)
right click and select open in new window to enlarge
cur-dev Sf on the left, Stockfish-XI-r3 on the right

[nnnnnnnn]

I have not looked at the patch - does it account for the difference between
score cp 0
in a table base, which should have associated wdl of

wdl 0 1000 0

and score cp 0 in a middlegame position, which is entirely different.

(What about when the PV of position not in the tablebase results in a tablebase draw - does wdl account for that too?)

[zullil]

I have not looked at the patch - does it account for the difference between
score cp 0
in a table base, which should have associated wdl of

wdl 0 1000 0

and score cp 0 in a middlegame position, which is entirely different.

(What about when the PV of position not in the tablebase results in a tablebase draw - does wdl account for that too?)

No. It fails to assign 0 1000 0 to root positions that are tablebase draws.

Code: Select all

$ ./stockfish 
Stockfish 010720 64 BMI2 by T. Romstad, M. Costalba, J. Kiiski, G. Linscott
uci
id name Stockfish 010720 64 BMI2
id author T. Romstad, M. Costalba, J. Kiiski, G. Linscott

option name Debug Log File type string default 
option name Contempt type spin default 24 min -100 max 100
option name Analysis Contempt type combo default Both var Off var White var Black var Both
option name Threads type spin default 1 min 1 max 512
option name Hash type spin default 16 min 1 max 33554432
option name Clear Hash type button
option name Ponder type check default false
option name MultiPV type spin default 1 min 1 max 500
option name Skill Level type spin default 20 min 0 max 20
option name Move Overhead type spin default 10 min 0 max 5000
option name Slow Mover type spin default 100 min 10 max 1000
option name nodestime type spin default 0 min 0 max 10000
option name UCI_Chess960 type check default false
option name UCI_AnalyseMode type check default false
option name UCI_LimitStrength type check default false
option name UCI_Elo type spin default 1350 min 1350 max 2850
option name UCI_ShowWDL type check default true
option name SyzygyPath type string default <empty>
option name SyzygyProbeDepth type spin default 1 min 1 max 100
option name Syzygy50MoveRule type check default true
option name SyzygyProbeLimit type spin default 7 min 0 max 7
uciok
setoption name SyzygyPath value /home/louis/Documents/Chess/tb/src
info string Found 510 tablebases
position fen 4k3/4b3/4p3/8/8/4N3/4B3/4K3 w - - 0 1
d

 +---+---+---+---+---+---+---+---+
 |   |   |   |   | k |   |   |   | 8
 +---+---+---+---+---+---+---+---+
 |   |   |   |   | b |   |   |   | 7
 +---+---+---+---+---+---+---+---+
 |   |   |   |   | p |   |   |   | 6
 +---+---+---+---+---+---+---+---+
 |   |   |   |   |   |   |   |   | 5
 +---+---+---+---+---+---+---+---+
 |   |   |   |   |   |   |   |   | 4
 +---+---+---+---+---+---+---+---+
 |   |   |   |   | N |   |   |   | 3
 +---+---+---+---+---+---+---+---+
 |   |   |   |   | B |   |   |   | 2
 +---+---+---+---+---+---+---+---+
 |   |   |   |   | K |   |   |   | 1
 +---+---+---+---+---+---+---+---+
   a   b   c   d   e   f   g   h

Fen: 4k3/4b3/4p3/8/8/4N3/4B3/4K3 w - - 0 1
Key: 27AD1ED89BF41C9C
Checkers: 
Tablebases WDL: Draw (Success)
Tablebases DTZ:    0 (Success)
go depth 12
info depth 1 seldepth 2 multipv 1 score cp 0 wdl 106 788 106 nodes 32 nps 10666 tbhits 21 time 3 pv e2b5 e8f7
info depth 2 seldepth 3 multipv 1 score cp 0 wdl 106 788 106 nodes 72 nps 24000 tbhits 21 time 3 pv e1d2 e8d7
info depth 3 seldepth 4 multipv 1 score cp 0 wdl 106 788 106 nodes 178 nps 59333 tbhits 21 time 3 pv e1d2 e8d7 e2b5 d7d6
info depth 4 seldepth 5 multipv 1 score cp 0 wdl 106 788 106 nodes 786 nps 196500 tbhits 21 time 4 pv e1d1 e8d7 d1c2 e7b4
info depth 5 seldepth 6 multipv 1 score cp 0 wdl 106 788 106 nodes 1539 nps 384750 tbhits 21 time 4 pv e2d3 e7c5 d3b5 e8f7 e3f1 c5g1
info depth 6 seldepth 8 multipv 1 score cp 0 wdl 106 788 106 nodes 2645 nps 529000 tbhits 21 time 5 pv e1d1 e8d7 d1c2 e7b4 e3f1 d7d6
info depth 7 seldepth 9 multipv 1 score cp 0 wdl 106 788 106 nodes 6814 nps 973428 tbhits 21 time 7 pv e1d1 e8d7 e3g4 e7h4 g4e5 d7d6 e5d3 h4f6
info depth 8 seldepth 11 multipv 1 score cp 0 wdl 106 788 106 nodes 10396 nps 1299500 tbhits 21 time 8 pv e1d1 e8d7 e3f1 e7b4 d1c2 d7d6 f1g3 b4e1
info depth 9 seldepth 15 multipv 1 score cp 0 wdl 106 788 106 nodes 48464 nps 1794962 tbhits 21 time 27 pv e2b5 e8f7 e1f1 e7c5 e3d1 f7e7 d1b2 c5d6 b2d3 d6h2
info depth 10 seldepth 17 multipv 1 score cp 0 wdl 106 788 106 nodes 67261 nps 1868361 tbhits 21 time 36 pv e2b5 e8f7 e1f1 e7c5 e3g4 c5b4 b5a4 b4d6 a4b3 d6c7 f1g2 f7e7
info depth 11 seldepth 18 multipv 1 score cp 0 wdl 106 788 106 nodes 150434 nps 2060739 tbhits 21 time 73 pv e2b5 e8f7 e1f1 e7c5 e3g4 c5b4 b5a4 b4d6 a4b3 d6a3 f1g2 f7e7 g2f3 a3c5
info depth 12 seldepth 20 multipv 1 score cp 0 wdl 106 788 106 nodes 362970 nps 2226809 tbhits 21 time 163 pv e1d1 e7b4 e2a6 e8e7 e3g2 b4c5 g2h4 e7d6 h4f3 d6d5 f3d2 c5d4
bestmove e1d1 ponder e7b4

[hgm]

It also fails to assign that to the initial position, right?

[zullil]

As many of know, NN engines have been reporting WDL data with Scoring Percentage for some time. A while back, I took their formula to convert to a centipawn and applied it the centipawn evaluation to come u

It is worth noting that Stockfish's WDL is a function of both the evaluation and the current game ply.

https://github.com/official-stockfish/S ... dfdf670392

[zullil]

It also fails to assign that to the initial position, right?

Right:

info depth 46 seldepth 63 multipv 1 score cp 52 wdl 197 749 54 nodes 6395029891 nps 1637910 hashfull 1000 tbhits 0 time 3904382 pv c2c4 e7e6 b1c3 d7d5 d2d4 g8f6 c4d5 e6d5 c1g5 f8e7 e2e3 e8g8 g1f3 c7c6 f1d3 f8e8 e1g1 h7h6 g5f4 c8e6 h2h3 b8d7 f4h2 a8c8 d1b1 c6c5 f1d1 d8b6 a2a4 e7d6 h2e5 a7a6 d4c5 b6c5 e5d4 c5c7 a4a5 d7e5 d3e2 e5f3 e2f3 d6e5 d4a7 g7g6 c3d5 f6d5 f3d5 e6d5 d1d5 c7c2 b1c2

[Milos]

As many of know, NN engines have been reporting WDL data with Scoring Percentage for some time. A while back, I took their formula to convert to a centipawn and applied it the centipawn evaluation to come u

It is worth noting that Stockfish's WDL is a function of both the evaluation and the current game ply.

https://github.com/official-stockfish/S ... dfdf670392

Yes and that is good. What is not good is that in case of TB draw it doesn't recognize it, i.e. it still reports statistics with some percentage of win and draw changes.

[zullil]

As many of know, NN engines have been reporting WDL data with Scoring Percentage for some time. A while back, I took their formula to convert to a centipawn and applied it the centipawn evaluation to come u

It is worth noting that Stockfish's WDL is a function of both the evaluation and the current game ply.

https://github.com/official-stockfish/S ... dfdf670392

Yes and that is good. What is not good is that in case of TB draw it doesn't recognize it, i.e. it still reports statistics with some percentage of win and draw changes.

I agree about tablebases. Reporting a 20% chance that the opening position is winning for White strikes me as rather silly. But maybe that's what happens at Fishtest where time controls are very short.

[nnnnnnnn]

As many of know, NN engines have been reporting WDL data with Scoring Percentage for some time. A while back, I took their formula to convert to a centipawn and applied it the centipawn evaluation to come u

It is worth noting that Stockfish's WDL is a function of both the evaluation and the current game ply.

https://github.com/official-stockfish/S ... dfdf670392

That makes it useless for analysis of positions or studies, for which the ply is immaterial or unknown. It would be extremely useful to have a wdl feature that accounted not for "ply" but for the material on the board (for which "ply" is likely a proxy) and whether the value derives from a TB (or some other known certain draw such as perpetual).

[zullil]

As many of know, NN engines have been reporting WDL data with Scoring Percentage for some time. A while back, I took their formula to convert to a centipawn and applied it the centipawn evaluation to come u

It is worth noting that Stockfish's WDL is a function of both the evaluation and the current game ply.

https://github.com/official-stockfish/S ... dfdf670392

That makes it useless for analysis of positions or studies, for which the ply is immaterial or unknown. It would be extremely useful to have a wdl feature that accounted not for "ply" but for the material on the board (for which "ply" is likely a proxy) and whether the value derives from a TB (or some other known certain draw such as perpetual).

I suggest you post this at https://groups.google.com/g/fishcooking?pli=1