Performance of Syzygy and Scorpio

[Laskos]

Please try again. If it doesn't work, I have to dig deeper to see which other option should be set. I compiled using the XP toolset for Visual Studio 2012 but maybe there are other options that copying a project won't bring.

In Properties:

Use de MFC in static library --- ON
In C/C++:
--code generation
---Runtyme library Multi-threaded(/MT)

Probe.

I have that as Multi-threaded DLL for Scorpio as well, that worked. I suppose that Kai is quiet means the new compiles work.
Thanks

Can you guess from below if I set-up correctly the SF Scorpio:

Code: Select all

Games Completed = 352 of 2000 (Avg game length = 8.537 sec)
Settings = Gauntlet/64MB/10000ms+100ms/M 700000cp for 1000 moves, D 150000 moves/EPD:C:\LittleBlitzer\360.epd(360)
Time = 1108 sec elapsed, 5190 sec remaining
 1.  Shredder 12 Nalimov      	202.5/352	177-124-51  	(L: m=124 t=0 i=0 a=0)	(D: r=3 i=12 f=30 s=6 a=0)	(tpm=14.8 d=1.16 nps=9006)
 2.  SF Scorpio               	73.0/176	59-89-28  	(L: m=89 t=0 i=0 a=0)	(D: r=0 i=3 f=23 s=2 a=0)	(tpm=228.9 d=33.05 nps=1146880)
 3.  SF NO TB                 	76.5/176	65-88-23  	(L: m=88 t=0 i=0 a=0)	(D: r=3 i=9 f=7 s=4 a=0)	(tpm=222.0 d=26.98 nps=2463556)

As you see, SF Scorpio performs no better than SF NO TB, it's something wrong I am doing or my "hard TB wins" file is bad for WDL? I might use another TB wins EPD suite, with more varied positions.

[Daniel Shawul]

I am not sure how you set it up. First you might then try Scorpio. Toga performed better so Scorpio should too. Then we will go to Stockfish.

[syzygy]

I might use another TB wins EPD suite, with more varied positions.

Most interesting would be endgame positions with more than 5 pieces. But I don't have a good test suite for that.

[Laskos]

I might use another TB wins EPD suite, with more varied positions.

Most interesting would be endgame positions with more than 5 pieces. But I don't have a good test suite for that.

I do have a set of 6-7 men positions, although some are imbalanced and many are not clear wins or draws. I will try to test on them later (there will be no "perfect score"), but now my girlfriend wants to go out to eat something.

[syzygy]

Syzygy WDL doesn't help SF at all.

That's a bit surprising as SF in such positions only probes WDL after a capture or pawn move. The probes after a capture into a 4-piece position probably won't help compared to default SF (since default SF most likely evaluates those 4-piece positions perfectly anyway), but those after pawn moves should be of benefit.

Actually, SF+WDL currently does not probe after pawn moves in this situation. It only probes after captures. So for positions with high DTC (distance to conversion) SF+WDL will not do much better than default SF. (It should still help to avoid playing an immediate drawing move from a won position.)

This is a bit annoying to fix, since SF insists on the 50-move counter being hidden from most of the engine.

I've fixed it anyway. The downside is that SF will have a bit too much preference for pushing pawns, but this only occurs if the position on the board is in the tablebases and DTZ is not available. I suppose http://abrok.eu/stockfish_syzygy/ will have a new binary before you've returned home

[ouachita]

Most interesting would be endgame positions with more than 5 pieces. But I don't have a good test suite for that.

I've got a 700 game endgame d/b with around 700 games with 3 to 18 pieces. Don't recall precisely, but most are wins.

[Adam Hair]

I have gotten results similar to Kai's:

Code: Select all

Program                                    +   -   Games   Score   Av.Op.  Draws

  1 Gaviota GTB                            9   9  6000    52.9 %   2390    5.9 %
  2 Stockfish Syzygy none                 21  21  1000    47.9 %   2410    4.2 %
  3 Stockfish Scorpio none                21  21  1000    47.6 %   2410    4.8 %
  4 Stockfish Scorpio EGBB  cache 256     21  21  1000    47.1 %   2410    5.7 %
  5 Stockfish Scorpio EGBB cache 16       21  21  1000    46.8 %   2410    6.5 %
  6 Stockfish Syzygy WDL probe depth 1    21  21  1000    46.5 %   2410    7.1 %
  7 Stockfish Syzygy WDL probe depth 4    21  21  1000    46.5 %   2410    7.1 %
   

I can try out Scorpio, but I will need a Linux compile. I have tried to compile it on my computer but ran into problems that I have not solved yet.

[Laskos]

I have gotten results similar to Kai's:

Code: Select all

Program                                    +   -   Games   Score   Av.Op.  Draws

  1 Gaviota GTB                            9   9  6000    52.9 %   2390    5.9 %
  2 Stockfish Syzygy none                 21  21  1000    47.9 %   2410    4.2 %
  3 Stockfish Scorpio none                21  21  1000    47.6 %   2410    4.8 %
  4 Stockfish Scorpio EGBB  cache 256     21  21  1000    47.1 %   2410    5.7 %
  5 Stockfish Scorpio EGBB cache 16       21  21  1000    46.8 %   2410    6.5 %
  6 Stockfish Syzygy WDL probe depth 1    21  21  1000    46.5 %   2410    7.1 %
  7 Stockfish Syzygy WDL probe depth 4    21  21  1000    46.5 %   2410    7.1 %
   

I can try out Scorpio, but I will need a Linux compile. I have tried to compile it on my computer but ran into problems that I have not solved yet.

Yes, I checked and re-checked my SF Scorpio, it seems to work as intended. The conclusion would by that Syzygy WDL and Scorpio are not helping SF to convert root TB wins. Maybe it would be better to switch them off when a TB position is reached.

I did another experiment. With 6-7 men positions of all kinds, where I don't have a perfect player, as I do not have 6-7 men TBs, I played a Round-Robin:

Code: Select all

    Program                            Score     

  1 SF 04.02 Syzygy                : 1533.5/3000
  2 SF 04.02 Syzygy WDL            : 1517.0/3000
  3 SF Scorpio                     : 1496.0/3000
  4 SF 04.02 NO TB                 : 1453.5/3000

Here Scorpio and Syzygy WDL do help, although not as much as the full Syzygy.

[syzygy]

I have gotten results similar to Kai's:

Code: Select all

Program                                    +   -   Games   Score   Av.Op.  Draws

  1 Gaviota GTB                            9   9  6000    52.9 %   2390    5.9 %
  2 Stockfish Syzygy none                 21  21  1000    47.9 %   2410    4.2 %
  3 Stockfish Scorpio none                21  21  1000    47.6 %   2410    4.8 %
  4 Stockfish Scorpio EGBB  cache 256     21  21  1000    47.1 %   2410    5.7 %
  5 Stockfish Scorpio EGBB cache 16       21  21  1000    46.8 %   2410    6.5 %
  6 Stockfish Syzygy WDL probe depth 1    21  21  1000    46.5 %   2410    7.1 %
  7 Stockfish Syzygy WDL probe depth 4    21  21  1000    46.5 %   2410    7.1 %
   

I can try out Scorpio, but I will need a Linux compile. I have tried to compile it on my computer but ran into problems that I have not solved yet.

Yes, I checked and re-checked my SF Scorpio, it seems to work as intended. The conclusion would by that Syzygy WDL and Scorpio are not helping SF to convert root TB wins. Maybe it would be better to switch them off when a TB position is reached.

Did you and Adam try the latest version on http://abrok.eu/stockfish_syzygy/ ? I don't see how WDL probing, the way it is done, could not be at least of some help with converting TB wins (with the exception of KBNK-like endgames).

[Max]

The conclusion would by that Syzygy WDL and Scorpio are not helping SF to convert root TB wins. Maybe it would be better to switch them off when a TB position is reached.

In some cases bitbases can hurt to defend against root TB positions.

[D]8/2K3p1/4k3/7r/8/8/R7/8 w - - 0 49

Stockfish with Scorpio egbbs defend this endgame not very well. White plays 1.Re2+ and after Re5 he trades the rooks with 2.Rxe5+.

[D]8/2K3p1/8/4k3/8/8/8/8 w - - 0 51

This makes it easy to win, even for weaker players like Beowulf as example. Stockfish with egbb gets mated soon.

Code: Select all

[Event "Computer Chess Game"]
[Site "Home"]
[Date "2014.02.06"]
[Round "-"]
[White "Stockfish 2014 (Scorpio)"]
[Black "Beowulf 2.4a"]
[Result "0-1"]
[TimeControl "40/60"]
[FEN "8/2K3p1/4k3/7r/8/8/R7/8 w - - 0 1"]
[SetUp "1"]

{--------------
. . . . . . . .
. . K . . . p .
. . . . k . . .
. . . . . . . r
. . . . . . . .
. . . . . . . .
R . . . . . . .
. . . . . . . .
white to play
--------------}
1. Re2+ {-99,90/76} Re5 {+1,49/10 1,0} 2. Rxe5+ {-100,63/100 0,2} Kxe5
{+111,77/17 0,4} 3. Kd7 {-103,02/69 0,8} g5 {+327,47/17 1,2} 4. Ke7
{-103,12/77 0,8} g4 {+6,31/2 0,2} 5. Kd7 {-103,29/100 0,1} g3 {+6,99/2 0,1}
6. Kc6 {-103,36/100 0,2} g2 {+7,58/2 0,2} 7. Kc5 {-104,65/100 0,1} g1=Q+
{+327,47/2 0,1} 8. Kc4 {-99,88/36 0,9} Qd4+ {+112,06/2 0,2} 9. Kb3
{-99,90/70 0,8} Kd5 {+111,80/3 0,2} 10. Ka2 {-99,92/100 0,2} Kc4
{+327,47/2 0,2} 11. Kb1 {-99,94/100 0,2} Kc3 {+327,47/2 0,2} 12. Ka1
{-99,96/100 0,2} Kb3+ {+327,48/2 0,2} 13. Kb1 {+0,00/1 0,2} Qb2#
{+327,66/2 0,1}
{Xboard adjudication: Checkmate} 0-1

In fact, this endgame should be won for black. But white can defend much harder, shown by Stockfish without Scorpio egbbs. Now white managed to draw against the same opponent!

Code: Select all

[Event "Computer Chess Game"]
[Site "Home"]
[Date "2014.02.06"]
[Round "-"]
[White "Stockfish 2014"]
[Black "Beowulf 2.4a"]
[Result "1/2-1/2"]
[TimeControl "40/60"]
[FEN "8/2K3p1/4k3/7r/8/8/R7/8 w - - 0 1"]
[SetUp "1"]

{--------------
. . . . . . . .
. . K . . . p .
. . . . k . . .
. . . . . . . r
. . . . . . . .
. . . . . . . .
R . . . . . . .
. . . . . . . .
white to play
--------------}
1. Ra6+ {-1,95/21} Kf7 {+1,45/10 0,9} 2. Ra7 {-2,11/25 0,9} Rf5
{+1,49/10 1,4} 3. Kc6+ {-2,35/24 1,0} Kf6 {+1,53/12 1,4} 4. Ra4
{-3,02/25 1,4} g5 {+1,63/11 1,4} 5. Kd7 {-3,76/25 2,1} Rd5+ {+1,74/10 1,1}
6. Kc6 {-4,77/25 4} Rd8 {+1,85/11 1,4} 7. Rb4 {-5,39/21 2,6} Rg8
{+2,05/11 1,5} 8. Kd5 {-5,56/21 1,3} g4 {+2,55/11 1,4} 9. Ke4
{-5,64/25 0,8} g3 {+2,81/10 1,7} 10. Rb6+ {-0,02/23 1,0} Ke7 {+0,18/11 1,2}
11. Rb1 {+0,00/32 0,7} Kd6 {+0,08/11 1,4} 12. Rd1+ {+0,00/37 0,8} Kc5
{+0,06/12 1,4} 13. Rc1+ {+0,00/39 0,9} Kb4 {+0,06/12 1,0} 14. Kf3
{+0,00/39 0,9} Rg7 {+0,04/11 1,1} 15. Rb1+ {+0,00/40 0,9} Kc3
{+0,07/12 1,3} 16. Rg1 {+0,00/45 0,8} Kd3 {+0,07/11 1,1} 17. Rxg3
{+0,00/38 0,9} Rf7+ {+0,05/10 1,4} 18. Kg4+ {+0,00/41 0,9} Ke4
{+0,06/11 1,3} 19. Rb3 {+0,00/40 1,0} Rg7+ {+0,06/11 1,5} 20. Kh3
{+0,00/46 1,0} Kd4 {+0,05/11 1,4} 21. Rb4+ {+0,00/39 0,9} Kd5
{+0,05/11 1,2} 22. Rg4 {+0,00/42 1,1} Rh7+ {+0,04/10 1,1} 23. Kg3
{+0,00/46 1,0} Rf7 {+0,04/11 1,5} 24. Rf4 {+0,00/48 1,1} Re7 {+0,02/10 1,3}
25. Kg4 {+0,00/45 1,1} Re1 {+0,01/11 1,5} 26. Rf5+ {+0,00/47 1,2} Ke4
{+0,02/11 1,0} 27. Rf4+ {+0,00/52 1,1} Ke5 {+0,01/12 1,5} 28. Rf5+
{+0,00/51 1,2} Kd6 {+0,00/12 1,5} 29. Rf3 {+0,00/43 1,1} Kc5 {+0,00/11 1,5}
30. Rf5+ {+0,00/55 1,3} Kc6 {+0,00/12 1,5} 31. Rf6+ {+0,00/54 1,2} Kd5
{+0,00/12 1,1} 32. Rf5+ {+0,00/68 1,3} Ke4 {+0,00/13 1,6} 33. Rf4+
{+0,00/67 1,4} Ke5 {+0,00/14 1,8} 34. Rf5+ {+0,00/72 1,5} Kd6
{+0,00/14 1,8} 35. Rf3 {+0,00/74 1,7} Kc5 {+0,00/13 1,8} 36. Rf5+
{+0,00/78 1,8} Kc6 {+0,00/14 1,8} 37. Rf6+ {+0,00/80 1,9} Kd5
{+0,00/14 1,9} 38. Rf5+ {+0,00/86 2,3}
{Draw by repetition} 1/2-1/2

Stockfish with Syzygy endgame bases managed to draw too.

Code: Select all

[Event "Computer Chess Game"]
[Site "Home"]
[Date "2014.02.06"]
[Round "-"]
[White "Stockfish 2014 (Syzygy)"]
[Black "Beowulf 2.4a"]
[Result "1/2-1/2"]
[TimeControl "40/60"]
[FEN "8/2K3p1/4k3/7r/8/8/R7/8 w - - 0 1"]
[SetUp "1"]

{--------------
. . . . . . . .
. . K . . . p .
. . . . k . . .
. . . . . . . r
. . . . . . . .
. . . . . . . .
R . . . . . . .
. . . . . . . .
white to play
--------------}
1. Ra6+ {-151,00/21} Kf7 {+1,45/10 0,9} 2. Ra7 {-151,00/25 0,9} Rf5
{+1,49/10 1,4} 3. Kc6+ {-151,00/24 1,0} Kf6 {+1,54/12 1,4} 4. Ra4
{-151,00/25 1,5} g5 {+1,61/11 1,4} 5. Kd7 {-151,00/25 2,2} Rd5+
{+1,72/10 1,0} 6. Kc6 {-151,00/25 4} Rd8 {+1,82/11 1,4} 7. Rb4
{-151,00/21 2,6} Rg8 {+2,02/11 1,4} 8. Kd5 {-151,00/20 0,9} g4
{+2,41/11 1,4} 9. Ke4 {-151,00/21 0,9} Kg5 {+1,79/10 1,4} 10. Rb1
{-151,00/23 0,7} Kh4 {+2,18/10 1,0} 11. Rd1 {-151,00/29 6} g3
{+2,38/11 1,4} 12. Kf3 {+0,00/1 0,2} Rf8+ {+1,79/11 3} 13. Kg2
{+0,00/1 0,2} Rf2+ {+1,79/12 1,3} 14. Kg1 {+0,00/32 0,8} Kh3 {+1,79/12 1,2}
15. Rd8 {+0,00/31 0,7} Rf4 {+1,79/12 1,2} 16. Rh8+ {+0,00/33 0,7} Rh4
{+1,79/14 1,1} 17. Rxh4+ {+0,00/100 0,6} Kxh4 {+2,56/2 0,2} 18. Kg2
{+0,00/1 0,1} Kg4 {+5,93/25 1,2} 19. Kg1 {+0,00/1 0,2} Kf5 {+5,93/23 1,5}
20. Kf1 {+0,00/100 0,2} Kg5 {+5,93/20 11} 21. Kg1 {+0,00/100 0,3} Kg4
{+2,68/23 1,0} 22. Kg2 {+0,00/1 0,2} Kf4 {+2,68/21 1,0} 23. Kg1
{+0,00/1 0,2} Kg5 {+0,00/23 0,2} 24. Kf1 {+0,00/100 0,2} Kf5 {+0,00/21 0,2}
25. Kg1 {+0,00/100 0,2} Ke5 {+0,00/21 0,2} 26. Kf1 {+0,00/100 0,2} Kf5
{+0,00/21 0,1} 27. Kg1 {+0,00/100 0,2} Ke5 {+0,00/21 0,2} 28. Kf1
{+0,00/100 0,2} Kf5 {+0,00/21 0,2}
{Draw by Repetition} 1/2-1/2

Also Stockfish with Gaviota bases managed to draw.

Code: Select all

Event "Computer Chess Game"]
[Site "Home"]
[Date "2014.02.06"]
[Round "-"]
[White "Stockfish DD (Gaviota)"]
[Black "Beowulf 2.4a"]
[Result "1/2-1/2"]
[TimeControl "40/60"]
[FEN "8/2K3p1/4k3/7r/8/8/R7/8 w - - 0 1"]
[SetUp "1"]

{--------------
. . . . . . . .
. . K . . . p .
. . . . k . . .
. . . . . . . r
. . . . . . . .
. . . . . . . .
R . . . . . . .
. . . . . . . .
white to play
--------------}
1. Ra6+ {-99,28/1} Kf7 {+1,45/10 0,9} 2. Ra7 {-99,30/1 0,3} Rf5
{+1,49/10 1,3} 3. Kb8+ {-99,32/1 0,2} Kf8 {+1,58/11 1,5} 4. Kc8
{-99,34/1 0,2} g5 {+1,71/11 1,3} 5. Ra4 {-99,36/1 0,2} Rf4 {+1,76/11 0,9}
6. Ra5 {-99,38/1 0,2} g4 {+1,91/12 1,3} 7. Kd7 {-99,40/1 0,2} g3
{+2,72/11 1,4} 8. Ke6 {+0,00/1 0,2} Rg4 {+2,86/11 1,4} 9. Ra1 {+0,00/1 0,2}
g2 {+2,87/12 1,4} 10. Rg1 {+0,00/1 0,2} Rg5 {+0,16/12 4} 11. Kf6
{+0,00/1 0,2} Rg7 {+0,08/11 1,4} 12. Ke5 {+0,00/1 0,2} Ke8 {+0,06/10 1,3}
13. Kd5 {+0,00/1 0,2} Rg4 {+0,16/11 0,9} 14. Ke6 {+0,00/1 0,2} Rg5
{+0,16/12 1,3} 15. Kf6 {+0,00/1 0,3} Rg8 {+0,08/11 1,4} 16. Kf5
{+0,00/1 0,2} Ke7 {+0,08/10 1,3} 17. Ke5 {+0,00/1 0,2} Rg4 {+0,08/10 1,3}
18. Kf5 {+0,00/1 0,2} Rg7 {+0,04/11 1,3} 19. Re1+ {+0,00/1 0,2} Kd6
{+0,14/12 1,3} 20. Rd1+ {+0,00/1 0,2} Kc5 {+0,14/13 1,3} 21. Rg1
{+0,00/1 0,3} Kc4 {+0,10/12 1,3} 22. Kf4 {+0,00/1 0,2} Kd3 {+0,08/11 1,0}
23. Kf3 {+0,00/1 0,3} Rg6 {+0,07/11 1,3} 24. Kf2 {+0,00/1 0,2} Rg5
{+0,06/10 1,1} 25. Rxg2 {+0,00/1 0,2} Rf5+ {+0,04/10 1,1} 26. Kg3
{+0,00/1 0,2} Ke3 {+0,05/11 1,4} 27. Rb2 {+0,00/1 0,2} Rg5+ {+0,06/11 1,4}
28. Kh4 {+0,00/1 0,2} Rg7 {+0,05/11 1,2} 29. Rb6 {+0,00/1 0,2} Kf4
{+0,05/11 1,5} 30. Kh3 {+0,00/1 0,2} Ke5 {+0,05/11 1,3} 31. Ra6
{+0,00/1 0,3} Kd5 {+0,05/11 1,4} 32. Ra2 {+0,00/1 0,2} Ke4 {+0,05/11 1,5}
33. Rg2 {+0,00/1 0,2} Rf7 {+0,05/12 1,6} 34. Kg3 {+0,00/1 0,2} Ke3
{+0,05/12 1,6} 35. Kh4 {+0,00/1 0,1} Rh7+ {+0,05/11 1,6} 36. Kg3
{+0,00/1 0,1} Ra7 {+0,04/12 1,7} 37. Kh4 {+0,00/1 0,2} Rf7 {+0,05/12 1,7}
38. Kg4 {+0,00/1 0,2} Rf4+ {+0,04/11 1,7} 39. Kh3 {+0,00/1 0,2} Rf6
{+0,04/11 1,3} 40. Rg5 {+0,00/1 0,1} Rf7 {+0,05/12 2,1} 41. Kh4
{+0,00/1 0,1} Kd4 {+0,04/11 1,4} 42. Rg6 {+0,00/1 0,1} Rf1 {+0,04/11 1,4}
43. Kh5 {+0,00/1 0,2} Kd5 {+0,04/11 1,4} 44. Kh6 {+0,00/1 0,1} Rf2
{+0,05/11 1,4} 45. Rg8 {+0,00/1 0,1} Ke6 {+0,05/11 1,4} 46. Rc8
{+0,00/1 0,1} Rg2 {+0,06/11 1,1} 47. Re8+ {+0,00/1 0,1} Kd7 {+0,05/13 1,4}
48. Re4 {+0,00/1 0,1} Kd6 {+0,05/12 1,4} 49. Re8 {+0,00/1 0,1} Kd5
{+0,05/12 1,4} 50. Rb8 {+0,00/1 0,1} Ke5 {+0,05/12 1,4} 51. Re8+
{+0,00/1 0,1} Kf6 {+0,05/13 1,4} 52. Kh5 {+0,00/1 0,1} Kf5 {+0,05/13 1,4}
53. Kh6 {+0,00/1 0,1} Rg1 {+0,05/11 1,5} 54. Re3 {+0,00/1 0,1} Rg6+
{+0,05/12 1,4} 55. Kh5 {+0,00/1 0,2} Rg8 {+0,05/13 1,4} 56. Kh6
{+0,00/1 0,1} Rg1 {+0,05/12 1,2} 57. Rf3+ {+0,00/1 0,2} Ke4 {+0,05/13 1,4}
58. Rf6 {+0,00/1 1,4} Kd5 {+0,05/12 1,4} 59. Rf5+ {+0,00/1 1:10} Ke6
{+0,05/13 42} 60. Rf4 {+0,00/1 0,1} Rg2 {+0,09/2 0,1} 61. Rd4 {+0,00/1 0,1}
Kf6 {+0,10/2 0,1} 62. Rh4 {+0,00/1 0,1} Rg6+ {+0,19/2 0,2} 63. Kh7
{+0,00/1 0,2} Rg7+ {+0,19/2 0,1} 64. Kh6 {+0,00/1 0,1} Rg2 {+0,14/2 0,1}
65. Kh7 {+0,00/1 0,2} Ke5 {+0,15/2 0,1} 66. Kh6 {+0,00/1 0,1} Rc2
{+0,19/2 0,1} 67. Rh5+ {+0,00/1 0,2} Ke4 {+0,17/2 0,1} 68. Rh1
{+0,00/1 0,1} Rc6+ {+0,19/2 0,2} 69. Kh5 {+0,00/1 0,2} Rc5+ {+0,16/2 0,1}
70. Kh6 {+0,00/1 0,1} Rc7 {+0,08/2 0,1} 71. Kg5 {+0,00/1 0,1} Rc3
{+0,05/2 0,1} 72. Rd1 {+0,00/1 0,2} Rg3+ {+0,05/2 0,2} 73. Kf6
{+0,00/1 0,2} Rf3+ {+0,08/2 0,1} 74. Ke7 {+0,00/1 0,1} Re3 {+0,00/3 0,1}
75. Rh1 {+0,00/1 0,1}
{Draw by fifty-move rule} 1/2-1/2

-Max