Removing Stockfish specific end game knowledge, use syzygy

[mohzus]

Hi,
I was wondering what would happen to Stockfish's strength if one removed its specific end game knowledge but use the syzygy table bases instead.
I've seen that Uri Blass tried something similar; he removed some end game knowledge -apparently rarely of any use- and tested this version against the dev version of that time. The reduced knowledge Stockfish did well at short time control and I think it was not tested at long time control. Here's the test: http://tests.stockfishchess.org/tests/v ... 3ba5ae705a. So a removal of 87 lines of code showed to be an improvement in terms of elo if I understand well. I know that removing all the end game knowledge would be harmful, but if I use the syzygy table bases I am not so sure about it.
I would like to hear your opinions on this.

For my part, as a non-programmer, I tried to remove all 5 pieces and less end game knowledge in Stockfish's code but I got errors when compiling. I personally have the 5 pieces syzygy EGTB (not the 6 pieces and I've seen 6 and 7 pieces end game specific knowledge in SF's code, I tried not to remove that).
Here's my engame.cpp file:

Code: Select all

http://speedy.sh/EN8d9/endgame.cpp

.
Here are the errors I got:

Code: Select all

g++ -o stockfish benchmark.o bitbase.o bitboard.o book.o endgame.o evaluate.o main.o material.o misc.o movegen.o movepick.o notation.o pawns.o position.o search.o tbprobe.o thread.o timeman.o tt.o uci.o ucioption.o -lgcov -lpthread -Wall -Wcast-qual -fno-exceptions -fno-rtti -fprofile-generate -ansi -pedantic -Wno-long-long -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -DUSE_BSFQ -flto
/tmp/ccGW1opD.ltrans2.ltrans.o&#58;(.rodata+0x28&#41;&#58; undefined reference to `Endgame<&#40;EndgameType&#41;21, ScaleFactor>&#58;&#58;operator&#40;)&#40;Position const&) const'
/tmp/ccGW1opD.ltrans2.ltrans.o&#58;(.rodata+0xa8&#41;&#58; undefined reference to `Endgame<&#40;EndgameType&#41;11, ScaleFactor>&#58;&#58;operator&#40;)&#40;Position const&) const'
/tmp/ccGW1opD.ltrans2.ltrans.o&#58;(.rodata+0x128&#41;&#58; undefined reference to `Endgame<&#40;EndgameType&#41;1, Value>&#58;&#58;operator&#40;)&#40;Position const&) const'
/tmp/ccGW1opD.ltrans7.ltrans.o&#58;(.rodata+0x248&#41;&#58; undefined reference to `Endgame<&#40;EndgameType&#41;13, ScaleFactor>&#58;&#58;operator&#40;)&#40;Position const&) const'
/tmp/ccGW1opD.ltrans7.ltrans.o&#58;(.rodata+0x2c8&#41;&#58; undefined reference to `Endgame<&#40;EndgameType&#41;0, Value>&#58;&#58;operator&#40;)&#40;Position const&) const'
collect2&#58; error&#58; ld returned 1 exit status
make&#91;2&#93;&#58; *** &#91;stockfish&#93; Error 1
make&#91;2&#93;&#58; Leaving directory `/home/bigtournevisse/Downloads/Stockfish-master/src'
make&#91;1&#93;&#58; *** &#91;gcc-profile-make&#93; Error 2
make&#91;1&#93;&#58; Leaving directory `/home/bigtournevisse/Downloads/Stockfish-master/src'
make&#58; *** &#91;profile-build&#93; Error 2

so I don't know whether I removed too few or too much.
If anyone is willing to help me to remove successfully all 5 pieces and less end game knowledge (and if that makes any sense, which isn't a given I know), I'm all in.

[Uri Blass]

Hi,
I was wondering what would happen to Stockfish's strength if one removed its specific end game knowledge but use the syzygy table bases instead.
I've seen that Uri Blass tried something similar; he removed some end game knowledge -apparently rarely of any use- and tested this version against the dev version of that time. The reduced knowledge Stockfish did well at short time control and I think it was not tested at long time control. Here's the test: http://tests.stockfishchess.org/tests/v ... 3ba5ae705a

Note that the result of the test does not mean that it is better at short time control but only that there is no big regression.

0 elo change has probability of 50% to pass.
If I remember correctly
some long time control test took many games with result that is close to 50% and after many games I stopped it because marco asked to stop it.

[syzygy]

I'll just state that imho it makes no sense to remove such useful code that does not cost any cpu cycles in positions where it is not needed.

It is not going to be measurable in Elo since the number of games where it makes a difference does not go above the noise level.

[Uri Blass]

I'll just state that imho it makes no sense to remove such useful code that does not cost any cpu cycles in positions where it is not needed.

It is not going to be measurable in Elo since the number of games where it makes a difference does not go above the noise level.

I disagree that it makes no sense to remove endgame code when the code is about how to win some won endgames like KBN vs K that it is possible to win by search without evaluation.

[mohzus]

Thank you very much guys for your opinions.
I have talked to a friend on FICS and he told me to remove/comment the lines 113 to 127 in endgame.cpp:

Code: Select all

//  add<KPK>("KPK");
//  add<KNNK>("KNNK");
//  add<KBNK>("KBNK");
//  add<KRKP>("KRKP");
//  add<KRKB>("KRKB");
//  add<KRKN>("KRKN");
//  add<KQKP>("KQKP");
//  add<KQKR>("KQKR");

//  add<KNPK>("KNPK");
//  add<KNPKB>("KNPKB");
//  add<KRPKR>("KRPKR");
//  add<KRPKB>("KRPKB");
//  add<KBPKB>("KBPKB");
//  add<KBPKN>("KBPKN");

I could compile without error.
The "bench" of this version is 8252600 while the original SF has a bench of 8347121.

I have tested this version of SF and it seems to work fine. Since I use the Syzygy I think I'm going to stick to this version until I find some bug or so.

[syzygy]

I disagree that it makes no sense to remove endgame code when the code is about how to win some won endgames like KBN vs K that it is possible to win by search without evaluation.

And lose the ability to win KBNK at sufficiently short time control. Bad idea, period.

Having KBNK code has zero cost. Zero.

[Uri Blass]

I disagree that it makes no sense to remove endgame code when the code is about how to win some won endgames like KBN vs K that it is possible to win by search without evaluation.

And lose the ability to win KBNK at sufficiently short time control. Bad idea, period.

Having KBNK code has zero cost. Zero.

The cost is longer code and I do not care about short time control that I never use.
The stockfish team prefer simplification of the code even with no elo gain.

The target of the developers is also not to have the best blitz player but have a program that scale well(this is the reason that the stockfish team use sprt(-1.5,4.5) at 15 seconds per game and sprt(0,6) 60 seconds per game so changes that are not scalable(help at 15+0.05 and do not help at 60+0.05 are probably not going to pass).

Note that if the stockfish team drop verification search only because it is a simplification and there is no proof that verification search helps then it is not consistent to keep knowledge about specific endgames when the advantage from them is near 0 elo points.

[syzygy]

I disagree that it makes no sense to remove endgame code when the code is about how to win some won endgames like KBN vs K that it is possible to win by search without evaluation.

And lose the ability to win KBNK at sufficiently short time control. Bad idea, period.

Having KBNK code has zero cost. Zero.

The cost is longer code and I do not care about short time control that I never use.
The stockfish team prefer simplification of the code even with no elo gain.

Luckily I don't have to reason with you. I think Marco has common sense and will apply it.

[Joerg Oster]

Well, the trend is to remove such 'bloat stuff'. Unfortunately.
If it was my engine, i would really be ashamed if it didn't know that KBBK is a draw and not a win, if the 2 bishops are on the same square color.

Of course, one can argue that this doesn't occur in game play very often. And yet ...

[mohzus]

I disagree that it makes no sense to remove endgame code when the code is about how to win some won endgames like KBN vs K that it is possible to win by search without evaluation.

And lose the ability to win KBNK at sufficiently short time control. Bad idea, period.

Having KBNK code has zero cost. Zero.

Sorry, I do not understand. I remove SF specific knowledge but I replace it with the 5 pieces Syzygy TB's. Stockfish should find such mates instantly, right?

Because I've just tried it and it finds the checkmate instantly:

Code: Select all

setoption name SyzygyProbeLimit value 5
setoption name SyzygyPath value /media/bigtournevisse/0def13e7-1105-40e3-9aac-44393a0f6ac3/home/bigtournevisse/Downloads/syzygy
info string Found 145 tablebases.
setoption name Threads value 2
position fen 8/8/8/8/3k4/8/N7/KB6
isready
readyok
go infinite
info depth 1 seldepth 1 score cp -15100 nodes 71 nps 3227 tbhits 5 time 22 multipv 1 pv d4e5
info depth 2 seldepth 2 score cp -15100 nodes 107 nps 4652 tbhits 5 time 23 multipv 1 pv d4e5 a1b2
info depth 3 seldepth 3 score cp -15100 nodes 158 nps 6869 tbhits 5 time 23 multipv 1 pv d4e5 a1b2 e5f4
info depth 4 seldepth 4 score cp -15100 nodes 328 nps 14260 tbhits 5 time 23 multipv 1 pv d4e5 a1b2 e5f4 b2c1
info depth 5 seldepth 5 score cp -15100 nodes 475 nps 19791 tbhits 5 time 24 multipv 1 pv d4e5 a1b2 e5f4 b2c3 f4g4
info depth 6 seldepth 6 score cp -15100 nodes 907 nps 36280 tbhits 5 time 25 multipv 1 pv d4e5 a1b2 e5f4 b2c3 f4e5 a2c1
info depth 7 seldepth 7 score cp -15100 nodes 1452 nps 53777 tbhits 5 time 27 multipv 1 pv d4e5 a1b2 e5f4 b2c3 f4e5 a2c1 e5f4
info depth 8 seldepth 8 score cp -15100 nodes 2236 nps 77103 tbhits 5 time 29 multipv 1 pv d4e5 a1b2 e5f4 b2c3 f4e5 a2c1 e5d5 c3b3
info depth 9 seldepth 10 score cp -15100 nodes 4702 nps 134342 tbhits 5 time 35 multipv 1 pv d4e5 a1b2 e5f4 b2c3 f4e5 c3c4 e5e6 c4d4 e6d6 a2c1
info depth 10 seldepth 12 score cp -15100 nodes 8360 nps 203902 tbhits 5 time 41 multipv 1 pv d4e5 a1b2 e5f4 b2c3 f4e5 c3c4 e5e6 c4d4 e6d6 a2c1 d6e6 b1c2 e6d6
info depth 11 seldepth 12 score cp -15100 nodes 10885 nps 253139 tbhits 5 time 43 multipv 1 pv d4e5 a1b2 e5f4 b2c3 f4e5 c3c4 e5e6 c4d4 e6d6 a2c1 d6e6 b1c2 e6d6
info depth 12 seldepth 12 score cp -15100 nodes 16427 nps 335244 tbhits 5 time 49 multipv 1 pv d4e5 a1b2 e5f4 b2c3 f4e5 c3c4 e5e6 c4d4 e6d6 a2c1 d6e6 b1c2 e6d6
info depth 13 seldepth 13 score cp -15100 nodes 27637 nps 476500 tbhits 5 time 58 multipv 1 pv d4e5 a1b2 e5f4 b2c3 f4e5 c3c4 e5e6 c4d4 e6d6 a2c1 d6e6 b1c2 e6d6
info depth 14 seldepth 14 score cp -15100 nodes 42407 nps 588986 tbhits 5 time 72 multipv 1 pv d4e5 a1b2 e5f4 b2c3 f4e5 c3c4 e5e6 c4d4 e6d6 a2c1 d6e6 b1c2 e6d6 d4c4 d6e6
info depth 15 seldepth 15 score cp -15100 nodes 61101 nps 745134 tbhits 5 time 82 multipv 1 pv d4e5 a1b2 e5f4 b2c3 f4e5 c3c4 e5e6 c4d4 e6d6 a2c1 d6e6 b1a2 e6f5 d4d5 f5f4 d5d4 f4g5 d4e4
info depth 16 seldepth 18 score cp -15100 nodes 107632 nps 1025066 tbhits 5 time 105 multipv 1 pv d4e5 a1b2 e5f4 b2c3 f4e5 c3c4 e5e6 c4d4 e6d6 a2c1 d6e6 b1a2 e6f5 c1d3 f5g5 d4e5 g5h4 d3c1
info depth 17 seldepth 18 score cp -15100 nodes 146551 nps 1241957 tbhits 5 time 118 multipv 1 pv d4e5 a1b2 e5f4 b2c3 f4e5 c3c4 e5e6 c4d4 e6d6 a2c1 d6e6 b1a2 e6f5 c1d3 f5g5 d4e5 g5g4 d3c1
info depth 18 seldepth 19 score cp -15100 nodes 245583 nps 1637220 tbhits 5 time 150 multipv 1 pv d4e5 a1b2 e5f4 b2c3 f4e5 c3c4 e5e6 c4d4 e6d6 a2c1 d6e6 b1a2 e6f5 c1d3 f5g5 d4e5 g5g4 e5e4 g4h4 d3c1 h4g5
info depth 19 seldepth 20 score cp -15100 nodes 386680 nps 2035157 tbhits 5 time 190 multipv 1 pv d4e5 a1b2 e5f4 b2c3 f4e5 c3c4 e5e6 c4d4 e6d6 a2c1 d6e6 b1a2 e6f5 c1d3 f5g5 d4e5 g5g4 e5e4 g4g5 e4e5
info depth 20 seldepth 21 score cp -15100 nodes 846492 nps 2748350 tbhits 5 time 308 multipv 1 pv d4e5 a1b2 e5f4 b2c3 f4e5 c3c4 e5e6 c4c5 e6e5 a2b4 e5f4 c5d4 f4f3 b4d5 f3g4 d4e4 g4h5 e4e3 h5g4 e3e4
info depth 21 seldepth 21 score cp -15100 nodes 1071280 nps 2919019 tbhits 5 time 367 multipv 1 pv d4e5 a1b2 e5f4 b2c3 f4e5 c3c4 e5e6 c4c5 e6e5 a2b4 e5f4 c5d4 f4f3 b4d5 f3e2 d4c3 e2f3 c3d3 f3f2 d3e4 f2e2 d5c3 e2f2
info depth 22 seldepth 23 score cp -15100 nodes 1568211 nps 3193912 tbhits 5 time 491 multipv 1 pv d4e5 a1b2 e5f4 b2c3 f4e5 c3c4 e5e6 c4c5 e6e5 a2b4 e5f4 c5d4 f4f3 d4e5 f3e3 b1e4 e3d2 e5d4 d2e2 e4c2 e2f3 c2b1 f3g4 d4e5 g4f3 b4d5 f3g3 e5e4 g3g4 d5f6 g4g5 e4e5 g5h4 e5f4
info depth 23 seldepth 24 score cp -15100 nodes 2304971 nps 3399662 tbhits 5 time 678 multipv 1 pv d4e5 a1b2 e5f4 a2b4 f4g5 b4d5 g5h4 b2c3 h4g5 c3d4 g5h6 d4e5 h6g7 d5b4 g7f7 b1f5 f7e7 f5e6 e7e8 e5d6 e8f8 b4d5 f8g7 d6e5 g7h6
info depth 24 seldepth 24 score cp -15100 nodes 3128090 nps 3562744 tbhits 5 time 878 multipv 1 pv d4e5 a1b2 e5f4 a2b4 f4g5 b4d5 g5h4 b2c3 h4g5 c3d4 g5h6 d4e5 h6g7 d5b4 g7f7 b1f5 f7e7 f5e6 e7e8 e5d6 e8f8 b4d5 f8g7 d6e7
info depth 25 seldepth 25 score cp -15100 nodes 4213551 nps 3676746 tbhits 5 time 1146 multipv 1 pv d4e5 a1b2 e5f4 a2b4 f4g5 b4d5 g5h4 b2c3 h4g5 c3d4 g5h6 d4e5 h6g7 d5b4 g7f7 b1f5 f7e7 f5e6 e7e8 e5d6 e8f8 e6f5 f8f7 f5b1 f7f6 b1c2 f6g5 d6e5 g5g4 c2d3 g4g3 e5e4 g3f2 d3c2 f2e2 c2b1 e2d2 b4a2 d2e2 e4f4 e2d2 f4f3
info depth 26 seldepth 26 score cp -15100 nodes 5937078 nps 3796085 tbhits 5 time 1564 multipv 1 pv d4e5 a1b2 e5f4 a2b4 f4g5 b4d5 g5h4 b2c3 h4g5 c3d4 g5h6 d4e5 h6g7 d5b4 g7f7 b1f5 f7g7 b4d5 g7f7 e5d6 f7g7 d6e6 g7h6 e6f6 h6h5 f5e6 h5h4
info depth 27 seldepth 27 score cp -15100 nodes 8086924 nps 3887944 tbhits 5 time 2080 multipv 1 pv d4e5 a1b2 e5f4 a2b4 f4g5 b4d5 g5h4 b2c3 h4g5 c3d4 g5h6 d4e5 h6g7 d5b4 g7f7 b1f5 f7g7 f5e6 g7g6 b4c6 g6g5 c6e7 g5h4 e5f4
info depth 28 currmove d4e3 currmovenumber 2
info depth 28 seldepth 30 score cp -15100 nodes 13747319 nps 4014988 tbhits 5 time 3424 multipv 1 pv d4e5 a1b2 e5f4 a2b4 f4g5 b4d5 g5h4 b2c3 h4g5 c3d4 g5h6 d4e5 h6g7 b1c2 g7f7 c2f5 f7e8 d5b4 e8f7 f5c2 f7g7 b4d5
info depth 29 currmove d4e5 currmovenumber 1
info depth 29 currmove d4d5 currmovenumber 2
info depth 29 currmove d4c4 currmovenumber 4
info depth 29 currmove d4c5 currmovenumber 5
info depth 29 seldepth 33 score cp -15100 nodes 23211869 nps 4086596 tbhits 5 time 5680 multipv 1 pv d4e5 a1b2 e5f4 a2b4 f4g5 b4d5 g5h4 b2c3 h4g5 c3d4 g5h6 d4e5 h6g7 b1c2 g7f7 c2a4 f7g7 a4e8 g7f8 e8g6 f8g7 e5f5 g7h6 d5f6 h6g7 f6h5 g7f8 f5f6 f8g8 g6b1
info depth 30 currmove d4e5 currmovenumber 1
info depth 30 currmove d4e3 currmovenumber 2
info depth 30 currmove d4c5 currmovenumber 4
info depth 30 currmove d4d5 currmovenumber 5
info depth 30 seldepth 33 score cp -15100 nodes 35117807 nps 3969459 tbhits 5 time 8847 multipv 1 pv d4e5 a1b2 e5f4 a2b4 f4g5 b4d5 g5h4 b2c3 h4g5 c3d4 g5h6 d4e5 h6g7 b1c2 g7f7 c2a4 f7g7 a4e8 g7f8 e8g6 f8g7 e5f5 g7h6 d5f6 h6g7 f6h5 g7f8 f5f6 f8g8 f6e6
info depth 31 currmove d4e5 currmovenumber 1
info depth 31 currmove d4d5 currmovenumber 2
info depth 31 seldepth 34 score cp -15100 nodes 56409578 nps 4041090 tbhits 5 time 13959 multipv 1 pv d4e5 a1b2 e5f4 a2b4 f4g5 b4d5 g5h4 b2c3 h4g5 c3d4 g5h6 d4e5 h6g7 b1c2 g7f7 c2a4 f7g7 a4e8 g7f8 e8g6 f8g7 g6e8
info depth 32 currmove d4e5 currmovenumber 1
stop