The value of endgame tablebases

[Colin-G]

I was wondering how many engines there are in the top 100 of CCRL 40/4 rating list (All engines, best version only) that do not use any tablebases, Nalimov, syzygy, gaviota, etc.
Is there any value for the increase in elo obtained by using tablebases?
I have seen several engine matches over the years where the result was "not as it should have been", due to tablebases not being used.
I was watching the end of this game, played on my laptop this morning, between the latest ProDeo v2.2, which does not use tablebases, and the very old (from 2003) and much weaker engine, Amy 0.83, which does.
I was just about to push the button to call it a draw, when ProDeo blundered by playing 72.Ra2 in the position shown. Amy immediatly knew it had a won game with its 5 man Nalimov tbs.
[d]8/8/R2K2p1/8/1r5k/8/6P1/8 w - - 0 72
[pgn]
Event "Arena tournament"]
[Site "HP2010"]
[Date "2017.01.24"]
[Round "1"]
[White "ProDeo 2.2"]
[Black "Amy083"]
[Result "0-1"]
[ECO "C42r"]
[TimeControl "40/240:40/240:40/240"]
[PlyCount "197"]

1.e4 e5 2.Nf3 Nf6 3.Nxe5 d6 4.Nf3 Nxe4 5.d4 d5 6.Bd3 Bd6 7.O-O O-O 8.c4 c6 9.Qc2 Na6 10.a3 Bg4 11.Ne5 Bxe5 12.dxe5 Nac5 13.f3 Nxd3 14.Qxd3 Nc5 15.Qd4 Nb3 16.Qxg4 Nxa1 17.Bh6 g6 18.Bxf8 Kxf8 19.Qf4 dxc4 20.Nc3 Nc2 21.Rd1 Qb6+ 22.Kh1 Qe3 23.Qf6 Kg8 24.Qe7 b5 25.Ne4 Qe2 26.Rg1 Nd4 27.Ng5 Rf8 28.Nxh7 Ra8 29.Qh4 Kg7 30.Qf6+ Kg8 31.Ng5 Rf8 32.Nxf7 Rxf7 33.Qd8+ Rf8 34.Qxd4 a6 35.a4 Rf5 36.Qd6 Kg7 37.e6 Rxf3 38.h4 Rf2 39.Qxc6 Qg4 40.Qb7+ Kh6 41.Qe7 bxa4 42.Qd8 Kh7 43.Qe7+ Kh8 44.Qe8+ Kg7 45.Qe7+ Kh6 46.Qd8 Kg7 47.Qe7+ Kg8 48.Kh2 Qf4+ 49.Kh1 Qg3 50.Qd8+ Rf8 51.Qd4 Rf4 52.Qd8+ Kh7 53.Qe7+ Kh6 54.Qg5+ Qxg5 55.hxg5+ Kxg5 56.Re1 Rf8 57.Rc1 Rh8+ 58.Kg1 Re8 59.Rxc4 Rxe6 60.Rxa4 Kf5 61.Kf2 Rb6 62.b4 Ke5 63.Ke3 Kd5 64.Kd3 Ke6 65.Kd4 Kf5 66.Kd5 Kg5 67.Kc5 Rb5+ 68.Kc4 Rb6 69.Kd5 Rb5+ 70.Kd6 Kh4 71.Rxa6 Rxb4 72.Ra2?? Re4 73.Kd5 Re8 74.Rb2 g5 75.Kc4 Kg3 76.Rd2 Re1 77.Ra2 g4 78.Kd4 Rg1 79.Ke4 Rxg2 80.Ra8 Rb2 81.Rg8 Rb4+ 82.Ke3 Kg2 83.Rf8 Rb3+ 84.Ke2 g3 85.Rh8 Rb5 86.Rg8 Re5+ 87.Kd3 Kf3 88.Rf8+ Kg4 89.Rf1 g2 90.Ra1 Kf4 91.Rg1 Kf3 92.Kd4 Rh5 93.Kd3 Rh4 94.Kc3 Kf2 95.Rc1 g1=Q 96.Rxg1 Kxg1 97.Kd3 Kf2 98.Kd2 Rh3 99.Kc1
White resigns
0-1
[/pgn]
Arena GUI, 40 moves in 4 minutes repeating, cpu AMD Athlon II P320 2.1GHz, 3GB ram, 128MB hash, OS Windows 7 64bit

[Norm Pollock]

With 6 pieces on the board how can you be sure it was the Nalimov 5-piece tbs that saw the blunder instead of the engine itself?

[Guenther]

I was wondering how many engines there are in the top 100 of CCRL 40/4 rating list (All engines, best version only) that do not use any tablebases, Nalimov, syzygy, gaviota, etc.
Is there any value for the increase in elo obtained by using tablebases?
I have seen several engine matches over the years where the result was "not as it should have been", due to tablebases not being used.
I was watching the end of this game, played on my laptop this morning, between the latest ProDeo v2.2, which does not use tablebases, and the very old (from 2003) and much weaker engine, Amy 0.83, which does.
I was just about to push the button to call it a draw, when ProDeo blundered by playing 72.Ra2 in the position shown. Amy immediatly knew it had a won game with its 5 man Nalimov tbs.
[d]8/8/R2K2p1/8/1r5k/8/6P1/8 w - - 0 72
[pgn]
Event "Arena tournament"]
[Site "HP2010"]
[Date "2017.01.24"]
[Round "1"]
[White "ProDeo 2.2"]
[Black "Amy083"]
[Result "0-1"]
[ECO "C42r"]
[TimeControl "40/240:40/240:40/240"]
[PlyCount "197"]

1.e4 e5 2.Nf3 Nf6 3.Nxe5 d6 4.Nf3 Nxe4 5.d4 d5 6.Bd3 Bd6 7.O-O O-O 8.c4 c6 9.Qc2 Na6 10.a3 Bg4 11.Ne5 Bxe5 12.dxe5 Nac5 13.f3 Nxd3 14.Qxd3 Nc5 15.Qd4 Nb3 16.Qxg4 Nxa1 17.Bh6 g6 18.Bxf8 Kxf8 19.Qf4 dxc4 20.Nc3 Nc2 21.Rd1 Qb6+ 22.Kh1 Qe3 23.Qf6 Kg8 24.Qe7 b5 25.Ne4 Qe2 26.Rg1 Nd4 27.Ng5 Rf8 28.Nxh7 Ra8 29.Qh4 Kg7 30.Qf6+ Kg8 31.Ng5 Rf8 32.Nxf7 Rxf7 33.Qd8+ Rf8 34.Qxd4 a6 35.a4 Rf5 36.Qd6 Kg7 37.e6 Rxf3 38.h4 Rf2 39.Qxc6 Qg4 40.Qb7+ Kh6 41.Qe7 bxa4 42.Qd8 Kh7 43.Qe7+ Kh8 44.Qe8+ Kg7 45.Qe7+ Kh6 46.Qd8 Kg7 47.Qe7+ Kg8 48.Kh2 Qf4+ 49.Kh1 Qg3 50.Qd8+ Rf8 51.Qd4 Rf4 52.Qd8+ Kh7 53.Qe7+ Kh6 54.Qg5+ Qxg5 55.hxg5+ Kxg5 56.Re1 Rf8 57.Rc1 Rh8+ 58.Kg1 Re8 59.Rxc4 Rxe6 60.Rxa4 Kf5 61.Kf2 Rb6 62.b4 Ke5 63.Ke3 Kd5 64.Kd3 Ke6 65.Kd4 Kf5 66.Kd5 Kg5 67.Kc5 Rb5+ 68.Kc4 Rb6 69.Kd5 Rb5+ 70.Kd6 Kh4 71.Rxa6 Rxb4 72.Ra2?? Re4 73.Kd5 Re8 74.Rb2 g5 75.Kc4 Kg3 76.Rd2 Re1 77.Ra2 g4 78.Kd4 Rg1 79.Ke4 Rxg2 80.Ra8 Rb2 81.Rg8 Rb4+ 82.Ke3 Kg2 83.Rf8 Rb3+ 84.Ke2 g3 85.Rh8 Rb5 86.Rg8 Re5+ 87.Kd3 Kf3 88.Rf8+ Kg4 89.Rf1 g2 90.Ra1 Kf4 91.Rg1 Kf3 92.Kd4 Rh5 93.Kd3 Rh4 94.Kc3 Kf2 95.Rc1 g1=Q 96.Rxg1 Kxg1 97.Kd3 Kf2 98.Kd2 Rh3 99.Kc1
White resigns
0-1
[/pgn]
Arena GUI, 40 moves in 4 minutes repeating, cpu AMD Athlon II P320 2.1GHz, 3GB ram, 128MB hash, OS Windows 7 64bit

Sorry Colin, but the 5men cannot always help in 6men positions and your position is an example ;-)
After 73...Re8?? it is a draw again, but obviously ProDeo made another wrong move at least. (I haven't checked it further.)

In any case it is an interesting endgame especially for programs not using tablebases.

[pedrox]

DanaSah 6.4 with 6 men Scorpio bitbases: detects in 0 seconds a won with Ra2.

Code: Select all

FEN: 8/8/3K2p1/8/1r5k/8/R5P1/8 b - - 1 72

Danasah640:
 1	00:00	 2	2	-0,25	g6g5
 1	00:00	 22	22	-0,12	b4d4 d6e5
 2	00:00	 134	134	-0,26	g6g5 a2a7
 2	00:00	 243	243	-0,16	h4g4 a2a7
 2	00:00	 295	295	-0,15	h4g5 a2a7
 3	00:00	 413	413	-0,15	h4g5 a2a7 b4b2
 4	00:00	 1k	1k	-0,09	h4g5 a2a5 g5f4 a5a7 b4b2
 5	00:00	 3k	3k	-0,02	h4g5 g2g3 b4b7 d6e5 b7e7 e5d4
 6	00:00	 6k	6k	-0,14	h4g5 d6c5 b4b7 c5d5 b7d7 d5e6 d7h7
 6	00:00	 13k	1.336k	-0,07	b4b6 d6c5 b6f6 c5d5 f6f5 d5d4 h4g3 a2a6
 7	00:00	 17k	1.743k	-0,07	b4b6 d6c5 b6f6 c5d5 h4g3 d5d4 f6f5 a2a6
 7	00:00	 23k	2.267k	+48,29	g6g5
 

5 men Scorpio bitbases: here need 9 seconds and more depth.

Code: Select all

FEN: 8/8/3K2p1/8/1r5k/8/R5P1/8 b - - 1 72

Danasah640:
 1	00:00	 2	2	-0,25	g6g5
 1	00:00	 22	22	-0,12	b4d4 d6e5
 2	00:00	 134	134	-0,26	g6g5 a2a7
 2	00:00	 243	243	-0,16	h4g4 a2a7
 2	00:00	 295	295	-0,15	h4g5 a2a7
 3	00:00	 413	413	-0,15	h4g5 a2a7 b4b2
 4	00:00	 1k	1k	-0,09	h4g5 a2a5 g5f4 a5a7 b4b2
 5	00:00	 3k	3k	-0,02	h4g5 g2g3 b4b7 d6e5 b7e7 e5d4
 6	00:00	 6k	6k	-0,14	h4g5 d6c5 b4b7 c5d5 b7d7 d5e6 d7h7
 6	00:00	 13k	13k	-0,07	b4b6 d6c5 b6f6 c5d5 f6f5 d5d4 h4g3 a2a6
 7	00:00	 18k	18k	-0,07	b4b6 d6c5 b6f6 c5d5 h4g3 d5d4 f6f5 a2a6
 7	00:00	 36k	1.209k	-0,06	b4b8 d6c7 b8h8 c7c6 h4g3 c6d5 h8h5 d5d4
 8	00:00	 53k	1.335k	-0,02	b4b8 d6c5 h4g5 c5d5 b8c8 a2a7 c8c2 g2g3
 9	00:00	 92k	1.028k	-0,02	b4b8 d6c5 h4g4 c5d5 b8h8 a2a4 g4g5 a4a7 h8h2 g2g3
 10	00:00	 129k	1.078k	-0,02	b4b8 d6c5 h4g4 c5d5 b8h8 a2a6 g4g5 a6a7 h8h2 g2g3
 10	00:00	 190k	1.120k	 0,00	b4b5 d6c6 b5f5 a2e2 h4g3 c6d6 f5f2 e2f2 g3f2
 11	00:00	 205k	1.136k	 0,00	b4b5 d6c6 b5f5 a2e2 h4g3 c6d6 f5f2 e2f2 g3f2
 12	00:00	 261k	1.133k	 0,00	b4b5 d6c6 b5f5 a2e2 h4g3 c6d6 f5f2 e2f2 g3f2
 13	00:00	 368k	1.188k	 0,00	b4b5 d6c6 b5f5 a2e2 h4g3 c6d6 f5f2 e2f2
 14	00:00	 458k	1.174k	 0,00	b4b5 d6c6 b5f5 a2e2 h4g3 c6d6 f5f2 e2f2
 15	00:00	 553k	1.151k	 0,00	b4b5 d6c6 b5f5 a2e2 h4g3 c6d6 f5f2 e2f2
 16	00:00	 711k	1.185k	 0,00	b4b5 d6c6 b5f5 a2e2 h4g3 c6d6 f5f2 e2f2
 17	00:00	 931k	1.193k	 0,00	b4b5 d6c6 b5f5 a2e2 h4g3 c6d6 f5f2 e2f2
 18	00:01	 1.261k	1.225k	 0,00	b4b5 d6c6 b5f5 a2e2 h4g3 c6d6 f5f2 e2f2
 19	00:01	 1.782k	1.204k	 0,00	b4b5 d6c6 b5f5 a2e2 h4g3 c6d6 f5f2 e2f2
 20	00:04	 5.401k	1.219k	 0,00	b4b5 a2a4 h4g3 d6e6 b5f5 a4a2 f5f2 a2f2
 20	00:09	 11.863k	1.247k	+42,54	b4f4 d6e5 g6g5 a2b2 h4g4 e5e6 g4g3 e6e5 f4f1 e5d4 f1f2 b2f2

DanaSah with 5 men of Scorpio bitbases can find the position won because the engine does not use the bitbases in a static mode like the GUI, the engine uses the bitbases also during the search and if the position leads to a forced reduction it will enter in 5 men bitbases or even 4 men.

With bitbases or tablebases my engine returns a value around +50 for a win, if the engine is who discovers the win without help would have a value maybe of +10 or +15. These values may change depending on the engine and tablebases or bitbases used.

[pedrox]

The use of tablebases or bitbases makes the engine appear much more intelligent in the endings, but the gain in elo is small.

Tablebases can not be used in all conditions on the search, since access to them slows down the search and what is gained on the one hand is lost on the other. Only tablebases win 0 elo.

The bitbases can be used during the search, it could be used in conjunction with the tablebases, for example Gaviota does the 2 things, and perhaps with 5 pieces bitbases you get about 15 elo points.

I think there is a study already done in the forum.

[cdani]

In my tests with Andscacs I measured like 15 elo in several thousands of games with 5-men, but maybe in long time control is more, or less...

[Ozymandias]

maybe in long time control is more, or less...

Less, I've measured up to a 100 ELO increase when making the jump to 6-men, at hyperbullet, with H4Tactical. That's the record.

[yurikvelo]

maybe in long time control is more, or less...

Less, I've measured up to a 100 ELO increase when making the jump to 6-men, at hyperbullet, with H4Tactical. That's the record.

ELO-gain is dependent on internal amount and precise of engine EG-knowledge.

TOP-3 engines play TB-less very well, and adjudicate 6-men at root position quite well. H5 even prefer its internal 6-men ajdudication over Syzygy, to decrease Syzygy probing and speed penalty.

[Jouni]

In 3000 games bullet selftest I got (5 piece syzygy):

SF 8 +11 ELO
Houdini 5 +14 ELO
Komodo 10 +7 ELO

Equal to 2-3 great patches .

[mjlef]

In 3000 games bullet selftest I got (5 piece syzygy):

SF 8 +11 ELO
Houdini 5 +14 ELO
Komodo 10 +7 ELO

Equal to 2-3 great patches .

That is a very useful test. If you do the same for 6 piece, I would love to see the results.

Were you results with a fast SSD or regular HD? Note for 5 piece it does not matter so much since the whole WLD files set is only about 500 megs, which most machines cache in memory anyway. It should matter more for 6 piece.