EGTB Testsuite

[Kirill Kryukov]

file mon and process monitor.
they will show you if and which (tb) files are accessed,and if its the engine or the gui accessing them.

Best
Fonzy

OK I see your idea now. That's great, but not what I am looking for here. I am looking for positions that will reveal whether an engine is using EGTB or not. Disk activity or access to the files are indirect evidence of EGTB usage. I want direct evidence in form of engine output.

[Kirill Kryukov]

Hii Kirill,
to see whether a new engine is installed correctly and uses tbs or egbbs I normally use this position:
[D]4k3/4p3/8/8/3p4/8/4N3/4KB2 w - -

engines with egbbs responds with high value:
14/24 0:01 +52.51 1.Sxd4 Kd7 2.Ke2 Kd6 3.Kd3 Kd5 4.Ke3 e6 (1.683.792) 1

engines with egtbs in search responds with a mate score:
6.11 0:00 +M29 1.Sxd4 e5 2.Se6 Ke7 3.Lc4 (460) 46
Bester Zug: Se2xd4 Zeit: 0:00.625 min K/s: 46.000 Knoten: 460

and engines with no access with about +6:
Engine: Ktulu 8 (64 MB)
von Rahman Paidar
12.00 0:01 +6.97 1.Sxd4 e5 2.Sf5 e4 3.Lc4 e3 4.Ke2 Kd7 5.Ld5 Kc7 6.Le6 Kb6 7.Lc4 Kc7 8.Ld5 Kd7 9.La2 Kc7 10.Lf7 Kc6 (2.657.411) 1736

Hi Werner! Thanks for reply. This is nice, I also use similar positions for testing EGTB access:
[D]k6N/1p5B/7p/7p/7p/7p/7p/7K w - - 0 1
[D]k7/1p6/7p/7p/7p/7p/1P5p/1RK5 w - - 0 1
and variants with fewer pawns.

[Kirill Kryukov]

Here is a position that is solved instantly with the R+P vs R tablebase, but no programs solve it without them:

[D]k7/P6R/3K4/8/7P/2r5/8/8 w - - 0 1
Rh8+! is the only way to win.

Thanks Robin! This very close to what I am looking for! It would be perfect if there was only one winning move.

[smirobth]

Here is a position that is solved instantly with the R+P vs R tablebase, but no programs solve it without them:

k7/P6R/3K4/8/7P/2r5/8/8 w - - 0 1
Rh8+! is the only way to win.

6 piece tablebases say that Rh6 and Rh5 are also ways to win....

Here is a position that fixes the problem of multiple moves winning:
[D]k7/P6R/2K5/8/7P/1r6/8/8 w - - 0 1
Now only Rh8 wins, but programs without access to the R+P vs R tablebase don't play it.

[Kirill Kryukov]

Here is a position that is solved instantly with the R+P vs R tablebase, but no programs solve it without them:

k7/P6R/3K4/8/7P/2r5/8/8 w - - 0 1
Rh8+! is the only way to win.

6 piece tablebases say that Rh6 and Rh5 are also ways to win....

Here is a position that fixes the problem of multiple moves winning:
[D]k7/P6R/2K5/8/7P/1r6/8/8 w - - 0 1
Now only Rh8 wins, but programs without access to the R+P vs R tablebase don't play it.

Thanks!! About what I am looking for! Unfortunately some engines can solve it without EGTB (Pepito 1.59: 1 sec, Hamsters 0.6: 5 sec), I feel your first position is more hard somehow, although this one has only one winning move and longer distance to mate.

Thanks and please post if you can think of any other positions! I am starting to collect them here.

[Mike S.]

Here is a position that fixes the problem of multiple moves winning:
[D]k7/P6R/2K5/8/7P/1r6/8/8 w - - 0 1
Now only Rh8 wins, but programs without access to the R+P vs R tablebase don't play it.

Analysis by Toga II 3.1.2SE (no bitbases, no Nalimovs):

1.h5 Rb6+ 2.Kd5 Rd6+ 3.Ke4 Rd4+ 4.Kf5 Rd7 5.Kg6 Rxh7 6.Kxh7 Kxa7
+- (3.87) Depth: 3/8 00:00:00
+- (1.95) Depth: 5/16 00:00:00
(...)
1.Rh8+ Kxa7 2.h5 Rc3+ 3.Kb5 Rb3+ 4.Kc5 Rh3 5.h6 Kb7 6.Rh7+ Ka6 7.Kd4 Rh4+ 8.Ke5 Kb5 9.Rh8 Rh5+ 10.Ke6 Kc4
+- (2.11) Depth: 8/22 00:00:00 151kN
+- (2.02) Depth: 11/26 00:00:00 435kN
1.h5 Rb6+ 2.Kd5 Rd6+ 3.Ke5 Re6+ 4.Kf4 Rf6+ 5.Ke4 Re6+ 6.Kd3 Rd6+ 7.Kc2 Rc6+ 8.Kd2 Rd6+ 9.Ke3 Re6+ 10.Kf3 Rf6+ 11.Ke2 Re6+ 12.Kd1 Re1+ 13.Kd2 Re7 14.Rh8+ Kxa7 15.h6 Kb6
+- (2.08) Depth: 11/30 00:00:00 498kN
+- (2.00) Depth: 15/34 00:00:02 2027kN
1.Rh8+ Kxa7 2.h5 Rc3+ 3.Kd6 Kb7 4.h6 Rc1 5.Ke5 Rc6 6.Kf4 Rc4+ 7.Kf5 Rc5+ 8.Ke6 Rc6+ 9.Kd5 Rg6 10.Kd4 Rg4+ 11.Ke5 Rg6 12.Rh7+ Kb6 13.Kf4 Ka6 14.Kf5 Rc6 15.Kg4 Rc4+ 16.Kg3 Rc6
+- (2.02) Depth: 15/34 00:00:03 3379kN
+- (2.03) Depth: 23/46 00:01:14 86604kN

Even here, the first move in itself is not a 100% reliable proof that endgame tables are accessed. Toga was quickly switching between various first moves during the first 3 seconds (only) though, but anyway, it means it is possible that another engine could immediatly choose Rh8+ without any tables, too... Maybe the "diagnosis" needs to take the evaluation into account.

[smirobth]

Here is a position that is solved instantly with the R+P vs R tablebase, but no programs solve it without them:

k7/P6R/3K4/8/7P/2r5/8/8 w - - 0 1
Rh8+! is the only way to win.

6 piece tablebases say that Rh6 and Rh5 are also ways to win....

Here is a position that fixes the problem of multiple moves winning:
[D]k7/P6R/2K5/8/7P/1r6/8/8 w - - 0 1
Now only Rh8 wins, but programs without access to the R+P vs R tablebase don't play it.

Thanks!! About what I am looking for! Unfortunately some engines can solve it without EGTB (Pepito 1.59: 1 sec, Hamsters 0.6: 5 sec), I feel your first position is more hard somehow, although this one has only one winning move and longer distance to mate.

Thanks and please post if you can think of any other positions! I am starting to collect them here.

Probably the first position was better since othes have pointed out that some programs find the correct move to solve the second one. Both positions involve Black having a desperado rook. The first position I posted had multiple winning moves, but it is harder for programs to see that they have to immediately give up the a-pawn and I don't know of any that will without TBs access. After 1.h5? immediately going for the desperado with 1...Rc6+? allows White to escape the desperado after 2.Kd7 Rd6+ 3.Ke7 Re6+ 4.Kf7 Rf6+ 5.Kg7 and now 5...Rg6+ can be answered by 6.hxg6. So Black's only way to draw after 1.h5? is 1...Rd3+ 2. Ke7 Re3+ 3. Kf7 Rf3+ 4. Kg7 Rg3+ 5. Kf8 Rg8+ 6. Ke7 Re8+ 7. Kd7 Rg8 =

However in the second position, after 1.h5? Black has the immediate desperado move 1...Rb6+ since 2.Kc7?? can be answered by 2...Rb7+ and Black wins.

[Kirill Kryukov]

Probably the first position was better since othes have pointed out that some programs find the correct move to solve the second one. Both positions involve Black having a desperado rook. The first position I posted had multiple winning moves, but it is harder for programs to see that they have to immediately give up the a-pawn and I don't know of any that will without TBs access. After 1.h5? immediately going for the desperado with 1...Rc6+? allows White to escape the desperado after 2.Kd7 Rd6+ 3.Ke7 Re6+ 4.Kf7 Rf6+ 5.Kg7 and now 5...Rg6+ can be answered by 6.hxg6. So Black's only way to draw after 1.h5? is 1...Rd3+ 2. Ke7 Re3+ 3. Kf7 Rf3+ 4. Kg7 Rg3+ 5. Kf8 Rg8+ 6. Ke7 Re8+ 7. Kd7 Rg8 =

However in the second position, after 1.h5? Black has the immediate desperado move 1...Rb6+ since 2.Kc7?? can be answered by 2...Rb7+ and Black wins.

I now found more engines that put 1.Rh8+ to the first line within a few seconds, so I have to consider this second position imperfect too, unfortunately. (Even if those engines don't really understand the idea completely and even change their preference to other moves later).

Details are here as before.

[smirobth]

Probably the first position was better since othes have pointed out that some programs find the correct move to solve the second one. Both positions involve Black having a desperado rook. The first position I posted had multiple winning moves, but it is harder for programs to see that they have to immediately give up the a-pawn and I don't know of any that will without TBs access. After 1.h5? immediately going for the desperado with 1...Rc6+? allows White to escape the desperado after 2.Kd7 Rd6+ 3.Ke7 Re6+ 4.Kf7 Rf6+ 5.Kg7 and now 5...Rg6+ can be answered by 6.hxg6. So Black's only way to draw after 1.h5? is 1...Rd3+ 2. Ke7 Re3+ 3. Kf7 Rf3+ 4. Kg7 Rg3+ 5. Kf8 Rg8+ 6. Ke7 Re8+ 7. Kd7 Rg8 =

However in the second position, after 1.h5? Black has the immediate desperado move 1...Rb6+ since 2.Kc7?? can be answered by 2...Rb7+ and Black wins.

I now found more engines that put 1.Rh8+ to the first line within a few seconds, so I have to consider this second position imperfect too, unfortunately. (Even if those engines don't really understand the idea completely and even change their preference to other moves later).

Details are here as before.

Have you found any programs that play any of the three winning moves in the first position (that has three possible winning moves)? That one should be much harder.

Also wouldn't it be just as effective for you purposes (but much easier) to just find positions where programs with TBS access show a mate score, and those without don't?

[Kirill Kryukov]

Have you found any programs that play any of the three winning moves in the first position (that has three possible winning moves)? That one should be much harder.

No, though I did not check many, and I did not let them think for more than a few seconds.

Also wouldn't it be just as effective for you purposes (but much easier) to just find positions where programs with TBS access show a mate score, and those without don't?

No, because it won't help with bitbases. Also because it's easy to test a program on a test suite with existing software, but I don't know any GUI that can test a program considering "showing mate score" vs "not showing mate score" differences.