EGTB Testsuite

[smirobth]

Here is a position that is solved instantly with the R+P vs R tablebase, but no programs solve it without them:

k7/P6R/3K4/8/7P/2r5/8/8 w - - 0 1
Rh8+! is the only way to win.

6 piece tablebases say that Rh6 and Rh5 are also ways to win....

Here is a position that fixes the problem of multiple moves winning:
[D]k7/P6R/2K5/8/7P/1r6/8/8 w - - 0 1
Now only Rh8 wins, but programs without access to the R+P vs R tablebase don't play it.

[Kirill Kryukov]

Here is a position that is solved instantly with the R+P vs R tablebase, but no programs solve it without them:

k7/P6R/3K4/8/7P/2r5/8/8 w - - 0 1
Rh8+! is the only way to win.

6 piece tablebases say that Rh6 and Rh5 are also ways to win....

Here is a position that fixes the problem of multiple moves winning:
[D]k7/P6R/2K5/8/7P/1r6/8/8 w - - 0 1
Now only Rh8 wins, but programs without access to the R+P vs R tablebase don't play it.

Thanks!! About what I am looking for! Unfortunately some engines can solve it without EGTB (Pepito 1.59: 1 sec, Hamsters 0.6: 5 sec), I feel your first position is more hard somehow, although this one has only one winning move and longer distance to mate.

Thanks and please post if you can think of any other positions! I am starting to collect them here.

[Mike S.]

Here is a position that fixes the problem of multiple moves winning:
[D]k7/P6R/2K5/8/7P/1r6/8/8 w - - 0 1
Now only Rh8 wins, but programs without access to the R+P vs R tablebase don't play it.

Analysis by Toga II 3.1.2SE (no bitbases, no Nalimovs):

1.h5 Rb6+ 2.Kd5 Rd6+ 3.Ke4 Rd4+ 4.Kf5 Rd7 5.Kg6 Rxh7 6.Kxh7 Kxa7
+- (3.87) Depth: 3/8 00:00:00
+- (1.95) Depth: 5/16 00:00:00
(...)
1.Rh8+ Kxa7 2.h5 Rc3+ 3.Kb5 Rb3+ 4.Kc5 Rh3 5.h6 Kb7 6.Rh7+ Ka6 7.Kd4 Rh4+ 8.Ke5 Kb5 9.Rh8 Rh5+ 10.Ke6 Kc4
+- (2.11) Depth: 8/22 00:00:00 151kN
+- (2.02) Depth: 11/26 00:00:00 435kN
1.h5 Rb6+ 2.Kd5 Rd6+ 3.Ke5 Re6+ 4.Kf4 Rf6+ 5.Ke4 Re6+ 6.Kd3 Rd6+ 7.Kc2 Rc6+ 8.Kd2 Rd6+ 9.Ke3 Re6+ 10.Kf3 Rf6+ 11.Ke2 Re6+ 12.Kd1 Re1+ 13.Kd2 Re7 14.Rh8+ Kxa7 15.h6 Kb6
+- (2.08) Depth: 11/30 00:00:00 498kN
+- (2.00) Depth: 15/34 00:00:02 2027kN
1.Rh8+ Kxa7 2.h5 Rc3+ 3.Kd6 Kb7 4.h6 Rc1 5.Ke5 Rc6 6.Kf4 Rc4+ 7.Kf5 Rc5+ 8.Ke6 Rc6+ 9.Kd5 Rg6 10.Kd4 Rg4+ 11.Ke5 Rg6 12.Rh7+ Kb6 13.Kf4 Ka6 14.Kf5 Rc6 15.Kg4 Rc4+ 16.Kg3 Rc6
+- (2.02) Depth: 15/34 00:00:03 3379kN
+- (2.03) Depth: 23/46 00:01:14 86604kN

Even here, the first move in itself is not a 100% reliable proof that endgame tables are accessed. Toga was quickly switching between various first moves during the first 3 seconds (only) though, but anyway, it means it is possible that another engine could immediatly choose Rh8+ without any tables, too... Maybe the "diagnosis" needs to take the evaluation into account.

[smirobth]

Here is a position that is solved instantly with the R+P vs R tablebase, but no programs solve it without them:

k7/P6R/3K4/8/7P/2r5/8/8 w - - 0 1
Rh8+! is the only way to win.

6 piece tablebases say that Rh6 and Rh5 are also ways to win....

Here is a position that fixes the problem of multiple moves winning:
[D]k7/P6R/2K5/8/7P/1r6/8/8 w - - 0 1
Now only Rh8 wins, but programs without access to the R+P vs R tablebase don't play it.

Thanks!! About what I am looking for! Unfortunately some engines can solve it without EGTB (Pepito 1.59: 1 sec, Hamsters 0.6: 5 sec), I feel your first position is more hard somehow, although this one has only one winning move and longer distance to mate.

Thanks and please post if you can think of any other positions! I am starting to collect them here.

Probably the first position was better since othes have pointed out that some programs find the correct move to solve the second one. Both positions involve Black having a desperado rook. The first position I posted had multiple winning moves, but it is harder for programs to see that they have to immediately give up the a-pawn and I don't know of any that will without TBs access. After 1.h5? immediately going for the desperado with 1...Rc6+? allows White to escape the desperado after 2.Kd7 Rd6+ 3.Ke7 Re6+ 4.Kf7 Rf6+ 5.Kg7 and now 5...Rg6+ can be answered by 6.hxg6. So Black's only way to draw after 1.h5? is 1...Rd3+ 2. Ke7 Re3+ 3. Kf7 Rf3+ 4. Kg7 Rg3+ 5. Kf8 Rg8+ 6. Ke7 Re8+ 7. Kd7 Rg8 =

However in the second position, after 1.h5? Black has the immediate desperado move 1...Rb6+ since 2.Kc7?? can be answered by 2...Rb7+ and Black wins.

[Kirill Kryukov]

Probably the first position was better since othes have pointed out that some programs find the correct move to solve the second one. Both positions involve Black having a desperado rook. The first position I posted had multiple winning moves, but it is harder for programs to see that they have to immediately give up the a-pawn and I don't know of any that will without TBs access. After 1.h5? immediately going for the desperado with 1...Rc6+? allows White to escape the desperado after 2.Kd7 Rd6+ 3.Ke7 Re6+ 4.Kf7 Rf6+ 5.Kg7 and now 5...Rg6+ can be answered by 6.hxg6. So Black's only way to draw after 1.h5? is 1...Rd3+ 2. Ke7 Re3+ 3. Kf7 Rf3+ 4. Kg7 Rg3+ 5. Kf8 Rg8+ 6. Ke7 Re8+ 7. Kd7 Rg8 =

However in the second position, after 1.h5? Black has the immediate desperado move 1...Rb6+ since 2.Kc7?? can be answered by 2...Rb7+ and Black wins.

I now found more engines that put 1.Rh8+ to the first line within a few seconds, so I have to consider this second position imperfect too, unfortunately. (Even if those engines don't really understand the idea completely and even change their preference to other moves later).

Details are here as before.

[smirobth]

Probably the first position was better since othes have pointed out that some programs find the correct move to solve the second one. Both positions involve Black having a desperado rook. The first position I posted had multiple winning moves, but it is harder for programs to see that they have to immediately give up the a-pawn and I don't know of any that will without TBs access. After 1.h5? immediately going for the desperado with 1...Rc6+? allows White to escape the desperado after 2.Kd7 Rd6+ 3.Ke7 Re6+ 4.Kf7 Rf6+ 5.Kg7 and now 5...Rg6+ can be answered by 6.hxg6. So Black's only way to draw after 1.h5? is 1...Rd3+ 2. Ke7 Re3+ 3. Kf7 Rf3+ 4. Kg7 Rg3+ 5. Kf8 Rg8+ 6. Ke7 Re8+ 7. Kd7 Rg8 =

However in the second position, after 1.h5? Black has the immediate desperado move 1...Rb6+ since 2.Kc7?? can be answered by 2...Rb7+ and Black wins.

I now found more engines that put 1.Rh8+ to the first line within a few seconds, so I have to consider this second position imperfect too, unfortunately. (Even if those engines don't really understand the idea completely and even change their preference to other moves later).

Details are here as before.

Have you found any programs that play any of the three winning moves in the first position (that has three possible winning moves)? That one should be much harder.

Also wouldn't it be just as effective for you purposes (but much easier) to just find positions where programs with TBS access show a mate score, and those without don't?

[Kirill Kryukov]

Have you found any programs that play any of the three winning moves in the first position (that has three possible winning moves)? That one should be much harder.

No, though I did not check many, and I did not let them think for more than a few seconds.

Also wouldn't it be just as effective for you purposes (but much easier) to just find positions where programs with TBS access show a mate score, and those without don't?

No, because it won't help with bitbases. Also because it's easy to test a program on a test suite with existing software, but I don't know any GUI that can test a program considering "showing mate score" vs "not showing mate score" differences.

[Dirt]

Have you found any programs that play any of the three winning moves in the first position (that has three possible winning moves)? That one should be much harder.

The new Toga found Rh8+ on my system in just over 30 minutes. A fast system should be able to cut that down to under 10 minutes, but that's still comfortably long enough for a good test; unless some other engine is quite a bit faster.

[Kirill Kryukov]

Come on! No any more positions? I know it's more hard than chatting about cloning and moral issues, still you can try!

Here I maintain the requirements and all positions collected so far.

All contribution will have full credit. When complete (or when it has anything useful at all ), this test suite will be used to test EGTB support in engines and interfaces. So come and leave your name in the history book of computer chess.

I think this task requires analytical and composition skills, and also mastery in using engines and tablebases. I'll understand that it is hard to do, but at least someone could try making a good endgame test?

The best position so far is submitted by Robin Smith:
[D]k7/P6R/3K4/8/7P/2r5/8/8 w - - bm Rh8+; dm 36;

Here is the best I myself could come up with:
[D]k6N/1p5B/7p/7p/7p/7p/7p/7K w - - bm Kxh2; dm 39;

Best regards
Kirill

[Graham Banks]

I prefer Robin's because it's a realistic position.