EGTB Testsuite

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Harvey Williamson, bob

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
User avatar
Kirill Kryukov
Posts: 492
Joined: Sun Mar 19, 2006 3:12 am

Re: EGTB Testsuite

Post by Kirill Kryukov » Sun Feb 03, 2008 6:31 am

F. Bluemers wrote:file mon and process monitor.
they will show you if and which (tb) files are accessed,and if its the engine or the gui accessing them.

Best
Fonzy
OK I see your idea now. That's great, but not what I am looking for here. I am looking for positions that will reveal whether an engine is using EGTB or not. Disk activity or access to the files are indirect evidence of EGTB usage. I want direct evidence in form of engine output.

User avatar
Kirill Kryukov
Posts: 492
Joined: Sun Mar 19, 2006 3:12 am

Re: EGTB Testsuite

Post by Kirill Kryukov » Sun Feb 03, 2008 7:02 am

Werner wrote:Hii Kirill,
to see whether a new engine is installed correctly and uses tbs or egbbs I normally use this position:
[D]4k3/4p3/8/8/3p4/8/4N3/4KB2 w - -

engines with egbbs responds with high value:
14/24 0:01 +52.51 1.Sxd4 Kd7 2.Ke2 Kd6 3.Kd3 Kd5 4.Ke3 e6 (1.683.792) 1

engines with egtbs in search responds with a mate score:
6.11 0:00 +M29 1.Sxd4 e5 2.Se6 Ke7 3.Lc4 (460) 46
Bester Zug: Se2xd4 Zeit: 0:00.625 min K/s: 46.000 Knoten: 460

and engines with no access with about +6:
Engine: Ktulu 8 (64 MB)
von Rahman Paidar
12.00 0:01 +6.97 1.Sxd4 e5 2.Sf5 e4 3.Lc4 e3 4.Ke2 Kd7 5.Ld5 Kc7 6.Le6 Kb6 7.Lc4 Kc7 8.Ld5 Kd7 9.La2 Kc7 10.Lf7 Kc6 (2.657.411) 1736
Hi Werner! Thanks for reply. This is nice, I also use similar positions for testing EGTB access:
[D]k6N/1p5B/7p/7p/7p/7p/7p/7K w - - 0 1
[D]k7/1p6/7p/7p/7p/7p/1P5p/1RK5 w - - 0 1
and variants with fewer pawns. :-)

User avatar
Kirill Kryukov
Posts: 492
Joined: Sun Mar 19, 2006 3:12 am

Re: EGTB Testsuite

Post by Kirill Kryukov » Sun Feb 03, 2008 7:04 am

smirobth wrote:Here is a position that is solved instantly with the R+P vs R tablebase, but no programs solve it without them:

[D]k7/P6R/3K4/8/7P/2r5/8/8 w - - 0 1
Rh8+! is the only way to win.
Thanks Robin! This very close to what I am looking for! It would be perfect if there was only one winning move.

User avatar
smirobth
Posts: 2307
Joined: Wed Mar 08, 2006 7:41 pm
Location: Brownsville Texas USA

Re: EGTB Testsuite

Post by smirobth » Sun Feb 03, 2008 7:08 am

George Tsavdaris wrote:
smirobth wrote:Here is a position that is solved instantly with the R+P vs R tablebase, but no programs solve it without them:

k7/P6R/3K4/8/7P/2r5/8/8 w - - 0 1
Rh8+! is the only way to win.
6 piece tablebases say that Rh6 and Rh5 are also ways to win....
Here is a position that fixes the problem of multiple moves winning:
[D]k7/P6R/2K5/8/7P/1r6/8/8 w - - 0 1
Now only Rh8 wins, but programs without access to the R+P vs R tablebase don't play it.
- Robin Smith

User avatar
Kirill Kryukov
Posts: 492
Joined: Sun Mar 19, 2006 3:12 am

Re: EGTB Testsuite

Post by Kirill Kryukov » Sun Feb 03, 2008 7:30 am

smirobth wrote:
George Tsavdaris wrote:
smirobth wrote:Here is a position that is solved instantly with the R+P vs R tablebase, but no programs solve it without them:

k7/P6R/3K4/8/7P/2r5/8/8 w - - 0 1
Rh8+! is the only way to win.
6 piece tablebases say that Rh6 and Rh5 are also ways to win....
Here is a position that fixes the problem of multiple moves winning:
[D]k7/P6R/2K5/8/7P/1r6/8/8 w - - 0 1
Now only Rh8 wins, but programs without access to the R+P vs R tablebase don't play it.
Thanks!! About what I am looking for! Unfortunately some engines can solve it without EGTB (Pepito 1.59: 1 sec, Hamsters 0.6: 5 sec), I feel your first position is more hard somehow, although this one has only one winning move and longer distance to mate.

Thanks and please post if you can think of any other positions! I am starting to collect them here.

User avatar
Mike S.
Posts: 1461
Joined: Thu Mar 09, 2006 4:33 am

Re: EGTB Testsuite

Post by Mike S. » Sun Feb 03, 2008 7:52 am

smirobth wrote:Here is a position that fixes the problem of multiple moves winning:
[D]k7/P6R/2K5/8/7P/1r6/8/8 w - - 0 1
Now only Rh8 wins, but programs without access to the R+P vs R tablebase don't play it.
Analysis by Toga II 3.1.2SE (no bitbases, no Nalimovs):

1.h5 Rb6+ 2.Kd5 Rd6+ 3.Ke4 Rd4+ 4.Kf5 Rd7 5.Kg6 Rxh7 6.Kxh7 Kxa7
+- (3.87) Depth: 3/8 00:00:00
+- (1.95) Depth: 5/16 00:00:00
(...)
1.Rh8+ Kxa7 2.h5 Rc3+ 3.Kb5 Rb3+ 4.Kc5 Rh3 5.h6 Kb7 6.Rh7+ Ka6 7.Kd4 Rh4+ 8.Ke5 Kb5 9.Rh8 Rh5+ 10.Ke6 Kc4
+- (2.11) Depth: 8/22 00:00:00 151kN
+- (2.02) Depth: 11/26 00:00:00 435kN
1.h5 Rb6+ 2.Kd5 Rd6+ 3.Ke5 Re6+ 4.Kf4 Rf6+ 5.Ke4 Re6+ 6.Kd3 Rd6+ 7.Kc2 Rc6+ 8.Kd2 Rd6+ 9.Ke3 Re6+ 10.Kf3 Rf6+ 11.Ke2 Re6+ 12.Kd1 Re1+ 13.Kd2 Re7 14.Rh8+ Kxa7 15.h6 Kb6
+- (2.08) Depth: 11/30 00:00:00 498kN
+- (2.00) Depth: 15/34 00:00:02 2027kN
1.Rh8+ Kxa7 2.h5 Rc3+ 3.Kd6 Kb7 4.h6 Rc1 5.Ke5 Rc6 6.Kf4 Rc4+ 7.Kf5 Rc5+ 8.Ke6 Rc6+ 9.Kd5 Rg6 10.Kd4 Rg4+ 11.Ke5 Rg6 12.Rh7+ Kb6 13.Kf4 Ka6 14.Kf5 Rc6 15.Kg4 Rc4+ 16.Kg3 Rc6
+- (2.02) Depth: 15/34 00:00:03 3379kN
+- (2.03) Depth: 23/46 00:01:14 86604kN

Even here, the first move in itself is not a 100% reliable proof that endgame tables are accessed. Toga was quickly switching between various first moves during the first 3 seconds (only) though, but anyway, it means it is possible that another engine could immediatly choose Rh8+ without any tables, too... Maybe the "diagnosis" needs to take the evaluation into account.
Regards, Mike

User avatar
smirobth
Posts: 2307
Joined: Wed Mar 08, 2006 7:41 pm
Location: Brownsville Texas USA

Re: EGTB Testsuite

Post by smirobth » Sun Feb 03, 2008 8:45 am

Kirill Kryukov wrote:
smirobth wrote:
George Tsavdaris wrote:
smirobth wrote:Here is a position that is solved instantly with the R+P vs R tablebase, but no programs solve it without them:

k7/P6R/3K4/8/7P/2r5/8/8 w - - 0 1
Rh8+! is the only way to win.
6 piece tablebases say that Rh6 and Rh5 are also ways to win....
Here is a position that fixes the problem of multiple moves winning:
[D]k7/P6R/2K5/8/7P/1r6/8/8 w - - 0 1
Now only Rh8 wins, but programs without access to the R+P vs R tablebase don't play it.
Thanks!! About what I am looking for! Unfortunately some engines can solve it without EGTB (Pepito 1.59: 1 sec, Hamsters 0.6: 5 sec), I feel your first position is more hard somehow, although this one has only one winning move and longer distance to mate.

Thanks and please post if you can think of any other positions! I am starting to collect them here.
Probably the first position was better since othes have pointed out that some programs find the correct move to solve the second one. Both positions involve Black having a desperado rook. The first position I posted had multiple winning moves, but it is harder for programs to see that they have to immediately give up the a-pawn and I don't know of any that will without TBs access. After 1.h5? immediately going for the desperado with 1...Rc6+? allows White to escape the desperado after 2.Kd7 Rd6+ 3.Ke7 Re6+ 4.Kf7 Rf6+ 5.Kg7 and now 5...Rg6+ can be answered by 6.hxg6. So Black's only way to draw after 1.h5? is 1...Rd3+ 2. Ke7 Re3+ 3. Kf7 Rf3+ 4. Kg7 Rg3+ 5. Kf8 Rg8+ 6. Ke7 Re8+ 7. Kd7 Rg8 =

However in the second position, after 1.h5? Black has the immediate desperado move 1...Rb6+ since 2.Kc7?? can be answered by 2...Rb7+ and Black wins.
- Robin Smith

User avatar
Kirill Kryukov
Posts: 492
Joined: Sun Mar 19, 2006 3:12 am

Re: EGTB Testsuite

Post by Kirill Kryukov » Sun Feb 03, 2008 10:19 am

smirobth wrote:Probably the first position was better since othes have pointed out that some programs find the correct move to solve the second one. Both positions involve Black having a desperado rook. The first position I posted had multiple winning moves, but it is harder for programs to see that they have to immediately give up the a-pawn and I don't know of any that will without TBs access. After 1.h5? immediately going for the desperado with 1...Rc6+? allows White to escape the desperado after 2.Kd7 Rd6+ 3.Ke7 Re6+ 4.Kf7 Rf6+ 5.Kg7 and now 5...Rg6+ can be answered by 6.hxg6. So Black's only way to draw after 1.h5? is 1...Rd3+ 2. Ke7 Re3+ 3. Kf7 Rf3+ 4. Kg7 Rg3+ 5. Kf8 Rg8+ 6. Ke7 Re8+ 7. Kd7 Rg8 =

However in the second position, after 1.h5? Black has the immediate desperado move 1...Rb6+ since 2.Kc7?? can be answered by 2...Rb7+ and Black wins.
I now found more engines that put 1.Rh8+ to the first line within a few seconds, so I have to consider this second position imperfect too, unfortunately. (Even if those engines don't really understand the idea completely and even change their preference to other moves later).

Details are here as before.

User avatar
smirobth
Posts: 2307
Joined: Wed Mar 08, 2006 7:41 pm
Location: Brownsville Texas USA

Re: EGTB Testsuite

Post by smirobth » Sun Feb 03, 2008 10:33 am

Kirill Kryukov wrote:
smirobth wrote:Probably the first position was better since othes have pointed out that some programs find the correct move to solve the second one. Both positions involve Black having a desperado rook. The first position I posted had multiple winning moves, but it is harder for programs to see that they have to immediately give up the a-pawn and I don't know of any that will without TBs access. After 1.h5? immediately going for the desperado with 1...Rc6+? allows White to escape the desperado after 2.Kd7 Rd6+ 3.Ke7 Re6+ 4.Kf7 Rf6+ 5.Kg7 and now 5...Rg6+ can be answered by 6.hxg6. So Black's only way to draw after 1.h5? is 1...Rd3+ 2. Ke7 Re3+ 3. Kf7 Rf3+ 4. Kg7 Rg3+ 5. Kf8 Rg8+ 6. Ke7 Re8+ 7. Kd7 Rg8 =

However in the second position, after 1.h5? Black has the immediate desperado move 1...Rb6+ since 2.Kc7?? can be answered by 2...Rb7+ and Black wins.
I now found more engines that put 1.Rh8+ to the first line within a few seconds, so I have to consider this second position imperfect too, unfortunately. (Even if those engines don't really understand the idea completely and even change their preference to other moves later).

Details are here as before.
Have you found any programs that play any of the three winning moves in the first position (that has three possible winning moves)? That one should be much harder.

Also wouldn't it be just as effective for you purposes (but much easier) to just find positions where programs with TBS access show a mate score, and those without don't?
- Robin Smith

User avatar
Kirill Kryukov
Posts: 492
Joined: Sun Mar 19, 2006 3:12 am

Re: EGTB Testsuite

Post by Kirill Kryukov » Sun Feb 03, 2008 12:18 pm

smirobth wrote:Have you found any programs that play any of the three winning moves in the first position (that has three possible winning moves)? That one should be much harder.
No, though I did not check many, and I did not let them think for more than a few seconds. :-)
smirobth wrote:Also wouldn't it be just as effective for you purposes (but much easier) to just find positions where programs with TBS access show a mate score, and those without don't?
No, because it won't help with bitbases. Also because it's easy to test a program on a test suite with existing software, but I don't know any GUI that can test a program considering "showing mate score" vs "not showing mate score" differences.

Post Reply