Zugzwang Test Suite ZPTS by Robert Allgeuer

[phenri]

Here are Zugzwang Test Suite ZPTS by Robert Allgeuer from Computer Chess Club Archives http://www.stmintz.com/ccc/index.php?id=392369

3k4/8/4K3/2R5/8/8/8/8 w - - bm Rc1 Rc2 Rc3 Rc4 Rc6; c0 "CCC post"; id "ZPTS.01";
1k6/7R/2K5/8/8/8/8/8 w - - bm Re7 Rf7 Rg7 Rh1 Rh2 Rh3 Rh4 Rh5 Rh6 Rh8+; c0 "CCC post"; id "ZPTS.02";
8/3k4/8/8/3PK3/8/8/8 w - - bm Kd5; c0 "CCC post"; id "ZPTS.03";
2k5/8/1K1P4/8/8/8/8/8 w - - bm Kc6; c0 "white wins"; id "ZPTS.04";
8/8/8/4N3/8/7p/8/5K1k w - - bm Ng4; c0 "white mates"; id "ZPTS.05";
8/8/1p1K4/Pp6/2k1p3/8/1P6/8 w - - bm a6; c0 "Kubbel 1927, 1 ... e3 2 a7 e2 3 a8/Q e1/Q 4 Qd5+ Kb4 5 Qd3!, white wins"; id "ZPTS.06";
8/1p5k/1P1p4/3p4/3Pp2p/2K1P2p/7P/8 w - - bm Kb2; c0 "1 Kb2! Kg8 2 Ka1!!, draw"; id "ZPTS.07";
8/4N3/8/8/8/2b1p3/p1K1P3/k7 w - - bm Nc6; c0 "white mates in 3"; id "ZPTS.08";
8/8/7p/2R5/4pp1K/8/8/3k2b1 w - - bm Rc4; c0 "Kricheli 1986, 1 Rc4! e3 2 Rd4+! Kc2 3 Rxf4 e2 4 Re4 Kd2 5 Rxe2+ Kxe2 6 Kg4!, draw"; id "ZPTS.09";
4KBkr/7p/6PP/4P3/8/3P1p2/8/8 w - - bm g7; c0 "CCC post 27.4.2004 by Gerd Isenberg, white wins"; id "ZPTS.10";
8/p7/1p6/p7/kq1Q4/8/K7/8 w - - bm Qd3; c0 "post in Avler chess forum, mate in 13: 1.Qd3 a6 2.Qd7+ Qb5 3.Qd4+ Qb4 4.Qd3 Qb2+ 5.Kxb2 Kb4 6.Qc3+ Kb5 7.Qc7 Kb4 8.Qc6 a4 9.Kc2 a3 10.Qc3+ Ka4 11.Qc4+ Ka5 12.Kb3 b5 13.Qc7#"; id "ZPTS.11";
8/8/p3R3/1p5p/1P5p/6rp/5K1p/7k w - - bm Re1+; c0 "CCC post by Eduard Nemeth, mate in 7"; id "ZPTS.12";
8/5b2/p2k4/1p1p1p1p/1P1K1P1P/2P1PB2/8/8 w - - bm Bd1 Be2; c0 "CCC post by Sune Fischer, Averbakh 1954, white wins: 1.Be2 Be8 [1.-Bg6 2.Bd3 Bh7 3.Bf1 leads to instant zugzwang, be it after 3.-Bg6 4.Bg2 Bf7 5.Bf3 or after 3.-Bg8 4.Be2 Bf7 5.Bf3] 2.Bd3 Bg6 3.Bc2 Bh7 4.Bb3! Bg8 5.Bd1 Bf7 6.Bf3! and so on"; id "ZPTS.13";
6k1/3p4/P2P4/8/5Kp1/1p4Q1/p5p1/b7 w - - bm Qxg2; c0 "Kasparyan 1959, 1 ... Be5+ 2 Kf5! a1/Q 3 a7! Qxa7 4 Kg6! Qa1! 5 Qd5! Kh8 6 Qe4!!, white wins"; id "ZPTS.14";
1r4RK/2n5/7k/8/8/8/8/8 b - - bm Ne8; c0 "Polgar - Kasparov, 1996: mate in 8: 1. ... Ne8 2.Rf8 Kg6 3.Rg8+ Kf7 4.Kh7 Rb5 5.Rf8+ Kxf8 6.Kg6 Rc5 7.Kh6 Kf7 8.Kh7 Rh5#"; id "ZPTS.15";
k2N2K1/8/8/8/5R2/3n4/3p4/8 w - - bm Rf7; c0 "CCC post by Ed Schroeder, Troitzky"; id "ZPTS.16";
8/8/1p1r1k2/p1pPN1p1/P3KnP1/1P6/8/3R4 b - - bm Nxd5; c0 "CCC post 27.4.2004 by Gerd Isenberg"; id "ZPTS.17";
8/6B1/p5p1/Pp4kp/1P5r/5P1Q/4q1PK/8 w - - bm Qxh4+; c0 "CCC post 10.9.2004 by Alvaro Begue"; id "ZPTS.18";
n1QBq1k1/5p1p/5KP1/p7/8/8/8/8 w - - bm Bc7; c0 "CCC post, mate in 12: 1.Bc7 Qxc8 2.gxf7+ Kh8 3.Be5 Qc5 4.Bb2 Nc7 5.Ba1 a4 6.Bb2 a3 7.Ba1 a2 8.Bb2 a1Q 9.Bxa1 Nd5+ 10.Ke6+ Nc3 11.Bxc3+ Qe5+ 12.Bxe5#"; id "ZPTS.19";
3nQ1k1/p2P2p1/1p6/8/5q1P/8/PP6/1K6 b - - bm Kh7; c0 "CCC post, draw, but h5 after Qf8 wins for white"; id "ZPTS.20";
8/8/8/1B6/6p1/8/4KPpp/3N2kr w - - bm Kd3 Ke3; c0 "CCC post by Tim Foden, id MES.831, white wins, 1. Kd3 g3 2. f4 Kf1 3. Kd2+ Kg1 4. Bd7 Kf1 5. Bh3 Rg1 6. Bg4 Rh1 7. Be2+ Kg1 8. Nc3 Kf2 9. Ne4+ Kg1 10. Ng5 Kf2 11. Nh3#"; id "ZPTS.21";
4B3/8/p7/k2N4/7p/K6p/PP5P/2q5 w - - bm Ba4; c0 "CCC post by Tim Foden, id CCC.347609, white wins"; id "ZPTS.22";
6Q1/8/8/7k/8/8/3p1pp1/3Kbrrb w - - bm Qg7; c0 "CCC post by Joachim Rang, mate in 4"; id "ZPTS.23";
8/8/8/2p5/1pp5/brpp4/1pprp2P/qnkbK3 w - - bm h3; c0 "CCC post by Joachim Rang, mate in 15, h4 is only draw"; id "ZPTS.24";
8/1B6/8/5p2/8/8/5Qrq/1K1R2bk w - - bm Qa7; c0 "CCC post by Tim Foden, id CCC.321759, mate in 3"; id "ZPTS.25";
8/3p1p2/5Ppp/K2R2bk/4pPrr/6Pp/4B2P/3N4 w - - bm Nc3; c0 "CCC post by Tim Foden, id CCC.321751, mate in 4: 1.Nc3 e3 2.Rb5 d6 3.Nd5 Bxf4 4.Nxf4#"; id "ZPTS.26";
8/8/p5p1/p2N3p/k2P3P/5P2/KP1qB3/8 w - - bm f4; c0 "white wins, derived from the more difficult ZPTS.28, Zugzwang after f4"; id "ZPTS.27";
8/p5pq/8/p2N3p/k2P3P/8/KP3PB1/8 w - - bm Be4; c0 "Kubbel 1925, 1.Be4!! Qh6 2.Bd3!! Qd2 [2...Qd6 3.b3#] 3.Be2 g6 [3...Qc2 4.Bd1 Qxd1 5.Nc3 -; 3...a6 4.f3 g6 5.f4 -] 4.f3 a6 5.f4 Qc2 6.Bd1, Zugzwang after f4, white wins"; id "ZPTS.28";
8/5p2/4b1p1/7R/5K1P/2r3B1/7N/4b1k1 w - - bm Nf3+; c0 "Noam Elkies 1984, 1 Nf3+! Rxf3+! 2 Kxf3 Bg4+! 3 Kf4!! Bxg3+ 4 Kxg4 gxh5+ 5 Kh3!! Kf1 6 Kxg3 Ke2 7 Kf4 f6 8 Kf5 Kf3 9 Kxf6 Kg4 10 Ke5 Kh4 11 Kf4, draw"; id "ZPTS.29";
5R2/2K5/1pP5/4k2p/3pp3/2p4N/B4N1b/n1R1B2b w - - bm Rxc3; c0 "CCC post by Tim Foden, id CCC.321966, mate in 4"; id "ZPTS.30";

I have converted to PGN, cbh, epd and attached for download.

Zugzwang Positions Test Suite - ZPTS.7z

[phenri]

And here are some engines tested
Conditions:
Maximum time = 10s.
i7-4cores-512MB

Sorted from worst to best

Stockfish 100214 is the worst

Stockfish 100214 64 SSE4.2-512MB

Code: Select all

1. ZPTS.01,  ZPTS   Solved in 0.03s/7; Solved: 1
2. ZPTS.02,  ZPTS   Solved in 0.02s/7; Solved: 2
3. ZPTS.03,  ZPTS   Solved in 0s/7; Solved: 3
4. ZPTS.04,  ZPTS   Solved in 0s/7; Solved: 4
5. ZPTS.05,  ZPTS   Solved in 0.02s/7; Solved: 5
6. ZPTS.06,  ZPTS   > 10s.
7. ZPTS.07,  ZPTS   Solved in 0.02s/16; Solved: 6
8. ZPTS.08,  ZPTS   Solved in 0s/7; Solved: 7
9. ZPTS.09,  ZPTS   Solved in 0.02s/9; Solved: 8
10. ZPTS.10,  ZPTS   > 10s.
11. ZPTS.11,  ZPTS   > 10s.
12. ZPTS.12,  ZPTS   Solved in 0.19s/24; Solved: 9
13. ZPTS.13,  ZPTS   Solved in 0.02s/7; Solved: 10
14. ZPTS.14,  ZPTS   > 10s.
15. ZPTS.15,  ZPTS   Solved in 4.18s/34; Solved: 11
16. ZPTS.16,  ZPTS   Solved in 0.03s/11; Solved: 12
17. ZPTS.17,  ZPTS   > 10s.
18. ZPTS.18,  ZPTS   > 10s.
19. ZPTS.19,  ZPTS   > 10s.
20. ZPTS.20,  ZPTS   Solved in 0.30s/19; Solved: 13
21. ZPST.21,  ZPTS   > 10s.
22. ZPST.22,  ZPTS   > 10s.
23. ZPTS.23,  ZPTS   > 10s.
24. ZPTS.24,  ZPTS   > 10s.
25. ZPST.25,  ZPTS   Solved in 0.05s/14; Solved: 14
26. ZPST.26,  ZPTS   > 10s.
27. ZPTS.27,  ZPTS   > 10s.
28. ZPTS.28,  ZPTS   > 10s.
29. ZPTS.29,  ZPTS   Solved in 4.27s/29; Solved: 15
30. ZPST.30,  ZPTS   Solved in 3.15s/19; Solved: 16
Result: 16 out of 30 = 53.3%. Average time = 0.76s / 14.00

Stockfish 2.3.1 JA 64bit SSE4.2-512MB

Code: Select all

1. ZPTS.01,  ZPTS   Solved in 0.02s/7; Solved: 1
2. ZPTS.02,  ZPTS   Solved in 0s/7; Solved: 2
3. ZPTS.03,  ZPTS   Solved in 0s/7; Solved: 3
4. ZPTS.04,  ZPTS   Solved in 0.02s/7; Solved: 4
5. ZPTS.05,  ZPTS   Solved in 0s/7; Solved: 5
6. ZPTS.06,  ZPTS   Solved in 0.11s/12; Solved: 6
7. ZPTS.07,  ZPTS   Solved in 0.02s/17; Solved: 7
8. ZPTS.08,  ZPTS   Solved in 0.02s/18; Solved: 8
9. ZPTS.09,  ZPTS   Solved in 0s/7; Solved: 9
10. ZPTS.10,  ZPTS   Solved in 0.19s/16; Solved: 10
11. ZPTS.11,  ZPTS   Solved in 2.68s/25; Solved: 11
12. ZPTS.12,  ZPTS   Solved in 0.03s/12; Solved: 12
13. ZPTS.13,  ZPTS   Solved in 0.02s/7; Solved: 13
14. ZPTS.14,  ZPTS   > 10s.
15. ZPTS.15,  ZPTS   Solved in 0.16s/19; Solved: 14
16. ZPTS.16,  ZPTS   > 10s.
17. ZPTS.17,  ZPTS   Solved in 0.13s/14; Solved: 15
18. ZPTS.18,  ZPTS   > 10s.
19. ZPTS.19,  ZPTS   > 10s.
20. ZPTS.20,  ZPTS   Solved in 2.50s/22; Solved: 16
21. ZPST.21,  ZPTS   > 10s.
22. ZPST.22,  ZPTS   Solved in 0.02s/7; Solved: 17
23. ZPTS.23,  ZPTS   Solved in 0.09s/24; Solved: 18
24. ZPTS.24,  ZPTS   > 10s.
25. ZPST.25,  ZPTS   > 10s.
26. ZPST.26,  ZPTS   > 10s.
27. ZPTS.27,  ZPTS   Solved in 2.20s/27; Solved: 19
28. ZPTS.28,  ZPTS   > 10s.
29. ZPTS.29,  ZPTS   Solved in 0s/7; Solved: 20
30. ZPST.30,  ZPTS   Solved in 2s/14; Solved: 21
Result: 21 out of 30 = 70.0%. Average time = 0.48s / 13.47

Stockfish 101113 64 SSE4.2-512MB

Code: Select all

1. ZPTS.01,  ZPTS   Solved in 0.02s/7; Solved: 1
2. ZPTS.02,  ZPTS   Solved in 0.02s/7; Solved: 2
3. ZPTS.03,  ZPTS   Solved in 0s/7; Solved: 3
4. ZPTS.04,  ZPTS   Solved in 0s/7; Solved: 4
5. ZPTS.05,  ZPTS   Solved in 0s/7; Solved: 5
6. ZPTS.06,  ZPTS   Solved in 0.70s/23; Solved: 6
7. ZPTS.07,  ZPTS   Solved in 0s/13; Solved: 7
8. ZPTS.08,  ZPTS   Solved in 0.02s/7; Solved: 8
9. ZPTS.09,  ZPTS   Solved in 0.17s/16; Solved: 9
10. ZPTS.10,  ZPTS   Solved in 2.92s/29; Solved: 10
11. ZPTS.11,  ZPTS   Solved in 10.72s/36; Solved: 11
12. ZPTS.12,  ZPTS   Solved in 0.03s/17; Solved: 12
13. ZPTS.13,  ZPTS   Solved in 0s/7; Solved: 13
14. ZPTS.14,  ZPTS   > 10s.
15. ZPTS.15,  ZPTS   Solved in 0.17s/22; Solved: 14
16. ZPTS.16,  ZPTS   > 10s.
17. ZPTS.17,  ZPTS   Solved in 1.48s/21; Solved: 15
18. ZPTS.18,  ZPTS   Solved in 3.87s/27; Solved: 16
19. ZPTS.19,  ZPTS   > 10s.
20. ZPTS.20,  ZPTS   Solved in 1.12s/23; Solved: 17
21. ZPST.21,  ZPTS   > 10s.
22. ZPST.22,  ZPTS   Solved in 0.02s/15; Solved: 18
23. ZPTS.23,  ZPTS   Solved in 0.08s/27; Solved: 19
24. ZPTS.24,  ZPTS   > 10s.
25. ZPST.25,  ZPTS   Solved in 4.28s/27; Solved: 20
26. ZPST.26,  ZPTS   Solved in 0.25s/19; Solved: 21
27. ZPTS.27,  ZPTS   Solved in 3.51s/37; Solved: 22
28. ZPTS.28,  ZPTS   > 10s.
29. ZPTS.29,  ZPTS   Solved in 0.02s/7; Solved: 23
30. ZPST.30,  ZPTS   Solved in 1.95s/16; Solved: 24
Result: 24 out of 30 = 80.0%. Average time = 1.30s / 17.66

Critter 1.6a 64-bit-512MB

Code: Select all

1. ZPTS.01,  ZPTS   Solved in 0.02s/7; Solved: 1
2. ZPTS.02,  ZPTS   Solved in 0s/7; Solved: 2
3. ZPTS.03,  ZPTS   Solved in 0s/7; Solved: 3
4. ZPTS.04,  ZPTS   Solved in 0.02s/7; Solved: 4
5. ZPTS.05,  ZPTS   Solved in 0.02s/7; Solved: 5
6. ZPTS.06,  ZPTS   Solved in 0.09s/13; Solved: 6
7. ZPTS.07,  ZPTS   Solved in 0.02s/18; Solved: 7
8. ZPTS.08,  ZPTS   Solved in 0s/8; Solved: 8
9. ZPTS.09,  ZPTS   Solved in 0.09s/12; Solved: 9
10. ZPTS.10,  ZPTS   Solved in 0.05s/11; Solved: 10
11. ZPTS.11,  ZPTS   Solved in 0.02s/9; Solved: 11
12. ZPTS.12,  ZPTS   Solved in 0.03s/13; Solved: 12
13. ZPTS.13,  ZPTS   Solved in 0s/8; Solved: 13
14. ZPTS.14,  ZPTS   Solved in 7.60s/17; Solved: 14
15. ZPTS.15,  ZPTS   Solved in 0.02s/8; Solved: 15
16. ZPTS.16,  ZPTS   > 10s.
17. ZPTS.17,  ZPTS   Solved in 0.20s/14; Solved: 16
18. ZPTS.18,  ZPTS   Solved in 0.03s/11; Solved: 17
19. ZPTS.19,  ZPTS   > 10s.
20. ZPTS.20,  ZPTS   > 10s.
21. ZPST.21,  ZPTS   Solved in 1.19s/24; Solved: 18
22. ZPST.22,  ZPTS   Solved in 0s/7; Solved: 19
23. ZPTS.23,  ZPTS   Solved in 0.03s/13; Solved: 20
24. ZPTS.24,  ZPTS   Solved in 0.05s/19; Solved: 21
25. ZPST.25,  ZPTS   Solved in 0.03s/9; Solved: 22
26. ZPST.26,  ZPTS   Solved in 0.11s/14; Solved: 23
27. ZPTS.27,  ZPTS   Solved in 0.02s/13; Solved: 24
28. ZPTS.28,  ZPTS   Solved in 3.21s/25; Solved: 25
29. ZPTS.29,  ZPTS   Solved in 0s/7; Solved: 26
30. ZPST.30,  ZPTS   Solved in 1.48s/14; Solved: 27
Result: 27 out of 30 = 90.0%. Average time = 0.53s / 11.92

Sting SF 4-512MB

Code: Select all

1. ZPTS.01,  ZPTS   Solved in 0.25s/7; Solved: 1
2. ZPTS.02,  ZPTS   Solved in 0.13s/7; Solved: 2
3. ZPTS.03,  ZPTS   Solved in 0.11s/7; Solved: 3
4. ZPTS.04,  ZPTS   Solved in 0.09s/7; Solved: 4
5. ZPTS.05,  ZPTS   Solved in 0.08s/7; Solved: 5
6. ZPTS.06,  ZPTS   Solved in 0.30s/17; Solved: 6
7. ZPTS.07,  ZPTS   Solved in 0.09s/7; Solved: 7
8. ZPTS.08,  ZPTS   Solved in 0.11s/7; Solved: 8
9. ZPTS.09,  ZPTS   Solved in 0.11s/7; Solved: 9
10. ZPTS.10,  ZPTS   Solved in 0.22s/15; Solved: 10
11. ZPTS.11,  ZPTS   Solved in 0.11s/16; Solved: 11
12. ZPTS.12,  ZPTS   Solved in 0.16s/17; Solved: 12
13. ZPTS.13,  ZPTS   Solved in 0.09s/7; Solved: 13
14. ZPTS.14,  ZPTS   > 10s.
15. ZPTS.15,  ZPTS   Solved in 0.11s/10; Solved: 14
16. ZPTS.16,  ZPTS   > 10s.
17. ZPTS.17,  ZPTS   Solved in 0.36s/16; Solved: 15
18. ZPTS.18,  ZPTS   Solved in 0.81s/20; Solved: 16
19. ZPTS.19,  ZPTS   Solved in 6.37s/19; Solved: 17
20. ZPTS.20,  ZPTS   Solved in 0.27s/15; Solved: 18
21. ZPST.21,  ZPTS   Solved in 2.43s/34; Solved: 19
22. ZPST.22,  ZPTS   Solved in 0.09s/11; Solved: 20
23. ZPTS.23,  ZPTS   Solved in 0.09s/11; Solved: 21
24. ZPTS.24,  ZPTS   Solved in 0.11s/7; Solved: 22
25. ZPST.25,  ZPTS   Solved in 1.28s/19; Solved: 23
26. ZPST.26,  ZPTS   Solved in 0.12s/12; Solved: 24
27. ZPTS.27,  ZPTS   Solved in 0.11s/10; Solved: 25
28. ZPTS.28,  ZPTS   Solved in 2.42s/31; Solved: 26
29. ZPTS.29,  ZPTS   Solved in 2.48s/24; Solved: 27
30. ZPST.30,  ZPTS   Solved in 0.30s/14; Solved: 28
Result: 28 out of 30 = 93.3%. Average time = 0.68s / 13.60

[Michel]

Thanks!

[Vinvin]

Thanks, Paul !
That clearly shows that the latest Stockfish versions have troubles with zugzwang !

The hardest (even for Sting)
#14 SF Doesn't see after Qxa7, Kg6 is winning (except in multi-pv).
[d]6k1/P2p4/3P4/4bK2/6p1/1p6/6Q1/q7 b - - 0 3

[zullil]

Thanks, Paul !
That clearly shows that the latest Stockfish versions have troubles with zugzwang !

The hardest (even for Sting)
#14 SF Doesn't see after Qxa7, Kg6 is winning (except in multi-pv).
[d]6k1/P2p4/3P4/4bK2/6p1/1p6/6Q1/q7 b - - 0 3

[d]6k1/q2p4/3P4/4bK2/6p1/1p6/6Q1/8 w - - 0 4

The very latest SF sees mate-in-8 quickly. Marco has reverted the commit that caused SF to fail on many zugzwang positions.

Code: Select all

info depth 28 seldepth 27 score mate 8 nodes 4627829 nps 3204867 time 1444 multipv 1 pv f5g6 a7a1 g2d5 g8h8 d5e4 a1a5 e4h1 e5h2 h1h2 a5h5 h2h5 h8g8 h5h7 g8f8 h7f7

[Uri Blass]

Thanks, Paul !
That clearly shows that the latest Stockfish versions have troubles with zugzwang !

The hardest (even for Sting)
#14 SF Doesn't see after Qxa7, Kg6 is winning (except in multi-pv).
[d]6k1/P2p4/3P4/4bK2/6p1/1p6/6Q1/q7 b - - 0 3

[d]6k1/q2p4/3P4/4bK2/6p1/1p6/6Q1/8 w - - 0 4

The very latest SF sees mate-in-8 quickly. Marco has reverted the commit that caused SF to fail on many zugzwang positions.

Code: Select all

info depth 28 seldepth 27 score mate 8 nodes 4627829 nps 3204867 time 1444 multipv 1 pv f5g6 a7a1 g2d5 g8h8 d5e4 a1a5 e4h1 e5h2 h1h2 a5h5 h2h5 h8g8 h5h7 g8f8 h7f7

Marco's conclusions are not orrect.
He claims that the verification search does not change ELO level
but there is zero evidence for it.

passing twice SPRT(-4,0) only means no significant regression by removing verification search and there is no similiar proof that adding verification search does not reduce the elo.

In order to have a similiar proof we need to run SPRT(0,4) twice and get negative result twice both at 15+0.05 and 60+0.05 when one positive result stop the test and means removing verification search.

[Uri Blass]

And here are some engines tested
Conditions:
Maximum time = 10s.
i7-4cores-512MB

Sorted from worst to best

Stockfish 100214 is the worst

Where do you get stockfish 100214?

http://abrok.eu/stockfish/
The latest version that is earlier than 12.2 is the version from 9.2

Author: Henri Wiechers
Date: Sun Feb 9 19:42:24 2014 +0100
Timestamp: 1391971344

Faster handling of king captures in Position::see

Another SEE speed up that passed the SPRT short TC test!

LLR: 2.96 (-2.94,2.94) [-1.50,4.50]
Total: 81337 W: 15060 L: 14745 D: 51532

No functional change.

running this version I get the following analysis for position number 6 that you claim that stockfish 9.2 did not solve(Edit maybe you ran it with 4 threads and with 4 threads you do not get deterministic results so it is bad for comparison)

I tried it some times with 4 threads and in the worst case I got a6 at depth 39 after almost 10 seconds

I used my default 1024 mbyte hash
changing the hash to 512 mbytes also does not help stockfish not to solve position number 6 and in most cases it could solve the position in less than 10 seconds but there was one failure when it suggested axb6 even at depth 53 after more than a minute.

[d]8/8/1p1K4/Pp6/2k1p3/8/1P6/8 w - - 0 1

Stockfish_14020919_x64_modern_sse42:
1/1 00:00 28 28,000 +1.05 a5-a6
2/2 00:00 73 73,000 +0.32 b2-b3+ Kc4-d4 a5xb6
3/3 00:00 184 92,000 +3.05 a5-a6 e4-e3 a6-a7
4/4 00:00 263 131,500 +0.87 a5-a6 e4-e3 a6-a7 e3-e2
5/5 00:00 600 300,000 +1.55 b2-b3+ Kc4-d3 a5xb6 e4-e3 b6-b7
6/6 00:00 1,339 669,500 +0.16 b2-b3+ Kc4xb3 a5-a6 e4-e3 a6-a7 e3-e2
7/8 00:00 1,817 908,500 +0.09 a5xb6 e4-e3 b6-b7 e3-e2 b7-b8Q e2-e1Q Qb8-g8+ Kc4-b4 Qg8-g4+ Kb4-b3
8/10 00:00 2,546 848,666 +0.10 a5xb6 e4-e3 b6-b7 e3-e2 b7-b8Q e2-e1Q Qb8-g8+ Kc4-b4 Qg8-d5 Qe1-g3+ Kd6-e6
9/15 00:00 12,798 1,828,285 -0.03 a5xb6 e4-e3 b6-b7 e3-e2 b7-b8Q e2-e1Q Qb8-g8+ Kc4-b4 Qg8-d5 Qe1-g3+ Qd5-e5 Qg3xe5+ Kd6xe5
10/16 00:00 21,866 1,987,818 0.00 a5-a6 e4-e3 a6-a7 e3-e2 a7-a8Q e2-e1Q Qa8-a2+ Kc4-b4 Qa2-a3+ Kb4-c4 Qa3-a2+
11/16 00:00 34,359 2,021,117 0.00 a5-a6 e4-e3 a6-a7 e3-e2 a7-a8Q e2-e1Q Qa8-a2+ Kc4-b4 Qa2-a3+ Kb4-c4 Qa3-a2+
12/16 00:00 56,772 1,669,764 0.00 a5-a6 e4-e3 a6-a7 e3-e2 a7-a8Q e2-e1Q Qa8-a2+ Kc4-b4 Qa2-a3+ Kb4-c4 Qa3-a2+
13/16 00:00 81,283 1,806,288 0.00 a5-a6 e4-e3 a6-a7 e3-e2 a7-a8Q e2-e1Q Qa8-a2+ Kc4-b4 Qa2-a3+ Kb4-c4 Qa3-a2+
14/16 00:00 125,839 1,772,380 0.00 a5-a6 e4-e3 a6-a7 e3-e2 a7-a8Q e2-e1Q Qa8-a2+ Kc4-b4 Qa2-a3+ Kb4-c4 Qa3-a2+
15/16 00:00 188,383 1,982,978 0.00 a5-a6 e4-e3 a6-a7 e3-e2 a7-a8Q e2-e1Q Qa8-a2+ Kc4-b4 Qa2-a3+ Kb4-c4 Qa3-a2+
16/16 00:00 252,169 2,066,959 0.00 a5-a6 e4-e3 a6-a7 e3-e2 a7-a8Q e2-e1Q Qa8-a2+ Kc4-b4 Qa2-a3+ Kb4-c4 Qa3-a2+
17/16 00:00 330,519 2,203,460 0.00 a5-a6 e4-e3 a6-a7 e3-e2 a7-a8Q e2-e1Q Qa8-a2+ Kc4-b4 Qa2-a3+ Kb4-c4 Qa3-a2+
18/16 00:00 454,049 2,340,458 0.00 a5-a6 e4-e3 a6-a7 e3-e2 a7-a8Q e2-e1Q Qa8-a2+ Kc4-b4 Qa2-a3+ Kb4-c4 Qa3-a2+
19/16 00:00 605,639 2,412,904 0.00 a5-a6 e4-e3 a6-a7 e3-e2 a7-a8Q e2-e1Q Qa8-a2+ Kc4-b4 Qa2-a3+ Kb4-c4 Qa3-a2+
20/16 00:00 810,667 2,509,804 0.00 a5-a6 e4-e3 a6-a7 e3-e2 a7-a8Q e2-e1Q Qa8-a2+ Kc4-b4 Qa2-a3+ Kb4-c4 Qa3-a2+
21/16 00:00 1,102,008 2,580,814 0.00 a5-a6 e4-e3 a6-a7 e3-e2 a7-a8Q e2-e1Q Qa8-a2+ Kc4-b4 Qa2-a3+ Kb4-c4 Qa3-a2+
22/16 00:00 1,441,447 2,644,856 0.00 a5-a6 e4-e3 a6-a7 e3-e2 a7-a8Q e2-e1Q Qa8-a2+ Kc4-b4 Qa2-a3+ Kb4-c4 Qa3-a2+
23/16 00:01 1,993,004 2,700,547 0.00 a5-a6 e4-e3 a6-a7 e3-e2 a7-a8Q e2-e1Q Qa8-a2+ Kc4-b4 Qa2-a3+ Kb4-c4 Qa3-a2+
24/37 00:01 4,427,956 2,969,789 +51.83 a5-a6 e4-e3 a6-a7 e3-e2 a7-a8Q Kc4-d3 Qa8-d5+ Kd3-c2 Qd5-e4+ Kc2-d2 Kd6-c6 e2-e1Q Qe4-b4+ Kd2-d1 Qb4xe1+ Kd1xe1 Kc6xb5 Ke1-d2 Kb5xb6 Kd2-d3 b2-b4 Kd3-e4 Kb6-c6 Ke4-e5 b4-b5 Ke5-f5 Kc6-d5 Kf5-g6 b5-b6 Kg6-h7 b6-b7

[phenri]

I just compile the 120214 version and I tested it twice, as well as with the syzygy. I also do a second test for the version 101113

What is strange is that I do not always get the same result, it seems random for some positions.

Code: Select all

ZPTS,xx	101113	2nd try	120214	2nd try	+syzygy	2nd try
ZPTS.01	Solved	Solved	Solved	Solved	Solved	Solved
ZPTS.02	Solved	Solved	Solved	Solved	Solved	Solved
ZPTS.03	Solved	Solved	Solved	Solved	Solved	Solved
ZPTS.04	Solved	Solved	Solved	Solved	Solved	Solved
ZPTS.05	Solved	Solved	Solved	Solved	Solved	Solved
ZPTS.06	Solved	Solved	Solved	Solved	Solved	Solved
ZPTS.07	Solved	Solved	Solved	Solved	Solved	Solved
ZPTS.08	Solved	Solved	Solved	Solved	Solved	Solved
ZPTS.09	Solved	Solved	Solved	Solved	Solved	Solved
ZPTS.10	Solved	Solved	Solved	Solved	Solved	Solved
ZPTS.11	Solved	> 10s.	> 10s.	> 10s.	Solved	> 10s.
ZPTS.12	Solved	Solved	Solved	Solved	Solved	Solved
ZPTS.13	Solved	Solved	Solved	Solved	Solved	Solved
ZPTS.14	> 10s.	> 10s.	> 10s.	> 10s.	> 10s.	> 10s.
ZPTS.15	Solved	Solved	Solved	Solved	Solved	Solved
ZPTS.16	> 10s.	> 10s.	> 10s.	Solved	Solved	Solved
ZPTS.17	Solved	Solved	Solved	Solved	Solved	Solved
ZPTS.18	Solved	> 10s.	> 10s.	> 10s.	> 10s.	> 10s.
ZPTS.19	> 10s.	> 10s.	> 10s.	> 10s.	> 10s.	> 10s.
ZPTS.20	Solved	Solved	Solved	Solved	Solved	Solved
ZPST.21	> 10s.	> 10s.	> 10s.	> 10s.	> 10s.	> 10s.
ZPST.22	Solved	Solved	Solved	Solved	Solved	Solved
ZPTS.23	Solved	Solved	Solved	Solved	Solved	Solved
ZPTS.24	> 10s.	> 10s.	> 10s.	> 10s.	> 10s.	> 10s.
ZPST.25	Solved	> 10s.	> 10s.	> 10s.	Solved	Solved
ZPST.26	Solved	Solved	> 10s.	> 10s.	> 10s.	> 10s.
ZPTS.27	Solved	Solved	Solved	Solved	Solved	Solved
ZPTS.28	> 10s.	> 10s.	> 10s.	> 10s.	> 10s.	> 10s.
ZPTS.29	Solved	Solved	Solved	Solved	Solved	Solved
ZPST.30	Solved	Solved	Solved	Solved	Solved	Solved
Result:	24/30	 21/30	 20/30	 21/30	 23/30	 22/30
Percent:  80.0%.	70.0%.	66.6%.	70.0%.	76.6%.	73.3%

[Uri Blass]

The reason is that you use more than 1 core.
The only good test for comparison is with one core unless you want to repeat the same position 10 times and find the number of times that it get it correctly.

[phenri]

And here are some engines tested
Conditions:
Maximum time = 10s.
i7-4cores-512MB

Sorted from worst to best

Stockfish 100214 is the worst

Where do you get stockfish 100214?

100214 is the date/timestamp when I fetched/merged/compiled the source code from Marco repository

Must refer to the last commit before that date.