[D] 4Q3/5ppk/7p/r7/P7/6Pb/1q3B1P/4R1K1 w - - 3 36
My engine is having a hard time avoiding Qxf7 here. I disabled nullmove and it couldn't avoid it still. What could be the problem?
How fast do other amateur engines could avoid Qxf7?
Test position for amateur engines!
Moderators: hgm, Rebel, chrisw
-
- Posts: 803
- Joined: Mon Jul 17, 2006 5:53 am
- Full name: Edsel Apostol
-
- Posts: 6401
- Joined: Thu Mar 09, 2006 8:30 pm
- Location: Chicago, Illinois, USA
Re: Test position for amateur engines!
Gaviota avoids Qxf7 all the time.Edsel Apostol wrote:[D] 4Q3/5ppk/7p/r7/P7/6Pb/1q3B1P/4R1K1 w - - 3 36
My engine is having a hard time avoiding Qxf7 here. I disabled nullmove and it couldn't avoid it still. What could be the problem?
How fast do other amateur engines could avoid Qxf7?
I think it is more interesting to see how long does it take to find Rf5 after Qxf7. It is not as "instantaneously" obvious as it seems. The engine should understand that after 1. Qxf7 Rf5 2.Qa7 Re5 3. Rxe5 Qxe5 White is lost.
[D]8/5Qpk/7p/r7/P7/6Pb/1q3B1P/4R1K1 b - - 0 37 bm Rxf5
2.7 for gaviota to see Rf5.
Code: Select all
15 1 0.0 -8.12 Qxf2+ 20.Qxf2 Rxa4
24 1 0.0 +0.30 Rxa4
103 2 0.0 +0.25 Rxa4 20.Qd5
967 3 0.0 +0.32 Rxa4 20.Qd5 Qc3
2609 4 0.0 +0.30 Rxa4 20.Qd5 Ra1 21.Qe4+ Kg8 22.Qe8+ Kh7
7009 5 0.0 +0.28 Rxa4 20.Qd5 Qc3 21.Kh1 Ra5
69479 6 0.2 +0.36 Rxa4 20.Qd5 Ra5 21.Qe4+ Bf5 22.Qf4 Qb3
81332 6: 0.3 +0.36 Rxa4 20.Qd5 Ra5 21.Qe4+ Bf5 22.Qf4 Qb3
292206 7 0.9 +0.49 Rxa4 20.Qd5 Qc2 21.Qh5 Ra2 22.Qf7 Rb2
400037 7: 1.2 +0.49 Rxa4 20.Qd5 Qc2 21.Qh5 Ra2 22.Qf7 Rb2
868045 8 2.4 +0.50 Rxa4 20.Qf3 Qc2 21.Qe2 Qc6 22.Be3 Re4
23.Ra1 Re5 24.Qd3+ Kg8
994912 8 2.7 :-) Rf5
1070470 8 2.9 :-) Rf5
1203799 8 3.3 :-) Rf5
1348001 8 3.7 +5.82 Rf5 20.Qa7 Re5 21.Rf1 Qe2 22.Rb1 Qf3
23.Qb7 Re1+ 24.Rxe1 Qxb7
1468997 8: 4.0 +5.82 Rf5 20.Qa7 Re5 21.Rf1 Qe2 22.Rb1 Qf3
23.Qb7 Re1+ 24.Rxe1 Qxb7
1628634 9 4.5 :-( Rf5
1958256 9 5.4 :-(
2073021 9 5.7 +5.82 Rf5 20.Qa7 Re5 21.Rf1 Qe2 22.Rb1 Qf3
23.Qb7 Re1+ 24.Rxe1 Qxb7
2077212 9: 5.7 +5.82 Rf5 20.Qa7 Re5 21.Rf1 Qe2 22.Rb1 Qf3
23.Qb7 Re1+ 24.Rxe1 Qxb7
2451486 10 6.7 +5.82 Rf5 20.Qa7 Re5 21.Rf1 Qe2 22.Rb1 Qf3
23.Qb7 Re1+ 24.Rxe1 Qxb7
3412855 10: 9.2 +5.82 Rf5 20.Qa7 Re5 21.Rf1 Qe2 22.Rb1 Qf3
23.Qb7 Re1+ 24.Rxe1 Qxb7
5476519 11 14.5 :-) Rf5
8004629 11 20.8 +6.82 Rf5 20.Qxf5+ Bxf5 21.g4 Bxg4 22.h4 Qb4
23.Ra1 Bf3 24.Kh2 Qf4+ 25.Kg1 Bc6
11430232 11: 30.0 +6.82 Rf5 20.Qxf5+ Bxf5 21.g4 Bxg4 22.h4 Qb4
23.Ra1 Bf3 24.Kh2 Qf4+ 25.Kg1 Bc6
15209367 12 39.8 +7.10 Rf5 20.Qxf5+ Bxf5 21.g4 Bxg4 22.Re3
Qa1+ 23.Be1 Qxa4 24.Bc3 Qa7 25.Kf2 g5
26.Bf6 Qa2+ 27.Kg1
28335584 12: 73.9 +7.10 Rf5 20.Qxf5+ Bxf5 21.g4 Bxg4 22.Re3
Qa1+ 23.Be1 Qxa4 24.Bc3 Qa7 25.Kf2 g5
26.Bf6 Qa2+ 27.Kg1
-
- Posts: 128
- Joined: Thu Mar 09, 2006 5:14 pm
- Location: Los Angeles, CA
Re: Test position for amateur engines!
Amateur likewise avoids Qxf7, then sees Rf5 in about 5 sec:
Code: Select all
11 8 139 1621561 Rxa4 Qd5 Qf6
12 10 385 4563078 Rxa4 Qf3
12 522 459 5391964 Rf5 Qa7 Re5 Rd1
13 702 776 8920212 Rf5 Qxf5 Bxf5 g4 Bxg4 a5 Qa2 Re7 Bf3 Re3
14 718 889 10191188 Rf5 Qxf5 Bxf5 g4 Bxg4 h4
15 721 1081 12509088 Rf5 Qxf5 Bxf5 g4 Bxg4 h4 Qc2 a5 Bh3 Re3 Qc1 Kh2 Qf1 Rg3 Qxf2 Kxh3 Qf5 Kg2 Qxa5
-
- Posts: 344
- Joined: Wed Sep 23, 2009 5:56 pm
- Location: Germany
Re: Test position for amateur engines!
The problem in Twisted Logic seems to be that 1...Rf5 is "pruned away" as useless. Even in the position after 1.Qxf7 the engine has no clue and evaluates the position as approximately equal after Rxa4 at depth 18. But in the position after 1.Qxf7 Rf5 the engine realizes that White is losing at depth 11. So with no pruning at all, the problem should be seen at depth 12 from the position after Qxf7 and at depth 13 from the starting position. As it is not seen, the problem seems to be the pruning of Rf5.
To my surprise,. Stockfish has the same problem at the first plies, but avoids Qxf7 at depth 15 after 2 fail-lows at depth 14 and 15.
To my surprise,. Stockfish has the same problem at the first plies, but avoids Qxf7 at depth 15 after 2 fail-lows at depth 14 and 15.
-
- Posts: 20943
- Joined: Mon Feb 27, 2006 7:30 pm
- Location: Birmingham, AL
Re: Test position for amateur engines!
It takes Crafty almost 1/2 second to discover that this move is bad:Edsel Apostol wrote:[D] 4Q3/5ppk/7p/r7/P7/6Pb/1q3B1P/4R1K1 w - - 3 36
My engine is having a hard time avoiding Qxf7 here. I disabled nullmove and it couldn't avoid it still. What could be the problem?
How fast do other amateur engines could avoid Qxf7?
Code: Select all
12-> 0.26 -0.13 1. Qxf7 Rxa4 2. Qf3 Qc2 3. Qe2 Qg6
4. Qd2 Be6 5. Qd6 Re4 6. Be3 Bc4 (s=2)
13 0.39 -1 1. Qxf7?
13 0.40 -3 1. Qxf7?
13 0.42 -M 1. Qxf7?
13 0.52 -6.98 1. Qxf7 Rf5 2. Qxf5+ Bxf5 3. g4 Bxg4
4. h4 Qa3 5. Re4 Bd1 6. Rc4 Bb3 7.
Rc7 Qxa4
13 1.28 -1.30 1. Qe4+ f5 2. Qh4 Bg4 3. Qe7 Rxa4 4.
Qe5 Qb7 5. Qe7 Qb3 6. Bc5 Bf3 7. Qd7
Be4
13 1.76 -1.09 1. Qc6 Qa2 2. Qe4+ f5 3. Qh4 Bg4 4.
Qd8 Qxa4 5. Qd4 Qa3 6. Qc4 Bf3 7. Bd4
Be4
13 2.12 -1.05 1. Qe2 Qxe2 2. Rxe2 Rxa4 3. Bb6 Rb4
4. Bc5 Rc4 5. Bd6 Be6 6. Kg2 Rd4 7.
Bf4
13-> 2.12 -1.05 1. Qe2 Qxe2 2. Rxe2 Rxa4 3. Bb6 Rb4
4. Bc5 Rc4 5. Bd6 Be6 6. Kg2 Rd4 7.
Bf4 (s=9)
14 2.20 -1.05 1. Qe2 Qxe2 2. Rxe2 Rxa4 3. Bb6 Rb4
4. Bc5 Rc4 5. Bd6 Be6 6. Kg2 Kg6 7.
h3 Bd5+ 8. Kh2 (s=8)
-
- Posts: 344
- Joined: Wed Sep 23, 2009 5:56 pm
- Location: Germany
Re: Test position for amateur engines!
Well, at least on my computer TL 20090922 has a real problem with this position. Even at ply 20 (I know this 'even' probably sounds ridiculous to people with a cluster ), which is reached after 4:10 minutes here, it does not see that Qxf7 is very bad. I have aborted the search here, but this means that it would probably play that move in a tournament game.
-
- Posts: 20943
- Joined: Mon Feb 27, 2006 7:30 pm
- Location: Birmingham, AL
Re: Test position for amateur engines!
What I usually do here is use a little human chess knowledge and walk down the PV a couple of moves (down the PV it thinks is best) and then see why it can't see the problem that is brewing around f2. You want a small tree to debug, not a huge one.metax wrote:Well, at least on my computer TL 20090922 has a real problem with this position. Even at ply 20 (I know this 'even' probably sounds ridiculous to people with a cluster ), which is reached after 4:10 minutes here, it does not see that Qxf7 is very bad. I have aborted the search here, but this means that it would probably play that move in a tournament game.
Here's an idea, assuming you can run Crafty, which spots this at 12 ply if I remember the output I posted.
Run down the PV for 4 full moves, and then feed the resulting position to Crafty (set noise to 0 so that you can see all output). Limit Crafty to something shallow using the sd=n command so that the tree will be very small. Keep walking down the PV until Crafty suddenly fails low (you have already played Qxf7 here so it is going to fail low eventually).
Now take that same position and feed it to your program and see how long it takes it to fail low. You will have a very simple PV output from Crafty, so you should be able to figure out what you are missing when you see your search output.
The idea is to keep shrinking the tree until you are debugging a 2-3-4 ply search which is very easy to work with, rather than a 20 ply search which is very difficult to debug.
-
- Posts: 261
- Joined: Wed Mar 08, 2006 8:49 pm
Re: Test position for amateur engines!
ProDeo Hera: core i7
10 00:00 1.374.429 1.374.429 -0.24 Qxf7 Rf5 Qa7 Re5 Rxe5 Qxe5 Qd4 Qe2 Qa1 Qf3 Qb1+ g6
11 00:01 2.774.058 2.774.058 -4.53 Qxf7 Rf5 Qa7 Re5 Rxe5 Qxe5 Qd4 Qe2 Qa1 Qf3 Qf1 Bxf1 Kxf1 11 00:02 4.926.565 2.463.282 -0.69 Re2 Qb3 Re3 Qxa4 Qxa4 Rxa4 Re7 Ra1+ Re1 Ra4 Bb6 Rb4 Bc5 Rb2 Re8 Kg6 Re7
11 00:02 6.222.010 3.111.005 -0.62 Qe2 Qxe2 Rxe2 Rxa4 Bb6 Bf5 Kf2 Kg6 Ke3 Ra1 Bd4 Rb1
12 00:04 9.627.058 2.259.872 -0.62 Qe2 Qxe2 Rxe2 Rxa4 Bb6 Be6 Kf2 Rb4 Ba5 Rb1 Rc2 Ra1 Bb6 Bb3 Kf3
13 00:05 11.472.751 2.181.131 -0.61 Qe2 Qxe2 Rxe2 Rxa4 Bb6 Rb4 Bc5 Rc4 Bd6 Rd4 Bc5 Rd5 Bb4 Be6
14 00:06 16.167.164 2.391.592 -0.59 Qe2 Qxe2 Rxe2 Rxa4 Bb6 Rb4 Bc5 Rb1+ Kf2 Kg6 Kf3 Rd1 Re7 Rd2 Bf2 Be6
15 00:11 28.840.595 2.403.382 -0.64 Qe2 Qb3 Qe4+ Bf5 Qf4 Qxa4 Qxa4 Rxa4 Re7 Ra2 Rxf7 Ra1+ Kg2 Bg6 Ra7 Be4+ Kh3 h5
10 00:00 1.374.429 1.374.429 -0.24 Qxf7 Rf5 Qa7 Re5 Rxe5 Qxe5 Qd4 Qe2 Qa1 Qf3 Qb1+ g6
11 00:01 2.774.058 2.774.058 -4.53 Qxf7 Rf5 Qa7 Re5 Rxe5 Qxe5 Qd4 Qe2 Qa1 Qf3 Qf1 Bxf1 Kxf1 11 00:02 4.926.565 2.463.282 -0.69 Re2 Qb3 Re3 Qxa4 Qxa4 Rxa4 Re7 Ra1+ Re1 Ra4 Bb6 Rb4 Bc5 Rb2 Re8 Kg6 Re7
11 00:02 6.222.010 3.111.005 -0.62 Qe2 Qxe2 Rxe2 Rxa4 Bb6 Bf5 Kf2 Kg6 Ke3 Ra1 Bd4 Rb1
12 00:04 9.627.058 2.259.872 -0.62 Qe2 Qxe2 Rxe2 Rxa4 Bb6 Be6 Kf2 Rb4 Ba5 Rb1 Rc2 Ra1 Bb6 Bb3 Kf3
13 00:05 11.472.751 2.181.131 -0.61 Qe2 Qxe2 Rxe2 Rxa4 Bb6 Rb4 Bc5 Rc4 Bd6 Rd4 Bc5 Rd5 Bb4 Be6
14 00:06 16.167.164 2.391.592 -0.59 Qe2 Qxe2 Rxe2 Rxa4 Bb6 Rb4 Bc5 Rb1+ Kf2 Kg6 Kf3 Rd1 Re7 Rd2 Bf2 Be6
15 00:11 28.840.595 2.403.382 -0.64 Qe2 Qb3 Qe4+ Bf5 Qf4 Qxa4 Qxa4 Rxa4 Re7 Ra2 Rxf7 Ra1+ Kg2 Bg6 Ra7 Be4+ Kh3 h5
-
- Posts: 2273
- Joined: Mon Sep 29, 2008 1:50 am
Re: Test position for amateur engines!
Not a good result for the latest version of GnuChess this time. On a Pentium 4 (which is very slow) GnuChess 5.07.17testb need precisely one hour to avoid Qxf7 at depth 16 (the fail low occurs after 40 minutes).
I wonder how the other engines manage to find it so quickly... It is after all quite
deep and involves a few relatively quiet moves. This version of GnuChess does LMR which is probably the culprit.
I am going to test with the original version of GnuChess which just does a full width search with extensions.
I wonder how the other engines manage to find it so quickly... It is after all quite
deep and involves a few relatively quiet moves. This version of GnuChess does LMR which is probably the culprit.
I am going to test with the original version of GnuChess which just does a full width search with extensions.
-
- Posts: 613
- Joined: Sun Jan 18, 2009 7:03 am
Re: Test position for amateur engines!
Interesting position! There are silent moves (like Re5) which can get pruned away.
Here is how current Stockfish development version survives here. On depth 12 it sees that sth is badly wrong, but because Stockfish doesn't resolve fail lows immediately, it sees the way out on depth 13.
2 -0.12 00:00 259 Qxf7 Rxa4
3 -0.16 00:00 778 Qxf7 Rxa4 Qf3
4 -0.28 00:00 1585 Qxf7 Rxa4 Re8 Ra1+ Re1
5 -1.01 00:00 4207 Qxf7 Rxa4 Qh5 Ra2 Qf3
5 -0.08 00:00 6791 Qe4+ g6 Bd4 Qa2 Re2 Qxa4
6 -0.48 00:00 14553 Qe4+ f5 Qc6 Qb4 Ra1 Qe4 Qxe4 fxe4
7 -0.61 00:00 22240 Qe4+ f5 Qc6 Qa2 Bd4 Qxa4 Qxa4 Rxa4 Bc5 Bg4
8 -0.53 00:00 38200 Qe4+ f5 Qc6 Qa2 Bd4 Qxa4 Qb6 Qc2 Qb7
9 -0.73 00:00 69700 Qe4+ f5 Qc6 Qa2 Re7 Rxa4 Qc3 Rg4 Qc6 Qb2
9 -0.48 00:00 81692 Qxf7 Rxa4 Qf3 Qc2 Qd1 Ra2 Qxc2+ Rxc2 Bd4 Kg6
10 -0.44 00:00 115773 Qxf7 Rxa4 Qf3 Qc2 Qe2 Ra2 Qxc2+ Rxc2 Re7 Kg6 Bd4
11 -0.32 00:00 185087 Qxf7 Rxa4 Qf3 Qc2 Qe2 Ra2 Qxc2+ Rxc2 Re7 Rd2 Be3
Re2
12 < -0.61 00:00 282325 Qxf7 Rf5 Qa7 Re5 Rd1 Qc2 Qd4 Qc6 Qd3+ Rf5
13 < -1.86 00:00 448441 Qxf7 Rf5 Qa7 Re5 Rd1 Qc2 Qd4 Qc6 Qd3+ Rf5
13 -0.97 00:00 1123k Qe4+ f5 Qh4 Bg4 h3 Bf3 Qf4 Be4 g4 Rxa4 gxf5 Qc2
f6 Bd5
14 -1.13 00:01 1846k Qe4+ f5 Qh4 Bg4 h3 Bf3 Qd4 Qxd4 Bxd4 Rxa4 Kf2
Rxd4 Kxf3 Kg6 Re7 Kf6 Ke3 Kxe7 Kxd4
14 -1.05 00:03 4294k Qe2 Qxe2 Rxe2 Rxa4 Bc5 Kg6 Kf2 Bg4 Rd2 Re4 Rb2
Rc4 Bd6 Rc6 Bf4
15 -1.01 00:04 4771k Qe2 Qxe2 Rxe2 Rxa4 Bc5 Kg6 Kf2 Bg4 Rb2 Re4 Bd6 f6
Bc5 Kf5 Rd2 Re5
16 -1.17 00:09 11200k Qe2 Qb3 Qe4+ Bf5 Qe3 Qxa4 Qc3 f6 Qd2 Qa2 Qxa2
Rxa2 Kg2 Kg6 Re7 h5 Kf3 Bg4+ Kg2
Here is how current Stockfish development version survives here. On depth 12 it sees that sth is badly wrong, but because Stockfish doesn't resolve fail lows immediately, it sees the way out on depth 13.
2 -0.12 00:00 259 Qxf7 Rxa4
3 -0.16 00:00 778 Qxf7 Rxa4 Qf3
4 -0.28 00:00 1585 Qxf7 Rxa4 Re8 Ra1+ Re1
5 -1.01 00:00 4207 Qxf7 Rxa4 Qh5 Ra2 Qf3
5 -0.08 00:00 6791 Qe4+ g6 Bd4 Qa2 Re2 Qxa4
6 -0.48 00:00 14553 Qe4+ f5 Qc6 Qb4 Ra1 Qe4 Qxe4 fxe4
7 -0.61 00:00 22240 Qe4+ f5 Qc6 Qa2 Bd4 Qxa4 Qxa4 Rxa4 Bc5 Bg4
8 -0.53 00:00 38200 Qe4+ f5 Qc6 Qa2 Bd4 Qxa4 Qb6 Qc2 Qb7
9 -0.73 00:00 69700 Qe4+ f5 Qc6 Qa2 Re7 Rxa4 Qc3 Rg4 Qc6 Qb2
9 -0.48 00:00 81692 Qxf7 Rxa4 Qf3 Qc2 Qd1 Ra2 Qxc2+ Rxc2 Bd4 Kg6
10 -0.44 00:00 115773 Qxf7 Rxa4 Qf3 Qc2 Qe2 Ra2 Qxc2+ Rxc2 Re7 Kg6 Bd4
11 -0.32 00:00 185087 Qxf7 Rxa4 Qf3 Qc2 Qe2 Ra2 Qxc2+ Rxc2 Re7 Rd2 Be3
Re2
12 < -0.61 00:00 282325 Qxf7 Rf5 Qa7 Re5 Rd1 Qc2 Qd4 Qc6 Qd3+ Rf5
13 < -1.86 00:00 448441 Qxf7 Rf5 Qa7 Re5 Rd1 Qc2 Qd4 Qc6 Qd3+ Rf5
13 -0.97 00:00 1123k Qe4+ f5 Qh4 Bg4 h3 Bf3 Qf4 Be4 g4 Rxa4 gxf5 Qc2
f6 Bd5
14 -1.13 00:01 1846k Qe4+ f5 Qh4 Bg4 h3 Bf3 Qd4 Qxd4 Bxd4 Rxa4 Kf2
Rxd4 Kxf3 Kg6 Re7 Kf6 Ke3 Kxe7 Kxd4
14 -1.05 00:03 4294k Qe2 Qxe2 Rxe2 Rxa4 Bc5 Kg6 Kf2 Bg4 Rd2 Re4 Rb2
Rc4 Bd6 Rc6 Bf4
15 -1.01 00:04 4771k Qe2 Qxe2 Rxe2 Rxa4 Bc5 Kg6 Kf2 Bg4 Rb2 Re4 Bd6 f6
Bc5 Kf5 Rd2 Re5
16 -1.17 00:09 11200k Qe2 Qb3 Qe4+ Bf5 Qe3 Qxa4 Qc3 f6 Qd2 Qa2 Qxa2
Rxa2 Kg2 Kg6 Re7 h5 Kf3 Bg4+ Kg2
Joona Kiiski