test positions for weak playing engines only

[Joost Buijs]

Not too easy. For instance Skipper can't solve WAC.002:

[d] 8/7p/5k2/5p2/p1p2P2/Pr1pPK2/1P1R3P/8 b - -

WAC.002 needs some understanding of passed pawns and is one of the more difficult ones.
I use WAC mainly to test if something is broken in my engine, at 1 sec. per move it normally scores 297/298 out of 300, when this number gets lower I know something is wrong.
There are at least ~250 very easy positions in WAC that even the weaker engines will solve at 1 sec. per move.
There are several versions of WAC out there, many positions are revised over the years, there is one called 'wacnew.epd' which is the one I use.
WAC is only about tactics, if you want something positional there is the STS test with 1500 positions.
The Arasan test is also very good, but probably too difficult for Skipper at the moment.

[Henk]

WAC #2 is known to be difficult. Null move can create problems in solving this.

--Jon

This time Skipper had null move pruning disabled.

[Joost Buijs]

WAC #2 is known to be difficult. Null move can create problems in solving this.

--Jon

This time Skipper had null move pruning disabled.

The difficulty with this position is that after the Rxb2 Rxb2 c3 white has a number of checks with the rook that will push the promotion of the black pawns over the horizon.
You can give the phalanx c3 d3 at least the value of a rook, but that is dangerous, or you have to look deep enough to see that these checks are leading to nowhere.

[Robert Pope]

Not too easy. For instance Skipper can't solve WAC.002:

[d] 8/7p/5k2/5p2/p1p2P2/Pr1pPK2/1P1R3P/8 b - -

WAC.002 seems not very easy. How does Skipper on the others ?

I first have to write a function that test them in a row. Testing them one by one is not that practical. WAC 001,003,004 no problem for Skipper.

[Also Skipper does not show PV all the time. So I have to fix that too if I have time.]

Take a look at epd2wb. That's what I use to test positions. About 1/3 of the WAC positions should be very easy for any properly running engine - things like mate in 2. The rest range in difficulty, up to a handful of challenging ones. It's a good benchmark to look at periodically for the sub-2000 engines, because you can make very visible progress against them as your engine improves.

[JVMerlino]

Do you know a list of fens which are positions any weak playing engine should be able to solve ?

I assume you mean "solve relatively quickly", as in within 30 seconds. Here are several that I use to make sure Myrddin isn't broken. Myrddin finds most of these in less than 5 seconds, but is only rated about 2360 CCRL.

This is Fine 70, perfect for testing if your transposition table is broken
8/k7/3p4/p2P1p2/P2P1P2/8/8/K7 w - - 0 1 bm Kb1

This is a mate in 7 that your engine should get very quickly if you have extensions for both check and single reply
7k/3b3p/1p6/3p1P1P/2p3r1/2P1R3/7r/2RK4 b - - 0 40 bm Rg1, Ba4

Two easy to find mate in 5 positions
R7/5k2/3p4/4p3/1P3p1R/8/2P4K/8 w - - 0 45 bm Rh6
6k1/8/5pB1/4b2p/3rP2P/2K1R1P1/1r2R3/8 b - - 0 57 bm Rd7, Rd8

Mate in 10 that isn't too hard to find
3R4/2r5/6rp/2k5/p1p2p1P/P1Rp1P2/1P6/5K2 b - - bm Rcg7

Find the move that delays mate as long as possible. Good for engines that have a series of aspiration windows before using a full-window search
8/3kp3/Q7/4p3/4K3/6p1/PP4P1/8 b - - 0 43 bm Ke8

And then several others that have different uses
r4r2/p1q1n1kp/2n1ppp1/8/3P2N1/3BPP2/2Q2P1P/R3K2R w KQ - 0 19 bm h3, h4
2q3k1/R2bprbp/3n2p1/3P4/P2NP3/5P2/1r3BPP/3Q1RK1 b - - 0 21 bm Nxe4
4r1k1/2r1pp1p/pp4p1/3b4/PB1b2N1/8/5PPP/3RR1K1 b - - 0 25 bm Bg7
1r5k/q1p3p1/3b1p1p/8/p1P4N/3Q1PP1/2P3K1/7R b - - 0 30 am a3
4r1k1/pp1b1pp1/1q1b1p2/3P4/2Q2PP1/P1N5/RP2N3/7K b - - 0 22 bm Qf2
r1bqr1k1/p2p1pp1/1p5p/2pP3Q/1n2p3/1P2P1P1/PB1P1PP1/R3KB1R w KQ - bm Bg7
3b2k1/3Prp1p/R5p1/p7/2Bp4/1P3P2/P5PP/6K1 w - - 0 40 bm Bb5
1k2r3/p2BPb1p/5P2/2p2P2/Pp6/5P2/1P1B3P/n3K3 w - - 0 32 bm Be6, Bf4
6k1/4bpp1/R7/2p1P3/6qP/1P4B1/7P/1R4K1 b - - 0 36 bm c4
8/8/4kp1p/R7/4N2P/p4rP1/6K1/3b4 w - - 0 52 bm Nf2
r4rk1/1p2ppbp/pq1p2p1/3P4/1nP3n1/2N2N2/PP2QPPP/R1B2RK1 b - - 0 18 bm a5
8/pR5p/4k3/2P1B3/5P1r/4P3/P2r4/2R3K1 w - - 0 34 am Qxa7 bm Rb2, Rg7
2r1r1k1/5p2/p3pbpB/1p1pP3/3p2P1/n1q2B2/2P2PK1/4R2R w - - bm exf6
3r1r1k/1b4pp/ppn1p3/4Pp1R/Pn5P/3P4/4QP2/1qB1NKR1 w - - bm Rxh7

jm

[Dann Corbit]

Here is what I have:
[d]rn1qkb1r/pp2pppp/5n2/3p1b2/3P4/2N1P3/PP3PPP/R1BQKBNR w KQkq - acd 38; acs 3223; bm Qb3; c0 "level: med-12"; c3 "Qb3"; ce 10; id "CCR.01"; pm Qb3 {391} Nf3 {10} Bb5+ {5} Bd3 {5}; pv Qb3 Nc6 Qxb7 Bd7 Qb3 Rb8 Qd1 e5 dxe5 Nxe5 Be2 Qc7 a3 Be7 Nf3 O-O O-O Bd6 h3 Rfc8 Nd4 Qb7 Ra2 Nc6 Ndb5 Be7 Bd2 a6 Nd4 Bd6 Re1 Qa7 Bc1 Nxd4 Qxd4 Bc5 Qd3 d4 exd4 Bxd4;

Not exactly a glowing centipawn evaluation. Maybe your move is close.

[Dann Corbit]

I have this:
[d]3r1k2/4npp1/1ppr3p/p6P/P2PPPP1/1NR5/5K2/2R5 w - - acd 34; acs 650; bm d5; c0 "Bogoliubov - Spielmann"; c1 "Bogoliubov - Spielmann"; c3 "d5"; c4 "(Bratko-Kopec) bk002"; ce 222; id "BK.02"; pm d5; pv d5 cxd5 e5 R6d7 Nd4 Rb8 Ke3;

[Maarten Claessens]

This is a mate in 7 that your engine should get very quickly if you have extensions for both check and single reply
7k/3b3p/1p6/3p1P1P/2p3r1/2P1R3/7r/2RK4 b - - 0 40 bm Rg1, Ba4

That's a nice one, Waduuttie solves it in 386 nodes:

Code: Select all

WaDuuttie> sb 7k/3b3p/1p6/3p1P1P/2p3r1/2P1R3/7r/2RK4 b - - 0 40

WaDuuttie> st 1

WaDuuttie> on
ply      nodes  score  pv
  1        386    +M7  g4g1 e3e1 d7a4 c1c2 a4c2 d1c1 g1e1 c1b2 c2f5 b2a3
                       e1a1 a3b4 h2b2
  2       1521    +M7  g4g1 e3e1 d7a4 c1c2 a4c2 d1c1 g1e1 c1b2 c2f5 b2a3
                       e1a1 a3b4 h2b2
  3       3547    +M7  g4g1 e3e1 d7a4 c1c2 a4c2 d1c1 g1e1 c1b2 c2f5 b2a3
                       e1a1 a3b4 h2b2
  4       5666    +M7  g4g1 e3e1 d7a4 c1c2 a4c2 d1c1 g1e1 c1b2 c2f5 b2a3
                       e1a1 a3b4 h2b2
Computer's move: g4g1 (15 msecs)

I guess engines like Spark can improve on that !

[Dann Corbit]

This is a mate in 7 that your engine should get very quickly if you have extensions for both check and single reply
7k/3b3p/1p6/3p1P1P/2p3r1/2P1R3/7r/2RK4 b - - 0 40 bm Rg1, Ba4

That's a nice one, Waduuttie solves it in 386 nodes:

Code: Select all

WaDuuttie> sb 7k/3b3p/1p6/3p1P1P/2p3r1/2P1R3/7r/2RK4 b - - 0 40

WaDuuttie> st 1

WaDuuttie> on
ply      nodes  score  pv
  1        386    +M7  g4g1 e3e1 d7a4 c1c2 a4c2 d1c1 g1e1 c1b2 c2f5 b2a3
                       e1a1 a3b4 h2b2
  2       1521    +M7  g4g1 e3e1 d7a4 c1c2 a4c2 d1c1 g1e1 c1b2 c2f5 b2a3
                       e1a1 a3b4 h2b2
  3       3547    +M7  g4g1 e3e1 d7a4 c1c2 a4c2 d1c1 g1e1 c1b2 c2f5 b2a3
                       e1a1 a3b4 h2b2
  4       5666    +M7  g4g1 e3e1 d7a4 c1c2 a4c2 d1c1 g1e1 c1b2 c2f5 b2a3
                       e1a1 a3b4 h2b2
Computer's move: g4g1 (15 msecs)

I guess engines like Spark can improve on that !

Takes Chest a lot more nodes (and 6 seconds):
7k/3b3p/1p6/3p1P1P/2p3r1/2P1R3/7r/2RK4 b - - acn 3832236; acs 6; bm Ba4+ Rg1+; ce 32754; dm 7; pv Ba4+ Rc2 Bxc2+ Kc1 Rg1+ Re1 Rxe1+ Kb2 Bd3+ Ka3 Ra1+ Kb4 Rb2#;

[flok]

https://chessprogramming.wikispaces.com/Test-Positions

[d] rn1qkb1r/pp2pppp/5n2/3p1b2/3P4/2N1P3/PP3PPP/R1BQKBNR w KQkq - 0 1

I tried the first one of "CCR one hour test" but Skipper thinks Nf3 is better than Qb3.

These testing strings are interesting!
For this Embla found d1-b3 at depth 3.