EPD destruction tests, part 2

[chrisw]

Code: Select all

rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - 0 1; startpos ; should be OK, all EPDs below assume normal chess (not 960)

rnbqkbnr/nnnnnnnn/8/8/8/8/NNNNNNNN/RNBQKBNR w KQkq - 0 1; 18 knights; should be OK (but see HGM on max possible promotions if no pieces captured)
rnbqkbnr/bbbbbbbb/8/8/8/8/BBBBBBBB/RNBQKBNR w KQkq - 0 1; 18 bishops; should be OK (technically these are not allowed under normal chess rules)
rnbqkbnr/rrrrrrrr/8/8/8/8/RRRRRRRR/RNBQKBNR w KQkq - 0 1; 18 rooks; should be OK (ditto)
rnbqkbnr/qqqqqqqq/8/8/8/8/QQQQQQQQ/RNBQKBNR w KQkq - 0 1; 18 queens; should be OK (ditto)

rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR  w  KQkq  -  0  1 ; extra white space ; should be not a problem
  rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - 0 1; leading spaces ; should be not a problem
nrbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - 0 1; castle status ; should be overridable
rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - 400 500; movenums ; should be overridable
rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - 10 5; movenums ; should be overridable
krr5/8/8/8/8/8/8/4K3 w kq - 0 1; two castling rooks on same side of black king; ; should be overridable

rnbqkbnr/ppppNNpp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - 0 1; white 18 pieces; well if engines can handle it, leave it ok, else cull
rnbqkbnr/pppppppp/8/8/8/8/PPPPPnPP/RNBQKBNR w KQkq - 0 1; black 17 pieces; well if engines can handle it, leave it ok, else cull
nnnnknnn/nnnnnnnn/8/8/8/8/NNNNNNNN/NNNNKNNN w - - 0 1; 30 knights ; well, if engine can handle impossible promoted piece counts, leave it as ok else cull
rnbnkbnn/bbbbnnnn/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - 0 1; too many promoted pieces; cull, or if engine can handle it, leave as OK
qqqqkqqq/qqqqqqqq/8/8/8/8/QQQQQQQQ/QQQQKQQQ w - - 0 1; 30 queens ; well, if engine can handle impossible promoted piece counts, leave it as ok else cull

BROKEN** rqrkbn2/1pBppppp/8/8/8/8/8/4K2R w K - 0 1 moves e1g1 a8a7 g1f2 a7a6 f1h1 f2e1 a6a8; f2e1 NOT a repetitio ** EPD WIP **

rnbqkbn1/ppppppppr/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - 0 1; extra piece on rank, cull
rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR K KQkq - 0 1; colour, cull
rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQAb - 0 1; castle status, cull (or override)
rnbqkbnr.pppppppp.8.8.8.8.PPPPPPPP.RNBQKBNR w KQkq - 0 1; slash chars ; cull EPD
rnbqkbnr/ppppippp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - 0 1; bad character; cull
rnbqk0nr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - 0 1; bad numeric; cull
rnbqkbnr/pppppppp/8/6/8/8/PPPPPPPP/RNBQKBNR w KQkq - 0 1; square count too low; cull
N7/8/2KQ2rp/6k1/4p3p/2p4P/4PP2/5N2 w - - 0 1; square count too high; cull
rnkqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - 0 1; 3 kings; cull
rnbq1bnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - 0 1; 1 king; cull
pnbqkbnr/ppprpppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - 0 1; pawn back rank; cull

blablabl/blablabl/blablabl/blablabl/blablabl/blablabl/blablabl/blablabl w - - 0 1; garbage
hello have a nice day w - - 0 1; garbage

1B6/prpb2p1/2KPp3/qp1p4/Q1k5/nRP3p1/BRPP2Pp/BN6 w - -;bad EPD, Dann Corbitt
1N4B1/2p2p2/2Q3k1/p2p1RPp/P4P1b/7P/2PPPrrK/8 w - -;bad EPD, Dann Corbitt
1N6/p7/5R2/nkR1r3/rp1QB3/qNpp4/1p1n1p2/1K2b3 w - -;bad EPD, Dann Corbitt
1Q2Rb2/1NB2kp1/4P2R/2K2P2/2p1P3/P7/n1bnPPP1/r2q4 w - -;bad EPD, Dann Corbitt
1Q6/3P1R2/2N1kPpp/5BPK/1N1pp1pb/1n1b1Pp1/4P3/8 w - -;bad EPD, Dann Corbitt
1b1knR1r/2pR3r/2b5/7K/4QBp1/1p1p3B/pn5p/7q w - -;bad EPD, Dann Corbitt
1b6/pp6/1pN5/np4R1/1QNk2b1/R4K2/3r4/1n1r4 w - -;bad EPD, Dann Corbitt
1bk1N3/1p1R2p1/P6p/r6p/r6p/2R1p2B/4Pp2/5Kqb w - -;bad EPD, Dann Corbitt
1r1K1R2/2P1P3/2N4q/Bp4p1/6np/4k3/1bP1p1p1/1r1n1N1b w - -;bad EPD, Dann Corbitt
2K5/3bBp1Q/5p2/5n2/2pN2q1/k5R1/n2rp3/8 w - -;bad EPD, Dann Corbitt
2Q1NR2/5B2/3p2p1/p2RPk2/1p1p4/4BP1N/4pPpP/1rq3Kb w - -;bad EPD, Dann Corbitt
2Q3k1/4b3/2K2q2/8/2B3r1/8/8/8 w - -;bad EPD, Dann Corbitt
3N4/2p3p1/p1R4n/Rq2BkPb/2N1rPP1/2pQ3B/3nK3/2r1b3 w - -;bad EPD, Dann Corbitt
3R4/Pp6/qK6/rp6/1R2p3/k3p2P/PQ2N3/BBN5 w - -;bad EPD, Dann Corbitt
3RB3/1b4Bp/1b2p2p/r3kpK1/p2R2p1/n1qP2P1/1PPN2P1/2r5 w - -;bad EPD, Dann Corbitt
3b3b/2ppRp1p/2p2KpQ/1P5P/5k1R/2P2PBq/4pPP1/6NN w - -;bad EPD, Dann Corbitt
3n4/2Q1p3/p3p3/p2k1NP1/p7/2KP1B2/8/4b3 w - -;bad EPD, Dann Corbitt
3nn3/2q1P3/3P2P1/B3P1b1/2Q2nP1/2q5/kn1P1qr1/2K5 w - -;bad EPD, Dann Corbitt
4B1N1/1p3N1p/1Rn1k1pr/1K4pr/6Pp/1Q5P/1Pp1b3/4R3 w - -;bad EPD, Dann Corbitt
4R3/1N3K2/pb4p1/1Pp2NP1/B1q1k3/1P5Q/bBP3P1/4n3 w - -;bad EPD, Dann Corbitt
4q3/2p2K2/2N2N1B/r2R1kRr/3p4/2P3P1/p7/4Q3 w - -;bad EPD, Dann Corbitt
5Q2/Q5P1/1q4KP/1P6/p3Q3/P4NpB/1P3ppp/RQ3kqn w - -;bad EPD, Dann Corbitt
5b2/2Q5/2pBp3/1N1k1p2/5P2/2K1Pb2/2P1P1R1/1n1R3B w - -;bad EPD, Dann Corbitt
5b2/2pQ3n/3kP2p/2pP4/b1pKP3/8/8/8 w - -;bad EPD, Dann Corbitt
5b2/8/2p4q/1rB1nn2/pkR3r1/1NR5/1p4K1/1N4Q1 w - -;bad EPD, Dann Corbitt
6B1/Q3n1n1/P2B1P1p/Rpp3pP/1p1p1rKp/1k1B3P/3p4/BrN1R2q w - -;bad EPD, Dann Corbitt
6bn/1Np1Np2/4pPp1/Qp2P1Pp/kq2P2R/bp6/1p6/1Kr5 w - -;bad EPD, Dann Corbitt
7Q/8/8/6p1/2N3Bk/pp3n1P/r2B1RRP/qK3n2 w - -;bad EPD, Dann Corbitt
8/1K4Q1/3k3b/p6B/2N1bn2/3N3p/7p/6R1 w - -;bad EPD, Dann Corbitt
8/1p6/5p2/2pP1ppn/P1R4k/2pK2rp/n1N1P3/8 w - -;bad EPD, Dann Corbitt
8/1p6/K7/8/8/2B5/1k6/8 w - -;bad EPD, Dann Corbitt
8/1qn5/5B2/8/2R5/k5n1/2R1P1pp/1KQ3rr w - -;bad EPD, Dann Corbitt
8/2N5/1p6/bkp1p3/8/3pP3/Q1pK4/2N5 w - -;bad EPD, Dann Corbitt
8/3N4/8/p7/1B6/k1Q5/1R2p1p1/rnN1K1rb w - -;bad EPD, Dann Corbitt
8/3k1p2/4P3/8/8/6p1/3P1K2/4B3 w - -;bad EPD, Dann Corbitt
8/3p1pq1/2p2b2/8/8/1B2B3/4pk1p/K1Rr1r2 w - -;bad EPD, Dann Corbitt
8/3pp1bp/6pk/6Pp/1pp3rP/qK6/8/r7 w - -;bad EPD, Dann Corbitt
8/4q1p1/4p3/5rKP/4kn1R/4bRNQ/4P1PB/5rNn w - -;bad EPD, Dann Corbitt
8/8/1N2P2n/Q1NpB1p1/ppPk4/Kp2pPP1/1R2P3/8 w - -;bad EPD, Dann Corbitt
8/8/4P1B1/8/4Qp2/3K1P2/1P3P2/3rk1R1 w - -;bad EPD, Dann Corbitt
8/8/8/1k3q2/8/3K4/8/1Q6 w - -;bad EPD, Dann Corbitt
8/8/8/5R1p/7p/6pK/5Ppq/5rQk w - -;bad EPD, Dann Corbitt
8/8/8/8/RP5P/p6r/P5Qp/K5qk w - -;bad EPD, Dann Corbitt
8/B2p1P1r/B7/7q/8/8/1pN1PPP1/2RQ1k1K w - -;bad EPD, Dann Corbitt
8/P3p1Q1/4p2B/2bpPk2/3p2p1/R2b2N1/r1P1P3/KB1N4 w - -;bad EPD, Dann Corbitt
8/R2k4/8/8/8/6Q1/K3r3/5q2 w - -;bad EPD, Dann Corbitt
8/kq6/8/8/8/8/8/1K4Q1 w - -;bad EPD, Dann Corbitt
Kq5r/5BPn/3p2R1/5k2/2pP1p1R/5P2/5N2/1Q6 w - -;bad EPD, Dann Corbitt
Q4Q2/qk2P2n/NN4P1/1K4pb/Q5q1/qq6/4p3/2n5 w - -;bad EPD, Dann Corbitt
Q4k2/2N1Bp1P/4Kp1P/5P2/8/1P6/8/n1Rq4 w - -;bad EPD, Dann Corbitt
RBRqBb2/nQr2pp1/2pK1p2/1k6/bPpp2p1/1P1P2P1/3P4/8 w - -;bad EPD, Dann Corbitt
Rb3b2/p1Bp1Rp1/P1pK1p2/nb6/p1k4P/2PP2P1/B1P1P3/rN6 w - -;bad EPD, Dann Corbitt
bn2rb2/1rP4R/1KBkQP2/3P1n2/1N6/3NR3/8/6q1 w - -;bad EPD, Dann Corbitt
k5Rq/4RNP1/5PPK/6N1/5B1r/7B/8/8 w - -;bad EPD, Dann Corbitt
n5K1/7R/1pk2PN1/4N2R/PP2B3/1bP1P3/7B/8 w - -;bad EPD, Dann Corbitt
q1KNNn1n/8/8/8/8/2brrb2/1RR5/3QQkB1 w - -;bad EPD, Dann Corbitt
r2R1kb1/2R5/4P1B1/3pP3/1p1N4/1p2K1pN/1PP3Q1/n5br w - -;bad EPD, Dann Corbitt
r6k/1K2q1Q1/8/8/8/8/8/8 w - -;bad EPD, Dann Corbitt
r7/2N5/1bk5/p2P4/p1np4/B7/2K4r/8 w - -;bad EPD, Dann Corbitt
rnb1k2r/pppp1ppp/8/4Q3/4P3/4q3/P1P2PPP/4KB1R w - -;bad EPD, Dann Corbitt

1R6/1brk2p1/4p2p/p1P1Pp2/P7/6P1/1P4P1/2R3K1 w - - 0 1 bm b8b7; epd should be OK
4r1k1/p1qr1p2/2pb1Bp1/1p5p/3P1n1R/1B3P2/PP3PK1/2Q4R w - - 0 1 bm c1f4; epd should be OK
r1b2rk1/ppq1bppp/2p1pn2/8/2NP4/2N1P3/PP2BPPP/2RQK2R w K - 0 1 bm e1g1; epd should be OK
r3r1k1/1bppq2p/1p1n1bp1/3P1p2/p4N1P/P1PB1P2/1P1Q1BP1/1K1R3R w - - 0 1 bm g2g4; epd should be OK
r1bqrk2/pp1n1n1p/3p1p2/P1pP1P1Q/2PpP1NP/6R1/2PB4/4RBK1 w - - 0 1 bm h5f7; epd should be OK
rn1q1rk1/p2p1pb1/bp3np1/2pP2Bp/7P/P1N2N2/1PQ1PPP1/R3KB1R w KQ - 0 1 bm g2g4; epd should be OK
N7/8/2KQ2rp/6k1/3p3p/2p4P/4PP2/5N2 w - - 0 1 bm f2f4; epd should be OK
r1bqk2r/ppn1bppp/2n5/2p1p3/8/2NP1NP1/PP1BPPBP/R2Q1RK1 b kq - 0 1 bm e8g8; epd should be OK
8/2p2kb1/np1p2p1/pN1Pp1Pp/2P1P2P/1P6/P4BK1/8 w - - 0 1 bm b5c7; epd should be OK
4r1rk/p3qpp1/1pnp1n1p/5P2/P1PPP3/4Q2P/2BB2R1/6RK w - - 0 1 bm g2g7; epd should be OK
rn1q1rk1/pp2bppp/1n2p1b1/8/2pPP3/1BN1BP2/PP2N1PP/R2Q1RK1 w - - 0 1 bm b3c2; epd should be OK
r4r1k/1p4qp/b1pNRbp1/2P5/3p1P1P/PQ4P1/3R2BK/8 w - - 0 1 bm d2d1; epd should be OK
2b2r1k/4q2p/3p2pQ/2pBp3/8/6P1/1PP2P1P/R5K1 w - - 0 1 bm a1a7; epd should be OK
r2q1r2/1b2bpkp/n3p1p1/2ppP1P1/p6R/1PN1BQR1/NPP2P1P/4K3 w - - 0 1 bm f3f6; epd should be OK
r1b1k2r/ppppqppp/8/2bP4/3p4/6P1/PPQPPPBP/R1B2RK1 b kq - 0 1 bm e8g8; epd should be OK
rnbqkb1r/p3pppp/1p6/2ppP3/3N4/2P5/PPP1QPPP/R1B1KB1R w KQkq - 0 1 bm e5e6; epd should be OK

[bob]

Some comments: I ran 'em all through Crafty. Rather than posting a huge file, I had Crafty print out the number of the EPD string (line number in the file, ignoring blank lines)

EPD #8. I do not believe castling status should be "overridable". That's a pretty important part of a position. Crafty calls this an ERROR as opposed to just a WARNING.

EPD #9. I do not see how the 50 move counter can be "overridable". This is again an important part of the position, since (for Crafty at least) it is used to trigger a 50 move draw. 400 would be an instant draw if the side on move wanted. A human might overrun the 50 move if he wants, and I suppose both sides could ignore it. But a program?? unlikely except for the "mate on half-move 100" exception.

EPD #11. How can two rooks on same side of king be overridable (castling status?)

EPD #17 is outside the EPD specs. Crafty handles this correctly, but EPD spec has nothing to handle this, it is up to the engine. BTW this is an illegal position because the side not on move is in check. Note the WHITE bishop at C7.

EPD #20 (I won't comment on more of these). Bad castling status should not be "overridable" whatever that means. It needs to be right.

EPD #28 Adding the pawn removed a rook which generates a second error, wrong castling status.

EPD #31 is "out there" with both kings being in check in addition to missing move number warning. 31-91 have wrong king in check if I didn't overlook a good one.

The rest beyond those check out OK.

Did find 4 problem positions to debug. I ran this through my evtest mechanism, where I call Evaluate to get a score, then "flip" the board so that white pieces become black as they are moved swapping sides of the board. I also do a "flop" where I reflect left to right. So normal, flop, flip, flop to give me the same position but on different orientations. Each call to evaluate should be the same value. On four of 'em it was not meaning I have a small asymmetry problem in my eval to run down...

Off to do this for a bit...

OK. Done. Had two typos in a large connected pawn matrix. Was supposed to be 100% symmetric, but was not. Probably made zero difference in terms of Elo, but it didn't belong there.

[chrisw]

Well, it’s not supposed to be a list from god, just a list of EPDs which flag up some specific situations. Plus some ideas of what to do with EPDs that MIGHT crash your engine.

Override: each to his own, if the EPD is internally inconsistent (eg castling flags or Ep status), then programmer will decide, if he wants to flag the EPD as defective, or repair it. Some “errors” are reparable.

50 move rule, flag override if already beyond 100. Well, again, up to programmer to decide whether to overrule or not. Personally, I’ld assume an EPD typo and repair it. But again, better to do something because an engine may start exhibiting undesired behaviours if this counter is out of range.

EPD 11, still contains it’s 960 comment from, I think Lucas, the actual normal chess status of #11 is inconsistent castling flags, I think, without checking it.

Many of the EPDs are from Dann Corbitt’s list, mine flags them all up as ‘bad’ but I didn’t check why.

I’ll update the list for future use over the next few days. If anyone want to add any, feel free to append.

And, on exploding qsearch, mine still has it, even with the 18 or 20 rooks EPD, so some work for me there ...

Some comments: I ran 'em all through Crafty. Rather than posting a huge file, I had Crafty print out the number of the EPD string (line number in the file, ignoring blank lines)

EPD #8. I do not believe castling status should be "overridable". That's a pretty important part of a position. Crafty calls this an ERROR as opposed to just a WARNING.

EPD #9. I do not see how the 50 move counter can be "overridable". This is again an important part of the position, since (for Crafty at least) it is used to trigger a 50 move draw. 400 would be an instant draw if the side on move wanted. A human might overrun the 50 move if he wants, and I suppose both sides could ignore it. But a program?? unlikely except for the "mate on half-move 100" exception.

EPD #11. How can two rooks on same side of king be overridable (castling status?)

EPD #17 is outside the EPD specs. Crafty handles this correctly, but EPD spec has nothing to handle this, it is up to the engine. BTW this is an illegal position because the side not on move is in check. Note the WHITE bishop at C7.

EPD #20 (I won't comment on more of these). Bad castling status should not be "overridable" whatever that means. It needs to be right.

EPD #28 Adding the pawn removed a rook which generates a second error, wrong castling status.

EPD #31 is "out there" with both kings being in check in addition to missing move number warning. 31-91 have wrong king in check if I didn't overlook a good one.

The rest beyond those check out OK.

Did find 4 problem positions to debug. I ran this through my evtest mechanism, where I call Evaluate to get a score, then "flip" the board so that white pieces become black as they are moved swapping sides of the board. I also do a "flop" where I reflect left to right. So normal, flop, flip, flop to give me the same position but on different orientations. Each call to evaluate should be the same value. On four of 'em it was not meaning I have a small asymmetry problem in my eval to run down...

Off to do this for a bit...

OK. Done. Had two typos in a large connected pawn matrix. Was supposed to be 100% symmetric, but was not. Probably made zero difference in terms of Elo, but it didn't belong there.