I'm wondering how those of you who are developing your own chess engines are making sure they are error free. I initially thought it was fairly simple; however, after testing my engine's accuracy I constantly found errors. At first my engine could examine about 80,000 legal moves/sec-using an 8th gen i7 Intel chip, about 2.7GHz. I used 250 chess puzzles where the solution was known ranging from 3 - 9 move checkmates. I probably found around 1,000 errors over 11 months, bringing the speed of my engine down to a little under 20,000 legal moves/sec. It was a very humbling experience. I never imagined it would be that challenging.
So, my question is, what are you using to test your engines if its not chess puzzles, with known solution?
Test accuracy of chess engines
Moderators: hgm, Rebel, chrisw
-
- Posts: 313
- Joined: Tue Aug 03, 2021 2:41 pm
- Full name: Bill Beame
-
- Posts: 4833
- Joined: Sun Aug 10, 2008 3:15 pm
- Location: Philippines
Re: Test accuracy of chess engines
Play engine vs engine matches. If it is stronger it is more accurate.
Or test your engine with positions having more solutions and different points like in MEA.
Or test your engine with positions having more solutions and different points like in MEA.
-
- Posts: 1357
- Joined: Wed Mar 08, 2006 10:15 pm
- Location: San Francisco, California
Re: Test accuracy of chess engines
First of all, are you certain that your move generation is 100% correct? Implementing perft and testing on a lot of tricky positions could find any number of errors. Try the ones on the computerchess wiki, and then these as well:Chessnut1071 wrote: ↑Sun Aug 15, 2021 6:37 am I'm wondering how those of you who are developing your own chess engines are making sure they are error free. I initially thought it was fairly simple; however, after testing my engine's accuracy I constantly found errors. At first my engine could examine about 80,000 legal moves/sec-using an 8th gen i7 Intel chip, about 2.7GHz. I used 250 chess puzzles where the solution was known ranging from 3 - 9 move checkmates. I probably found around 1,000 errors over 11 months, bringing the speed of my engine down to a little under 20,000 legal moves/sec. It was a very humbling experience. I never imagined it would be that challenging.
So, my question is, what are you using to test your engines if its not chess puzzles, with known solution?
r3k2r/8/8/8/3pPp2/8/8/R3K1RR b KQkq e3 0 1 // perft(6) = 485,647,607
r3k2r/Pppp1ppp/1b3nbN/nP6/BBP1P3/q4N2/Pp1P2PP/R2Q1RK1 w kq - 0 1 // perft(6) = 706,045,033
8/7p/p5pb/4k3/P1pPn3/8/P5PP/1rB2RK1 b - d3 0 28 // perft(6) = 38,633,283
8/3K4/2p5/p2b2r1/5k2/8/8/1q6 b - - 1 67 // perft(7) = 493,407,574
rnbqkb1r/ppppp1pp/7n/4Pp2/8/8/PPPP1PPP/RNBQKBNR w KQkq f6 0 3 // perft(6) = 244,063,299
r3k2r/p1ppqpb1/bn2pnp1/3PN3/1p2P3/2N2Q1p/PPPBBPPP/R3K2R w KQkq - // Perft(5) = 193,690,690
8/p7/8/1P6/K1k3p1/6P1/7P/8 w - - // Perft(8) = 8,103,790
n1n5/PPPk4/8/8/8/8/4Kppp/5N1N b - - // Perft(6) = 71,179,139
r3k2r/p6p/8/B7/1pp1p3/3b4/P6P/R3K2R w KQkq - // Perft(6) = 77,054,993
8/2p5/3p4/KP5r/1R3p1k/8/4P1P1/8 w - - // Perft(7) = 178,633,661
8/5p2/8/2k3P1/p3K3/8/1P6/8 b - - // Perft(8) = 64,451,405
r3k2r/pb3p2/5npp/n2p4/1p1PPB2/6P1/P2N1PBP/R3K2R w KQkq - // Perft(5) = 29,179,893
Next, I'm quite concerned about your engine's speed. 20K moves/sec is extremely slow, as even most of the weaker engines can do at least 100K. Can you describe a bit about your engine's architecture? Language? Board representation? Basic search algorithm?
John Merlino - Myrddin chess engine