OpenCL perft() Technical Issues

Discussion of chess software programming and technical issues.

Moderator: Ras

User avatar
sje
Posts: 4675
Joined: Mon Mar 13, 2006 7:43 pm

Revisiting WAC

Post by sje »

Revisiting WAC

WAC is a test suite which uses the set of 300 positions taken from the book Win at Chess. A FEN file with these positions can be found at: https://dl.dropboxusercontent.com/u/316 ... EN/WAC.fen

Oscar will need the capability of processing work unit files made from FEN records, so to that end I've installed a first attempt at automated FEN batch file processing into the program. The initial test file is WAC.fen as referenced above, and the test is to calculate the perft(n) for each position and report the total of these calculations for each depth.

For depths from zero to five, the results are:

Code: Select all

Record count: 300   Depth: 0   Total: 300
Record count: 300   Depth: 1   Total: 11962
Record count: 300   Depth: 2   Total: 393416
Record count: 300   Depth: 3   Total: 15796091
Record count: 300   Depth: 4   Total: 551178617
Record count: 300   Depth: 5   Total: 22363709083
I believe that these numbers are correct, but an independent confirmation would certainly be appreciated.
Top
User avatar
Ajedrecista
Posts: 2087
Joined: Wed Jul 13, 2011 9:04 pm
Location: Madrid, Spain.

Re: Revisiting WAC.

Post by Ajedrecista »

Hello Steven:
sje wrote:Revisiting WAC

WAC is a test suite which uses the set of 300 positions taken from the book Win at Chess. A FEN file with these positions can be found at: https://dl.dropboxusercontent.com/u/316 ... EN/WAC.fen

Oscar will need the capability of processing work unit files made from FEN records, so to that end I've installed a first attempt at automated FEN batch file processing into the program. The initial test file is WAC.fen as referenced above, and the test is to calculate the perft(n) for each position and report the total of these calculations for each depth.

For depths from zero to five, the results are:

Code: Select all

Record count: 300   Depth: 0   Total: 300
Record count: 300   Depth: 1   Total: 11962
Record count: 300   Depth: 2   Total: 393416
Record count: 300   Depth: 3   Total: 15796091
Record count: 300   Depth: 4   Total: 551178617
Record count: 300   Depth: 5   Total: 22363709083
I believe that these numbers are correct, but an independent confirmation would certainly be appreciated.
Please take a look at this old thread:

Looking for suggestions for perft test positions

Specially, take a glance at this Richard Vida's post:

Re: Another idea for perft position sources
rvida wrote:depth 1: 11962
depth 2: 393416
depth 3: 15784129
depth 4: 550761277
depth 5: 22347090274
Your results differ with depth > 2 (you noted it at that time). Bad thing. But now you get the same sum for depth 5 than in 2011:

Re: Another idea for perft position sources

Marcel van Kervinck suggested you a set of more than 6000 positions:

Re: Looking for suggestions for perft test positions

And Tom Likens gave his results on this 6000+ positions:

Re: Looking for suggestions for perft test positions

In reality, there are 6838 positions like it is said here, here and here.

By the way, Tom's and Marcel's results matched, as Tom pointed out. That thread could be a gold mine for your interests. Good luck!

Regards from Spain.

Ajedrecista.
Top
User avatar
sje
Posts: 4675
Joined: Mon Mar 13, 2006 7:43 pm

Re: Revisiting WAC.

Post by sje »

Yes, I did recall my earlier perft() runs on WAC but couldn't locate the results.

To check and debug differences between the outputs of different programs requires the use of a common format. I will be using the existing work unit format; it's easy to generate and easy to parse. Because all the data is generated by the same formatting routine, a simple text comparison between two files is sufficient to spot any differences.

To recap:

Code: Select all

Work unit input (7 fields): FEN occurrences
Work unit output (9 fields): FEN occurrences perft(7) occurrences*perft(7)
Soon, I'll have Oscar process the 68 record work unit wu7.964; the output should match Symbolic's output byte-for-byte.

Since Oscar will be made open source under the GPL, others may use it to debug their programs -- or to spot bugs in Oscar itself.
Top
User avatar
Ajedrecista
Posts: 2087
Joined: Wed Jul 13, 2011 9:04 pm
Location: Madrid, Spain.

Re: Revisiting WAC.

Post by Ajedrecista »

Hello again:
sje wrote:Yes, I did recall my earlier perft() runs on WAC but couldn't locate the results.
I checked the first 30 positions of WAC with JetChess (only depth 3) and all the results agree with Richard's results. I assume that JetChess is right so I recommend you to check each one of the first 30 positions and if your results do not match with the first 30 Richard's results at depth 3 (also my results), then you have a problem. Ideally, it would be better to check all the positions but I have better things to do that check manually each position (it is the only way I know to do it).

Furthermore, I isolated depth 5 results by Richard and sum them. I obtained the same total than him. I mean, he did not go wrong in the sum.

I am not claiming anybody is right, only that the first 30 results at depth 3 seem correct in Richard's post. Good luck, Steven.

Regards from Spain.

Ajedrecista.
Top
User avatar
sje
Posts: 4675
Joined: Mon Mar 13, 2006 7:43 pm

... and its pal, WCSAC

Post by sje »

WCSAC is a set of 1,001 positions taken from the book 1,001 Winning Chess Sacrifices and Combinations. The FEN file can be found here: https://dl.dropboxusercontent.com/u/316 ... /WCSAC.fen

Oscar says:

Code: Select all

Record count: 1001   Depth: 0   Total: 1001
Record count: 1001   Depth: 1   Total: 40745
Record count: 1001   Depth: 2   Total: 1345186
Record count: 1001   Depth: 3   Total: 55205442
Record count: 1001   Depth: 4   Total: 1934097171
Record count: 1001   Depth: 5   Total: 80370577863
If your program says something else, then please post the details.
Top
User avatar
sje
Posts: 4675
Joined: Mon Mar 13, 2006 7:43 pm

Re: Revisiting WAC.

Post by sje »

Ajedrecista wrote:I checked the first 30 positions of WAC with JetChess (only depth 3) and all the results agree with Richard's results.
Here's what Oscar has to say (WAC 1-30, depth=3):

Code: Select all

2rr3k/pp3pp1/1nnqbN1p/3pN3/2pP4/2P3Q1/PPB4P/R4RK1 w - - 0 1 109684
8/7p/5k2/5p2/p1p2P2/Pr1pPK2/1P1R3P/8 b - - 0 1 3587
5rk1/1ppb3p/p1pb4/6q1/3P1p1r/2P1R2P/PP1BQ1P1/5RKN w - - 0 1 73374
r1bq2rk/pp3pbp/2p1p1pQ/7P/3P4/2PB1N2/PP3PPR/2KR4 w - - 0 1 53839
5k2/6pp/p1qN4/1p1p4/3P4/2PKP2Q/PP3r2/3R4 b - - 0 1 29743
7k/p7/1R5K/6r1/6p1/6P1/8/8 w - - 0 1 1867
rnbqkb1r/pppp1ppp/8/4P3/6n1/7P/PPPNPPP1/R1BQKBNR b KQkq - 0 1 24285
r4q1k/p2bR1rp/2p2Q1N/5p2/5p2/2P5/PP3PPP/R5K1 w - - 0 1 36726
3q1rk1/p4pp1/2pb3p/3p4/6Pr/1PNQ4/P1PB1PP1/4RRK1 b - - 0 1 55363
2br2k1/2q3rn/p2NppQ1/2p1P3/Pp5R/4P3/1P3PPP/3R2K1 w - - 0 1 65915
r1b1kb1r/3q1ppp/pBp1pn2/8/Np3P2/5B2/PPP3PP/R2Q1RK1 w kq - 0 1 49802
4k1r1/2p3r1/1pR1p3/3pP2p/3P2qP/P4N2/1PQ4P/5R1K b - - 0 1 24494
5rk1/pp4p1/2n1p2p/2Npq3/2p5/6P1/P3P1BP/R4Q1K w - - 0 1 52666
r2rb1k1/pp1q1p1p/2n1p1p1/2bp4/5P2/PP1BPR1Q/1BPN2PP/R5K1 w - - 0 1 67039
1R6/1brk2p1/4p2p/p1P1Pp2/P7/6P1/1P4P1/2R3K1 w - - 0 1 8851
r4rk1/ppp2ppp/2n5/2bqp3/8/P2PB3/1PP1NPPP/R2Q1RK1 w - - 0 1 46564
1k5r/pppbn1pp/4q1r1/1P3p2/2NPp3/1QP5/P4PPP/R1B1R1K1 w - - 0 1 61453
R7/P4k2/8/8/8/8/r7/6K1 w - - 0 1 2335
r1b2rk1/ppbn1ppp/4p3/1QP4q/3P4/N4N2/5PPP/R1B2RK1 w - - 0 1 58230
r2qkb1r/1ppb1ppp/p7/4p3/P1Q1P3/2P5/5PPP/R1B2KNR b kq - 0 1 41756
5rk1/1b3p1p/pp3p2/3n1N2/1P6/P1qB1PP1/3Q3P/4R1K1 w - - 0 1 69971
r1bqk2r/ppp1nppp/4p3/n5N1/2BPp3/P1P5/2P2PPP/R1BQK2R w KQkq - 0 1 49617
r3nrk1/2p2p1p/p1p1b1p1/2NpPq2/3R4/P1N1Q3/1PP2PPP/4R1K1 w - - 0 1 77371
6k1/1b1nqpbp/pp4p1/5P2/1PN5/4Q3/P5PP/1B2B1K1 b - - 0 1 80294
3R1rk1/8/5Qpp/2p5/2P1p1q1/P3P3/1P2PK2/8 b - - 0 1 5267
3r2k1/1p1b1pp1/pq5p/8/3NR3/2PQ3P/PP3PP1/6K1 b - - 0 1 56584
7k/pp4np/2p3p1/3pN1q1/3P4/Q7/1r3rPP/2R2RK1 w - - 0 1 72484
1r1r2k1/4pp1p/2p1b1p1/p3R3/RqBP4/4P3/1PQ2PPP/6K1 b - - 0 1 69183
r2q2k1/pp1rbppp/4pn2/2P5/1P3B2/6P1/P3QPBP/1R3RK1 w - - 0 1 81710
1r3r2/4q1kp/b1pp2p1/5p2/pPn1N3/6P1/P3PPBP/2QRR1K1 w - - 0 1 64525
Do any of these differ from your values?
Top
petero2
Posts: 722
Joined: Mon Apr 19, 2010 7:07 pm
Location: Sweden
Full name: Peter Osterlund

Re: Revisiting WAC

Post by petero2 »

sje wrote:For depths from zero to five, the results are:

Code: Select all

Record count: 300   Depth: 0   Total: 300
Record count: 300   Depth: 1   Total: 11962
Record count: 300   Depth: 2   Total: 393416
Record count: 300   Depth: 3   Total: 15796091
Record count: 300   Depth: 4   Total: 551178617
Record count: 300   Depth: 5   Total: 22363709083
I believe that these numbers are correct, but an independent confirmation would certainly be appreciated.
I got the same results and also:

Code: Select all

Depth: 6   Total: 809264359148
Depth: 7   Total: 33043845819034
Top
User avatar
sje
Posts: 4675
Joined: Mon Mar 13, 2006 7:43 pm

Re: Revisiting WAC.

Post by sje »

For Oscar's complete output for WAC depth=3, see: https://dl.dropboxusercontent.com/u/316 ... N/WAC3.sum
Top
User avatar
sje
Posts: 4675
Joined: Mon Mar 13, 2006 7:43 pm

Re: Revisiting WAC

Post by sje »

petero2 wrote:I got the same results and also:

Code: Select all

Depth: 6   Total: 809264359148
Depth: 7   Total: 33043845819034
Good.

Back in the Old Days, some workers would edit the position data in the test suites for various reasons, but neglect to inform others of the changes. So different versions of the same suite would make any calculation results hazardous to compare. Now we have the web and can be more sure of using canonical data.
Top
petero2
Posts: 722
Joined: Mon Apr 19, 2010 7:07 pm
Location: Sweden
Full name: Peter Osterlund

Re: ... and its pal, WCSAC

Post by petero2 »

sje wrote:

Code: Select all

Record count: 1001   Depth: 0   Total: 1001
Record count: 1001   Depth: 1   Total: 40745
Record count: 1001   Depth: 2   Total: 1345186
Record count: 1001   Depth: 3   Total: 55205442
Record count: 1001   Depth: 4   Total: 1934097171
Record count: 1001   Depth: 5   Total: 80370577863
I got the same results and also:

Code: Select all

Depth: 6   Total: 2921082171758
Top

Return to “Computer Chess Club: Programming and Technical Discussions”