Are the perft results wrong on the CPW?

Discussion of chess software programming and technical issues.

Moderator: Ras

User avatar
phhnguyen
Posts: 1524
Joined: Wed Apr 21, 2010 4:58 am
Location: Australia
Full name: Nguyen Hong Pham

Re: Are the perft results wrong on the CPW?

Post by phhnguyen »

Sapling wrote: Sun Feb 09, 2025 9:53 am
phhnguyen wrote: Sun Feb 09, 2025 8:22 am
Clearly, the information in CPW was not updated from that page. CPW has some extra information such as Captures, E.p., Castles, Promotions which are not on the page.

This information was updated by Alessandro Iavicoli without pointing to the source. Perhaps, it is from his own information. If you know the source, I will update CPW.
If you're interested in adding more information to the table I've filled out full statistics to startpos perft(11) and i'm currently 20% into perft(12).
Yes, I will update CPW with them. Some information verifies previous ones and some will wait to be verified in the future. That is why I need clear sources of previous data.
Sapling wrote: Sun Feb 09, 2025 9:53 am Either way I do think there should be a note as to what the checks / double checks / discovery checks are actually counting in the CPW table as after perft(7) people get the wrong results because they are counting differently, even though their engine is generating moves correctly.
That is a problem for CPW editors: how can we update them?
BTW, what do you mean by "wrong" with "counting differently"? Wrong method or bug in coding? How do you know? Compare to what? Any third source data?
Sapling wrote: Sun Feb 09, 2025 9:53 am And I think this defeats the purpose of perft.
The information about checks, double checks... are extra ones for Perfts. They are fun and may be useful for someone. It is easy to remove them but not fun, IMHO!
Sapling wrote: Sun Feb 09, 2025 9:53 am This week I will also start filling out a similar table for Kiwipete.
IMO, extra information and/or too-depth Perft for Kiwipete positions or any non-starting ones are not useful since Perfts of them are for debugging purposes only. Of course, you could do what you want and someone may find them useful later!
https://banksiagui.com
The most features chess GUI, based on opensource Banksia - the chess tournament manager
Sapling
Posts: 21
Joined: Sun Oct 13, 2024 7:31 pm
Location: UK
Full name: Tim Jones

Re: Are the perft results wrong on the CPW?

Post by Sapling »

phhnguyen wrote: Wed Feb 12, 2025 9:42 am BTW, what do you mean by "wrong" with "counting differently"? Wrong method or bug in coding? How do you know? Compare to what? Any third source data?
They are correct (at least for the start pos & kiwipete), but many others have had problems with the double and discovered checks diverging from the results from their own engines.

Their move generation is correct, and the values in the table are correct. But they are showing different stats, and therefore lead to this confusion that there is some weird quirk with perft checks.

To quote my response earlier in the chat, this is what the CPW is showing
Checks as the sum of all checks of any kind (including single, double, discovered, mate)
Double Checks only count when it's NOT a mate
Discovery Checks only count when it's NOT a mate
But other people are are including double & discovered checks WHEN MATED in their perft results. I think this is additionally confusing because peoples results match the table exactly until they get to a depth where mated checks (depth 5 in kiwipete) occur, and thus their results differ

This confusion is even mentioned on the wiki itself for Kiwipete:
also known as Kiwipete by Peter McKenzie [7]. The number of double-checks in depth 5 is discussed in Talkchess [8] and may be 2645 instead of 2637.
I'll have to dig them out but i've seen quite a few posts where people are getting hung up on this discrepancy, they always end up shrugging and moving on, but I think the frustration can be mitigated by just mentioning mates don't count as a double / discovered check but do count as direct single checks. Or updating the stats to include all mates to align more with what engines naturally report.

For a third source check here:
https://wismuth.com/chess/statistics-games.html
The Grand Chess Tree - Distributed volunteer computing project
Sapling - 3380 ELO [CCRL] UCI chess engine
alessandro
Posts: 52
Joined: Tue Aug 12, 2014 11:21 am
Location: Lund
Full name: Alessandro Iavicoli

Re: Are the perft results wrong on the CPW?

Post by alessandro »

Sapling wrote: Wed Jan 29, 2025 7:05 pm First time poster here, when I was building Sapling I really enjoyed coding & optimizing the move generator, ofc it's bearing on the strength of an engine is very minimal but, Afterall I was building the engine primarily for fun.

I've just started a new project called 'The Grand Chess Tree' - I'll create another post about it when it's in a more polished state, but suffice to say it's a specialized move generation engine.

So far I've traversed to perft(9), but I'm noticing from depth 7 the discovered checks and mates are not matching the results in the CPW for the initial position

depth(7) all match except mates:
mine vs cpw
mates: 435,765 vs 435,767
then for depth(8) the discovered start to diverge also:
discovered: 847,043 vs 847,039
mates: 9,851,894 vs 9,852,036

One thing to note is that the checks, discovered checks and mates are all calculated separately from the move generation, and all other counts seem to match which indicates a miss match in the classification of a leaf node.

I'm wondering if anyone else has ran a perft with discovered & mate counts past depth 7 so I can confer?
AdaChess can generate those stats also via the divide command, that you can use to explore moves and lines where the node counters doesn't match.
--
AdaChess - Smart Chess Engine - https://github.com/adachess/AdaChess

Image