No standard specification for Perft

Discussion of chess software programming and technical issues.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Michael Sherwin
Posts: 3191
Joined: Fri May 26, 2006 1:00 am
Location: WY, USA
Full name: Michael Sherwin

No standard specification for Perft

Post by Michael Sherwin » Fri Apr 19, 2019 3:06 pm

I went to the chess programming wiki to see if perft has been standardized. It is not. It can be what any author wants it to be. That means the perft of one author's engine may not correlate well to that of another author's engine. If we are going to use a common command name to measure performance it should mean the same thing on all engines that comply with the "standard". For example in my engines I make and unmake all generated moves because it makes no sense to me to have the performance of the move generator skewed by depth - 1 make/unmake functions. So if an author wants to just get the performance of the move generator then it should be done by iterating through a list of fen positions and summing the time spent in the move generator. So what should perft measure, mg performance or search accuracy and search performance? If the later then possibly the captures only should be tacked onto the leafs. Afterall in any serious engine there is a qsearch. And the qsearch is what really matters the most when when evaluating performance of the system minus the evaluation.
If you are on a sidewalk and the covid goes beep beep
Just step aside or you might have a bit of heat
Covid covid runs through the town all day
Can the people ever change their ways
Sherwin the covid's after you
Sherwin if it catches you you're through

User avatar
xr_a_y
Posts: 1104
Joined: Sat Nov 25, 2017 1:28 pm
Location: France

Re: No standard specification for Perft

Post by xr_a_y » Fri Apr 19, 2019 4:26 pm

If the move generator only generates pseudo valid moves, then leaf node have to be validated and thus probably make/validated/unmake or copy/make/validated.

Am I wrong ?

Robert Pope
Posts: 516
Joined: Sat Mar 25, 2006 7:27 pm

Re: No standard specification for Perft

Post by Robert Pope » Fri Apr 19, 2019 4:27 pm

Perft is intended to measure/validate move generation and tree travel accuracy. Everything else is stuff that other people have decided to measure about it, but really aren't its focus. It really isn't a performance metric. And qsearch is entirely irrelevant to it.

abulmo2
Posts: 253
Joined: Fri Dec 16, 2016 10:04 am
Contact:

Re: No standard specification for Perft

Post by abulmo2 » Fri Apr 19, 2019 4:52 pm

perft is a debugging tool, and the important thing is the number it returns, not the way you get it. If bulk counting makes debugging faster, I wonder what is wrong with it?
Richard Delorme

Michael Sherwin
Posts: 3191
Joined: Fri May 26, 2006 1:00 am
Location: WY, USA
Full name: Michael Sherwin

Re: No standard specification for Perft

Post by Michael Sherwin » Fri Apr 19, 2019 5:38 pm

xr_a_y wrote:
Fri Apr 19, 2019 4:26 pm
If the move generator only generates pseudo valid moves, then leaf node have to be validated and thus probably make/validated/unmake or copy/make/validated.

Am I wrong ?
That is true. However, the wiki says to only generate legal moves.
If you are on a sidewalk and the covid goes beep beep
Just step aside or you might have a bit of heat
Covid covid runs through the town all day
Can the people ever change their ways
Sherwin the covid's after you
Sherwin if it catches you you're through

Michael Sherwin
Posts: 3191
Joined: Fri May 26, 2006 1:00 am
Location: WY, USA
Full name: Michael Sherwin

Re: No standard specification for Perft

Post by Michael Sherwin » Fri Apr 19, 2019 5:46 pm

Robert Pope & abulmo2, no argument from me on that except it should be called Accut for accuracy test instead of Perft for performance test. Apparently, whomever first wrote Perft was concerned with raw performance with an accurate count as a side effect bonus that became the focus for some. So, it is those that came after, that redefined what the purpose of Perft is.
If you are on a sidewalk and the covid goes beep beep
Just step aside or you might have a bit of heat
Covid covid runs through the town all day
Can the people ever change their ways
Sherwin the covid's after you
Sherwin if it catches you you're through

Robert Pope
Posts: 516
Joined: Sat Mar 25, 2006 7:27 pm

Re: No standard specification for Perft

Post by Robert Pope » Fri Apr 19, 2019 6:00 pm

Michael Sherwin wrote:
Fri Apr 19, 2019 5:46 pm
Robert Pope & abulmo2, no argument from me on that except it should be called Accut for accuracy test instead of Perft for performance test. Apparently, whomever first wrote Perft was concerned with raw performance with an accurate count as a side effect bonus that became the focus for some. So, it is those that came after, that redefined what the purpose of Perft is.
No, it was the opposite. Perft first showed up in Crafty, as a way to validate that the Movegen/Make/Unmake was done properly. Other engines copied the function, since it is a useful feature to have. It was other people who came after and were obsessed with maximizing perft "speed" and such, so they started doing things like hashing subtrees to speed it up.

And, if you invent a command for your engine, you're the one who gets to decide what it is named. :)

Robert Pope
Posts: 516
Joined: Sat Mar 25, 2006 7:27 pm

Re: No standard specification for Perft

Post by Robert Pope » Fri Apr 19, 2019 6:04 pm

Michael Sherwin wrote:
Fri Apr 19, 2019 5:38 pm
xr_a_y wrote:
Fri Apr 19, 2019 4:26 pm
If the move generator only generates pseudo valid moves, then leaf node have to be validated and thus probably make/validated/unmake or copy/make/validated.

Am I wrong ?
That is true. However, the wiki says to only generate legal moves.
It doesn't say to only generate legal moves, it says to only count legal moves. Make pseudolegal and validate is a perfectly fine way to do that, as the wiki explicitly describes.

Michael Sherwin
Posts: 3191
Joined: Fri May 26, 2006 1:00 am
Location: WY, USA
Full name: Michael Sherwin

Re: No standard specification for Perft

Post by Michael Sherwin » Fri Apr 19, 2019 6:56 pm

Robert Pope wrote:
Fri Apr 19, 2019 6:00 pm
Michael Sherwin wrote:
Fri Apr 19, 2019 5:46 pm
Robert Pope & abulmo2, no argument from me on that except it should be called Accut for accuracy test instead of Perft for performance test. Apparently, whomever first wrote Perft was concerned with raw performance with an accurate count as a side effect bonus that became the focus for some. So, it is those that came after, that redefined what the purpose of Perft is.
No, it was the opposite. Perft first showed up in Crafty, as a way to validate that the Movegen/Make/Unmake was done properly. Other engines copied the function, since it is a useful feature to have. It was other people who came after and were obsessed with maximizing perft "speed" and such, so they started doing things like hashing subtrees to speed it up.

And, if you invent a command for your engine, you're the one who gets to decide what it is named. :)
From the wiki.
Supposably, perft was first implemented within the Cobol program RSCE-1 by R.C. Smith, submitted to the USCF for evaluation, and subject of an 1978 Computerworld article
If you are on a sidewalk and the covid goes beep beep
Just step aside or you might have a bit of heat
Covid covid runs through the town all day
Can the people ever change their ways
Sherwin the covid's after you
Sherwin if it catches you you're through

Robert Pope
Posts: 516
Joined: Sat Mar 25, 2006 7:27 pm

Re: No standard specification for Perft

Post by Robert Pope » Fri Apr 19, 2019 7:34 pm

Huh. Though if you read the actual article about RCSE-1, all it says is that it calculated the number of depth 3 leaf positions at 8902, which is equivalent to what perft(3) does, not that they invented or implemented the perft command.

Somebody back in 1500 wrote down all 20 legal opening moves. That doesn't mean they invented perft(1).

Maybe someone with actual historical experience can correct me, but I'm pretty sure the actual command was first implemented in Crafty, not not RCSE-1 or Belle.

Post Reply