No standard specification for Perft

[Michael Sherwin]

I went to the chess programming wiki to see if perft has been standardized. It is not. It can be what any author wants it to be. That means the perft of one author's engine may not correlate well to that of another author's engine. If we are going to use a common command name to measure performance it should mean the same thing on all engines that comply with the "standard". For example in my engines I make and unmake all generated moves because it makes no sense to me to have the performance of the move generator skewed by depth - 1 make/unmake functions. So if an author wants to just get the performance of the move generator then it should be done by iterating through a list of fen positions and summing the time spent in the move generator. So what should perft measure, mg performance or search accuracy and search performance? If the later then possibly the captures only should be tacked onto the leafs. Afterall in any serious engine there is a qsearch. And the qsearch is what really matters the most when when evaluating performance of the system minus the evaluation.

[xr_a_y]

If the move generator only generates pseudo valid moves, then leaf node have to be validated and thus probably make/validated/unmake or copy/make/validated.

Am I wrong ?

[Robert Pope]

Perft is intended to measure/validate move generation and tree travel accuracy. Everything else is stuff that other people have decided to measure about it, but really aren't its focus. It really isn't a performance metric. And qsearch is entirely irrelevant to it.

[abulmo2]

perft is a debugging tool, and the important thing is the number it returns, not the way you get it. If bulk counting makes debugging faster, I wonder what is wrong with it?

[Michael Sherwin]

If the move generator only generates pseudo valid moves, then leaf node have to be validated and thus probably make/validated/unmake or copy/make/validated.

Am I wrong ?

That is true. However, the wiki says to only generate legal moves.

[Michael Sherwin]

Robert Pope & abulmo2, no argument from me on that except it should be called Accut for accuracy test instead of Perft for performance test. Apparently, whomever first wrote Perft was concerned with raw performance with an accurate count as a side effect bonus that became the focus for some. So, it is those that came after, that redefined what the purpose of Perft is.

[Robert Pope]

Robert Pope & abulmo2, no argument from me on that except it should be called Accut for accuracy test instead of Perft for performance test. Apparently, whomever first wrote Perft was concerned with raw performance with an accurate count as a side effect bonus that became the focus for some. So, it is those that came after, that redefined what the purpose of Perft is.

No, it was the opposite. Perft first showed up in Crafty, as a way to validate that the Movegen/Make/Unmake was done properly. Other engines copied the function, since it is a useful feature to have. It was other people who came after and were obsessed with maximizing perft "speed" and such, so they started doing things like hashing subtrees to speed it up.

And, if you invent a command for your engine, you're the one who gets to decide what it is named.

[Robert Pope]

If the move generator only generates pseudo valid moves, then leaf node have to be validated and thus probably make/validated/unmake or copy/make/validated.

Am I wrong ?

That is true. However, the wiki says to only generate legal moves.

It doesn't say to only generate legal moves, it says to only count legal moves. Make pseudolegal and validate is a perfectly fine way to do that, as the wiki explicitly describes.

[Michael Sherwin]

Robert Pope & abulmo2, no argument from me on that except it should be called Accut for accuracy test instead of Perft for performance test. Apparently, whomever first wrote Perft was concerned with raw performance with an accurate count as a side effect bonus that became the focus for some. So, it is those that came after, that redefined what the purpose of Perft is.

No, it was the opposite. Perft first showed up in Crafty, as a way to validate that the Movegen/Make/Unmake was done properly. Other engines copied the function, since it is a useful feature to have. It was other people who came after and were obsessed with maximizing perft "speed" and such, so they started doing things like hashing subtrees to speed it up.

And, if you invent a command for your engine, you're the one who gets to decide what it is named.

From the wiki.

Supposably, perft was first implemented within the Cobol program RSCE-1 by R.C. Smith, submitted to the USCF for evaluation, and subject of an 1978 Computerworld article

[Robert Pope]

Huh. Though if you read the actual article about RCSE-1, all it says is that it calculated the number of depth 3 leaf positions at 8902, which is equivalent to what perft(3) does, not that they invented or implemented the perft command.

Somebody back in 1500 wrote down all 20 legal opening moves. That doesn't mean they invented perft(1).

Maybe someone with actual historical experience can correct me, but I'm pretty sure the actual command was first implemented in Crafty, not not RCSE-1 or Belle.