No standard specification for Perft
Moderators: hgm, Rebel, chrisw
-
- Posts: 3196
- Joined: Fri May 26, 2006 3:00 am
- Location: WY, USA
- Full name: Michael Sherwin
No standard specification for Perft
I went to the chess programming wiki to see if perft has been standardized. It is not. It can be what any author wants it to be. That means the perft of one author's engine may not correlate well to that of another author's engine. If we are going to use a common command name to measure performance it should mean the same thing on all engines that comply with the "standard". For example in my engines I make and unmake all generated moves because it makes no sense to me to have the performance of the move generator skewed by depth - 1 make/unmake functions. So if an author wants to just get the performance of the move generator then it should be done by iterating through a list of fen positions and summing the time spent in the move generator. So what should perft measure, mg performance or search accuracy and search performance? If the later then possibly the captures only should be tacked onto the leafs. Afterall in any serious engine there is a qsearch. And the qsearch is what really matters the most when when evaluating performance of the system minus the evaluation.
If you are on a sidewalk and the covid goes beep beep
Just step aside or you might have a bit of heat
Covid covid runs through the town all day
Can the people ever change their ways
Sherwin the covid's after you
Sherwin if it catches you you're through
Just step aside or you might have a bit of heat
Covid covid runs through the town all day
Can the people ever change their ways
Sherwin the covid's after you
Sherwin if it catches you you're through
-
- Posts: 1871
- Joined: Sat Nov 25, 2017 2:28 pm
- Location: France
Re: No standard specification for Perft
If the move generator only generates pseudo valid moves, then leaf node have to be validated and thus probably make/validated/unmake or copy/make/validated.
Am I wrong ?
Am I wrong ?
-
- Posts: 558
- Joined: Sat Mar 25, 2006 8:27 pm
Re: No standard specification for Perft
Perft is intended to measure/validate move generation and tree travel accuracy. Everything else is stuff that other people have decided to measure about it, but really aren't its focus. It really isn't a performance metric. And qsearch is entirely irrelevant to it.
-
- Posts: 433
- Joined: Fri Dec 16, 2016 11:04 am
- Location: France
- Full name: Richard Delorme
Re: No standard specification for Perft
perft is a debugging tool, and the important thing is the number it returns, not the way you get it. If bulk counting makes debugging faster, I wonder what is wrong with it?
Richard Delorme
-
- Posts: 3196
- Joined: Fri May 26, 2006 3:00 am
- Location: WY, USA
- Full name: Michael Sherwin
Re: No standard specification for Perft
That is true. However, the wiki says to only generate legal moves.
If you are on a sidewalk and the covid goes beep beep
Just step aside or you might have a bit of heat
Covid covid runs through the town all day
Can the people ever change their ways
Sherwin the covid's after you
Sherwin if it catches you you're through
Just step aside or you might have a bit of heat
Covid covid runs through the town all day
Can the people ever change their ways
Sherwin the covid's after you
Sherwin if it catches you you're through
-
- Posts: 3196
- Joined: Fri May 26, 2006 3:00 am
- Location: WY, USA
- Full name: Michael Sherwin
Re: No standard specification for Perft
Robert Pope & abulmo2, no argument from me on that except it should be called Accut for accuracy test instead of Perft for performance test. Apparently, whomever first wrote Perft was concerned with raw performance with an accurate count as a side effect bonus that became the focus for some. So, it is those that came after, that redefined what the purpose of Perft is.
If you are on a sidewalk and the covid goes beep beep
Just step aside or you might have a bit of heat
Covid covid runs through the town all day
Can the people ever change their ways
Sherwin the covid's after you
Sherwin if it catches you you're through
Just step aside or you might have a bit of heat
Covid covid runs through the town all day
Can the people ever change their ways
Sherwin the covid's after you
Sherwin if it catches you you're through
-
- Posts: 558
- Joined: Sat Mar 25, 2006 8:27 pm
Re: No standard specification for Perft
No, it was the opposite. Perft first showed up in Crafty, as a way to validate that the Movegen/Make/Unmake was done properly. Other engines copied the function, since it is a useful feature to have. It was other people who came after and were obsessed with maximizing perft "speed" and such, so they started doing things like hashing subtrees to speed it up.Michael Sherwin wrote: ↑Fri Apr 19, 2019 7:46 pm Robert Pope & abulmo2, no argument from me on that except it should be called Accut for accuracy test instead of Perft for performance test. Apparently, whomever first wrote Perft was concerned with raw performance with an accurate count as a side effect bonus that became the focus for some. So, it is those that came after, that redefined what the purpose of Perft is.
And, if you invent a command for your engine, you're the one who gets to decide what it is named.
-
- Posts: 558
- Joined: Sat Mar 25, 2006 8:27 pm
Re: No standard specification for Perft
It doesn't say to only generate legal moves, it says to only count legal moves. Make pseudolegal and validate is a perfectly fine way to do that, as the wiki explicitly describes.Michael Sherwin wrote: ↑Fri Apr 19, 2019 7:38 pmThat is true. However, the wiki says to only generate legal moves.
-
- Posts: 3196
- Joined: Fri May 26, 2006 3:00 am
- Location: WY, USA
- Full name: Michael Sherwin
Re: No standard specification for Perft
From the wiki.Robert Pope wrote: ↑Fri Apr 19, 2019 8:00 pmNo, it was the opposite. Perft first showed up in Crafty, as a way to validate that the Movegen/Make/Unmake was done properly. Other engines copied the function, since it is a useful feature to have. It was other people who came after and were obsessed with maximizing perft "speed" and such, so they started doing things like hashing subtrees to speed it up.Michael Sherwin wrote: ↑Fri Apr 19, 2019 7:46 pm Robert Pope & abulmo2, no argument from me on that except it should be called Accut for accuracy test instead of Perft for performance test. Apparently, whomever first wrote Perft was concerned with raw performance with an accurate count as a side effect bonus that became the focus for some. So, it is those that came after, that redefined what the purpose of Perft is.
And, if you invent a command for your engine, you're the one who gets to decide what it is named.
Supposably, perft was first implemented within the Cobol program RSCE-1 by R.C. Smith, submitted to the USCF for evaluation, and subject of an 1978 Computerworld article
If you are on a sidewalk and the covid goes beep beep
Just step aside or you might have a bit of heat
Covid covid runs through the town all day
Can the people ever change their ways
Sherwin the covid's after you
Sherwin if it catches you you're through
Just step aside or you might have a bit of heat
Covid covid runs through the town all day
Can the people ever change their ways
Sherwin the covid's after you
Sherwin if it catches you you're through
-
- Posts: 558
- Joined: Sat Mar 25, 2006 8:27 pm
Re: No standard specification for Perft
Huh. Though if you read the actual article about RCSE-1, all it says is that it calculated the number of depth 3 leaf positions at 8902, which is equivalent to what perft(3) does, not that they invented or implemented the perft command.
Somebody back in 1500 wrote down all 20 legal opening moves. That doesn't mean they invented perft(1).
Maybe someone with actual historical experience can correct me, but I'm pretty sure the actual command was first implemented in Crafty, not not RCSE-1 or Belle.
Somebody back in 1500 wrote down all 20 legal opening moves. That doesn't mean they invented perft(1).
Maybe someone with actual historical experience can correct me, but I'm pretty sure the actual command was first implemented in Crafty, not not RCSE-1 or Belle.