What is a good perft speed?

[imnoumni]

Hi everyone. I got interested in chess programming and spent the past few weeks on perft.

I have it generating the correct values and wanted know what "fast" is. I don't know what to benchmark against so downloaded Stockfish 16 AVX2 and got the following:

Code: Select all

$ echo "go perft 7" | eval time stockfish-ubuntu-x86-64-avx2
Stockfish 16 by the Stockfish developers (see AUTHORS file)
... moves ...
Nodes searched: 3195901860

real    0m13.746s
user    0m13.717s
sys     0m0.028s

I ran my own perft. It's single-threaded, no hashing. Just pumping through the move generator and counting legal moves.

Code: Select all

$ time ./main divide startpos 7                                                                                                                        
... moves ...
perft 3195901860 nps 639087301 time 5.000728

real    0m5.003s
user    0m5.003s
sys     0m0.000s

So my program takes roughly 37% of the time that Stockfish does to calculate perft of startpos to depth 7. Is this "good" ? Stockfish does a lot more that my code doesn't do, I'm sure.

Not sure how to compare.

[kranium]

Hi everyone. I got interested in chess programming and spent the past few weeks on perft.

I have it generating the correct values and wanted know what "fast" is. I don't know what to benchmark against so downloaded Stockfish 16 AVX2 and got the following:

Code: Select all

$ echo "go perft 7" | eval time stockfish-ubuntu-x86-64-avx2
Stockfish 16 by the Stockfish developers (see AUTHORS file)
... moves ...
Nodes searched: 3195901860

real    0m13.746s
user    0m13.717s
sys     0m0.028s

I ran my own perft. It's single-threaded, no hashing. Just pumping through the move generator and counting legal moves.

Code: Select all

$ time ./main divide startpos 7                                                                                                                        
... moves ...
perft 3195901860 nps 639087301 time 5.000728

real    0m5.003s
user    0m5.003s
sys     0m0.000s

So my program takes roughly 37% of the time that Stockfish does to calculate perft of startpos to depth 7. Is this "good" ? Stockfish does a lot more that my code doesn't do, I'm sure.

Not sure how to compare.

Hi Imno-
Yes it's very good

I've shared my list here for you:
https://github.com/FireFather/perft

As you can see 5 secs for perft 7 is fast indeed, as the top 4:
gargantua
zerologic
osama
paladin
are dedicated perft engines
as are 7 & 8:
perft &
surge

The list is far from complete...it's just something simple I've compiled over the past few years.

[lithander]

Most chess engines use pseudo-legal move generation. If you use the same movegen for perft you can not bulk-count at the end of the tree. You have to play each move and check it's legality. In that "class" you can expect a result in the ballpark of 30s on modern hardware for your movegen to be considered fast. (single threaded, no hashing)

If you want to reach top speeds you have to do legal move generation and bulk counting as you're probably doing! That's the other class of perft implementations. Maybe give https://github.com/RedBedHed/Charon a try on your PC to compare? I bookmarked that one a while ago thinking "whoo that seems fast"

[Ajedrecista]

Hello:

Welcome to TalkChess!

I am not a programmer, so please take my answer with care. I think that the main objective of perft is correctness, so I would not care too much about perft speeds if you are programming a chess engine unless you want to focus on a perft counter.

Having said that, there are several techniques to boost perft speeds: hash, bulk counting, multi-thread... Even a lone number of speed is misleading because it heavily depends on the hardware: a very fast perft counter running on a very slow hardware can be slower than a slow perft counter running on very fast hardware. I think you should stick to your hardware and run benchmarks, as you already did with SF.

------------

Here is a short list with interesting perft counters that work on Windows. Probably some of them do not run under Ubuntu.

There is a fast, single-core dedicated perft counter called JetChess:

https://web.archive.org/web/20080901011 ... /jetchess/
https://web.archive.org/web/20150402020 ... .0.0.0.zip

Which is a user friendly GUI and will give you a very high standard, but please do not get frustrated if you do not get paired with it.

There was other even faster multi-core perft counter called gperft, but it is not available anymore.

Other fast perft counter is qperft, easily available; qperft counts recursively, this is, if you go to perft(3) of a given position, it computes perft(1), perft(2) and perft(3), not only perft(3).

So I ran benchmarks of Perft(7) of the starting position in my PC with SF 16, qperft, JetChess 1.0.0.0 and gperft 1.1 to compare:

Code: Select all

Perft(7) of the starting position: 3,195,901,860
Total times for each qperft run [Perft(1) + Perft(2) + ... + Perft(6) + Perft(7)]

=======================================================================
ENGINE / COUNTER        HASH (MB)        THREADS        TIME (M:SS.sss)
=======================================================================
gperft 1.1                 64               2           0:00.815
gperft 1.1                 64               2           0:00.866
gperft 1.1                128               1           0:01.465
gperft 1.1                 64               1           0:01.485
gperft 1.1                  0               2           0:03.494
qperft                    128               1           0:04.369
qperft                    256               1           0:04.433
JetChess 1.0.0.0           64               1           0:04.498
JetChess 1.0.0.0          128               1           0:04.565
JetChess 1.0.0.0          256               1           0:04.586
qperft                     64               1           0:04.679
qperft                     32               1           0:05.110
qperft                     16               1           0:05.600
gperft 1.1                  0               1           0:06.362
qperft                      0               1           0:23.922
SF 16 x86-64 Modern         ?               1           0:28 (manual time)

Hash, bulk counting... are game changers regarding perft speed, as you can see. Bigger hash in perft usually starts slower, then pays off with high perft values. Perft(7) is not that high and you can expect lower elapsed times with bigger hashes.

Making a wild guess, your PC is roughly twice as fast as mine regarding SF (it is true that you use AVX2 compile while I use Modern compile). Then, your engine scores above qperft without hash in my chart (a corrected time around 10.19 seconds in my chart with a rule of three of your times and my SF time). I would say that you are doing a good job. Again, my tip is please do not get obsessed with perft speeds.

------------

I guess you know CPW (Chessprogramming wiki):

https://www.chessprogramming.org/

Which is very useful for general chess programming, not only perft. Regarding perft, there are at least two interesting pages there:

https://www.chessprogramming.org/Perft
https://www.chessprogramming.org/Perft_Results

Where you can find a GitHub link to a very fast perft counter, which I have not tried, so I have not got any feedback for you:

https://github.com/ankan-ban/perft_gpu

Good luck with your project!

Regards from Spain.

Ajedrecista.

[imnoumni]

Thanks for the feedback and the links! Bookmarked. Really helps me understand what's happening.

Indeed, it's single-threaded, no hashing. I realized that in the last depth, when you only generate one move, it's the same as +1. I guess that's what's called bulk counting. But it's really only useful for boosting perft numbers. You'd need the actual moves in the actual chess engine.

I also did look into cache line behavior while I was doing this and kept in mind while designing the data. This probably gave a 5-10% boost. For those who work in Linux, I can see the results in perf stat as I am at 3.9 instructions per cycle and topdown-retiring 80% of instructions. There are still a lot of branch misses though, which I expect to be the bottleneck.

Gigantua is at the top of the perft list, so I compiled it (which took a looooong time) and got 1120 Mnps for startpos depth 7. That's almost twice as fast as mine. While reading the article on it, I got the impression that it's specialized for perft and not easily used in an actual chess engine, which is what I want to do. Still, quite an achievement.

I also noticed that different game positions give different Mnps results. Kiwipete gives particuarly high numbers. For kiwipete depth 5, Gigantua gets 1630Mnps and I get 1150Mnps on my laptop. What's the reason for big differences in perft speeds for different positions?

[Ras]

Stockfish does a lot more that my code doesn't do, I'm sure.

My move generator also collects the mobility data for eval. Plus data for static move ordering like captures / promotions with MVV-LVA, quiet moves as per history, or else as per distance to enemy king. Obviously, that takes a toll on perft, but is useful for an actual engine. The move generator and make/unmake are usually no relevant bottleneck anyway.

Most chess engines use pseudo-legal move generation. If you use the same movegen for perft you can not bulk-count at the end of the tree. You have to play each move and check it's legality.

I use a trick to avoid that: if a moving piece is not aligned to the king, or it is aligned but there are other pieces in that line, it cannot be pinned to the king. Exploiting that observation gave me a hybrid that improved both perft and regular search NPS, especially with a totally inefficient mailbox-and-loops design.

[imnoumni]

I use a trick to avoid that: if a moving piece is not aligned to the king, or it is aligned but there are other pieces in that line, it cannot be pinned to the king. Exploiting that observation gave me a hybrid that improved both perft and regular search NPS, especially with a totally inefficient mailbox-and-loops design.

But then, why not just do legal move generation?

[Ras]

But then, why not just do legal move generation?

Because in a full-blown engine, you get a beta cutoff after the first move most of the time. Any effort to make the remaining moves fully legal would be wasted.

[kranium]

I've updated the list
https://github.com/FireFather/perft-speeds

Now it's an editable table instead of a bitmap.

I'll be happy to add engines...
just contact me here firefather at telenet dot be
and send a copy of or point me to your engine.

[lithander]

Most chess engines use pseudo-legal move generation. If you use the same movegen for perft you can not bulk-count at the end of the tree. You have to play each move and check it's legality.

I use a trick to avoid that: if a moving piece is not aligned to the king, or it is aligned but there are other pieces in that line, it cannot be pinned to the king. Exploiting that observation gave me a hybrid that improved both perft and regular search NPS, especially with a totally inefficient mailbox-and-loops design.

I tried tricks like that but couldn't get any significant nps increase out of it (for search). The one you describe only works when not already in check. In practice my engine spends a lot of time on positions that are in check because they are more interesting. And then I found that computing pinned pieces on the fly is almost as expensive as the entire legality check. Instead of accurate pins you can do a legality checks for every potential pinned piece (e.g aligned but without considering other pieces) but then you check legality more often in vain. In the end it never worked out for me - didn't hurt, didn't help.