My Perft Results

Discussion of chess software programming and technical issues.

Moderator: Ras

Mike Sherwin
Posts: 965
Joined: Fri Aug 21, 2020 1:25 am
Location: Planet Earth, Sol system
Full name: Michael J Sherwin

Re: My Perft Results

Post by Mike Sherwin »

hgm wrote: Fri Nov 04, 2022 10:31 pm
Mike Sherwin wrote: Fri Nov 04, 2022 10:00 pm
hgm wrote: Fri Nov 04, 2022 9:19 am
Mike Sherwin wrote: Thu Nov 03, 2022 10:56 pmMy perft 6 takes about 3.5 seconds. It is written in C. And it does not use the fastest move generator, magic. Mine uses no tricks and another's that uses no tricks runs 50% faster than mine, but he uses magic. Crafty also uses magic and does perft 6 in 2 seconds. I don't think Bob uses any tricks because his perft has always cared about accuracy and not speed.
Qperft uses mailbox, and it calculates perft(6) in 0.7 sec on my 3.2GHz i7.
Is that not with all tricks active? What does it do with no tricks. With all permutations of tricks? As I have said many times before mine makes and unmakes all moves. So maybe my 30 million n/s is not so bad. And I need to test after adding magic. And it would be nice if there were well defined examples to compare results with.
Not making the moves is not a 'trick'. Making and UnMaking them without doing anything in between is just stupid. An engine would not make any moves in the leaves of the search tree. So why should perft do that. The purpose is to count them, not to make them.
Well my generator is pseudo legal and I know no other way than to make the move, call InCheck and then unmake the move for the last ply. If that is stupid :evil: then please explain.
JoAnnP38
Posts: 253
Joined: Mon Aug 26, 2019 4:34 pm
Location: Clearwater, Florida USA
Full name: JoAnn Peeler

Re: My Perft Results

Post by JoAnnP38 »

hgm wrote: Fri Nov 04, 2022 10:31 pm Not making the moves is not a 'trick'. Making and UnMaking them without doing anything in between is just stupid. An engine would not make any moves in the leaves of the search tree. So why should perft do that. The purpose is to count them, not to make them.
Is it though? No disrespect intended since I've only recently returned to chess programming after having dabbled in it slightly back in the 70s as a college student. I thought (and its certainly serving my purpose as such) that the Perft was created precisely to test how efficient and correct your move generation is. Now granted all the tricks at your disposal should be pulled out to avoid having to reevaluate a node (or branch of nodes) when actually doing a search in game. But if running the Perft test is simply there as a problem distinct unto itself and one not necessarily leading to a better chess engine then it starts to come off as another Volkswagon Dieselgate affair. Did I happen to mention that no disrespect is intended? This is just an enjoyable discussion and I know I have so much to learn from you guys.
JoAnnP38
Posts: 253
Joined: Mon Aug 26, 2019 4:34 pm
Location: Clearwater, Florida USA
Full name: JoAnn Peeler

Re: My Perft Results

Post by JoAnnP38 »

Ras wrote: Fri Nov 04, 2022 11:41 pm An engine would not count the moves in the leaves, either. It would call eval instead. I think the purpose of perft is to be as close to the engine move generator as possible as to allow checking its performance and correctness. So, if the engine design relies on make / check verification / unmake, then it's perfectly reasonable to use that same way also in perft. If however the engine uses some way to avoid this overhead, partially or completely, then it should do the same thing in perft.
I think counting the nodes is simply a method to substitute the simplest evaluation possible (i.e. all nodes evaluate to 1) so you can test how efficient your move generation and execution are. Personally, (and I could be snorting the wrong sort of medicinals) optimizing the Perft algorithm is not all that constructive unless it helps you build a better engine. I've even considered the notion of setting up a transposition table in my Perft just so I wouldn't have to recount nodes if I've seen that position before, but I've decided that really wouldn't help me optimize my move generation. I'm just so happy to have a means to introduce some early tests into my engine so I can make sure the move generation is solid before starting on other aspects.

Cheers!
JoAnnP38
Posts: 253
Joined: Mon Aug 26, 2019 4:34 pm
Location: Clearwater, Florida USA
Full name: JoAnn Peeler

Re: My Perft Results

Post by JoAnnP38 »

Mike Sherwin wrote: Sat Nov 05, 2022 2:39 am Well my generator is pseudo legal and I know no other way than to make the move, call InCheck and then unmake the move for the last ply. If that is stupid :evil: then please explain.
I'm with you there Mike, but I think I'm discovering that Perft is used for different things by different people and that's fine by me. I'm just glad I have a means to see and early indication on whether my move generation is up to snuff. I'll wait until I'm actually implementing the actual game search before pulling all the rabbits out of my hat. :wink: :D
User avatar
hgm
Posts: 28353
Joined: Fri Mar 10, 2006 10:06 am
Location: Amsterdam
Full name: H G Muller

Re: My Perft Results

Post by hgm »

Mike Sherwin wrote: Sat Nov 05, 2022 2:39 amWell my generator is pseudo legal and I know no other way than to make the move, call InCheck and then unmake the move for the last ply. If that is stupid :evil: then please explain.
The 'stupid' qualification only referred to the case where you would make and unmake moves without doing anything in between. Testing for check would be a valid excuse for doing it. But it would be a very inefficient method for testing move legality. And that could be justifiably be held against the method for move generation. Most moves can very well be tested for legality without making them.

E.g. moves of a non-royal piece can only be illegal when that piece was pinned. (Or when you were in check to begin with, but that should be handled by a special-case move generator.) To be pinned such pieces have to be aligned with the friendly King. Most pieces won't be so aligned, and it is quite cheap to test for (e.g. if(aligned[fromSqr-kingSqr]) ... ), and failing the test excludes that the move can be illegal. And thus removes the necessity for making and unmaking them just for check testing.

And you don't even have to do that test for every individual move: you can do it for all moves of the piece at once, as these all have the same fromSqr. So you could do it in the move generator before generating moves for that piece, complete the test by looking for a pinner in the few cases that is is aligned with its King, and fall back on a special case of move generation only for moves along the pin ray when they are actually pinned, instead of generating all their moves.

This is basically what Qperft does. Except instead of subjecting all the pieces it generates moves for to the alignment test, it subjects all opponent sliders to such a test. (There are only 5 of those, while it could have 16 pieces of its own.) Again alignment is not common, but if it occurs it tests for a single piece on the ray to King, and then that piece is pinned and exempted from normal move generation, and uses the on-ray move generator instead.

None of the above is a 'perft trick'; it is just a method for making a speedy move generator from which an engine benefits just as well.

The above optimizations do not apply to King moves; actually Qperft does make all King moves, even in 'bulk counting' mode, to test whether they are legal by an IsSquareAttacked() call on the toSqr after the move. In principle this is still wasteful. Making the move before doing the IsSquareAttacked test only makes a difference when the King would have 'stepped into its own shadow', and this is only possible when it was in check to begin with. Which probably is known, but even if not, fully making the move could simply be replaced by temporarily taking the King off board during the test, so that it cannot block any slider attacks on the square behind it. Even worse, doing tests for all King destinations separately is wasteful. Because once you found out a slider attacks one such square, it usually attacks more. And the same test could have revealed that.

E.g. you could subject every enemy piece an alignment test with the king neighborhood through a lookup in bulkAligned[sqr-kingSqr], which would give you a mask where each bit corresponds to a square adjacent to King, set to 1 if the piece is aligned with it. You could have a second set of bits in the same word to indicate whether these are distant attacks. For those you would have to vet the attack for not being blocked. But usually finding a blocker would block all attacks of that piece into the King neighborhood. The masks with the unblocked attacks for all pieces could be ORed together to get the illegal King destinations, and then you only generate King moves to the remaining squares.

Again, this would not be a 'perft trick', but just an optimization of the move generator.
JoAnnP38
Posts: 253
Joined: Mon Aug 26, 2019 4:34 pm
Location: Clearwater, Florida USA
Full name: JoAnn Peeler

Re: My Perft Results

Post by JoAnnP38 »

Over the past couple of days, I have come to know my profiler on a first name basis and his name is dotTrace!

Previously, I reported Perft(6) results with a performance of about 1 minute which was much too slow. For now, my goal is to get the performance to under 5 seconds for Perft(6) and I have made some progress, but I'm still not to my goal (under 5 seconds.) Here are my current results:

Perft(0) : ( Elapsed = 00:00:00.0000457, Mnps = 0.02 ) = 1
Perft(1) : ( Elapsed = 00:00:00.0006250, Mnps = 0.03 ) = 20
Perft(2) : ( Elapsed = 00:00:00.0008465, Mnps = 0.47 ) = 400
Perft(3) : ( Elapsed = 00:00:00.0037415, Mnps = 2.38 ) = 8902
Perft(4) : ( Elapsed = 00:00:00.0396298, Mnps = 4.98 ) = 197281
Perft(5) : ( Elapsed = 00:00:00.5199880, Mnps = 9.36 ) = 4865609
Perft(6) : ( Elapsed = 00:00:09.5101991, Mnps = 12.52 ) = 119060324

At this point I feel like I've squeezed all the water I can out of that stone, so I'm going to start focusing on search. Perhaps other ideas will develop as I flesh out my design for search. For anyone else who is using C#/.NET to write a chess engine here are some of my suggestions that I've learned over the past 48 hours:
  • Multidimensional arrays are slow compared to jagged arrays (i.e. int[,] versus int[][]). So never use them. Additionally, single dimensional arrays are faster than jagged array (so, don't use jagged arrays either.)
  • Don't use the .NET collections. They are too heavy weight. Roll your own, but just make sure the collection is just a thin verneer over a single dimensioned array.
  • Pinning an array in memory using unsafe/fixed can also give you a little more performance when iterating over an array (maybe 5-10%)
  • Keep an incrementally updated table that indicates if a square is attacked by either side. This sped up my IsSquareAttacked() method tremendously. It might have even been my best improvement.
  • Do not allocate objects on the heap. Let the garbage collector stay as dormant as possible.
Okay, that's about it for now. I'll probably start another thread that I will post updates to as I progress. The working name for my engine is loTechCS, unless someone else has already used that. I consider it an homage to the chess engine HiTech developed at Carnie Melon back in the 80s. Hans Berliner was always a hero of mine. Anyways, thanks for all the interesting feedback. I especially like that tidbit from H.G.Muller concerning keeping a list of pinned pieces and having a different move generator for those pieces. Much food for thought there!
dangi12012
Posts: 1062
Joined: Tue Apr 28, 2020 10:03 pm
Full name: Daniel Infuehr

Re: My Perft Results

Post by dangi12012 »

Mike Sherwin wrote: Sat Nov 05, 2022 2:39 am Well my generator is pseudo legal and I know no other way than to make the move, call InCheck and then unmake the move for the last ply. If that is stupid :evil: then please explain.
Do you need special code for castling? - the only move where the position is legal before and after, but yet the move might be illegal?

[fen]rn2k1nr/ppp3pp/3p4/4pp1b/8/8/PPPP1PPP/R3K1NR w KQkq - 0 1[/fen]
Worlds-fastest-Bitboard-Chess-Movegenerator
Daniel Inführ - Software Developer
JoAnnP38
Posts: 253
Joined: Mon Aug 26, 2019 4:34 pm
Location: Clearwater, Florida USA
Full name: JoAnn Peeler

Re: My Perft Results

Post by JoAnnP38 »

dangi12012 wrote: Sat Nov 05, 2022 10:48 am
Do you need special code for castling? - the only move where the position is legal before and after, but yet the move might be illegal?

[fen]rn2k1nr/ppp3pp/3p4/4pp1b/8/8/PPPP1PPP/R3K1NR w KQkq - 0 1[/fen]
I generate pseudo legal moves like Mike, but I don't check for the legality of castling until MakeMove(). I check specifically whether the king starts out in check or had to move through a squared attacked by the other side and if so I immediately Unmake the move.
User avatar
hgm
Posts: 28353
Joined: Fri Mar 10, 2006 10:06 am
Location: Amsterdam
Full name: H G Muller

Re: My Perft Results

Post by hgm »

Legality of castling can also be checked without making the move. Orthodox Chess pieces are such that the only way they could attack a square the King passes through if they already attack it before the move, as neither the Rook (because it is at the board edge) nor the King (which would be in check) can block a slider attack to such a square. So it just requires running IsSquareAttacked() on all squares passed through in the current position. In fact you should already know if the adjacent square is attacked, as this had to be tested for the normal King move to that square. So for orthodox castling only the to-square remains.

Of course when in check you would not generate castlings in the first place. Only normal King moves, and (if it is a single check) capture of the checker, and for single distant checks also interpositions.

Note that it is often counter-productive in engines to have a dedicated legality test at all. Many of my engines just search all pseudo-legal moves, and when the MVV/LVA-wise best capture in the daughter captures a King immediately have it return a +INF score (so the move is ignored in the parent). This reduces the overhead in testing to an absolute minimum. That more than the necessary amount of work has been done when a move turns out to be illegal hurts less than doing a much cheaper test on every move, as illegal moves are pretty rare.

So a dedicated legality test often is a perft-only feature, which would not be present in an engine, but needs to be done in perft for getting correct counts.
JVMerlino
Posts: 1396
Joined: Wed Mar 08, 2006 10:15 pm
Location: San Francisco, California

Re: My Perft Results

Post by JVMerlino »

JoAnnP38 wrote: Sat Nov 05, 2022 5:54 am
Ras wrote: Fri Nov 04, 2022 11:41 pm An engine would not count the moves in the leaves, either. It would call eval instead. I think the purpose of perft is to be as close to the engine move generator as possible as to allow checking its performance and correctness. So, if the engine design relies on make / check verification / unmake, then it's perfectly reasonable to use that same way also in perft. If however the engine uses some way to avoid this overhead, partially or completely, then it should do the same thing in perft.
I think counting the nodes is simply a method to substitute the simplest evaluation possible (i.e. all nodes evaluate to 1) so you can test how efficient your move generation and execution are. Personally, (and I could be snorting the wrong sort of medicinals) optimizing the Perft algorithm is not all that constructive unless it helps you build a better engine. I've even considered the notion of setting up a transposition table in my Perft just so I wouldn't have to recount nodes if I've seen that position before, but I've decided that really wouldn't help me optimize my move generation. I'm just so happy to have a means to introduce some early tests into my engine so I can make sure the move generation is solid before starting on other aspects.

Cheers!
Yes, these are all perfectly valid extensions to the original idea behind perft, which was simply to ensure that move generation and make/unmake were flawless. This also opened the door to comparing perft times, and therefore comparing the efficiency of various move generation and make/unmake methods. The one piece of advice I can give (which others are free to disagree with) is that once you are getting correct perft results at an acceptable speed (your goal of 5s is reasonable), spending a lot more time increasing the speed will only give a very small increase in playing strength.

As I mentioned, Myrddin's move generation is pretty slow. But even if I was able to double its speed (at this point an impossible task without a complete rewrite) I would estimate I would get only about 10 elo. Not worth the effort for this lazy programmer. And almost certainly not worth the effort for somebody like yourself who is writing their first engine. Besides, working on search and eval is much more fun (to me at least). :D