Checking TT move for legality

[syzygy]

You are not testing with 64 GB of hash.

Well forgive me for developing on a Windows PC using the X86 64-bit architecture. I normally only test with 256MB of hash table because Pedantic is: 1) single-threaded and 2) I know that CCRL primarily tests single-threaded engines with 256MB of hash table. Isn't this a more realistic scenario?

So this explains why you have not noticed frequent crashes. In my previous comments I mentioned my assumption of 2^32 TT entries.

[syzygy]

An old thread on the same topic:
http://talkchess.com/forum3/viewtopic.php?f=7&t=54941

A still older thread where the calculation I outlined above gave a result pretty close to the actual measurement:
http://talkchess.com/forum3/viewtopic.p ... 18#p469818

The first derivation of this calculation may have been given by Rémi Coulom in 1999:
https://www.stmintz.com/ccc/index.php?id=44382

[JoAnnP38]

So this explains why you have not noticed frequent crashes. In my previous comments I mentioned my assumption of 2^32 TT entries.

It's not that I haven't noticed frequent crashes. It's that I've not noticed ANY crashes.

[syzygy]

So this explains why you have not noticed frequent crashes. In my previous comments I mentioned my assumption of 2^32 TT entries.

It's not that I haven't noticed frequent crashes. It's that I've not noticed ANY crashes.

Since they are not frequent you don't notice them.

With 256Mb hash and 16-byte entries you use 24 bits of your hashkey to index into the table, which leaves 40 bits for disambiguation. If 32 bits meant 15 minutes at 10Mnps, then 40 bits means 64 hours. If your program only does 1Mnps, then it is 640 hours. I doubt that you would be able to link a non-reproducible crash every 640 hours of search time to an invalid tt move.

But again, your engine will not be very reliable once it is run with a large (but, today, entirely realistic) hashtable.

[syzygy]

Secondly if you have two positions with the same zobrist hash you would most likely have many more, that is they should cluster. The reason is very simple. Assume two positions have the same zobrist hash it is probable that they have at least one piece at the same square that has one or more legal moves in common. That will result in that one or more of their children to have the same zobrist hash. The clustering will have the effect that once it crashes it will crash many more times if it could and since your engine will only crash once per game it will have the nice side effect that getting the first crash will be less likely than pure random.

I agree that such a clustering effect could delay the occurrence of the first undetected collision without changing the average number of collisions per unit of time. E.g. if they always come in pairs, that probably doubles the expected time until the first collision. (At the same time, the probability that the first collision (or, rather, cluster of collisions) crashes the engine will be larger.)

I doubt that clustering has a very large effect (doubling of 15 mins to crash to 30 mins to crash is not a large effect), but to be sure this should be tested.

[hgm]

Note that validating TT moves and killers is a completely different things. For killers it is really important to test the pseudo-legality, and a sizable fraction (perhaps 10%) will not be pseudo-legal. TT moves are almost never illegal, and the only thing you should worry about is to weed out moves that would crash your engine. Even if you work with pseudo-legal move generation, allowing arbitrary mutations of the board might interfere with assumptions you made in the check test. And applying promotion moves to non-Pawns might lead to invalid board positions, e.g. with non-existent piece types in them, or without Kings.

Minimal requirements are:
* From-square should not be empty. (And since it isn't any harder to test, must contain a friendly piece.)
* To-square should not contain your own King. (And because this is tested just as easily, not contain any friendly piece.)
* Depending on how you handle King capture by generated moves, you might also not want to capture an enemy King, or abort with a +INF score when you do.
* If the move is a promotion, the from-square should contain a Pawn.
* If the move is a castling, make sure K & R are present, and their destinations empty.
In the latest two cases UnMake probably would not properly restore the position otherwise, even if it would not be a problem for MakeMove.