Move ordering transposition table question

[jmcd]

So I've recently decided to start another engine from scratch and I've been relearning everything as I go. Today I was adding move ordering to my engine and encountered a mysterious bug that took me a while to solve.
I separated the move ordering into stages (TT move, winning caps, quiets, losing caps) with captures and quiets generated in separate functions only if the node doesn't finish beforehand. After doing this, I started getting some illegal move errors, and I wasn't sure why at first. After debugging, I realized that the illegal moves were actually the transposition table moves that were sent to the move ordering object. These were collisions that were never actually caught because I hadn't checked to see if the TT move was one of the actual possible moves for the position.
My question is this- how are you supposed to get around this problem if you're doing staged move generation? Is there a way to verify if a TT move is actually valid without generating all the other possible moves?

[tcusr]

So I've recently decided to start another engine from scratch and I've been relearning everything as I go. Today I was adding move ordering to my engine and encountered a mysterious bug that took me a while to solve.
I separated the move ordering into stages (TT move, winning caps, quiets, losing caps) with captures and quiets generated in separate functions only if the node doesn't finish beforehand. After doing this, I started getting some illegal move errors, and I wasn't sure why at first. After debugging, I realized that the illegal moves were actually the transposition table moves that were sent to the move ordering object. These were collisions that were never actually caught because I hadn't checked to see if the TT move was one of the actual possible moves for the position.
My question is this- how are you supposed to get around this problem if you're doing staged move generation? Is there a way to verify if a TT move is actually valid without generating all the other possible moves?

you need to check if it is legal (or at least playable) by considering things like the moving piece (if there is actually a piece there), the color of the piece and if that kind of piece can move to the destination square.

[hgm]

If your TT would be functioning properly, retreiving an illegal move from it would be extremely rare. Like once every 4 billion probes. So even when running at 1Mnps, it would take on the order of an hour to get a collision resulting in the move that was intended for another position, somewhere in the tree. And normally that would not affect the move choice in the root at all, unless making the illegal move would irreversibly corrupt the game state and crash the engine that way.

[jmcd]

you need to check if it is legal (or at least playable) by considering things like the moving piece (if there is actually a piece there), the color of the piece and if that kind of piece can move to the destination square.

thanks, this is one of the solutions I was considering. would it be standard practice to put this move validity check in the universal do_move function, or should it only be used when probing the transposition table?

[Mergi]

you need to check if it is legal (or at least playable) by considering things like the moving piece (if there is actually a piece there), the color of the piece and if that kind of piece can move to the destination square.

thanks, this is one of the solutions I was considering. would it be standard practice to put this move validity check in the universal do_move function, or should it only be used when probing the transposition table?

Usually you would want to check the move validity right after retrieving it from the TT. It can stop you from making a beta cutoff based on incorrect information. If the move is invalid then you ignore the score retrieved from the TT as well.

[tcusr]

you need to check if it is legal (or at least playable) by considering things like the moving piece (if there is actually a piece there), the color of the piece and if that kind of piece can move to the destination square.

thanks, this is one of the solutions I was considering. would it be standard practice to put this move validity check in the universal do_move function, or should it only be used when probing the transposition table?

this kind of legality check is usually done only on the TT move when probing, in do_move i (and everyone else with a pseudo-legal generator) check if the move i played does not put my king in check.

the test doesn't have to be perfect and cover all edge cases, the most important thing is that the move that passed the test is reversible.
this means that, even if you capture your own piece, you are able to return back to your original state.
this mostly depends on how you structured your program, so if you're uncomfortable with this many collisions (bear in mind Stockfish gets thousands of these per second) you can write a big legality check, which is always faster than generating all moves.