Blunder check for sub-stockfish engines

[ydebilloez]

I was trying out blunder checking to validate my engine to find out the positions where the engine is making a bad move. Luckily, we now have stockfish 18 that can serve as a reference for perfect play.

I am looking for a tool that can take a PGN file, and generate an EPD file or FEN string together with the position where a bad move was played. Most tools give evaluation of the complete game with all evaluations for all moves, not the faulty move only. This creates a lot of burden to weed to the file and create test scripts that only refer to the position with the bad move played.

- Tried out Arena 3.10, but epd reporting requires manual processing of the log of the game to find out the position. On top of this, there is some error in it.
- Tried out pgn-extract together analyse, the results are not pointing out the the exact FEN position where an engine is failing.
- Tried out uci analyser, also with unsatisfactory results.
- Tried out Banksia, Scid, ... and a lot of other UI programs, ... no solution found.

Any clues?

[Dann Corbit]

The guy who wrote Toms Live Chess Viewer (Thomas McBurney) had a program that wouild find sudden shifts in eval. I don't remember the name of the program.
Jay Honnold on github has the TLCV code, but I don't see the other tool that was used to detect sudden eval swings.
Personally, I atomize the complete game into EPD records.
Then I do my own diff for evaluation swings.
I don't know of any simple tool that does it all in one step.

[Rebel]

I was trying out blunder checking to validate my engine to find out the positions where the engine is making a bad move. Luckily, we now have stockfish 18 that can serve as a reference for perfect play.

I am looking for a tool that can take a PGN file, and generate an EPD file or FEN string together with the position where a bad move was played. Most tools give evaluation of the complete game with all evaluations for all moves, not the faulty move only. This creates a lot of burden to weed to the file and create test scripts that only refer to the position with the bad move played.

- Tried out Arena 3.10, but epd reporting requires manual processing of the log of the game to find out the position. On top of this, there is some error in it.
- Tried out pgn-extract together analyse, the results are not pointing out the the exact FEN position where an engine is failing.
- Tried out uci analyser, also with unsatisfactory results.
- Tried out Banksia, Scid, ... and a lot of other UI programs, ... no solution found.

Any clues?

This is from my old (hacked) website - https://rebel13.nl/download/mri.html

Output example : https://rebel13.nl/output.htm

If you think it is useful I can make it available.

[cpeters]

This sounds like a task for the chess-artist only reversed/in an avoid-move-scenario, because it can create puzzles.
https://github.com/fsmosca/chess-artist

Maybe ask him, if he's interested in implementing such a function.

greetings

[Norm Pollock]

The guy who wrote Toms Live Chess Viewer (Thomas McBurney) had a program that wouild find sudden shifts in eval. I don't remember the name of the program.
...

Thomas McBurney's "Game Analyser" can be downloaded here:

https://www.mediafire.com/file/mme95j0i ... r.zip/file

[ydebilloez]

Thanks for all the answers, still examining the options. In the meantime, I made a script I use at this moment based on uci-analyse, pgn-extract and perl/regex. It does the job to some extend.

Code: Select all

#!/bin/bash
# comment list of games with bad moves and blunders only

FNAME=badgames.pgn
PGNDIR=../belofte.builds/testfiles

echo "Run analyse-PGN for Black (can be White or Both)"
cd analyse-pgn
./apgn ${PGNDIR}/${FNAME} --player Black
cd ..

echo "Remove variations"
cd pgn-extract
./pgn-extract ${PGNDIR}/analyzed-${FNAME} -o ${PGNDIR}/short-${FNAME} -V

echo "Cleanup evaluations that do not interest us, !, !!, !!! and ?"

perl -0777 -pe 's/\{[^}]*(?:good|questionable|inaccurate|mistake|!!!|!!)[^}]*\}//g' \
     ${PGNDIR}/short-${FNAME} \
     > ${PGNDIR}/annotated-${FNAME}
cd ..

echo "Finished, files: analyzed-${FNAME}, short-${FNAME}, annotated-${FNAME}"