Automated tool to append "bm" PV to EPD file.

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Rebel, chrisw

Ferdy
Posts: 4833
Joined: Sun Aug 10, 2008 3:15 pm
Location: Philippines

Re: Automated tool to append "bm" PV to EPD file.

Post by Ferdy »

MikeGL wrote:I wonder if there is already a tool that can parse an EPD file and
append SF9 PV analysis as 'bm' on each position in EPD file.
I don't know what you are up to, but chess artist can analyze and output like below.

Code: Select all

r1bk1b1r/ppp2ppp/2p5/4Pn2/8/5N2/PPP2PPP/RNB2RK1 w - - acd 17; acs 10; bm Nc3; ce 30; Ae "Brainfish 280816 64 POPCNT";
Do you really need the uci pv?

chess artist:
https://github.com/fsmosca/chess-artist

You may try the released exe file.

Code: Select all

a. Command line
chess-artist -infile a.epd -outfile out_a.epd -eng Branfish.exe -engoptions "Hash value 128, Threads value 1" -movetime 10000 -eval search
Ferdy
Posts: 4833
Joined: Sun Aug 10, 2008 3:15 pm
Location: Philippines

Re: Automated tool to append "bm" PV to EPD file.

Post by Ferdy »

Jesse Gersenson wrote:This is just speculation, as I am can not able to tell the difference between EPD and FEN...
FEN:

Code: Select all

1B2N3/8/3P1P2/5r1p/5p2/2b2k2/8/7K w - - 0 1
EPD:

Code: Select all

1B2N3/8/3P1P2/5r1p/5p2/2b2k2/8/7K w - -

1B2N3/8/3P1P2/5r1p/5p2/2b2k2/8/7K w - - fmvn 1; hmvc 0;

1B2N3/8/3P1P2/5r1p/5p2/2b2k2/8/7K w - - bm d7;
Dann Corbit
Posts: 12538
Joined: Wed Mar 08, 2006 8:57 pm
Location: Redmond, WA USA

Re: Automated tool to append "bm" PV to EPD file.

Post by Dann Corbit »

Beware about Arena:
It's output can become decoupled from the EPD file you are analyzing and (for instance) three outputs in a row could be for the same problem instead of for 3 different problems. Or it can skip a problem altogether and all sorts of horrible things like that. The faster you set the time control, the worse the problem becomes.

For that reason, I modify open source programs to write analysis to a log file with the position together with the analysis.

Sometimes, though, you don't have the source (e.g. Houdini) so you have to use an algorithm to tie the log file to the EPD records (I have programs I wrote to do this). But in such a case, I always validate the legality of the selected move as a cross check to ensure that we did not become decoupled.

Other GUIs have similar problems or only write out non-standard looking analysis
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.
MikeGL
Posts: 1010
Joined: Thu Sep 01, 2011 2:49 pm

Re: Automated tool to append "bm" PV to EPD file.

Post by MikeGL »

Ferdy wrote:
MikeGL wrote:I wonder if there is already a tool that can parse an EPD file and
append SF9 PV analysis as 'bm' on each position in EPD file.
I don't know what you are up to, but chess artist can analyze and output like below.

Code: Select all

r1bk1b1r/ppp2ppp/2p5/4Pn2/8/5N2/PPP2PPP/RNB2RK1 w - - acd 17; acs 10; bm Nc3; ce 30; Ae "Brainfish 280816 64 POPCNT";
Do you really need the uci pv?

chess artist:
https://github.com/fsmosca/chess-artist

You may try the released exe file.

Code: Select all

a. Command line
chess-artist -infile a.epd -outfile out_a.epd -eng Branfish.exe -engoptions "Hash value 128, Threads value 1" -movetime 10000 -eval search
Noted. Already downloaded your tool and included it on my tool box for
my future analysis of my personal PGN and EPD games.
Thanks for the link.

Dann Corbit wrote:Beware about Arena:
It's output can become decoupled from the EPD file you are analyzing and (for instance) three outputs in a row could be for the same problem instead of for 3 different problems. Or it can skip a problem altogether and all sorts of horrible things like that. The faster you set the time control, the worse the problem becomes.

For that reason, I modify open source programs to write analysis to a log file with the position together with the analysis.

Sometimes, though, you don't have the source (e.g. Houdini) so you have to use an algorithm to tie the log file to the EPD records (I have programs I wrote to do this). But in such a case, I always validate the legality of the selected move as a cross check to ensure that we did not become decoupled.

Other GUIs have similar problems or only write out non-standard looking analysis
Thanks for the warning. Would analyze using longer time controls, probably 15 to 20 seconds.
I told my wife that a husband is like a fine wine; he gets better with age. The next day, she locked me in the cellar.
Dann Corbit
Posts: 12538
Joined: Wed Mar 08, 2006 8:57 pm
Location: Redmond, WA USA

Re: Automated tool to append "bm" PV to EPD file.

Post by Dann Corbit »

Thanks for the warning. Would analyze using longer time controls, probably 15 to 20 seconds.
At 12 minutes, failures are rare.
At 10-20 seconds you will see lots of failures on overnight runs.
Not every run will fail, but many of them will.
I find that 3.0 fails less than 3.51 but both of them fail.
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.
Jesse Gersenson
Posts: 593
Joined: Sat Aug 20, 2011 9:43 am

Re: Automated tool to append "bm" PV to EPD file.

Post by Jesse Gersenson »

Dann Corbit wrote:... become decoupled.
I wrote another version of my above script which only launches the engine once, and then feeds it positions. For whatever reason it always outputs the total number of positions - 1. 100 positions in gives 99 PV's out. But I do get 100 bestmove lines. Also, the PV appear out of order!? The stockfish benchmark code must have a better solution.

The script I posted above, on my machine, will break if time per move is < 70ms.
The work around I came up with, available here, http://termbin.com/eabt , was to put the PV output in a variable.

Would users typically use one core per position, or many cores per position, on this type of problem set?
Dann Corbit
Posts: 12538
Joined: Wed Mar 08, 2006 8:57 pm
Location: Redmond, WA USA

Re: Automated tool to append "bm" PV to EPD file.

Post by Dann Corbit »

Jesse Gersenson wrote:
Dann Corbit wrote:... become decoupled.
I wrote another version of my above script which only launches the engine once, and then feeds it positions. For whatever reason it always outputs the total number of positions - 1. 100 positions in gives 99 PV's out. But I do get 100 bestmove lines. Also, the PV appear out of order!? The stockfish benchmark code must have a better solution.

The script I posted above, on my machine, will break if time per move is < 70ms.
The work around I came up with, available here, http://termbin.com/eabt , was to put the PV output in a variable.

Would users typically use one core per position, or many cores per position, on this type of problem set?
I generally use a large number of cores at long time control, but sometimes people will use small core counts at extremely high speed. I have done that also, but never with Arena compiled data.
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.