Best EPD Testing Software

[can00336]

I'm trying to run the arasan18 epd test suite on various engines. I have tried Shredder Classic and Arena, both crash too often for various reasons.
epd2wb seems to work best, but I have issues with Hakkapeliitta and Senpai crashing as well as Hiarcs 14 not taking uci options.
Also, I can't get sub one second move times in epd2wb for some reason.

Anyone have a better suggestion?
Thanks.

[Dann Corbit]

Use Arena.
You can run an EPD test suite with that.
I think that ChessGui can do it also, but I did not try it myself.

[Roger Brown]

Use Arena.
You can run an EPD test suite with that.
I think that ChessGui can do it also, but I did not try it myself.

Hello Dann,

The op indicated Arena crashing when running epd. I have experienced that issue.

ChessGUI can definitely do epd testing with several engines queued up.

Sub one second timing - not so sure but it is free...

Later.

[Dann Corbit]

Use Arena.
You can run an EPD test suite with that.
I think that ChessGui can do it also, but I did not try it myself.

Hello Dann,

The op indicated Arena crashing when running epd. I have experienced that issue.

ChessGUI can definitely do epd testing with several engines queued up.

Sub one second timing - not so sure but it is free...

Later.

I use older versions of Arena because they are more stable.
I did not try the latest version yet.
I have experienced occasional decoupling with Arena, where the engine stops writing to the log file, even though it is still running (and if you attach to the chess engine with a debugger you can see that it is still creating output).
The decoupling problem happens in all versions of Arena, but it is more rare in older versions. I guess that it has to do with I/O buffering, but that is only a wild guess.

I did not experience crashes with Epd2wb, but I probably did not test a broad enough spectrum of engines.

It is a fairly complicated problem to make every WB and UCI engine communicate in a stable way for EPD analysis, especially since I have seen a lot of variance in WB output.

Not sure, for instance, if every WB output parser can handle:

for fail high
and:

for fail low. There are some engines that do it.

[Roger Brown]

I did not experience crashes with Epd2wb, but I probably did not test a broad enough spectrum of engines.

It is a fairly complicated problem to make every WB and UCI engine communicate in a stable way for EPD analysis, especially since I have seen a lot of variance in WB output.

Not sure, for instance, if every WB output parser can handle:

for fail high
and:

for fail low. There are some engines that do it.

Hello Dann,

Never experienced crashes with epd2wb (slightly misnamed, since the latest version can handle uci engines as well) either.

Would one of those face engines be Mint?



Later.

[michiguel]

I did not experience crashes with Epd2wb, but I probably did not test a broad enough spectrum of engines.

It is a fairly complicated problem to make every WB and UCI engine communicate in a stable way for EPD analysis, especially since I have seen a lot of variance in WB output.

Not sure, for instance, if every WB output parser can handle:

for fail high
and:

for fail low. There are some engines that do it.

Hello Dann,

Never experienced crashes with epd2wb (slightly misnamed, since the latest version can handle uci engines as well) either.

Would one of those face engines be Mint?



Later.

Mint and the rest of the good ones. I see that you failed high Roger. <fail high>

Miguel

[Ferdy]

I'm trying to run the arasan18 epd test suite on various engines. I have tried Shredder Classic and Arena, both crash too often for various reasons.
epd2wb seems to work best, but I have issues with Hakkapeliitta and Senpai crashing as well as Hiarcs 14 not taking uci options.
Also, I can't get sub one second move times in epd2wb for some reason.

Anyone have a better suggestion?
Thanks.

Try polyglot it can do sub second.

If you are not fine with it, I will create one, just specify what output you need. I only support uci engines.

[can00336]

Use Arena.
You can run an EPD test suite with that.
I think that ChessGui can do it also, but I did not try it myself.

I have tried three different versions of Arena. They all stop communicating with the engine at some random point during long runs.

I haven't tried ChessGui, but I would prefer a cmd line interface, if possible.
Thanks for the suggestion!

[can00336]

I use older versions of Arena because they are more stable.
I did not try the latest version yet.
I have experienced occasional decoupling with Arena, where the engine stops writing to the log file, even though it is still running (and if you attach to the chess engine with a debugger you can see that it is still creating output).
The decoupling problem happens in all versions of Arena, but it is more rare in older versions. I guess that it has to do with I/O buffering, but that is only a wild guess.

I did not experience crashes with Epd2wb, but I probably did not test a broad enough spectrum of engines.

It is a fairly complicated problem to make every WB and UCI engine communicate in a stable way for EPD analysis, especially since I have seen a lot of variance in WB output.

Not sure, for instance, if every WB output parser can handle:

for fail high
and:

for fail low. There are some engines that do it.

epd2wb is not crashing, it is the other engines. epd2wb is pretty much exactly what I want (minus sub one second move times) were it not for some engine incompatibilities.

[can00336]

Try polyglot it can do sub second.

If you are not fine with it, I will create one, just specify what output you need. I only support uci engines.

I can't seem to figure out how to get polyglot to do sub one second move times. Also, Hakkapeliitta and Senpai fail to run no matter what I put in my ini file. The ini files are rather bothersome.

To be honest, I was looking for a generic epd testing tool exactly like your STS Rating tool: www.talkchess.com/forum/viewtopic.php?t=56653
Your STS Rating tool works on every single UCI engine I have tried without fail. It would be great if you could make it into a more generic testing platform.

If this is something that interests you, I would suggest testing it using arasan18.epd as it has both "bm", best move, and "am", avoid move situations.
Regarding output, you could do what you do now just simplified since you don't have scores or different themes. It might also be helpful to output the current number of correct moves during the run as well as a percentage correct.
Here is a detailed output sample from epd2wb if you want to go overboard: https://goo.gl/dbnoCm

Thanks!