problem running epd tests in scid vs pc

[chesskobra]

While running some epd test suites in scid vs pc, I have the following problem: if the bm in the test suite is, for example, Bxh7 and the engine calculates Bxh7+, then the test fails. Many tests failed because bm does not have + in my test suite. Is there a simple solution?

I ran the tests in Arena, and they worked fine, but in Arena I didn't see how to set seconds per move, also I didn't see how to save epd with pv added. I have Ferdy's epd-analyzer, which I plan to try next.

An unrelated question is: I have a bunch of epd test suites collected over time, and for most of them I had noted the source. But I have a few test suites under a directory Holmes. It contains one with 628 tablebase positions, another with 3744 middlegame positions, another with 2826 endgame positions, a couple of mixed epds. I tried searching again without success. I was wondering if someone could point me to the source.

[Ajedrecista]

Hello:

[...]

An unrelated question is: I have a bunch of epd test suites collected over time, and for most of them I had noted the source. But I have a few test suites under a directory Holmes. It contains one with 628 tablebase positions, another with 3744 middlegame positions, another with 2826 endgame positions, a couple of mixed epds. I tried searching again without success. I was wondering if someone could point me to the source.

I can not answer your main question. However, googling "EPD Holmes 3744" was lucky enough to get the answer. You need The WayBack Machine, though:

computer_chess:wiki:download:epd_contents

Code: Select all

Epd File     #Pos     Gamestage     Author
------------------------------------------------------
[...]
eg           2826     endgame       (Andreas Herrmann)
[...]
mg           3744     middlegame    (Andreas Herrmann)
[...]
tb            628     endgame       (Andreas Herrmann)
[...]

Andreas Herrmann is the author of chess engine Holmes. Unfortunately, I was not able to find a source to download those EPD files, not even a mention at the old Holmes site.

Regards from Spain.

Ajedrecista.

[chesskobra]

Thanks a lot for tracking down the source.

[Ajedrecista]

Hello again:

I finally found the three EPD files in GitHub, just if somebody is interested:

https://github.com/sylvinus/chessathome ... itions/epd

https://github.com/sylvinus/chessathome ... epd/mg.epd
https://github.com/sylvinus/chessathome ... epd/eg.epd
https://github.com/sylvinus/chessathome ... epd/tb.epd

There is also other file called holmes.epd with 8,613 positions. Most of them seem to be from the other three EPD files, but there are others that not:

https://github.com/sylvinus/chessathome ... holmes.epd

Regards from Spain.

Ajedrecista.

[chesskobra]

Thanks for the links. The repo seems to have many epd test suites. There is also spillthetea collection. But unfortunately many test suites lack description as to how they were created, with what objective in mind. It is hard to curate these collections. Sometimes there are old discussions here on Talkchess (which is how I discovered Talkchess in the first place).

According to some comments in this forum, it seems that testing engines with test suites is obsolete and does not help. But I do collect them. I have now been running various test suites on an old laptop under Stockfish 17 and some other engines, and often there is huge difference between Stockfish and other engines. It is possible that many test suites had best moves from some version of Stockfish itself, but at least I hope the original best moves were obtained at long time controls. The comparison is still useful as a control experiment. So far I have tested Eigenman (3 test suites) with Stockfish 17, Laset 1.8 beta, Lc0 (networks 744706, j104.1-30). Eigenman test suite seems quite hard.