What's the best tactical test suite for the modern erA?

[johnhamlen65]

Dear lovely CCC community,

I'm currently working on a Tech 4 technology chess program to continue the lineage of these super-simple search monsters 50 years after Tech 2 competed in the 1st WCCC in 1974.
https://www.chessprogramming.org/Tech

Squirrel and I have been away from chess programming for a while: last code written in 1997, last completion in 2011, but one thing is clear is how easily modern engines on modern hardware make some of the hard test suites of yesteryear seem trivial!

Does anyone have and recommendations for a suite(s) of test positions which will test Tech 4's tactical ability i.e. very deep combinations which lead to material win or mate.

Many thanks in advance
John

[Viz]

The best usage of test suites during development is to not use test suites at all.

[Jouni]

Try this https://www.arasanchess.org/arasan21.epd . Good engine solves 180 with 15 sec limit and 4 cores.

[gaard]

The best usage of test suites during development is to not use test suites at all.

It seems counterintuitive that tactical test suites would not be useful for testing tactical ability. A few engines have tactical/mate finding modes that really do work. Not everyone is competing in the Elo arms race.

[ImNotStockfish]

Decent ones are Hard Talkchess 2020 (2021 revision), TTT1, TTT2, Arasan... You can combine them to get a better sample size but it will still be pretty small
Bad ones are ShashChess Hardpositions2022

[Viz]

The best usage of test suites during development is to not use test suites at all.

It seems counterintuitive that tactical test suites would not be useful for testing tactical ability. A few engines have tactical/mate finding modes that really do work. Not everyone is competing in the Elo arms race.

If OP wasn't involved in chess programming for 27 years he probably would want to actually learn how to develop a strong engine first and only then maybe putting in some extra oomph with suites or whatever. Well, ofc if he wants to have 2000 elo engine which would solve testsuites he can try, but in general it's a dead end and not an interesting one.

[shawn]

Modern engine development relies on scientific testing, which is part of the reason why they are so strong. I would suggest you to read https://www.chessprogramming.org/SPRT to learn more about this methodology. Testing suits are almost completely obsolete for top engines.

[johnhamlen65]

Try this https://www.arasanchess.org/arasan21.epd . Good engine solves 180 with 15 sec limit and 4 cores.

Thanks for the help Jouni!

I'll take a look at those tomorrow.

Cheers
John

[shawn]

The best usage of test suites during development is to not use test suites at all.

It seems counterintuitive that tactical test suites would not be useful for testing tactical ability. A few engines have tactical/mate finding modes that really do work. Not everyone is competing in the Elo arms race.

https://github.com/Matthies/RubiChess/w ... uter-chess

[johnhamlen65]

Modern engine development relies on scientific testing, which is part of the reason why they are so strong. I would suggest you to read https://www.chessprogramming.org/SPRT to learn more about this methodology. Testing suits are almost completely obsolete for top engines.

Thanks Shawn. Looks very interesting.

A scientific method for improving Elo. I'm sure Tech 4 could benefit from that! However, I believe testing suites are of more use for a technology program than a "standard" one. The only hope a Tech program has of defeating a top rated program is to find it's way to a position where there are tactics beyond its opponent's horizon.