'STS' Test Suite - Available for Testing

[swami]

Strategic Test Suite (STS) v1.0 is available for testing:

Beta version of the Test suite file will consist of 100 selected Undermining/Pawn Weakening positions - along with solutions.


It's still too early for me to release it, I'd probably release it once I get the test suite almost error- free. That's why I'm looking for prospective authors/testers willing to test their engines using it and report the results/ bugs back to me.

Please either send me a pm here or send me an e-mail (nswami15 at yahoo dot com) to obtain this test suite.

Thanks in advance.

Ps; This test suite is ideal for engines <2900.

[swami]

I'm working on building up the test suites that cover these complex strategic principles:

Undermining/Pawn Weakening (Completed!)
Blockade
Outpost
Prophylxaxis
Counter Play
Simplification
Positional Sacrifice
Weaknesses
Initiative

This is getting painstakingly slow... So If there's anyone who is willing to work as a team, then I would appreciate it and would give them credit for the work.

[swami]

Ps; This test suite is ideal for engines <2900.

This ain't suited to Rybka. Since this was _mostly_ basically taken from Rybka 3.0 4 CPU games! I think this is more ideal for testing engines other than Rybka.

I selected positions where pawn weakening/undermining happens in Rybka games. So essentially all of the 100 positions are basically _pawn_ moves.

Cut off was > +0.20. If difference in Rybka 2.2n2's analysis line from first best to second best is < 0.20 after about 1 minute of analysis then I'd not add it to the test suite. Only If it's >0.20 (and significant) then I'd add it.

Conditions for the test suite:

Time: anywhere ranging from 30 seconds to 1 minute each position.

[swami]

Re: Test Suite Progress:

Dann Corbitt is now in my team, hip hip hoooray!

He has been kind, hardworking, and has refined the test suite to include only sensible test positions. He says he found the test suite to be intensely exciting!

We have nearly finished the Undermining chapter. Now it is in the final testing stage, I will release it once that is done.

[Kempelen]

I have just read this topic. I had the same idea from long time ago, but I have not found time to execute it.
In what point is the work? I'm expecting

[swami]

I have just read this topic. I had the same idea from long time ago, but I have not found time to execute it.
In what point is the work? I'm expecting

Hi Fermin,

I've decided to release the test suite finally, this was planned to be released on New Year's but got delayed a bit.

Please download here:
http://computerchessblogger.googlepages ... tional.rar

Regards.

[Jouni]

Finally new testsuite, thanks! But isn't this TOO EASY: Naum and Grapefruit (1 CPU) got 90-91/100 with one minute limit only.

Jouni

[swami]

Finally new testsuite, thanks! But isn't this TOO EASY: Naum and Grapefruit (1 CPU) got 90-91/100 with one minute limit only.

Jouni

It could be easy for really stronger engines but do not forget that: Main objective is that it's designed and is more suitable to engines ranging from 1500-2700. Rybka, Naum and Fruit can solve many.

I'm surprised that they solved only 90/100 with 1 minute limit. They would probably have solved 100/100 in tactics. Anyway Thanks for posting the results.

[kgburcham]

It is interesting to see how Rybka compares to other programs on same hardware. In these types of positions Rybka has a big advantage.

STS Test Suite
Q6600 4x2.4
Hash 1024

time to solve=160 seconds


Rybka 3
Total Time 6:01m Solution Time: 2:11m

Zappa Mexico II
TotTime: 8:20m SolTime: 6:25m

Deep Sjeng WC2008 x64
TotTime: 19:27m SolTime: 16:38m

Deep Shredder 11 x64
TotTime: 20:07m SolTime: 18:08m

Deep Hiarcs Paderborn
88 out of 100. Average time = 4.77s / 10.98

[Cubeman]

Is there a PGN file of this test.I have a few programs on my PPC and can only use PGN.