STS (v4.0) - "Square Vacancy"

[swami]

Chapter 4 of the Strategic Test Suite: Square Vacancy is now available for download.

* Consists of carefully selected 100 questionairres on Square Vacancy

* All of the questions in this test suite are selected by me and thoroughly verified by Dann Corbitt. Each question has been analysed by top engines for hours. I initially selected about 200 problems and sent them to Dann, of which 100 has passed criteria. Dann then made the final selection.

* All the answers carry similar traits:

• (1) Occupying the vacant square near the opponent's territory to create more pressure in surrounding areas.
  (2) Taking complete control of the vacant square in an attempt to gain more space over the board
  (3) Queen is the piece that creates more pressure if it's placed on a vacant square in opponent's camp. Therefore, a lot of these tests are usually a Queen move

Download it! here:
http://sites.google.com/site/strategict ... re-vacancy




Feel free to report the results from your engine/ favourite engine, suggestions are welcome.

Special Thanks to Pedro Castro, Allard Siemelink, Zach Wegner for the testing and correcting mistakes throughout the process. Thanks to those who reported bugs and tested out the V1.0, V2.0 and V3.0.

Test Suite Release time: 14th of July, 2009
Swaminathan and Dann Corbit.

[michiguel]

Chapter 4 of the Strategic Test Suite: Square Vacancy is now available for download.

* Consists of carefully selected 100 questionairres on Square Vacancy

* All of the questions in this test suite are selected by me and thoroughly verified by Dann Corbitt. Each question has been analysed by top engines for hours. I initially selected about 200 problems and sent them to Dann, of which 100 has passed criteria. Dann then made the final selection.

* All the answers carry similar traits:

• (1) Occupying the vacant square near the opponent's territory to create more pressure in surrounding areas.
  (2) Taking complete control of the vacant square in an attempt to gain more space over the board
  (3) Queen is the piece that creates more pressure if it's placed on a vacant square in opponent's camp. Therefore, a lot of these tests are usually a Queen move

Download it! here:
http://sites.google.com/site/strategict ... re-vacancy




Feel free to report the results from your engine/ favourite engine, suggestions are welcome.

Special Thanks to Pedro Castro, Allard Siemelink, Zach Wegner for the testing and correcting mistakes throughout the process. Thanks to those who reported bugs and tested out the V1.0, V2.0 and V3.0.

Test Suite Release time: 14th of July, 2009
Swaminathan and Dann Corbit.

Gaviota 0.66.3, AMD 2x 4600 (2.4 Ghz), one cpu, 10 seconds/position, 16 MB hashtable

average nps: 360549
solved: 60
wrong : 40
ratio : 60.00%

Miguel

[pedrox]

AMD 2x 4600 (2.4 Ghz), one cpu, 10 seconds/position, 64 MB hashtable

DanaSah 4.24 --> 61/100
Caligula 0.6 --> 56/100

[swami]

It's always good to receive test results, thanks for the input guys! Keep them coming...!!

These results gives me the impression that difficulty is equivalent to those in previous suites, that's to say: Average. scores vary from engines to engines depending on their understanding in specific themes, that's good enough for me.

[bob]

Chapter 4 of the Strategic Test Suite: Square Vacancy is now available for download.

* Consists of carefully selected 100 questionairres on Square Vacancy

* All of the questions in this test suite are selected by me and thoroughly verified by Dann Corbitt. Each question has been analysed by top engines for hours. I initially selected about 200 problems and sent them to Dann, of which 100 has passed criteria. Dann then made the final selection.

* All the answers carry similar traits:

• (1) Occupying the vacant square near the opponent's territory to create more pressure in surrounding areas.
  (2) Taking complete control of the vacant square in an attempt to gain more space over the board
  (3) Queen is the piece that creates more pressure if it's placed on a vacant square in opponent's camp. Therefore, a lot of these tests are usually a Queen move

Download it! here:
http://sites.google.com/site/strategict ... re-vacancy




Feel free to report the results from your engine/ favourite engine, suggestions are welcome.

Special Thanks to Pedro Castro, Allard Siemelink, Zach Wegner for the testing and correcting mistakes throughout the process. Thanks to those who reported bugs and tested out the V1.0, V2.0 and V3.0.

Test Suite Release time: 14th of July, 2009
Swaminathan and Dann Corbit.

I ran this on my core2-duo 2.0ghz laptop, 30 secs / position. I did not do the different move bonus scoring stuff, just either found the "bm" or didn't find it.

Code: Select all

total positions searched..........         100
number right......................          82
number wrong......................          18
percentage right..................          82
percentage wrong..................          18
total nodes searched..............  4923033675
average search depth..............        12.6
nodes per second..................     3996390
total time........................       20:31

[Zach Wegner]

ZCT does rather well on this one at 10s/position:

Code: Select all

score=72/100 72.0%
avg.depth=8.5 avg.time=2.649 avg.time^2=16367.30

I'm also rerunning ZCT on every test suite at the suggested 7 minutes. I'll update once it's done (it should take a while...).

[swami]

ZCT does rather well on this one at 10s/position:

Code: Select all

score=72/100 72.0%
avg.depth=8.5 avg.time=2.649 avg.time^2=16367.30

I'm also rerunning ZCT on every test suite at the suggested 7 minutes. I'll update once it's done (it should take a while...).

That's pretty good results, Zach. What was the hardware used? Is it mp?

I'd very much like to see its results in all the test suites at 7 minutes!

[swami]

I ran this on my core2-duo 2.0ghz laptop, 30 secs / position. I did not do the different move bonus scoring stuff, just either found the "bm" or didn't find it.

Code: Select all

total positions searched..........         100
number right......................          82
number wrong......................          18
percentage right..................          82
percentage wrong..................          18
total nodes searched..............  4923033675
average search depth..............        12.6
nodes per second..................     3996390
total time........................       20:31

Thanks for the report. Would it do even better in that faster hardware you had where you once reported Crafty getting 95/100 in it. Was wondering how Crafty would do in the same test on that hardware

[Zach Wegner]

That's pretty good results, Zach. What was the hardware used? Is it mp?

Yep, 2.4 GHz Q6600, 4 cpus. 512M hash.

[vladstamate]

At 10sec per position, Plisk gets 44 correct answers out the 100.

Correct: 44
Wrong: 56
Percent: 44%

This was in Linux, using a 32bit compile of the engine running on 1CPU with 128Mb hash. The processor was a P4D 3.0Ghz.

Btw, I really appreciate the effort you (and Dann Corbit) put into the STS. They really help with engine tuning and testing/validation.

Regards,
Vlad.