STS (v4.0) - "Square Vacancy"

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Harvey Williamson, bob

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Post Reply
swami
Posts: 6535
Joined: Thu Mar 09, 2006 3:21 am

STS (v4.0) - "Square Vacancy"

Post by swami » Tue Jul 14, 2009 4:52 am

Chapter 4 of the Strategic Test Suite: Square Vacancy is now available for download.

* Consists of carefully selected 100 questionairres on Square Vacancy

* All of the questions in this test suite are selected by me and thoroughly verified by Dann Corbitt. Each question has been analysed by top engines for hours. I initially selected about 200 problems and sent them to Dann, of which 100 has passed criteria. Dann then made the final selection.

* All the answers carry similar traits:
  • (1) Occupying the vacant square near the opponent's territory to create more pressure in surrounding areas.
    (2) Taking complete control of the vacant square in an attempt to gain more space over the board
    (3) Queen is the piece that creates more pressure if it's placed on a vacant square in opponent's camp. Therefore, a lot of these tests are usually a Queen move
Download it! here:
http://sites.google.com/site/strategict ... re-vacancy

Image


Feel free to report the results from your engine/ favourite engine, suggestions are welcome.

Special Thanks to Pedro Castro, Allard Siemelink, Zach Wegner for the testing and correcting mistakes throughout the process. Thanks to those who reported bugs and tested out the V1.0, V2.0 and V3.0.

Test Suite Release time: 14th of July, 2009
Swaminathan and Dann Corbit.

User avatar
michiguel
Posts: 6371
Joined: Thu Mar 09, 2006 7:30 pm
Location: Chicago, Illinois, USA
Contact:

Re: STS (v4.0) - "Square Vacancy"

Post by michiguel » Tue Jul 14, 2009 7:07 am

swami wrote:Chapter 4 of the Strategic Test Suite: Square Vacancy is now available for download.

* Consists of carefully selected 100 questionairres on Square Vacancy

* All of the questions in this test suite are selected by me and thoroughly verified by Dann Corbitt. Each question has been analysed by top engines for hours. I initially selected about 200 problems and sent them to Dann, of which 100 has passed criteria. Dann then made the final selection.

* All the answers carry similar traits:
  • (1) Occupying the vacant square near the opponent's territory to create more pressure in surrounding areas.
    (2) Taking complete control of the vacant square in an attempt to gain more space over the board
    (3) Queen is the piece that creates more pressure if it's placed on a vacant square in opponent's camp. Therefore, a lot of these tests are usually a Queen move
Download it! here:
http://sites.google.com/site/strategict ... re-vacancy

Image


Feel free to report the results from your engine/ favourite engine, suggestions are welcome.

Special Thanks to Pedro Castro, Allard Siemelink, Zach Wegner for the testing and correcting mistakes throughout the process. Thanks to those who reported bugs and tested out the V1.0, V2.0 and V3.0.

Test Suite Release time: 14th of July, 2009
Swaminathan and Dann Corbit.
Gaviota 0.66.3, AMD 2x 4600 (2.4 Ghz), one cpu, 10 seconds/position, 16 MB hashtable

average nps: 360549
solved: 60
wrong : 40
ratio : 60.00%

Miguel

User avatar
pedrox
Posts: 938
Joined: Fri Mar 10, 2006 5:07 am
Location: Basque Country (Spain)
Contact:

Re: STS (v4.0) - "Square Vacancy"

Post by pedrox » Tue Jul 14, 2009 9:50 am

AMD 2x 4600 (2.4 Ghz), one cpu, 10 seconds/position, 64 MB hashtable

DanaSah 4.24 --> 61/100
Caligula 0.6 --> 56/100

swami
Posts: 6535
Joined: Thu Mar 09, 2006 3:21 am

Re: STS (v4.0) - "Square Vacancy"

Post by swami » Tue Jul 14, 2009 11:22 am

It's always good to receive test results, thanks for the input guys! Keep them coming...!!

These results gives me the impression that difficulty is equivalent to those in previous suites, that's to say: Average. scores vary from engines to engines depending on their understanding in specific themes, that's good enough for me. :)

bob
Posts: 20358
Joined: Mon Feb 27, 2006 6:30 pm
Location: Birmingham, AL

Re: STS (v4.0) - "Square Vacancy"

Post by bob » Tue Jul 14, 2009 1:32 pm

swami wrote:Chapter 4 of the Strategic Test Suite: Square Vacancy is now available for download.

* Consists of carefully selected 100 questionairres on Square Vacancy

* All of the questions in this test suite are selected by me and thoroughly verified by Dann Corbitt. Each question has been analysed by top engines for hours. I initially selected about 200 problems and sent them to Dann, of which 100 has passed criteria. Dann then made the final selection.

* All the answers carry similar traits:
  • (1) Occupying the vacant square near the opponent's territory to create more pressure in surrounding areas.
    (2) Taking complete control of the vacant square in an attempt to gain more space over the board
    (3) Queen is the piece that creates more pressure if it's placed on a vacant square in opponent's camp. Therefore, a lot of these tests are usually a Queen move
Download it! here:
http://sites.google.com/site/strategict ... re-vacancy

Image


Feel free to report the results from your engine/ favourite engine, suggestions are welcome.

Special Thanks to Pedro Castro, Allard Siemelink, Zach Wegner for the testing and correcting mistakes throughout the process. Thanks to those who reported bugs and tested out the V1.0, V2.0 and V3.0.

Test Suite Release time: 14th of July, 2009
Swaminathan and Dann Corbit.
I ran this on my core2-duo 2.0ghz laptop, 30 secs / position. I did not do the different move bonus scoring stuff, just either found the "bm" or didn't find it.

Code: Select all

total positions searched..........         100
number right......................          82
number wrong......................          18
percentage right..................          82
percentage wrong..................          18
total nodes searched..............  4923033675
average search depth..............        12.6
nodes per second..................     3996390
total time........................       20:31

User avatar
Zach Wegner
Posts: 1922
Joined: Wed Mar 08, 2006 11:51 pm
Location: Earth
Contact:

Re: STS (v4.0) - "Square Vacancy"

Post by Zach Wegner » Tue Jul 14, 2009 4:03 pm

ZCT does rather well on this one at 10s/position:

Code: Select all

score=72/100 72.0%
avg.depth=8.5 avg.time=2.649 avg.time^2=16367.30
I'm also rerunning ZCT on every test suite at the suggested 7 minutes. I'll update once it's done (it should take a while...).

swami
Posts: 6535
Joined: Thu Mar 09, 2006 3:21 am

Re: STS (v4.0) - "Square Vacancy"

Post by swami » Tue Jul 14, 2009 4:10 pm

Zach Wegner wrote:ZCT does rather well on this one at 10s/position:

Code: Select all

score=72/100 72.0%
avg.depth=8.5 avg.time=2.649 avg.time^2=16367.30
I'm also rerunning ZCT on every test suite at the suggested 7 minutes. I'll update once it's done (it should take a while...).
That's pretty good results, Zach. What was the hardware used? Is it mp?

I'd very much like to see its results in all the test suites at 7 minutes!

swami
Posts: 6535
Joined: Thu Mar 09, 2006 3:21 am

Re: STS (v4.0) - "Square Vacancy"

Post by swami » Tue Jul 14, 2009 4:13 pm

bob wrote: I ran this on my core2-duo 2.0ghz laptop, 30 secs / position. I did not do the different move bonus scoring stuff, just either found the "bm" or didn't find it.

Code: Select all

total positions searched..........         100
number right......................          82
number wrong......................          18
percentage right..................          82
percentage wrong..................          18
total nodes searched..............  4923033675
average search depth..............        12.6
nodes per second..................     3996390
total time........................       20:31
Thanks for the report. Would it do even better in that faster hardware you had where you once reported Crafty getting 95/100 in it. Was wondering how Crafty would do in the same test on that hardware :)

User avatar
Zach Wegner
Posts: 1922
Joined: Wed Mar 08, 2006 11:51 pm
Location: Earth
Contact:

Re: STS (v4.0) - "Square Vacancy"

Post by Zach Wegner » Tue Jul 14, 2009 4:52 pm

swami wrote:That's pretty good results, Zach. What was the hardware used? Is it mp?
Yep, 2.4 GHz Q6600, 4 cpus. 512M hash.

vladstamate
Posts: 161
Joined: Thu Jan 08, 2009 8:06 pm
Location: San Francisco, USA
Contact:

Re: STS (v4.0) - "Square Vacancy"

Post by vladstamate » Tue Jul 14, 2009 6:30 pm

At 10sec per position, Plisk gets 44 correct answers out the 100.

Correct: 44
Wrong: 56
Percent: 44%

This was in Linux, using a 32bit compile of the engine running on 1CPU with 128Mb hash. The processor was a P4D 3.0Ghz.

Btw, I really appreciate the effort you (and Dann Corbit) put into the STS. They really help with engine tuning and testing/validation.

Regards,
Vlad.

Post Reply