Zugzwang Test Suite ZPTS by Robert Allgeuer

[phenri]

I have a rhetorical question... There are tools that would test a large number of engines (over 50) simply by specifying the folder containing all?
Because it is very tedious with Fritz GUI.

Or can be a macro that will write the default settings automatically and launch the testsuite for each engine.

ie
Stockfish yxz.UCI

Code: Select all

[ENGINE]
Name=Stockfish yxz
Author=Tord Romstad, Marco Costalba and Joona Kiiski
Filename=C:\Engines\Stockfish\Stockfish xyz.exe
Priority=below normal
[OPTIONS]
Threads=4

Stockfish yxx.UCI
etc

[lech]

Hi Paul,
I think the two important positions should be added to this test:

4kr2/5p1K/3p1Q2/1p4P1/4P3/1PP5/7b/8 w - - bm Kh6; R. Reti 1926;
8/p7/qkpN1p2/8/1B1P4/PP3K2/5P2/8 w - - bm Ba5+; ???;

There is another very good position, but without only one winnig move:
8/8/8/8/1kB1q3/8/1PRb4/1K6 b - -; black must lose the first move (to get the same position but with the white move)

good work
regards
Marek

[velmarin]

Because it is very tedious with Fritz GUI.

Or can be a macro that will write the default settings automatically and launch the testsuite for each engine.

Yes , Fritz is tedious.

Arena if allowed to do so easily.
Engines, Automatic analisys.
From there it's easy to do and gives a detailed output...

[phenri]

Thank you for the test positions

[phenri]

Thank you, but it's still the same problem, you must add them one by one and configure them to use only the number of cores desired by default.
Another thing I find not friendly with arena, the output file

I found and I rewrote a script that will do a small part of the job.

Code: Select all

@echo off
:: Credits:
:: Internet and google search engine :)
:: http://stackoverflow.com/a/16461424
:: http://stackoverflow.com/a/17064031
:: P.Henri

@echo The script will create a file (.UCI) with default settings for each engine listed in the
@echo directory. And then move to the directory in question (CB).
@echo The script must be run in a folder containing only the engines concerned.
pause

setlocal enabledelayedexpansion

for /f "tokens=1 delims= " %%a in ('dir *.exe^|find "File(s)"') do (set numfiles=%%a)
set /a i=0
for /f %%f in ( 'dir /b /A-D %1"*.exe"' ) do ( 
  set /a i=i+1) & (
echo [ENGINE]
echo Name=%%~nf
echo Filename=%CD%\%%~nf.exe
echo Priority=below normal
echo [OPTIONS]
echo Threads=1
 ) %numfiles% > %%~nf.UCI
MOVE /-Y *.UCI %AppData%\ChessBase\Engines.UCI\

[Master Om]

Hi Paul,
I think the two important positions should be added to this test:

4kr2/5p1K/3p1Q2/1p4P1/4P3/1PP5/7b/8 w - - bm Kh6; R. Reti 1926;
8/p7/qkpN1p2/8/1B1P4/PP3K2/5P2/8 w - - bm Ba5+; ???;

There is another very good position, but without only one winnig move:
8/8/8/8/1kB1q3/8/1PRb4/1K6 b - -; black must lose the first move (to get the same position but with the white move)

good work
regards
Marek

[Event "?"]
[Site "?"]
[Date "????.??.??"]
[Round "?"]
[White "?"]
[Black "?"]
[Result "*"]
[Analysis "StockfishRW VD 64 SSE4.2"]
[AnalysisTime "36"]
[FEN "8/8/8/8/1kB1q3/8/1PRb4/1K6 b - - 0 1"]
[SetUp "1"]

1... Qh1+ 2. Ka2 Qa8+ 3. Kb1 Qe4 4. Bf7 Qf3 5. Rc4+ Kb5 6. Rc7 Kb6 7. Re7 Bc3
8. Kc2 Qf5+ 9. Kb3 Bxb2 10. Re6+ Ka5 11. Bg8 Qd3+ 12. Ka2 Qc4+ 13. Kxb2
{(0:00:36) 417068kN}{[%eval -14479,30,StockfishRW VD 64 SSE4.2][%meval
36s|TB:2130748]} *

[Event "?"]
[Site "?"]
[Date "????.??.??"]
[Round "?"]
[White "?"]
[Black "?"]
[Result "*"]
[Analysis "StockfishRW VD 64 SSE4.2"]
[AnalysisTime "58"]
[FEN "4kr2/5p1K/3p1Q2/1p4P1/4P3/1PP5/7b/8 w - - 0 1"]
[SetUp "1"]

1. Kh6 Be5 2. Kg7 Bh2 3. c4 bxc4 4. e5 Bxe5 5. bxc4 Bxf6+ 6. gxf6 d5 7. cxd5
{(0:00:58) 511868kN}{[%eval 140,30,StockfishRW VD 64 SSE4.2][%meval
58s|TB:721748]} *

[Event "?"]
[Site "?"]
[Date "????.??.??"]
[Round "?"]
[White "?"]
[Black "?"]
[Result "*"]
[Analysis "StockfishRW VD 64 SSE4.2"]
[AnalysisTime "16"]
[FEN "8/p7/qkpN1p2/8/1B1P4/PP3K2/5P2/8 w - - 0 1"]
[SetUp "1"]

1. Ba5+ Kxa5 2. Nc4+ Kb5 3. Kf4 c5 4. d5 f5 5. Kg5 f4 6. f3 Qc6 7. dxc6 Kxc6
8. Kxf4 Kd5 9. Ke3 Ke6 10. Ke4 Kf6 11. f4 Ke6 12. f5+ Kf6 13. Kf4 a6 14. Nd6
Ke7 15. Ke5 {(0:00:16) 172734kN}{[%eval 621,26,StockfishRW VD 64
SSE4.2][%meval 16s|TB:93277]} *

[phenri]

[Event "?"]
[Site "?"]
[Date "????.??.??"]
[Round "?"]
[White "?"]
[Black "?"]
[Result "*"]
[Analysis "StockfishRW VD 64 SSE4.2"]
[AnalysisTime "58"]
[FEN "4kr2/5p1K/3p1Q2/1p4P1/4P3/1PP5/7b/8 w - - 0 1"]
[SetUp "1"]

1. Kh6 Be5 2. Kg7 Bh2 3. c4 bxc4 4. e5 Bxe5 5. bxc4 Bxf6+ 6. gxf6 d5 7. cxd5
{(0:00:58) 511868kN}{[%eval 140,30,StockfishRW VD 64 SSE4.2][%meval
58s|TB:721748]} *

[Event "?"]
[Site "?"]
[Date "????.??.??"]
[Round "?"]
[White "?"]
[Black "?"]
[Result "*"]
[Analysis "StockfishRW VD 64 SSE4.2"]
[AnalysisTime "16"]
[FEN "8/p7/qkpN1p2/8/1B1P4/PP3K2/5P2/8 w - - 0 1"]
[SetUp "1"]

1. Ba5+ Kxa5 2. Nc4+ Kb5 3. Kf4 c5 4. d5 f5 5. Kg5 f4 6. f3 Qc6 7. dxc6 Kxc6
8. Kxf4 Kd5 9. Ke3 Ke6 10. Ke4 Kf6 11. f4 Ke6 12. f5+ Kf6 13. Kf4 a6 14. Nd6
Ke7 15. Ke5 {(0:00:16) 172734kN}{[%eval 621,26,StockfishRW VD 64
SSE4.2][%meval 16s|TB:93277]} *

Are you sure? Can you try again and take care of cleaning the hash table before?

[zullil]

[Analysis "StockfishRW VD 64 SSE4.2"]

Sure it's fast on these positions, because null move pruning has been removed from this engine.

[phenri]

Oh right! I was not aware of this last

[lech]

[Analysis "StockfishRW VD 64 SSE4.2"]

Sure it's fast on these positions, because null move pruning has been removed from this engine.

As I earlier wrote here, it may be an interesting solution (to cut null move search). How much stronger can be such a version?