Testing Stockfish 11-03-13. 480 Games.

[mcostalba]

Thanks for the explanation.

bench: 4507288 is what I needed.

[Tomcass]

Thanks for anticipating the right answer, Jesús!

Marco, in my next posts I will include the bench signature of Stockfish so that it will be easier for you to identify the version tested.

TESTING STOCKFISH 090713 (Bench 4507230)= 480 games

7 980 3.33 Ghz.
6 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2012
Time control: 4 min +0 sec/ game
No tablebases
Relative Speed: 28.66
Knodes per second: 13.759

Time control 4+0

Stockfish 090713 64 SSE4.2_ - Critter 1.6 64-bitx6_nob 16.0 - 24.0 +5/=22/-13 40.00%
Stockfish 090713 64 SSE4.2_ - Deep Rybka 4.1 SSE42 x64 25.5 - 14.5 +17/=17/-6 63.75%
Stockfish 090713 64 SSE4.2_ - Houdini 3 Pro x64_6 15.5 - 24.5 +6/=19/-15 38.75%

Time control 2+2

Stockfish 090713 64 SSE4.2_ - Critter 1.6 64-bitx6_nob 22.5 - 17.5 +12/=21/-7 56.25%
Stockfish 090713 64 SSE4.2_ - Deep Rybka 4.1 SSE42 x64 24.0 - 16.0 +15/=18/-7 60.00%
Stockfish 090713 64 SSE4.2_ - Houdini 3 Pro x64_6 15.5 - 24.5 +9/=13/-18 38.75%

240 GAMES x6 = http://www.mediafire.com/?l5resdvc3ms5zfa

Score using 6 real cores = 119.0-121.0 = 49.58%

i7 975 3.33 Ghz.
4 real cores
GUI: Fritz 12
Book: Fritz 12
Ponder: Off
No tablebases
Relative Speed: 20.62
Knodes per second: 9.899

Time control 4+0

Stockfish 090713 64 SSE4.2_ - Critter 1.6a 64-bitnob_4 21.5 - 18.5 +9/=25/-6 53.75%
Stockfish 090713 64 SSE4.2_ - Deep Rybka 4 x64 (x4) 24.0 - 16.0 +9/=30/-1 60.00%
Stockfish 090713 64 SSE4.2_ - Houdini 3 Pro x64_4 16.0 - 24.0 +4/=24/-12 40.00%

Time control 2+2

Stockfish 090713 64 SSE4.2_ - Critter 1.6a 64-bitnob_4 24.0 - 16.0 +12/=24/-4 60.00%
Stockfish 090713 64 SSE4.2_ - Deep Rybka 4 x64 (x4) 24.0 - 16.0 +11/=26/-3 60.00%
Stockfish 090713 64 SSE4.2_ - Houdini 3 Pro x64_4 16.5 - 23.5 +4/=25/-11 41.25%

240 GAMES x4 = http://www.mediafire.com/?mvbvg63xr9cd76b

Score using 4 cores = 126.0 – 114.0 = 52.50%

Global Score: = 245.0 – 235.0 = 51.04%
Against Critter 1.6: 52.50% Deep Rybka 4: 60.94% Houdini 3.0: 39.69%
Estimated ELO Performance: 3.101

Regards,

Tom.

[Tomcass]

Stockfish 020713 64 SSE4.2_ 480 GAMES - bench: 4769737

7 980 3.33 Ghz.
6 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2012c
No tablebases
Relative Speed: 28.66
Knodes per second: 13.759

Time control 4+0

Stockfish 200713 64 SSE4.2_ - Critter 1.6 64-bitx6_nob 20.5 - 19.5 +6/=29/-5 51.25%
Stockfish 200713 64 SSE4.2_ - Deep Rybka 4.1 SSE42 x64 (x6) 29.0 - 11.0 +21/=16/-3 72.50%
Stockfish 200713 64 SSE4.2_ - Houdini 3 Pro x64_6 13.5 - 26.5 +6/=15/-19 33.75%

Time control 2+2

Stockfish 200713 64 SSE4.2_ - Critter 1.6 64-bitx6_nob 20.5 - 19.5 +8/=25/-7 51.25%
Stockfish 200713 64 SSE4.2_ - Deep Rybka 4.1 SSE42 x64 26.0 - 14.0 +18/=16/-6 65.00%
Stockfish 200713 64 SSE4.2_ - Houdini 3 Pro x64_6 19.5 - 20.5 +8/=23/-9 48.75%

240 games x 6 cores = http://www.mediafire.com/?j9zp5e3w5940c3q
Global Score with 6 cores = 129.0-111.0 = 53.75%


i7 975 3.33 Ghz.
4 real cores
Ponder: Off
GUI: Fritz 11
Book: Perfect 2012c
No tablebases
Relative Speed: 20.62
Knodes per second: 9.899

Time control 4+0

Stockfish 200713 64 SSE4.2_ - Critter 1.6a 64-bitnob_4 18.5 - 21.5 +6/=25/-9 46.25%
Stockfish 200713 64 SSE4.2_ - Deep Rybka 4 x64 (x4) 25.0 - 15.0 +15/=20/-5 62.50%
Stockfish 200713 64 SSE4.2_ - Houdini 3 Pro x64_4 17.0 - 23.0 +6/=22/-12 42.50%

Time control 2+2

Stockfish 200713 64 SSE4.2_ - Critter 1.6a 64-bitnob_4 22.5 - 17.5 +11/=23/-6 56.25%
Stockfish 200713 64 SSE4.2_ - Deep Rybka 4 x64 (x4) 23.5 - 16.5 +15/=17/-8 58.75%
Stockfish 200713 64 SSE4.2_ - Houdini 3 Pro x64_4 16.5 - 23.5 +8/=17/-15 41.25%

240 games x 4 cores= http://www.mediafire.com/?3fxpn6g1u814kv4
Global Score with 4 cores= 123.0-117.0 = 51.25%


Global Score: 252.0 – 228.0 = 52.50%
Against Critter 1.6: 51.25% Deep Rybka 4: 64.69% Houdini 3.0: 41.56%
Estimated ELO Performance: 3111

Another very good score for Stockfish.

Regards,

Tom.

[Tomcass]

STOCKFISH 020813 480GAMES / BENCH: 4364793 / TIMESTAMP: 1375454905

7 980 3.33 Ghz.
6 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2012c
No tablebases
Relative Speed: 28.66
Knodes per second: 13.759

Time Control_4+0

Stockfish 020813 64 SSE4.2_ - Critter 1.6 64-bitx6_nob 20.5 - 19.5 +11/=19/-10 51.25%
Stockfish 020813 64 SSE4.2_ - Deep Rybka 4.1 SSE42 x64 26.0 - 14.0 +15/=22/-3 65.00%
Stockfish 020813 64 SSE4.2_ - Houdini 3 Pro x64_6 18.0 - 22.0 +8/=20/-12 45.00%

Time control_2+2

Stockfish 020813 64 SSE4.2_ - Critter 1.6 64-bitx6_nob 22.5 - 17.5 +12/=21/-7 56.25%
Stockfish 020813 64 SSE4.2_ - Deep Rybka 4.1 SSE42 x64 27.0 - 13.0 +18/=18/-4 67.50%
Stockfish 020813 64 SSE4.2_ - Houdini 3 Pro x64_6 16.5 - 23.5 +9/=15/-16 41.25%

http://www.mediafire.com/?wwnu4hwal6hf7st
Global Score with 6 cores: 130.5-109.5 = 54.37%


i7 975 3.33 Ghz.
4 real cores
Ponder: Off
GUI: Fritz 11
Book: Perfect 2012c
No tablebases
Relative Speed: 20.62
Knodes per second: 9.899

Time control 4+0

Stockfish 020813 64 SSE4.2_ - Critter 1.6a 64-bitnob_4 24.5 - 15.5 +16/-7/=17 61.25%
Stockfish 020813 64 SSE4.2_ - Deep Rybka 4 x64 (x4) 21.0 - 19.0 +12/-10/=18 52.50%
Stockfish 020813 64 SSE4.2_ - Houdini 3 Pro x64_4 20.0 - 20.0 +10/-10/=20 50.00%

Time control_2+2

Stockfish 020813 64 SSE4.2_ - Critter 1.6a 64-bitnob_4 23.0 - 17.0 +13/-7/=20 57.50%
Stockfish 020813 64 SSE4.2_ - Deep Rybka 4 x64 (x4) 23.0 - 17.0 +13/-7/=20 57.50%
Stockfish 020813 64 SSE4.2_ - Houdini 3 Pro x64_4 18.0 - 22.0 +7/-11/=22 45.00%

http://www.mediafire.com/?2jq57nf75j7gm4w
Global Score with 4 cores: 129.5 – 110.5 = 53.96%


Global Score: 260.0 – 220.0 = 54.17%
Against Critter 1.6: 56.56% Deep Rybka 4: 60.62% Houdini 3.0: 45.31%
Estimated ELO Performance: 3123

NEW BEST SCORE FOR STOCKFISH IN MY TESTS!.

This is a SUBSTANTIAL improvement for Stockfish. Only 49 Elo points below the monster Houdini 3.0 Pro. 7 ELO points better than the previous best in my tests. Well done, Stockfish team!!

Regards,

Tom.

[Tomcass]

STOCKFISH 030813 480GAMES / BENCH: 4424151 / TIMESTAMP: 1375547443

7 980 3.33 Ghz.
6 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2012c
No tablebases
Relative Speed: 28.66
Knodes per second: 13.759

Time Control_4+0

Stockfish 030813 64 SSE4.2_ - Critter 1.6 64-bitx6_nob 23.0 - 17.0 +10/=26/-4 57.50%
Stockfish 030813 64 SSE4.2_ - Deep Rybka 4.1 SSE42 x64 25.5 - 14.5 +15/=21/-4 63.75%
Stockfish 030813 64 SSE4.2_ - Houdini 3 Pro x64_6 16.5 - 23.5 +5/=23/-12 41.25%

Time control_2+2

Stockfish 030813 64 SSE4.2_ - Critter 1.6 64-bitx6_nob 23.5 - 16.5 +15/=17/-8 58.75%
Stockfish 030813 64 SSE4.2_ - Deep Rybka 4.1 SSE42 x64 25.5 - 14.5 +15/=21/-4 63.75%
Stockfish 030813 64 SSE4.2_ - Houdini 3 Pro x64_6 20.5 - 19.5 +11/=19/-10 51.25%

240 games using 6 cores= http://www.mediafire.com/?mqn1p71ie7ra8gg
Global Score with 6 cores: 134.5-105.5 = 56.04%

i7 975 3.33 Ghz.
4 real cores
Ponder: Off
GUI: Fritz 11
Book: Perfect 2012c
No tablebases
Relative Speed: 20.62
Knodes per second: 9.899

Time control 4+0

Stockfish 030813 64 SSE4.2_ - Critter 1.6a 64-bitnob_4 23.5 - 16.5 +13/-6/=21 58.75%
Stockfish 030813 64 SSE4.2_ - Deep Rybka 4 x64 (x4) 25.0 - 15.0 +17/-7/=16 62.50%
Stockfish 030813 64 SSE4.2_ - Houdini 3 Pro x64_4 14.0 - 26.0 +5/-17/=18 35.00%

Time control_2+2

Stockfish 030813 64 SSE4.2_ - Critter 1.6a 64-bitnob_4 23.5 - 16.5 +12/-5/=23 58.75%
Stockfish 030813 64 SSE4.2_ - Deep Rybka 4 x64 (x4) 28.5 - 11.5 +19/-2/=19 71.25%
Stockfish 030813 64 SSE4.2_ - Houdini 3 Pro x64_4 19.0 - 21.0 +10/-12/=18 47.50%

240 games using 4 cores = http://www.mediafire.com/?clfkl5xq3j1j2q9
Global Score with 4 cores: 133.5-107.5 = 55.62%

Global Score: 268.0- 212.0 = 55.83%
Against Critter 1.6: 58.44% Deep Rybka 4: 65.31% Houdini 3.0: 43.75%
Estimated ELO Performance: 3.135

AGAIN A NEW BEST SCORE FOR STOCKFISH!

The evolution of Stockfish is simply impressive. At the time controls of my tests, the score of
this version is roughly at Komodo 5.1r2 level (Estimated ELO 3.131). Only 37 ELO points below Houdini 3.0 Pro.

The Stockfish team is making a fantastic work. Thanks guys!.

Regards,

Tom.

[wolfman]

A confirmation : in my FTS list is surpasses Stockfish 3.

Rank Engine Version Compu TSR %MM AAD ATMM ART
9 Stockfish 20130802JS 64 i 3136 37,17 14,39 2,48 720
12 Stockfish 3 sse2 JA 64 i 3124 36,27 14,26 2,53 729

[beram]

Well it looks like the latest Stockfish really surpasses Komodo 5.1MP
I had this on my system: AMD X6 1090T 3200 Mhz

Code: Select all

Bram Testsuite 2.0 - 50 games, TC 4m 2s, each engine 4CPU, ponder off

Stockfish 030813 64 SSE4.2   - Houdini 3 x64                  23.0 - 27.0    +13/=20/-17   46.00%
Stockfish 030813 64 SSE4.2   - Komodo 5.1r2 64-bit            29.0 - 21.0    +15/=28/-7    58.00%

[ernest]

...really surpasses...

Stockfish 030813 64 SSE4.2 - Komodo 5.1r2 64-bit 29.0 - 21.0 +15/=28/-7 58.00%

Indeed 58%
But 95% error bar still is ­±9%

So time will tell...

[beram]

Well it looks like the latest Stockfish really surpasses Komodo 5.1MP
I had this on my system: AMD X6 1090T 3200 Mhz

Code: Select all

Bram Testsuite 2.0 - 50 games, TC 4m 2s, each engine 4CPU, ponder off

Stockfish 030813 64 SSE4.2   - Houdini 3 x64                  23.0 - 27.0    +13/=20/-17   46.00%
Stockfish 030813 64 SSE4.2   - Komodo 5.1r2 64-bit            29.0 - 21.0    +15/=28/-7    58.00%

Well Ernest, it 'really' looks like it....

Code: Select all

Noomentestsuite 2012, 60 games, TC 4m 2s, each engine 4CPU, ponder off
                                
1   Stockfish 030813 64 SSE4.2  +17/=34/-9 56.67%   34.0/60
2   Komodo 5.1r2 64-bit         +9/=34/-17 43.33%   26.0/60

[Tomcass]

STOCKFISH 030813= 400 GAMES AGAINST KOMODO 5.1r2 / BENCH: 4424151 / TIMESTAMP: 1375547443

i7 980 3.33 Ghz.
6 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2012c
No tablebases
Relative Speed: 28.66
Knodes per second: 13.759

Time Control_4+0

Stockfish 030813 64 bit - Komodo 5.1r2 64 bit 45.0-55.0 +16/=57/-26 45.00%

Time control_2+2

Stockfish 030813 64 bit - Komodo 5.1r2 64 bit 50.5-49.5 +23/=56/-22 50.50%

200 games using 6 cores= http://www.mediafire.com/?1vl9lc08ja528h3
Global Score with 6 cores: 95.5-104.5 = 47.75%

i7 975 3.33 Ghz.
4 real cores
Ponder: Off
GUI: Fritz 11
Book: Perfect 2012c
No tablebases
Relative Speed: 20.62
Knodes per second: 9.899

Time control 4+0

Stockfish 030813 64 bit - Komodo 5.1r2 64 bit 48.5-51.5 +22/=57/-25 48.50%

Time control_2+2

Stockfish 030813 64 bit - Komodo 5.1r2 64 bit 52.0-48.0 +24/=56/-20 52.00%

200 games using 4 cores = http://www.mediafire.com/?glt90mq4wz246zz
Global Score with 4 cores: 100.5-99.5 = 50.25%

GLOBAL SCORE FOR STOCKFISH 030813 AFTER 400 GAMES = 196.0-204.0 = 49.00% Estimated ELO performance = 3.124

Please note that in my tests SF 030813 performs much better at incremental time control that at a fixed TC mode. And Komodo is reported to be very good at incremental time controls.

4+0 score: 93.5-106.5 46.75%
2+2 score: 102.5-97.5 51.25%

We can confirm this fact by making the same segmentation for the previous posted 480 games standard test:

4+0 score : 127.5-112.5 53.12%
2+2 score : 140.5-99.5 58.54%

Regards from Barcelona.

Tom.