Testing Stockfish 11-03-13. 480 Games.

[Tomcass]

Stockfish 250813 = 480 GAMES.

Bench: 4728533 Timestamp: 1377448609

i7 980 3.33 Ghz.
6 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2012c
No tablebases
Hash 512
Relative Speed: 28.66
Knodes per second: 13.759

Time Control_4+0

Stockfish 250813 64 SSE4.2_ - Critter 1.6 64-bitx6_nob 19.5 - 20.5 +7/=25/-8 48.75%
Stockfish 250813 64 SSE4.2_ - Deep Rybka 4.1 SSE42 x64 28.0 - 12.0 +18/=20/-2 70.00%
Stockfish 250813 64 SSE4.2_ - Houdini 3 Pro x64_6 17.0 - 23.0 +11/=12/-17 42.50%

Time Control 2+2

201308SF250813ST66_2+2_120games-1 2013

Stockfish 250813 64 SSE4.2_ - Critter 1.6 64-bitx6_nob 25.5 - 14.5 +12/=27/-1 63.75%
Stockfish 250813 64 SSE4.2_ - Deep Rybka 4.1 SSE42 x64 29.5 - 10.5 +21/=17/-2 73.75%
Stockfish 250813 64 SSE4.2_ - Houdini 3 Pro x64_6 18.5 - 21.5 +8/=21/-11 46.25%

240 Games X6 = http://www.mediafire.com/?97kp2tucree91z4
Score using 6 cores = 138.0-102.0 = 57.50%

i7 975 3.33 Ghz.
4 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2012c
No tablebases
Hash 256
Relative Speed: 20.62
Knodes per second: 9.899

Time Control 4+0

Stockfish 250813 64 SSE4.2_ - Critter 1.6a 64-bitnob_4 21.0 - 19.0 +10/=22/-8 52.50%
Stockfish 250813 64 SSE4.2_ - Deep Rybka 4 x64 (x4) 29.0 - 11.0 +24/=10/-6 72.50%
Stockfish 250813 64 SSE4.2_ - Houdini 3 Pro x64_4 18.5 - 21.5 +9/=19/-12 46.25%

Time Control 2+2

Stockfish 250813 64 SSE4.2_ - Critter 1.6a 64-bitnob_4 26.5 - 13.5 +16/=21/-3 66.25%
Stockfish 250813 64 SSE4.2_ - Deep Rybka 4 x64 (x4) 29.0 - 11.0 +21/=16/-3 72.50%
Stockfish 250813 64 SSE4.2_ - Houdini 3 Pro x64 20.5 - 19.5 +7/=27/-6 51.25%

240 Games X 4 = http://www.mediafire.com/?77wnd52o227qal2
Score using 4 cores = 144.5-95.5 = 60.21%

Segmenting by Time Control:

Fixed = 53.33%
Incremental = 62.50% (New best score with incremental TC)

Global Score: 282.5–197.5 = 58.85% (New best score for Stockfish)

Against Critter 1.6: 57.81% Deep Rybka 4: 72.19%(New best score against Deep Rybka 4) Houdini 3.0: 46.56%

Estimated ELO Performance: 3156

And suddenly ... the Stockfish Development version apears to be closer and closer to Houdini 3 Pro. According to the estimated performance of this test, the 250813 Version is only 16 Elo points below Houdini 3.0 Pro (3.172) and:

25 points above Komodo 5.1r2
27 points above Stockfish 4.

This is a GREAT result for Stockfish. Congratulations to Marco and all the TEAM!. And the newer and interesting time management tools introduced by Uri Blass are not yet applied to this version. Let's test SF 26082013!.

Regards from Barcelona.

Tom.

[gladius]

This is a GREAT result for Stockfish. Congratulations to Marco and all the TEAM!. And the newer and interesting time management tools introduced by Uri Blass are not yet applied to this version. Let's test SF 26082013!.

Regards from Barcelona.

Tom.

Thanks for all the great tests Tom, it's really fun to watch them . I think that SF may have gotten a bit lucky in this run, there was only one small strength change vs SF4 in that version.

I'm looking forward to seeing how the TC changes do against other engines!

[mwyoung]

Stockfish 250813 = 480 GAMES.

Bench: 4728533 Timestamp: 1377448609

i7 980 3.33 Ghz.
6 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2012c
No tablebases
Hash 512
Relative Speed: 28.66
Knodes per second: 13.759

Time Control_4+0

Stockfish 250813 64 SSE4.2_ - Critter 1.6 64-bitx6_nob 19.5 - 20.5 +7/=25/-8 48.75%
Stockfish 250813 64 SSE4.2_ - Deep Rybka 4.1 SSE42 x64 28.0 - 12.0 +18/=20/-2 70.00%
Stockfish 250813 64 SSE4.2_ - Houdini 3 Pro x64_6 17.0 - 23.0 +11/=12/-17 42.50%

Time Control 2+2

201308SF250813ST66_2+2_120games-1 2013

Stockfish 250813 64 SSE4.2_ - Critter 1.6 64-bitx6_nob 25.5 - 14.5 +12/=27/-1 63.75%
Stockfish 250813 64 SSE4.2_ - Deep Rybka 4.1 SSE42 x64 29.5 - 10.5 +21/=17/-2 73.75%
Stockfish 250813 64 SSE4.2_ - Houdini 3 Pro x64_6 18.5 - 21.5 +8/=21/-11 46.25%

240 Games X6 = http://www.mediafire.com/?97kp2tucree91z4
Score using 6 cores = 138.0-102.0 = 57.50%

i7 975 3.33 Ghz.
4 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2012c
No tablebases
Hash 256
Relative Speed: 20.62
Knodes per second: 9.899

Time Control 4+0

Stockfish 250813 64 SSE4.2_ - Critter 1.6a 64-bitnob_4 21.0 - 19.0 +10/=22/-8 52.50%
Stockfish 250813 64 SSE4.2_ - Deep Rybka 4 x64 (x4) 29.0 - 11.0 +24/=10/-6 72.50%
Stockfish 250813 64 SSE4.2_ - Houdini 3 Pro x64_4 18.5 - 21.5 +9/=19/-12 46.25%

Time Control 2+2

Stockfish 250813 64 SSE4.2_ - Critter 1.6a 64-bitnob_4 26.5 - 13.5 +16/=21/-3 66.25%
Stockfish 250813 64 SSE4.2_ - Deep Rybka 4 x64 (x4) 29.0 - 11.0 +21/=16/-3 72.50%
Stockfish 250813 64 SSE4.2_ - Houdini 3 Pro x64 20.5 - 19.5 +7/=27/-6 51.25%

240 Games X 4 = http://www.mediafire.com/?77wnd52o227qal2
Score using 4 cores = 144.5-95.5 = 60.21%

Segmenting by Time Control:

Fixed = 53.33%
Incremental = 62.50% (New best score with incremental TC)

Global Score: 282.5–197.5 = 58.85% (New best score for Stockfish)

Against Critter 1.6: 57.81% Deep Rybka 4: 72.19%(New best score against Deep Rybka 4) Houdini 3.0: 46.56%

Estimated ELO Performance: 3156

And suddenly ... the Stockfish Development version apears to be closer and closer to Houdini 3 Pro. According to the estimated performance of this test, the 250813 Version is only 16 Elo points below Houdini 3.0 Pro (3.172) and:

25 points above Komodo 5.1r2
27 points above Stockfish 4.

This is a GREAT result for Stockfish. Congratulations to Marco and all the TEAM!. And the newer and interesting time management tools introduced by Uri Blass are not yet applied to this version. Let's test SF 26082013!.

Regards from Barcelona.

Tom.

A gain of 27 points since stockfish 4, very nice. And the lastest update by Uri claims 5 more elo by using better time management.

[Tomcass]

Stockfish 260813 = 480 GAMES

Bench: 4728533 Timestamp: 1377538198
(Please note that the Bench for 250813 and 260813 versions are the same, although I think there are functional changes –time management- between them. The Timestamp is different).

i7 980 3.33 Ghz.
6 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2012c
No tablebases
Hash 512
Relative Speed: 28.66
Knodes per second: 13.759

Time Control_4+0

Stockfish 260813 64 SSE4.2_ - Critter 1.6 64-bitx6_nob 25.0 - 15.0 +13/=24/-3 62.50%
Stockfish 260813 64 SSE4.2_ - Deep Rybka 4.1 SSE42 x64 26.0 - 14.0 +16/=20/-4 65.00%
Stockfish 260813 64 SSE4.2_ - Houdini 3 Pro x64_6 18.0 - 22.0 +6/=24/-10 45.00%

Time Control_2+2

Stockfish 260813 64 SSE4.2_ - Critter 1.6 64-bitx6_nob 24.0 - 16.0 +13/=22/-5 60.00%
Stockfish 260813 64 SSE4.2_ - Deep Rybka 4.1 SSE42 x64 30.0 - 10.0 +23/=14/-3 75.00%
Stockfish 260813 64 SSE4.2_ - Houdini 3 Pro x64_6 19.5 - 20.5 +9/=21/-10 48.75%

240 Games X6 = http://www.mediafire.com/?7qlrzwoiqacy525
Score using 6 cores = 142.5-97.5 = 59.37%

i7 975 3.33 Ghz.
4 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2012c
No tablebases
Hash 256
Relative Speed: 20.62
Knodes per second: 9.899

Time Control_4+0

Stockfish 260813 64 SSE4.2_ - Critter 1.6a 64-bitnob_4 21.5 - 18.5 +9/=25/-6 53.75%
Stockfish 260813 64 SSE4.2_ - Deep Rybka 4 x64 (x4) 27.0 - 13.0 +17/=20/-3 67.50%
Stockfish 260813 64 SSE4.2_ - Houdini 3 Pro x64_4 16.0 - 24.0 +9/=14/-17 40.00%

Time Control_2+2

Stockfish 260813 64 SSE4.2_ - Critter 1.6a 64-bitnob_4 22.5 - 17.5 +11/=23/-6 56.25%
Stockfish 260813 64 SSE4.2_ - Deep Rybka 4 x64 (x4) 29.0 - 11.0 +19/=20/-1 72.50%
Stockfish 260813 64 SSE4.2_ - Houdini 3 Pro x64_4 18.5 - 21.5 +8/=21/-11 46.25%

240 Games X4 = http://www.mediafire.com/?kre9h6xmaze892a
Score using 4 cores = 134.5-105.5 = 56.04%

Segmenting by Time Control:

Fixed = 55.42%
Incremental = 59.79%

Global Score: 277.0–203.0 = 57.71% (Second best score only after 250813 version)
Against Critter 1.6: 58.12% Deep Rybka 4: 70.00% Houdini 3.0: 45.00%
Estimated ELO Performance: 3148

Regards,

Tom.

[Tomcass]

This is a GREAT result for Stockfish. Congratulations to Marco and all the TEAM!. And the newer and interesting time management tools introduced by Uri Blass are not yet applied to this version. Let's test SF 26082013!.

Regards from Barcelona.

Tom.

Thanks for all the great tests Tom, it's really fun to watch them . I think that SF may have gotten a bit lucky in this run, there was only one small strength change vs SF4 in that version.

I'm looking forward to seeing how the TC changes do against other engines!

Thanks to you for your kind words, Gary. For me is a pleasure to test Stockfish Development versions and note its substantial improvement day after day.

By the way, following the suggestions of some friends I will introduce Komodo 5.1r2 in my tests and remove from them Deep Rybka 4, which today is clearly below Stockfish.

Best regards from Barcelona.

Tom

[Tomcass]

Following the suggestion of SigmaPT and other friends I have removed Deep Rybka 4 from my tests and introduced Komodo 5.1r2 instead. Thanks for your suggestion, my friends. My tests are much even and exciting -at least for me- now.

Stockfish 280813= 480 GAMES.

Bench: 4728533 Timestamp: 1377712211

i7 980 3.33 Ghz.
6 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2012c
No tablebases
Hash 512
Relative Speed: 28.66
Knodes per second: 13.759

Time Control_4+0

Stockfish 280813 64 SSE4.2_ - Critter 1.6a 64-bit_NOB 21.0 - 19.0 +6/=30/-4 52.50%
Stockfish 280813 64 SSE4.2_ - Houdini 3 Pro x64_6 20.0 - 20.0 +7/=26/-7 50.00%
Stockfish 280813 64 SSE4.2_ - Komodo 5.1r2 64-bitx6nob 18.5 - 21.5 +7/=23/-10 46.25%

Time Control_2+2

Stockfish 280813 64 SSE4.2_ - Critter 1.6a 64-bit_NOB 23.5 - 16.5 +11/=25/-4 58.75%
Stockfish 280813 64 SSE4.2_ - Houdini 3 Pro x64_6 18.0 - 22.0 +8/=20/-12 45.00%
Stockfish 280813 64 SSE4.2_ - Komodo 5.1r2 64-bitx6nob 20.5 - 19.5 +7/=27/-6 51.25%

240 Games X6= http://www.mediafire.com/?s5zbg7ht2mcft0u
Score using 6 cores = 121.5-118.5 = 50.62%

i7 975 3.33 Ghz.
4 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2012c
No tablebases
Hash 256
Relative Speed: 20.62
Knodes per second: 9.899

Time Control_4+0

Stockfish 280813 64 SSE4.2x - Critter 1.6a 64-bitnob_4 23.0 - 17.0 +11/=24/-5 57.50%
Stockfish 280813 64 SSE4.2x - Houdini 3 Pro x64_4 16.0 - 24.0 +4/=24/-12 40.00%
Stockfish 280813 64 SSE4.2x - Komodo 5.1r2 64-bitnob 19.0 - 21.0 +9/=20/-11 47.50%

Time Control_2+2

Stockfish 280813 64 SSE4.2x - Critter 1.6a 64-bitnob_4 22.5 - 17.5 +11/=23/-6 56.25%
Stockfish 280813 64 SSE4.2x - Houdini 3 Pro x64_4 20.5 - 19.5 +10/=21/-9 51.25%
Stockfish 280813 64 SSE4.2x - Komodo 5.1r2 64-bitnob 16.0 - 24.0 +7/=18/-15 40.00%

240 Games X4 = http://www.mediafire.com/?4xk5jds3d3882tr
Score using 4 cores = 117.0-123.0 = 48.75%

Segmenting the result by Time Control:

Fixed = 48.96%
Incremental = 50.42%

Global Score: 238.5–241.5 = 49.69%
Against Critter 1.6a (3093): 56.25% Houdini 3.0 Pro (3172): 46.56% Komodo 5.1r2 (3135): 46.25%
Average ELO of opponents= 3133
Estimated ELO Performance= 3131

The new invited engine Komodo 5.1r2 has been a very tough opponent, performing even better than Houdini 3.0Pro in this test. I expected around 52% for Stockfish and my guess has been not right this time. Let’s keep testing.

Regards,

Tom.

[ernest]

Global Score: 238.5–241.5 = 49.69%

What happened?
Seems a bit low, even with statistical noise...

[Uri Blass]

Global Score: 238.5–241.5 = 49.69%

What happened?
Seems a bit low, even with statistical noise...

Komodo replaced rybka so comparison the global score against previous versions is meaningless.

[Kohflote]

Hi,

Yes, we have lost the baseline to compare different versions of SF. It will be great if Tom could test SF 250813 version (the best version tested by him) against Komodo so that we could have a baseline for comparison again

Best wishes,
Kah Huat, Koh

[ernest]

Komodo replaced rybka

Gosh, didn't see that!

Thanks, Uri.