Testing Stockfish 11-03-13. 480 Games.

[Tomcass]

TESTING STOCKFISH 230415 BULLET = 1440 GAMES

(Please note that a new 8 cores computer has substituted my old 4 cores one)

6 real cores
Ponder: Off.
GUI: Fritz 14
Book: Perfect 2014t – Limit 8 moves. Sedat –
No tablebases. No RTB used.
Hash 512
Relative Speed: 29.54
Knodes per second: 14.177

Time Control 4 + 0

Stockfish 230415b 64 SSE42_ - Houdini 4 x64_st_X6_CT0 151.0 - 89.0 +77/=148/-15 62.92%
Stockfish 230415b 64 SSE42_ - Komodo 8 64-bit_6_NOB 146.5 - 92.5 +74/=145/-20 61.30%
Stockfish 230415b 64 SSE42_ - Gull 3 x64 XP 164.0 - 76.0 +102/=124/-14 68.33%

Score: 461.5 – 258.5 = 64.10%

720 Games:
http://www.mediafire.com/download/6i...0_720Games.pgn

8 real cores
Ponder: Off
GUI: Fritz 14
Book: Perfect 2014t – Limit 8 moves. Sedat –
No tablebases. No RTB used.
Hash 512
Relative Speed: 44.92
Knodes per Second: 21.562

Time Control: 2+2

Stockfish 230415b 64 SSE42_ - Houdini 4 Pro x64x8 167.5 - 72.5 +111/=113/-16 69.79%
Stockfish 230415b 64 SSE42_ - Komodo 8 64-bit_8 146.0 - 94.0 +71/=150/-19 60.83%
Stockfish 230415b 64 SSE42_ - Gull 3 x64 XP_8 165.5 - 74.5 +101/=129/-10 68.96%

Score: 479.0 – 241.9 = 66.53%

720 Games:
http://www.mediafire.com/view/x5zj0x...2_720Games.htm

Against : Houdini 4.0 St. Ct0 (3136) = 66.34% ; Komodo 8 (3153) = 61.06%, Gull 3 XP (3103) = 68.64%

GLOBAL SCORE: 940.5 – 499.5= 65.31%

Average Estimated Elo Opponents = 3131 Estimated Elo Performance= 3238

This is a new best score ever in my tests. Stockfish Bullet’s compiles are extremely strong.

Regards,

Tom

[Ozymandias]

(Please note that a new 8 cores computer has substituted my old 4 cores one)

Intel® Core™ i7-5960X?

[Tomcass]

Hi, Juan.

Yes, you are right. This is the new 8 cores computer I am using in my tests.

INT751 PROCESADOR INTEL CORE i7 5960X 3GHZ SK201-3 20MB 140W

Un abrazo!

Tom

[Ozymandias]

[ernest]

INT751 PROCESADOR INTEL CORE i7 5960X 3GHZ SK201-3 20MB 140W

No overclocking?

[Tomcass]

Hi Ernest,

I don't like hard overclocking. I have increased the clock only 10% from 3 to 3,3 Ghz. This speed can be increased by the computer -with no intervention from my side- through a turbo process until 3,8 Mhz. but this does not happen very often.

I intend that my computers last around 10 years and they usualy work 24 hours per day in my tests, besides other professional uses.

Kind regards from Barcelona.

Tom.

[Tomcass]

TESTING STOCKFISH 180515 DEVELOPMENT = 480 GAMES
Timestamp: 1431982770 Bench: 8098149

6 real cores
Ponder: Off.
GUI: Fritz 14
Book: Perfect 2014t – Limit 8 moves. Sedat –
No tablebases. No RTB used.
Hash 512
Relative Speed: 29.54
Knodes per second: 14.177

Time Control 4 + 0
Stockfish 180515 64 POPCNT_ - Komodo 9.01 64-bit_X6 20.5 - 19.5 +7/=27/-6 51.25%
Stockfish 180515 64 POPCNT_ - Houdini 4 x64_st_X6_CT0 27.0 - 13.0 +16/=22/-2 67.50%
Stockfish 180515 64 POPCNT_ - Gull 3 x64 XP 28.5 - 11.5 +19/=19/-2 71.25%
Score: 76.0 – 44.0 = 63.33%

Time Control 2 + 2
Stockfish 180515 64 POPCNT_ - Komodo 9.01 64-bit_X6 19.5 - 20.5 +4/=31/-5 48.75%
Stockfish 180515 64 POPCNT_ - Houdini 4 x64_st_X6_CT0 25.0 - 15.0 +11/=28/-1 62.50%
Stockfish 180515 64 POPCNT_ - Gull 3 x64 XP 28.0 - 12.0 +18/=20/-2 70.00%
Score: 72.5 – 47.5 = 60.42%

240 Games using 6 cores: 148.5 – 91.5= 61.87%
http://www.mediafire.com/download/oi...6_240Games.pgn

8 real cores
Ponder: Off
GUI: Fritz 14
Book: Perfect 2014t – Limit 8 moves. Sedat –
No tablebases. No RTB used.
Hash 512
Relative Speed: 44.92
Knodes per Second: 21.562

Time Control: 4 + 0
Stockfish 180515 64 BMI2x8 - Komodo 9.01 64-bit-X8 19.0 - 21.0 +5/=28/-7 47.50%
Stockfish 180515 64 BMI2x8 - Houdini 4 Pro x64_Ct0_8 23.0 - 17.0 +9/=28/-3 57.50%
Stockfish 180515 64 BMI2x8 - Gull 3 x64 XP_8 27.0 - 13.0 +17/=20/-3 67.50%
Score: 69.0 – 51.0 = 57.5%

Time Control: 2 + 2
Stockfish 180515 64 BMI2x8 - Komodo 9.01 64-bit-X8 22.5 - 17.5 +10/=25/-5 56.25%
Stockfish 180515 64 BMI2x8 - Houdini 4 Pro x64_Ct0_8 23.5 - 16.5 +13/=21/-6 58.75%
Stockfish 180515 64 BMI2x8 - Gull 3 x64 XP_8 25.0 - 15.0 +12/=26/-2 62.50%
Score: 71.0 – 49.0 = 59.17%

240 Games using 8 cores: 140.0 – 100.0 = 58.33%
http://www.mediafire.com/download/un...240GamesOK.pgn

Against : Komodo 9.01 (3219) = 50.94% Houdini 4 (3136) = 61.56%, Gull 3 XP (3103) = 67.81%

GLOBAL SCORE: 288.5 – 191.5 = 60.10%
Average Estimated Elo Opponents = 3153
Estimated Elo Performance= 3224

This is the best score for a Stockfish Development in my tests, only 14 ELO points below the best performer, SF 230415 Bullet. The previous best SF Development was 130315 version, with an Estimated Elo Performance of 3219 after 1440 games. I will extend this test also till 1440 Games in order to get a more precise vision of the strength of this version.

Regards,

Tom.

[voyagerOne]

Aren't you comparing apples with oranges?

One test you use Komodo8. (SF 230415 Bullet)
The latest test is Komodo9.

[Tomcass]

I don't think so. Obviously I use Komodo 8 with its ELO and Komodo 9.01 with its ELO. I have done the same system in the last 10 years of testing, when a new engine appears.

Here you have the calculation of the Average Estimated Elo of the Opponents:

Houdini 4.0 St. Ct0 (3136) Komodo 8 (3153) Gull 3 XP (3103)
Average Estimated Elo Opponents = 3131

Komodo 9.01 (3219) Houdini 4 (3136) Gull 3 XP (3103)
Average Estimated Elo Opponents = 3153

Tom.

[Tomcass]

TESTING STOCKFISH 180515 DEVELOPMENT = 960 GAMES (LEG 2 and 3)

Timestamp: 1431982770 Bench: 8098149

6 real cores
Ponder: Off.
GUI: Fritz 14
Book: Perfect 2014t – Limit 8 moves. Sedat –
No tablebases. No RTB used.
Hash 512
Relative Speed: 29.54
Knodes per second: 14.177

Time Control 4 + 0

Stockfish 180515 64 POPCNT_ - Komodo 9.01 64-bit_X6 37.5 - 42.5 +11/=53/-16 46.88%
Stockfish 180515 64 POPCNT_ - Houdini 4 x64_st_X6_CT0 48.0 - 32.0 +26/=44/-10 60.00%
Stockfish 180515 64 POPCNT_ - Gull 3 x64 XP 52.0 - 28.0 +31/=42/-7 65.00%

Time Control 2 + 2

Stockfish 180515 64 POPCNT_ - Komodo 9.01 64-bit_X6 33.5 - 46.5 +6/=55/-19 41.88%
Stockfish 180515 64 POPCNT_ - Houdini 4 x64_st_X6_CT0 51.5 - 28.5 +27/=49/-4 64.38%
Stockfish 180515 64 POPCNT_ - Gull 3 x64 XP 51.5 - 28.5 +28/=47/-5 64.38%

Score: 274.0 – 206.0 = 57.08%
480 Games using 6 cores:
http://www.mediafire.com/download/bb113 ... 0Games.pgn

8 real cores
Ponder: Off
GUI: Fritz 14
Book: Perfect 2014t – Limit 5 moves. Sedat –
No tablebases. No RTB used.
Hash 512
Relative Speed: 44.92
Knodes per Second: 21.562

Time Control: 4 + 0

Stockfish 180515 64 BMI2x8 - Komodo 9.01 64-bit-X8 38.5 - 41.5 +13/=51/-16 48.13%
Stockfish 180515 64 BMI2x8 - Houdini 4 Pro x64_Ct0_8 49.0 - 31.0 +20/=58/-2 61.25%
Stockfish 180515 64 BMI2x8 - Gull 3 x64 XP_8 57.0 - 23.0 +37/=40/-3 71.25%

Time Control 2 + 2

Stockfish 180515 64 BMI2x8 - Komodo 9.01 64-bit-X8 40.5 - 39.5 +15/=51/-14 50.63%
Stockfish 180515 64 BMI2x8 - Houdini 4 Pro x64_Ct0_8 49.0 - 31.0 +25/=48/-7 61.25%
Stockfish 180515 64 BMI2x8 - Gull 3 x64 XP_8 57.0 - 23.0 +35/=44/-1 71.25%

Score= 291.0 – 189.0 = 60.62%
480 Games using 8 cores:
http://www.mediafire.com/download/bsqon ... amesOK.pgn

GLOBAL SCORE AFTER 1440 GAMES= 853.5 – 586.5 = 59.27%

Against : Komodo 9.01 (3219) = 48.23% Houdini 4 (3136) = 61.67%, Gull 3 XP (3103) = 67.91%

Average Estimated Elo Opponents = 3153
Estimated Elo Performance= 3218

Unfortunately after 1440 games, STOCKFISH 180515 DEVELOPMENT has not confirmed the improvement found after the first 480 games leg. It has performed almost exactly equal than the best SF Development in my tests so far: STOCKFISH 130315 DEVELOPMENT (3219).

Regards from Barcelona,

Tom.