Testing Stockfish 11-03-13. 480 Games.

Discussion of computer chess matches and engine tournaments.

Moderator: Ras

Vinvin
Posts: 5297
Joined: Thu Mar 09, 2006 9:40 am
Full name: Vincent Lejeune

Re: Timestamps of Stockfish.

Post by Vinvin »

This version is a killer !! :twisted:
Top
Tomcass
Posts: 786
Joined: Sun Apr 16, 2006 9:09 pm

Re: Timestamps of Stockfish.

Post by Tomcass »

480 GAMES= STOCKFISH 09/06/2013 STANDARD

I7 980 3.33 Ghz.
6 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2012
Time control: 4 min +0 sec/ game
No tablebases

Stockfish 090613 64 SSE4.26 - Critter 1.6 64-bitx6_nob 19.0 - 21.0 +6/=26/-8 47.50%
Stockfish 090613 64 SSE4.26 - Deep Rybka 4.1 SSE42 x64 (x6) 21.5 - 18.5 +12/=19/-9 53.75%
Stockfish 090613 64 SSE4.26 - Houdini 3 Pro x64_6 14.5 - 25.5 +7/=15/-18 36.25%

I7 980 3.33 Ghz.
6 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2012
Time control: 2min +2 sec/ game
No tablebases
Stockfish 090613 64 SSE4.26 - Critter 1.6 64-bitx6_nob 20.5 - 19.5 +5/=31/-4 51.25%
Stockfish 090613 64 SSE4.26 - Deep Rybka 4.1 SSE42 x64 (x6) 22.0 - 18.0 +12/=20/-8 55.00%
Stockfish 090613 64 SSE4.26 - Houdini 3 Pro x64_6 15.5 - 24.5 +7/=17/-16 38.75%

X6 240 Games: http://www.mediafire.com/?wbbvrtolannegq5

Overall average with 6 cores: (113.0-127.0) = 47.08%

i7 975 3.33 Ghz.
4 real cores
GUI: Fritz 12
Book: Fritz 12
Time control: 4 min + 0 sec/ game
Ponder: Off
No tablebases

Stockfish 090613 64 SSE4.2_ - Critter 1.6a 64-bitnob_4 21.0 - 19.0 +7/=28/-5 52.50%
Stockfish 090613 64 SSE4.2_ - Deep Rybka 4 x64 (x4) 24.5 - 15.5 +16/=17/-7 61.25%
Stockfish 090613 64 SSE4.2_ - Houdini 3 Pro x64_4 17.5 - 22.5 +3/=29/-8 43.75%

i7 975 3.33 Ghz.
4 real cores
GUI: Fritz 12
Book: Fritz 12
Time control: 2 min + 2 sec/ game
Ponder: Off
No tablebases

Stockfish 090613 64 SSE4.2_ - Critter 1.6a 64-bitnob_4 20.5 - 19.5 +5/=31/-4 51.25%
Stockfish 090613 64 SSE4.2_ - Deep Rybka 4 x64 (x4) 22.0 - 18.0 +10/=24/-6 55.00%
Stockfish 090613 64 SSE4.2_ - Houdini 3 Pro x64_4 15.0 - 25.0 +5/=20/-15 37.50%

X4 240 games: http://www.mediafire.com/?82t66j26p2d9io3

Overall average with 4 cores: (120.5-119.5) = 50.21%

Global Score: 233.5-246.5 = 48.65%
Against Critter 1.6: 50.62% Deep Rybka 4: 56.25% Houdini 3.0: 39.06%
Estimated ELO Performance: 3.085


Regards,

Tom.
Top
Tomcass
Posts: 786
Joined: Sun Apr 16, 2006 9:09 pm

Re: Timestamps of Stockfish.

Post by Tomcass »

480 GAMES= STOCKFISH 13/06/2013 STANDARD

I7 980 3.33 Ghz.
6 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2012
Time control: 4 min +0 sec/ game
No tablebases

Stockfish 160613 64 SSE4.2_ - Critter 1.6 64-bitx6_nob 20.5 - 19.5 +7/=27/-6 51.25%
Stockfish 160613 64 SSE4.2_ - Deep Rybka 4.1 SSE42 x64 (x6) 22.5 - 17.5 +12/=21/-7 56.25%
Stockfish 160613 64 SSE4.2_ - Houdini 3 Pro x64_6 14.0 - 26.0 +5/=18/-17 35.00%

I7 980 3.33 Ghz.
6 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2012
Time control: 2min +2 sec/ game
No tablebases

Stockfish 160613 64 SSE4.2_ - Critter 1.6 64-bitx6_nob 23.5 - 16.5 +13/=21/-6 58.75%
Stockfish 160613 64 SSE4.2_ - Deep Rybka 4.1 SSE42 x64 (x6) 26.5 - 13.5 +17/=19/-4 66.25%
Stockfish 160613 64 SSE4.2_ - Houdini 3 Pro x64_6 16.0 - 24.0 +6/=20/-14 40.00%

X6 240 games = http://www.mediafire.com/?k8iwnjlhicjibjw

Overall average with 6 cores: (123.0-117.0) = 51.25%


i7 975 3.33 Ghz.
4 real cores
GUI: Fritz 12
Book: Fritz 12
Time control: 4 min + 0 sec/ game
Ponder: Off
No tablebases

Stockfish 160613 64 SSE4.2_ - Critter 1.6a 64-bitnob_4 23.0 - 17.0 +9/=28/-3 57.50%
Stockfish 160613 64 SSE4.2_ - Deep Rybka 4 x64 (x4) 21.0 - 19.0 +9/=24/-7 52.50%
Stockfish 160613 64 SSE4.2_ - Houdini 3 Pro x64_4 13.5 - 26.5 +3/=21/-16 33.75%



i7 975 3.33 Ghz.
4 real cores
GUI: Fritz 12
Book: Fritz 12
Time control: 2 min + 2 sec/ game
Ponder: Off
No tablebases

Stockfish 160613 64 SSE4.2_ - Critter 1.6a 64-bitnob_4 20.0 - 20.0 +7/=26/-7 50.00%
Stockfish 160613 64 SSE4.2_ - Deep Rybka 4 x64 (x4) 21.5 - 18.5 +10/=23/-7 53.75%
Stockfish 160613 64 SSE4.2_ - Houdini 3 Pro x64_4 17.0 - 23.0 +3/=28/-9 42.50%

X4 240 GAMES = http://www.mediafire.com/?4ub2c8nbqd90tw3

Overall average with 4 cores: (116.0-124.0) = 48.33%


Global Score: 239.0-241.0 = 49.79%

Against Critter 1.6: 54.37% Deep Rybka 4: 57.19% Houdini 3.0: 37.81%

Estimated ELO Performance: 3.093

Regards,

Tom.
Top
Tomcass
Posts: 786
Joined: Sun Apr 16, 2006 9:09 pm

Re: Timestamps of Stockfish.

Post by Tomcass »

480 GAMES= STOCKFISH 19/06/2013 STANDARD

I7 980 3.33 Ghz.
6 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2012
Time control: 4 min +0 sec/ game
No tablebases

Stockfish 190613 64 SSE4.2_ - Critter 1.6 64-bitx6_nob 21.0 - 19.0 +13/=16/-11 52.50%
Stockfish 190613 64 SSE4.2_ - Deep Rybka 4.1 SSE42 x64 26.0 - 14.0 +17/=18/-5 65.00%
Stockfish 190613 64 SSE4.2_ - Houdini 3 Pro x64_6 16.0 - 24.0 +6/=20/-14 40.00%

I7 980 3.33 Ghz.
6 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2012
Time control: 2min +2 sec/ game
No tablebases
Stockfish 190613 64 SSE4.2_ - Critter 1.6 64-bitx6_nob 18.0 - 22.0 +8/=20/-12 45.00%
Stockfish 190613 64 SSE4.2_ - Deep Rybka 4.1 SSE42 x64 25.0 - 15.0 +17/=16/-7 62.50%
Stockfish 190613 64 SSE4.2_ - Houdini 3 Pro x64_6 10.5 - 29.5 +1/=19/-20 26.25%

X6 240 games = http://www.mediafire.com/?96u0y8rgzt66vck
Overall average with 6 cores: (116.5-123.5) = 48.54%



i7 975 3.33 Ghz.
4 real cores
GUI: Fritz 12
Book: Fritz 12
Time control: 4 min + 0 sec/ game
Ponder: Off
No tablebases

20130618_Komodo5.1MPnob_2+2_x4 2013

Komodo 5.1 64-bitx4nob - Critter 1.6a 64-bitnob_4 24.0 - 16.0 +12/=24/-4 60.00%
Komodo 5.1 64-bitx4nob - Deep Rybka 4 x64 (x4) 23.0 - 17.0 +12/=22/-6 57.50%
Komodo 5.1 64-bitx4nob - Houdini 3 Pro x64_4 17.5 - 22.5 +6/=23/-11 43.75%




i7 975 3.33 Ghz.
4 real cores
GUI: Fritz 12
Book: Fritz 12
Time control: 2 min + 2 sec/ game
Ponder: Off
No tablebases

Stockfish 190613 64 SSE4.2_ - Critter 1.6a 64-bitnob_4 20.5 - 19.5 +7/=27/-6 51.25%
Stockfish 190613 64 SSE4.2_ - Deep Rybka 4 x64 (x4) 25.0 - 15.0 +14/=22/-4 62.50%
Stockfish 190613 64 SSE4.2_ - Houdini 3 Pro x64_4 15.5 - 24.5 +1/=29/-10 38.75%

X4 240 Games = http://www.mediafire.com/?nmhvs834a1ll7rv

Overall average with 4 cores: (125.5-114.5) = 52.29%


Global Score: 242.0-238.0 = 50.42%

Against Critter 1.6: 52.19% Deep Rybka 4: 61.87% Houdini 3.0: 37.19%

Estimated ELO Performance: 2.907

Regards,

Tom.
Top
Tomcass
Posts: 786
Joined: Sun Apr 16, 2006 9:09 pm

Re: Timestamps of Stockfish.

Post by Tomcass »

... wooops!

There was a mistake in my previous post.

The Estimated ELO Performance: was 3.097 (not 2.907 as I wrote wrongly). :oops:

Sorry!

Tom.
Top
Tomcass
Posts: 786
Joined: Sun Apr 16, 2006 9:09 pm

Re: Timestamps of Stockfish.

Post by Tomcass »

480 GAMES= STOCKFISH 220613 STANDARD

I7 980 3.33 Ghz.
6 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2012
Time control: 4 min +0 sec/ game
No tablebases

Stockfish 220613 64 SSE4.2_ - Critter 1.6 64-bitx6_nob 18.5 - 21.5 +8/=21/-11 46.25%
Stockfish 220613 64 SSE4.2_ - Deep Rybka 4.1 SSE42 x64 (x6) 24.5 - 15.5 +14/=21/-5 61.25%
Stockfish 220613 64 SSE4.2_ - Houdini 3 Pro x64_6 20.0 - 20.0 +11/=18/-11 50.00%

I7 980 3.33 Ghz.
6 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2012
Time control: 2min +2 sec/ game
No tablebases

Stockfish 220613 64 SSE4.2_ - Critter 1.6 64-bitx6_nob 18.5 - 21.5 +7/=23/-10 46.25%
Stockfish 220613 64 SSE4.2_ - Deep Rybka 4.1 SSE42 x64 (x6) 23.0 - 17.0 +13/=20/-7 57.50%
Stockfish 220613 64 SSE4.2_ - Houdini 3 Pro x64_6 19.5 - 20.5 +11/=17/-12 48.75%

X6 240 GAMES = http://www.mediafire.com/?ik6m9a5xp25y96s

Overall average with 6 cores: (124.0-116.00) = 51,67%

i7 975 3.33 Ghz.
4 real cores
GUI: Fritz 12
Book: Fritz 12
Time control: 4 min + 0 sec/ game
Ponder: Off
No tablebases

Stockfish 220613 64 SSE4.2S - Critter 1.6a 64-bitnob_4 22.5 - 17.5 +10/=25/-5 56.25%
Stockfish 220613 64 SSE4.2S - Deep Rybka 4 x64 (x4) 23.5 - 16.5 +11/=25/-4 58.75%
Stockfish 220613 64 SSE4.2S - Houdini 3 Pro x64_4 18.0 - 22.0 +6/=24/-10 45.00%

i7 975 3.33 Ghz.
4 real cores
GUI: Fritz 12
Book: Fritz 12
Time control: 2 min + 2 sec/ game
Ponder: Off
No tablebases

Stockfish 220613 64 SSE4.2S - Critter 1.6a 64-bitnob_4 22.0 - 18.0 +9/=26/-5 55.00%
Stockfish 220613 64 SSE4.2S - Deep Rybka 4 x64 (x4) 25.0 - 15.0 +11/=28/-1 62.50%
Stockfish 220613 64 SSE4.2S - Houdini 3 Pro x64_4 17.5 - 22.5 +6/=23/-11 43.75%

X4 240 GAMES = http://www.mediafire.com/?d6bz2mbh1lhxzmi

Overall average with 4 cores: (128.5-111.5) = 53.54%

Global Score: 252.5- 227.5 = 52.60%

Against Critter 1.6: 50.94% Deep Rybka 4: 60.00% Houdini 3.0: 46.87%

Estimated ELO Performance: 3.112

A very good score for Stockfish, the second best ever in my tests. Please pay attention to its impressive performance against Houdini 3.0 Pro.

Regards,

Tom.
Top
Tomcass
Posts: 786
Joined: Sun Apr 16, 2006 9:09 pm

Re: Timestamps of Stockfish.

Post by Tomcass »

STOCKFISH 230613 WITH CONTEMPT 27 = 480 GAMES

I7 980 3.33 Ghz.
6 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2012
Time control: 4 min +0 sec/ game
No tablebases

201306SF230616ST766_CT27_4+0_120games 2013

Stockfish 230613 64 SSE4.2X - Critter 1.6 64-bitx6_nob 18.5 - 21.5 +8/=21/-11 46.25%
Stockfish 230613 64 SSE4.2X - Deep Rybka 4.1 SSE42 x64 (x6) 24.0 - 16.0 +15/=18/-7 60.00%
Stockfish 230613 64 SSE4.2X - Houdini 3 Pro x64_6 20.0 - 20.0 +12/=16/-12 50.00%



I7 980 3.33 Ghz.
6 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2012
Time control: 2min +2 sec/ game
No tablebases

201306SF230616ST766_CT27_2+2_120games 2013

Stockfish 230613 64 SSE4.2X - Critter 1.6 64-bitx6_nob 22.0 - 18.0 +11/=22/-7 55.00%
Stockfish 230613 64 SSE4.2X - Deep Rybka 4.1 SSE42 x64 (x6) 23.0 - 17.0 +12/=22/-6 57.50%
Stockfish 230613 64 SSE4.2X - Houdini 3 Pro x64_6 13.5 - 26.5 +6/=15/-19 33.75%

X6 240 GAMES = http://www.mediafire.com/?b6vvl277tj6zc8t
Overall average with 6 cores: (121.0 – 119.0) = 50.42%


i7 975 3.33 Ghz.
4 real cores
GUI: Fritz 12
Book: Fritz 12
Time control: 4 min + 0 sec/ game
Ponder: Off
No tablebases

Stockfish 230613 64 SSE4.2X - Critter 1.6a 64-bitnob_4 22.0 - 18.0 +14/=16/-10 55.00%
Stockfish 230613 64 SSE4.2X - Deep Rybka 4 x64 (x4) 27.0 - 13.0 +17/=20/-3 67.50%
Stockfish 230613 64 SSE4.2X - Houdini 3 Pro x64_4 18.5 - 21.5 +11/=15/-14 46.25%


i7 975 3.33 Ghz.
4 real cores
GUI: Fritz 12
Book: Fritz 12
Time control: 2 min + 2 sec/ game
Ponder: Off

Stockfish 230613 64 SSE4.2_ - Critter 1.6a 64-bitnob_4 21.5 - 18.5 +9/=25/-6 53.75%
Stockfish 230613 64 SSE4.2_ - Deep Rybka 4 x64 (x4) 22.0 - 18.0 +11/=22/-7 55.00%
Stockfish 230613 64 SSE4.2_ - Houdini 3 Pro x64_4 14.5 - 25.5 +4/=21/-15 36.25%

X4 240 GAMES = http://www.mediafire.com/?hoglla9puqomdbh
Overall average with 4 cores: (125.5-114.5) = 52.29%

Global Score: 246.5-233.5 51.35%

Against Critter 1.6: 52.50% Deep Rybka 4: 60.00% Houdini 3.0: 41.56%
Estimated ELO Performance: 3.103


Fixed Time Control 4+0 = 54.17% (New best score at fixed TC).
Against Critter 1.6: 50.62% Deep Rybka 4: 63.75% Houdini 3.0: 48.12%

Incremental Time Control 2+2= 48.54%
Against Critter 1.6: 54.37% Deep Rybka 4: 56.25% Houdini 3.0: 35.00%

Contempt 27 seems very powerful at fixed time controls, but not at incremental ones. IMHO the difference is too big –after 480 games- to be due only to statistical noise. Any comment from statistical experts will be appreciated.

Regards,

Tom.
Top
Tomcass
Posts: 786
Joined: Sun Apr 16, 2006 9:09 pm

Re: Timestamps of Stockfish.

Post by Tomcass »

TESTING STOCKFISH 020713 (Timestamp: 1372742828)= 480 GAMES.

i7 980 3.33 Ghz.
6 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2012
Time control: 4 min +0 sec/ game
No tablebases
Relative Speed: 28.66
Knodes per second: 13.759

Time control 4+0

Stockfish 020713 64 SSE4.2_ - Critter 1.6 64-bitx6_nob 19.0 - 21.0 +7/=24/-9 47.50%
Stockfish 020713 64 SSE4.2_ - Deep Rybka 4.1 SSE42 x64 24.5 - 15.5 +16/=17/-7 61.25%
Stockfish 020713 64 SSE4.2_ - Houdini 3 Pro x64_6 17.0 - 23.0 +9/=16/-15 42.50%

Time control 2+2

Stockfish 020713 64 SSE4.2_ - Critter 1.6 64-bitx6_nob 23.5 - 16.5 +12/=23/-5 58.75%
Stockfish 020713 64 SSE4.2_ - Deep Rybka 4.1 SSE42 x64 25.5 - 14.5 +14/=23/-3 63.75%
Stockfish 020713 64 SSE4.2_ - Houdini 3 Pro x64_6 15.5 - 24.5 +6/=19/-15 38.75%

240 GAMES x6 = http://www.mediafire.com/?211yerd701ee6p8

Score using 6 real cores = 125.0-115.0 = 52.08%

i7 975 3.33 Ghz.
4 real cores
GUI: Fritz 12
Book: Fritz 12
Ponder: Off
No tablebases
Relative Speed: 20.62
Knodes per second: 9.899

Time control 4+0

Stockfish 020713 64 SSE4.2_ - Critter 1.6a 64-bitnob_4 23.0 - 17.0 +12/=22/-6 57.50%
Stockfish 020713 64 SSE4.2_ - Deep Rybka 4 x64 (x4) 28.0 - 12.0 +17/=22/-1 70.00%
Stockfish 020713 64 SSE4.2_ - Houdini 3 Pro x64_4 15.5 - 24.5 +5/=21/-14 38.75%

Time control 2+2

Stockfish 020713 64 SSE4.2_ - Critter 1.6a 64-bitnob_4 23.5 - 16.5 +10/=27/-3 58.75%
Stockfish 020713 64 SSE4.2_ - Deep Rybka 4 x64 (x4) 21.5 - 18.5 +10/=23/-7 53.75%
Stockfish 020713 64 SSE4.2_ - Houdini 3 Pro x64_4 18.5 - 21.5 +8/=21/-11 46.25%

240 GAMES x4 = http://www.mediafire.com/?9mc6cx0ud5ep9sm

Score using 4 cores = 130.0-110.0 = 54.17%

Global Score: 255.0 – 225.0 = 53.12%
Against Critter 1.6: 55.62% Deep Rybka 4: 62.19% Houdini 3.0: 41.56%
Estimated ELO Performance: 3116

New best score for Stockfish in my tests … although improving just a little bit over the previous best. (Sergei Bubka’s effect). :wink:

Please note that, according to my tests and having all respect to statistical noise, this score is only 2 estimated Elo points below Komodo 5.1r1 (3118).

Regards,

Tom.[/b]
Top
mcostalba
Posts: 2684
Joined: Sat Jun 14, 2008 9:17 pm

Re: Timestamps of Stockfish.

Post by mcostalba »

Thanks Tom for running this.

Can I suggest to use a standard way of getting the timestamp ?

Timestamp: 1372742828

Is un-useful to us because we don't now how you get it, so it is impossible to pin point the exact commit.

Could you please use:

stockfish bench

As a way to get the signature ?
Top
User avatar
Ajedrecista
Posts: 2122
Joined: Wed Jul 13, 2011 9:04 pm
Location: Madrid, Spain.

Re: Timestamps of Stockfish.

Post by Ajedrecista »

Hi Marco:
mcostalba wrote:Thanks Tom for running this.

Can I suggest to use a standard way of getting the timestamp ?

Timestamp: 1372742828

Is un-useful to us because we don't now how you get it, so it is impossible to pin point the exact commit.

Could you please use:

stockfish bench

As a way to get the signature ?
The autocompiles are downloaded from here:

Stockfish Development Versions
Stockfish Development Versions are build automatically if there are changes on the master branch in the git repository (https://github.com/mcostalba/Stockfish).
They are compiled with gcc 4.7/mingw on Ubuntu 12.10.
6.Jan.2013: now with link time optimization for windows too.
You can search there some info about each autocompile. For example, Timestamp: 1372742828 has the following information attached:
Author: Marco Costalba
Date: Tue Jul 2 07:27:08 2013 +0200
Timestamp: 1372742828

Merge branch 'master' into aspiration

bench: 4507288
I hope this is the info you was looking for. Good luck with the future development of SF!

Regards from Spain.

Ajedrecista.
Top

Return to “Computer Chess Club: Tournaments and Matches”