Testing Stockfish 11-03-13. 480 Games.

Discussion of computer chess matches and engine tournaments.

Moderator: Ras

Tomcass
Posts: 786
Joined: Sun Apr 16, 2006 9:09 pm

Re: Timestamps of Stockfish..SF_0308 - Komodo 51MP 58%

Post by Tomcass »

Kohflote wrote:Hi,

Yes, we have lost the baseline to compare different versions of SF. It will be great if Tom could test SF 250813 version (the best version tested by him) against Komodo so that we could have a baseline for comparison again :)

Best wishes,
Kah Huat, Koh
Hello, Kah Huat Koh.

I think that your suggestion of testing SF 250813 version (the best version till now in my tests) against Komodo to have a baseline for comparison is very good. I will do it tomorrow Monday. Thanks, my friend. :wink:

Regards,

Tom.
Top
Tomcass
Posts: 786
Joined: Sun Apr 16, 2006 9:09 pm

Re: Timestamps of Stockfish..SF_0308 - Komodo 51MP 58%

Post by Tomcass »

Stockfish 300813 = 480 GAMES

Bench: 4620975 Timestamp: 1377873738

i7 980 3.33 Ghz.
6 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2012c
No tablebases
Hash 512
Relative Speed: 28.66
Knodes per second: 13.759

Time Control_4+0

Stockfish 300813 64 SSE4.2_ - Critter 1.6a 64-bit_NOB 25.0 - 15.0 +17/=16/-7 62.50%
Stockfish 300813 64 SSE4.2_ - Houdini 3 Pro x64_6 23.0 - 17.0 +13/=20/-7 57.50%
Stockfish 300813 64 SSE4.2_ - Komodo 5.1r2 64-bitx6nob 21.0 - 19.0 +11/=20/-9 52.50%

Time Control_2+2

Stockfish 300813 64 SSE4.2_ - Critter 1.6a 64-bit_NOB 26.0 - 14.0 +16/=20/-4 65.00%
Stockfish 300813 64 SSE4.2_ - Houdini 3 Pro x64_6 21.0 - 19.0 +12/=18/-10 52.50%
Stockfish 300813 64 SSE4.2_ - Komodo 5.1r2 64-bitx6nob 19.0 - 21.0 +6/=26/-8 47.50%

240 Games x6 cores= http://www.mediafire.com/?pb4m0d59yvr1ydv
Score using 6 cores = 135.0-105.0 = 56.25%

i7 975 3.33 Ghz.
4 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2012c
No tablebases
Hash 256
Relative Speed: 20.62
Knodes per second: 9.899

Time Control_4+0

Stockfish 300813 64 SSE4.2_ - Critter 1.6a 64-bitnob_4 20.5 - 19.5 +7/=27/-6 51.25%
Stockfish 300813 64 SSE4.2_ - Houdini 3 Pro x64_4 17.0 - 23.0 +5/=24/-11 42.50%
Stockfish 300813 64 SSE4.2_ - Komodo 5.1r2 64-bitnob 20.5 - 19.5 +10/=21/-9 51.25%

Time Control_2+2
Stockfish 300813 64 SSE4.2_ - Critter 1.6a 64-bitnob_4 21.5 - 18.5 +10/=23/-7 53.75%
Stockfish 300813 64 SSE4.2_ - Houdini 3 Pro x64_4 18.5 - 21.5 +6/=25/-9 46.25%
Stockfish 300813 64 SSE4.2_ - Komodo 5.1r2 64-bitnob 20.0 - 20.0 +8/=24/-8 50.00%

240 Games X4 = http://www.mediafire.com/?1h3b3y8mujqsev8
Score using 4 cores= 118.0-122.0 = 49.17%

Segmenting the result by Time Control:

Fixed = 52.92%
Incremental = 52.50%

Global Score: 253.0–227.0 = 52.71%
Against Critter 1.6a (3093): 58.12% Houdini 3.0 Pro (3172): 49.69% Komodo 5.1r2 (3135): 50.31%

Average ELO of opponents= 3133
Estimated ELO Performance= 3152

According to this test the latest version of SF development is roughly 20 ELO points below Houdini 3.0 Pro. Brilliant!.

Regards,

Tom.
Top
Tomcass
Posts: 786
Joined: Sun Apr 16, 2006 9:09 pm

Re: Timestamps of Stockfish..SF_0308 - Komodo 51MP 58%

Post by Tomcass »

Tomcass wrote:
Kohflote wrote:Hi,

Yes, we have lost the baseline to compare different versions of SF. It will be great if Tom could test SF 250813 version (the best version tested by him) against Komodo so that we could have a baseline for comparison again :)

Best wishes,
Kah Huat, Koh
Hello, Kah Huat Koh.

I think that your suggestion of testing SF 250813 version (the best version till now in my tests) against Komodo to have a baseline for comparison is very good. I will do it tomorrow Monday. Thanks, my friend. :wink:

Regards,

Tom.
The 'baseline' suggested by Kah Huat Koh, introducing Komodo 5.1r2 instead of Deep Ryka 4:

160 Games Stockfish 250813 – Komodo 5.1r2

i7 980 3.33 Ghz.
6 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2012c
No tablebases
Hash 512
Relative Speed: 28.66
Knodes per second: 13.759

Time Control_4+0

1 Komodo 5.1r2 64-bitx6nob +8/=26/-6 52.50% 21.0/40
2 Stockfish 250813 64 SSE4.2_ +6/=26/-8 47.50% 19.0/40

Time Control_2+2

1 Stockfish 250813 64 SSE4.2_ +13/=18/-9 55.00% 22.0/40
2 Komodo 5.1r2 64-bitx6nob +9/=18/-13 45.00% 18.0/40

80 Games X6 = http://www.mediafire.com/?6fxgdo6tsnfykqv
Score using 6 cores (Stockfish) = 41.0-39.0 = 51.25%

i7 975 3.33 Ghz.
4 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2012c
No tablebases
Hash 256
Relative Speed: 20.62
Knodes per second: 9.899

Time Control_4+0

1 Komodo 5.1r2 64-bitnob +13/=18/-9 55.00% 22.0/40
2 Stockfish 250813 64 SSE4.2_ +9/=18/-13 45.00% 18.0/40

Time Control_2+2

1 Stockfish 250813 64 SSE4.2_ +15/=17/-8 58.75% 23.5/40
2 Komodo 5.1r2 64-bitnob +8/=17/-15 41.25% 16.5/40

80 Games X 4 =http://www.mediafire.com/?z598xsvufu4m5um
Score using 4 cores (Stockfish) 41.5-38.5 = 51.87%

Global Score: Stockfish 250813 – Komodo 5.1r2 82.5-77.5 = 51.56%


Stockfish 250813 = 480 GAMES (with Komodo5.1r2 instead of Deep Rybka 4)

Bench: 4728533 Timestamp: 1377448609


Stockfish 250813 64 SSE4.2_ - Critter 1.6 64-bitx6_nob 19.5 - 20.5 +7/=25/-8 48.75%
Stockfish 250813 64 SSE4.2_ - Houdini 3 Pro x64_6 17.0 - 23.0 +11/=12/-17 42.50%
Stockfish 250813 64 SSE4.2_ - Komodo 5.1r2 64-bitx6 19.0-21.0 +6/=26/-8 47.50%

Stockfish 250813 64 SSE4.2_ - Critter 1.6 64-bitx6_nob 25.5 - 14.5 +12/=27/-1 63.75%
Stockfish 250813 64 SSE4.2_ - Houdini 3 Pro x64_6 18.5 - 21.5 +8/=21/-11 46.25%
Stockfish 250813 64 SSE4.2_ - Komodo 5.1r2 64-bitx6 22.0-18.0 +13/=18/-9 55.00%

Stockfish 250813 64 SSE4.2_ - Critter 1.6a 64-bitnob_4 21.0 - 19.0 +10/=22/-8 52.50%
Stockfish 250813 64 SSE4.2_ - Houdini 3 Pro x64_4 18.5 - 21.5 +9/=19/-12 46.25%
Stockfish 250813 64 SSE4.2_ - Komodo 5.1r2 64-bitx6 18.0-22.0 +9/=18/-13 45.00%

Stockfish 250813 64 SSE4.2_ - Critter 1.6a 64-bitnob_4 26.5 - 13.5 +16/=21/-3 66.25%
Stockfish 250813 64 SSE4.2_ - Houdini 3 Pro x64 20.5 - 19.5 +7/=27/-6 51.25%
Stockfish 250813 64 SSE4.2_ - Komodo 5.1r2 64-bitx6 23.5-16.5 +15/=17/-8 58.75%

Global Score: 249.5-230.5 = 51.98%
Against Critter 1.6: 57.81% Houdini 3.0: 46.56% Komodo 5.1r2: 51.56%
Average ELO of opponents = 3133
Estimated ELO Performance: 3147

(Estimated ELO performance with Deep Rypka 4 = 3156)


Regards,

Tom.
Top
Kohflote
Posts: 240
Joined: Wed Sep 19, 2007 11:07 am
Location: Singapore

Re: Timestamps of Stockfish..SF_0308 - Komodo 51MP 58%

Post by Kohflote »

Hi Tom,

Thank you for your testing of SF 250813 with Komodo and again re-establishing the baseline for comparing different versions of SF :)

From your tests, the best version as of now is SF 300813.

Thank you & regards,
Kah Huat, Koh
Top
Tomcass
Posts: 786
Joined: Sun Apr 16, 2006 9:09 pm

Re: Timestamps of Stockfish..SF_0308 - Komodo 51MP 58%

Post by Tomcass »

Stockfish 070913 = 480 GAMES

Bench: 3864419 Timestamp: 1378549844

i7 980 3.33 Ghz.
6 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2012c
No tablebases
Hash 512
Relative Speed: 28.66
Knodes per second: 13.759

Time Control_4+0

Stockfish 070913 64 SSE4.2_ - Critter 1.6a 64-bit_NOB 22.5 - 17.5 +12/=21/-7 56.25%
Stockfish 070913 64 SSE4.2_ - Houdini 3 Pro x64_6 22.5 - 17.5 +12/=21/-7 56.25%
Stockfish 070913 64 SSE4.2_ - Komodo 5.1r2 64-bitx6nob 22.5 - 17.5 +12/=21/-7 56.25%

Time Control_2+2

Stockfish 070913 64 SSE4.2_ - Critter 1.6a 64-bit_NOB 24.5 - 15.5 +14/=21/-5 61.25%
Stockfish 070913 64 SSE4.2_ - Houdini 3 Pro x64_6 20.0 - 20.0 +9/=22/-9 50.00%
Stockfish 070913 64 SSE4.2_ - Komodo 5.1r2 64-bitx6nob 21.5 - 18.5 +11/=21/-8 53.75%

240 Games x6 cores= http://www.mediafire.com/?r4q79kb7eg1spg3
Score using 6 cores = 133.5-106.5 = 55.62%

i7 975 3.33 Ghz.
4 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2012c
No tablebases
Hash 256
Relative Speed: 20.62
Knodes per second: 9.899

Time Control_4+0

Stockfish 070913 64 SSE4.2_ - Critter 1.6a 64-bitnob_4 28.0 - 12.0 +18/=20/-2 70.00%
Stockfish 070913 64 SSE4.2_ - Houdini 3 Pro x64_4 20.0 - 20.0 +9/=22/-9 50.00%
Stockfish 070913 64 SSE4.2_ - Komodo 5.1r2 64-bitnob 21.5 - 18.5 +12/=19/-9 53.75%

Time Control_2+2

Stockfish 070913 64 SSE4.2_ - Critter 1.6a 64-bitnob_4 21.5 - 18.5 +10/=23/-7 53.75%
Stockfish 070913 64 SSE4.2_ - Houdini 3 Pro x64_4 22.0 - 18.0 +13/=18/-9 55.00%
Stockfish 070913 64 SSE4.2_ - Komodo 5.1r2 64-bitnob 21.0 - 19.0 +10/=22/-8 52.50%


240 Games X 4 cores= http://www.mediafire.com/?dikr51i1v9oj8nh
Score using 4 Cores = 134.5-105.5 =56.04%

Segmenting the result by Time Control:

Fixed = 57.08%
Incremental = 54.37%

Global Score: 267.5–212.5 = 55.73%
Against Critter 1.6a (3093): 60.31% Houdini 3.0 Pro (3172): 52.81% Komodo 5.1r2 (3135): 54.06%
Average ELO of opponents= 3133
Estimated ELO Performance= 3173

For the first time in my tests, a Stockfish version has performed a bit better than Houdini 3.0 Pro.

The improvement in the last three weeks has been impressive. Congratulations for this historical -from my viewpoint- score to Marco, Gary and all the Stockfish Team. :D

Regards from Barcelona.

Tom.
Top
Jouni
Posts: 3651
Joined: Wed Mar 08, 2006 8:15 pm
Full name: Jouni Uski

Re: Timestamps of Stockfish..SF_0308 - Komodo 51MP 58%

Post by Jouni »

Yes this progress is really SURPRISING and STUNNING!! When Marco announced SF going to Github I thought it days as top engine are soon over. How wrong you can be :) .
Jouni
Top
Tomcass
Posts: 786
Joined: Sun Apr 16, 2006 9:09 pm

Re: Timestamps of Stockfish..SF_0308 - Komodo 51MP 58%

Post by Tomcass »

Stockfish 090913 = 480 GAMES

Bench: 3864419 Timestamp: 1378646933

i7 980 3.33 Ghz.
6 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2012c
No tablebases
Hash 512
Relative Speed: 28.66
Knodes per second: 13.759

Time Control_4+0

Stockfish 090913 64 SSE4.2_ - Critter 1.6a 64-bit_NOB 22.0 - 18.0 +12/=20/-8 55.00%
Stockfish 090913 64 SSE4.2_ - Houdini 3 Pro x64_6 20.0 - 20.0 +9/=22/-9 50.00%
Stockfish 090913 64 SSE4.2_ - Komodo 5.1r2 64-bitx6nob 22.0 - 18.0 +12/=20/-8 55.00%

Time Control_2+2

Stockfish 090913 64 SSE4.2_ - Critter 1.6a 64-bit_NOB 23.5 - 16.5 +13/=21/-6 58.75%
Stockfish 090913 64 SSE4.2_ - Houdini 3 Pro x64_6 18.0 - 22.0 +10/=16/-14 45.00%
Stockfish 090913 64 SSE4.2_ - Komodo 5.1r2 64-bitnob 19.0 - 21.0 +8/=22/-10 47.50%

240 Games x6 cores= http://www.mediafire.com/?x5n8n3w2c7cwia6
Score using 6 cores = 124.5-115.5 = 51.87%

i7 975 3.33 Ghz.
4 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2012c
No tablebases
Hash 256
Relative Speed: 20.62
Knodes per second: 9.899

Time Control_4+0

Stockfish 090913 64 SSE4.2_ - Critter 1.6a 64-bitnob_4 22.5 - 17.5 +15/=15/-10 56.25%
Stockfish 090913 64 SSE4.2_ - Houdini 3 Pro x64_4 21.5 - 18.5 +9/=25/-6 53.75%
Stockfish 090913 64 SSE4.2_ - Komodo 5.1r2 64-bitnob 24.5 - 15.5 +13/=23/-4 61.25%

Time Control_2+2

Stockfish 090913 64 SSE4.2_ - Critter 1.6a 64-bitnob_4 22.5 - 17.5 +13/=19/-8 56.25%
Stockfish 090913 64 SSE4.2_ - Houdini 3 Pro x64_4 18.5 - 21.5 +9/=19/-12 46.25%
Stockfish 090913 64 SSE4.2_ - Komodo 5.1r2 64-bitnob 18.5 - 21.5 +7/=23/-10 46.25%

240 Games X 4 cores= http://www.mediafire.com/?0cte9n3j2fbbyjw
Score using 4 Cores = 128.0-112.0 = 53.33%

Segmenting the result by Time Control:

Fixed = 55.21% 132.5-107.5 (+25)
Incremental = 50.00% 120.0-120.0 (+0)

Global Score: 252.5–227.5 = 52.60%
Against Critter 1.6a (3093): 56.56% Houdini 3.0 Pro (3172): 48.75% Komodo 5.1r2 (3135): 52.50%

Average ELO of opponents= 3133
Estimated ELO Performance= 3151

Perhaps Stockfish has been a bit unlucky in this test … or probably it was a bit lucky in the previous one. Let’s keep testing to discover it.

Regards,

Tom.
Top
Masta
Posts: 28
Joined: Fri Jul 26, 2013 6:24 am

Re: Timestamps of Stockfish..SF_0308 - Komodo 51MP 58%

Post by Masta »

Tomcass wrote:Stockfish 090913 = 480 GAMES

Bench: 3864419 Timestamp: 1378646933

i7 980 3.33 Ghz.
6 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2012c
No tablebases
Hash 512
Relative Speed: 28.66
Knodes per second: 13.759

Time Control_4+0

Stockfish 090913 64 SSE4.2_ - Critter 1.6a 64-bit_NOB 22.0 - 18.0 +12/=20/-8 55.00%
Stockfish 090913 64 SSE4.2_ - Houdini 3 Pro x64_6 20.0 - 20.0 +9/=22/-9 50.00%
Stockfish 090913 64 SSE4.2_ - Komodo 5.1r2 64-bitx6nob 22.0 - 18.0 +12/=20/-8 55.00%

Time Control_2+2

Stockfish 090913 64 SSE4.2_ - Critter 1.6a 64-bit_NOB 23.5 - 16.5 +13/=21/-6 58.75%
Stockfish 090913 64 SSE4.2_ - Houdini 3 Pro x64_6 18.0 - 22.0 +10/=16/-14 45.00%
Stockfish 090913 64 SSE4.2_ - Komodo 5.1r2 64-bitnob 19.0 - 21.0 +8/=22/-10 47.50%

240 Games x6 cores= http://www.mediafire.com/?x5n8n3w2c7cwia6
Score using 6 cores = 124.5-115.5 = 51.87%

i7 975 3.33 Ghz.
4 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2012c
No tablebases
Hash 256
Relative Speed: 20.62
Knodes per second: 9.899

Time Control_4+0

Stockfish 090913 64 SSE4.2_ - Critter 1.6a 64-bitnob_4 22.5 - 17.5 +15/=15/-10 56.25%
Stockfish 090913 64 SSE4.2_ - Houdini 3 Pro x64_4 21.5 - 18.5 +9/=25/-6 53.75%
Stockfish 090913 64 SSE4.2_ - Komodo 5.1r2 64-bitnob 24.5 - 15.5 +13/=23/-4 61.25%

Time Control_2+2

Stockfish 090913 64 SSE4.2_ - Critter 1.6a 64-bitnob_4 22.5 - 17.5 +13/=19/-8 56.25%
Stockfish 090913 64 SSE4.2_ - Houdini 3 Pro x64_4 18.5 - 21.5 +9/=19/-12 46.25%
Stockfish 090913 64 SSE4.2_ - Komodo 5.1r2 64-bitnob 18.5 - 21.5 +7/=23/-10 46.25%

240 Games X 4 cores= http://www.mediafire.com/?0cte9n3j2fbbyjw
Score using 4 Cores = 128.0-112.0 = 53.33%

Segmenting the result by Time Control:

Fixed = 55.21% 132.5-107.5 (+25)
Incremental = 50.00% 120.0-120.0 (+0)

Global Score: 252.5–227.5 = 52.60%
Against Critter 1.6a (3093): 56.56% Houdini 3.0 Pro (3172): 48.75% Komodo 5.1r2 (3135): 52.50%

Average ELO of opponents= 3133
Estimated ELO Performance= 3151

Perhaps Stockfish has been a bit unlucky in this test … or probably it was a bit lucky in the previous one. Let’s keep testing to discover it.

Regards,

Tom.
No...this version is bugged with the patch "Move draw by material check " . It will have bad results because there was a bug triggering at depth 13. You need to try the new version Timestamp: 1379088502 and you will have good results again.
Top
Tomcass
Posts: 786
Joined: Sun Apr 16, 2006 9:09 pm

Re: Timestamps of Stockfish..SF_0308 - Komodo 51MP 58%

Post by Tomcass »

Thank you very much for your comment, Alexandre. I was surprised by the poor result of this test at incremental Time Control.

In about one hour I will post my test related to: Bench: 4554579 Timestamp: 1378968251. Everything OK now.

Regards,

Tom.
Top
Tomcass
Posts: 786
Joined: Sun Apr 16, 2006 9:09 pm

Re: Timestamps of Stockfish..SF_0308 - Komodo 51MP 58%

Post by Tomcass »

Stockfish 120913 = 480 GAMES

Bench: 4554579 Timestamp: 1378968251

i7 980 3.33 Ghz.
6 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2012c
No tablebases
Hash 512
Relative Speed: 28.66
Knodes per second: 13.759

Time Control_4+0

Stockfish 120913 64 SSE4.2_ - Critter 1.6a 64-bit_NOB 22.0 - 18.0 +13/=18/-9 55.00%
Stockfish 120913 64 SSE4.2_ - Houdini 3 Pro x64_6 22.5 - 17.5 +14/=17/-9 56.25%
Stockfish 120913 64 SSE4.2_ - Komodo 5.1r2 64-bitx6nob 22.5 - 17.5 +10/=25/-5 56.25%

Time Control_2+2

Stockfish 120913 64 SSE4.2_ - Critter 1.6a 64-bit_NOB 26.5 - 13.5 +15/=23/-2 66.25%
Stockfish 120913 64 SSE4.2_ - Houdini 3 Pro x64_6 20.5 - 19.5 +9/=23/-8 51.25%
Stockfish 120913 64 SSE4.2_ - Komodo 5.1r2 64-bitx6nob 23.0 - 17.0 +10/=26/-4 57.50%

240 Games x6 cores= http://www.mediafire.com/?f882gp447d1741w
Score using 6 cores = 137.0-103.0 = 57.08%

i7 975 3.33 Ghz.
4 real cores
Ponder: Off.
GUI: Fritz 12
Book: Perfect 2012c
No tablebases
Hash 256
Relative Speed: 20.62
Knodes per second: 9.899

Time Control_4+0

Stockfish 120913 64 SSE4.2_ - Critter 1.6a 64-bitnob_4 22.5 - 17.5 +12/=21/-7 56.25%
Stockfish 120913 64 SSE4.2_ - Houdini 3 Pro x64_4 19.5 - 20.5 +10/=19/-11 48.75%
Stockfish 120913 64 SSE4.2_ - Komodo 5.1r2 64-bitnob 21.5 - 18.5 +9/=25/-6 53.75%

Time Control_2+2

Stockfish 120913 64 SSE4.2_ - Critter 1.6a 64-bitnob_4 21.5 - 18.5 +13/=17/-10 53.75%
Stockfish 120913 64 SSE4.2_ - Houdini 3 Pro x64_4 19.0 - 21.0 +11/=16/-13 47.50%
Stockfish 120913 64 SSE4.2_ - Komodo 5.1r2 64-bitnob 22.5 - 17.5 +15/=15/-10 56.25%

240 Games X 4 cores= http://www.mediafire.com/?y1ux4fg29ix27xq
Score using 4 Cores = 126.5-113.5 = 52.71%

Segmenting the result by Time Control:

Fixed = 130.5-119.5 = 54.37%
Incremental = 133.0-107.0 = 55.42%

Global Score: 263.5-226.5 = 54.90%
Against Critter 1.6a (3093): 57.81% Houdini 3.0 Pro (3172): 50.94% Komodo 5.1r2 (3135): 55.94%

Average ELO of opponents= 3.133
Estimated ELO Performance= 3.167

Another very good score for Stockfish.

Best regards,

Tom.
Top

Return to “Computer Chess Club: Tournaments and Matches”