Testing Komodo9: 1440 Games

[Tomcass]

TESTING KOMODO 9 = 1440 GAMES

6 real cores

Ponder: Off.
GUI: Fritz 14
Book: Perfect 2014t – Limit 8 moves. Sedat –
No tablebases. No RTB used.
Hash 512
Relative Speed: 29.54
Knodes per second: 14.177

Time Control 4 + 0

Komodo 9 64-bit_6 - Houdini 4 x64_st_X6_CT0 71.5 - 48.5 +37/=69/-14 59.58%
Komodo 9 64-bit_6 - Gull 3 x64 XP 79.5 - 40.5 +48/=63/-9 66.25%
Komodo 9 64-bit_6 - Stockfish 6 64 POPCNT_X6_O 54.0 - 66.0 +16/=76/-28 45.00%

Time Control 2 + 2

Komodo 9 64-bit_6 - Houdini 4 x64_st_X6_CT0 67.0 - 53.0 +33/=68/-19 55.83%
Komodo 9 64-bit_6 - Gull 3 x64 XP 79.5 - 40.5 +44/=71/-5 66.25%
Komodo 9 64-bit_6 - Stockfish 6 64 POPCNT_X6_O 62.0 - 58.0 +22/=80/-18 51.67%

Score: 412.5 – 307.5 =57.29%

720 Games:
http://www.mediafire.com/download/z45jm ... 0Games.pgn

8 real cores

Ponder: Off
GUI: Fritz 14
Book: Perfect 2014t – Limit 8 moves. Sedat –
No tablebases. No RTB used.
Hash 1024
Relative Speed: 44.92
Knodes per Second: 21.562

Time Control: 4+0

Komodo 9 64-bit_8 - Houdini 4 Pro x64_Ct0_8 74.0 - 46.0 +42/=64/-14 61.67%
Komodo 9 64-bit_8 - Gull 3 x64 XP_8 77.5 - 42.5 +47/=61/-12 64.58%
Komodo 9 64-bit_8 - Stockfish 6 64 BMI2_8 56.5 - 63.5 +19/=75/-26 47.08%

Time Control 2 + 2
Komodo 9 64-bit_8 - Houdini 4 Pro x64_Ct0_8 80.0 - 40.0 +47/=66/-7 66.67%
Komodo 9 64-bit_8 - Gull 3 x64 XP_8 77.0 - 43.0 +46/=62/-12 64.17%
Komodo 9 64-bit_8 - Stockfish 6 64 BMI2_8 58.0 - 62.0 +14/=88/-18 48.33%

Score: 423.0 – 297.0 = 58.75%
720 Games:
http://www.mediafire.com/view/27bitwufi ... 0Games.htm

Against : Houdini 4.0 St. Ct0 (3136) = 60.94% ; Gull 3 XP (3103) = 65.31%; Stockfish 6 (3218) = 48.02%

GLOBAL SCORE: 836.5 – 603.5 = 58.09%

Average Estimated Elo Opponents = 3152

Estimated Elo Performance for Komodo 9 after 1.440 games = 3.209

Under my testing conditions, using 6 and 8 cores, and 4+0 and 2+2 Time Control, Komodo 9 performs

29 ELO points below the current leader in my tests, Stockfish Bullet 230415.
9 ELO points below Stockfish 6

The improvement over Komodo 8 is 56 Elo Points. Well done!. (Please note that K9 could have lost a small number of games because of its problems with the time management). I will repeat this test using 9.01 in the 720 games with 4 + 0 time control.

Congratulations to Larry and Mark for this substantial steep ahead.

Regards,

Tom.

[Vinvin]

TESTING KOMODO 9 = 1440 GAMES
...
29 ELO points below the current leader in my tests, Stockfish Bullet 230415.
9 ELO points below Stockfish 6
...

Please test K9 against Stockfish Bullet 230415 too ...

[Tomcass]

TESTING KOMODO 9 = 1440 GAMES
...
29 ELO points below the current leader in my tests, Stockfish Bullet 230415.
9 ELO points below Stockfish 6
...

Please test K9 against Stockfish Bullet 230415 too ...

I will, Vincent. Thanks for your suggestion!.

Tom.

[overlord]

Hello Tom,

second batch of the K9 games is impossible to download. Could you please fix it. Thanks.

Best regards to Barcelona

[Ajedrecista]

Hello Kvicala:

Hello Tom,

second batch of the K9 games is impossible to download. Could you please fix it. Thanks.

Best regards to Barcelona

The key is replace view by download in the URL:

http://www.mediafire.com/download/27bit ... 0Games.htm

The file has a size of 8.76 MB according to Mediafire. I did not try to download it but the download starts if you click on the download button.

Thanks to Tom for him effort.

Regards from Spain.

Ajedrecista.

[Tomcass]

Hello Kvicala:

Hello Tom,

second batch of the K9 games is impossible to download. Could you please fix it. Thanks.

Best regards to Barcelona

The key is replace view by download in the URL:

http://www.mediafire.com/download/27bit ... 0Games.htm

The file has a size of 8.76 MB according to Mediafire. I did not try to download it but the download starts if you click on the download button.

Thanks to Tom for him effort.

Regards from Spain.

Ajedrecista.

Thanks to Overlord for pointing out my mistake and to Jesús Muñoz for offering the right solution.

In a few hours I will finish the re-testing of Komodo 9, using the Komodo 9.01 version at 4 +0 Time Control.

Regards,

Tom.

[Tomcass]

TESTING KOMODO 9.01 = 1440 GAMES

I have not repeated the test at 2+2 because there is no change between Komodo9 and Komodo 9.01 at incremental time control. (If I am wrong please let me know). So that here you have the result for Komodo 9.01 after 1440 games.

6 real cores

Ponder: Off.
GUI: Fritz 14
Book: Perfect 2014t – Limit 8 moves. Sedat –
No tablebases. No RTB used.
Hash 512
Relative Speed: 29.54
Knodes per second: 14.177

Time Control 4 + 0
Komodo 9.01 64-bit_X6 - Houdini 4 x64_st_X6_CT0 72.5 - 47.5 +38/=69/-13 60.42%
Komodo 9.01 64-bit_X6 - Gull 3 x64 XP 84.0 - 36.0 +57/=54/-9 70.00%
Komodo 9.01 64-bit_X6 - Stockfish 6 64 POPCNT_X6_O 60.5 - 59.5 +20/=81/-19 50.42%
360 Games Komodo 9.01 4+0 6 cores:
http://www.mediafire.com/download/ltchj ... 0Games.pgn

Time Control 2 + 2
Komodo 9 64-bit_6 - Houdini 4 x64_st_X6_CT0 67.0 - 53.0 +33/=68/-19 55.83%
Komodo 9 64-bit_6 - Gull 3 x64 XP 79.5 - 40.5 +44/=71/-5 66.25%
Komodo 9 64-bit_6 - Stockfish 6 64 POPCNT_X6_O 62.0 - 58.0 +22/=80/-18 51.67%

Score: 425.5 – 294.5 = 59.10%


8 real cores

Ponder: Off
GUI: Fritz 14
Book: Perfect 2014t – Limit 8 moves. Sedat –
No tablebases. No RTB used.
Hash 1024
Relative Speed: 44.92
Knodes per Second: 21.562

Time Control: 4+0
Komodo 9.01 64-bit-X8 - Houdini 4 Pro x64_Ct0_8 76.5 - 43.5 +42/=69/-9 63.75%
Komodo 9.01 64-bit-X8 - Gull 3 x64 XP_8 83.5 - 36.5 +53/=61/-6 69.58%
Komodo 9.01 64-bit-X8 - Stockfish 6 64 BMI2_8 58.0 - 62.0 +18/=80/-22 48.33%

360 Games Komodo 9.01 4+0 8 cores:
http://www.mediafire.com/download/oag22 ... 0Games.htm

Time Control 2 + 2
Komodo 9 64-bit_8 - Houdini 4 Pro x64_Ct0_8 80.0 - 40.0 +47/=66/-7 66.67%
Komodo 9 64-bit_8 - Gull 3 x64 XP_8 77.0 - 43.0 +46/=62/-12 64.17%
Komodo 9 64-bit_8 - Stockfish 6 64 BMI2_8 58.0 - 62.0 +14/=88/-18 48.33%

Score: 433.0 – 287.0 = 64.14%

Against : Houdini 4.0 St. Ct0 (3136) = 61.67% ; Gull 3 XP (3103) = 67.50%; Stockfish 6 (3218) = 48.02%

GLOBAL SCORE: 858.5 – 581.5 = 59.62%

Average Estimated Elo Opponents = 3152
Estimated Elo Performance for Komodo 9 after 1.440 games = 3.219

Under my testing conditions, using 6 and 8 cores, and 4+0 and 2+2 Time Control, Komodo 9 performs

19 ELO points below the current leader in my tests, Stockfish Bullet 230415.
1 ELO better than Stockfish 6

The improvement of Komodo 9.01 over Komodo 8 is 66 Elo points! Quite impressive!. What a brillinat work by the Komodo team!.

Regards,

Tom.

[lkaufman]

TESTING KOMODO 9.01 = 1440 GAMES

I have not repeated the test at 2+2 because there is no change between Komodo9 and Komodo 9.01 at incremental time control. (If I am wrong please let me know). So that here you have the result for Komodo 9.01 after 1440 games.

6 real cores

Ponder: Off.
GUI: Fritz 14
Book: Perfect 2014t – Limit 8 moves. Sedat –
No tablebases. No RTB used.
Hash 512
Relative Speed: 29.54
Knodes per second: 14.177

Time Control 4 + 0
Komodo 9.01 64-bit_X6 - Houdini 4 x64_st_X6_CT0 72.5 - 47.5 +38/=69/-13 60.42%
Komodo 9.01 64-bit_X6 - Gull 3 x64 XP 84.0 - 36.0 +57/=54/-9 70.00%
Komodo 9.01 64-bit_X6 - Stockfish 6 64 POPCNT_X6_O 60.5 - 59.5 +20/=81/-19 50.42%
360 Games Komodo 9.01 4+0 6 cores:
http://www.mediafire.com/download/ltchj ... 0Games.pgn

Time Control 2 + 2
Komodo 9 64-bit_6 - Houdini 4 x64_st_X6_CT0 67.0 - 53.0 +33/=68/-19 55.83%
Komodo 9 64-bit_6 - Gull 3 x64 XP 79.5 - 40.5 +44/=71/-5 66.25%
Komodo 9 64-bit_6 - Stockfish 6 64 POPCNT_X6_O 62.0 - 58.0 +22/=80/-18 51.67%

Score: 425.5 – 294.5 = 59.10%


8 real cores

Ponder: Off
GUI: Fritz 14
Book: Perfect 2014t – Limit 8 moves. Sedat –
No tablebases. No RTB used.
Hash 1024
Relative Speed: 44.92
Knodes per Second: 21.562

Time Control: 4+0
Komodo 9.01 64-bit-X8 - Houdini 4 Pro x64_Ct0_8 76.5 - 43.5 +42/=69/-9 63.75%
Komodo 9.01 64-bit-X8 - Gull 3 x64 XP_8 83.5 - 36.5 +53/=61/-6 69.58%
Komodo 9.01 64-bit-X8 - Stockfish 6 64 BMI2_8 58.0 - 62.0 +18/=80/-22 48.33%

360 Games Komodo 9.01 4+0 8 cores:
http://www.mediafire.com/download/oag22 ... 0Games.htm

Time Control 2 + 2
Komodo 9 64-bit_8 - Houdini 4 Pro x64_Ct0_8 80.0 - 40.0 +47/=66/-7 66.67%
Komodo 9 64-bit_8 - Gull 3 x64 XP_8 77.0 - 43.0 +46/=62/-12 64.17%
Komodo 9 64-bit_8 - Stockfish 6 64 BMI2_8 58.0 - 62.0 +14/=88/-18 48.33%

Score: 433.0 – 287.0 = 64.14%

Against : Houdini 4.0 St. Ct0 (3136) = 61.67% ; Gull 3 XP (3103) = 67.50%; Stockfish 6 (3218) = 48.02%

GLOBAL SCORE: 858.5 – 581.5 = 59.62%

Average Estimated Elo Opponents = 3152
Estimated Elo Performance for Komodo 9 after 1.440 games = 3.219

Under my testing conditions, using 6 and 8 cores, and 4+0 and 2+2 Time Control, Komodo 9 performs

19 ELO points below the current leader in my tests, Stockfish Bullet 230415.
1 ELO better than Stockfish 6

The improvement of Komodo 9.01 over Komodo 8 is 66 Elo points! Quite impressive!. What a brillinat work by the Komodo team!.

Regards,

Tom.

Thank you. Your results at the sudden death time control show overall 56.32% for Komodo 9 compared to 60.42% for Komodo 9.1. This translates to an elo improvement for Komodo 9.1 of 29 elo points at sudden death, the only level affected by the update except for Aquarium users. This is more than I got in my own tests using LittleBlitzer; maybe the gain is GUI-dependent.
As for the overall gain of 66 elo over Komodo 8, I suppose a few elo of that was due to the new Sudden Death time management, since this was already poor on Komodo 8 and worse on Komodo 9 until we did the update to 9.01. We are pleasantly surprised that the reported gains are generally coming in higher than we found at bullet levels, which indicates that the changes scale well.

[Tomcass]

KOMODO 9.01 AGAINST FOUR STOCKFISH SELECTION: 640 GAMES.

My selection of Stockfish engines:
The latest SF Marco Zerbinati 090515
The latest SF Development from Abroak 090515 Timestamp: 1431176779
The latest SF Bullet 080515
The best performer Stockfish in my tests: Stockfish 230415b also compiled by Bullet

6 real cores
Ponder: Off.
GUI: Fritz 14
Book: Perfect 2014t – Limit 8 moves. Sedat –
No tablebases. No RTB used.
Hash 512
Relative Speed: 29.54
Knodes per second: 14.177

Time Control 4 + 0

Komodo 9.01 64-bit_X6 - SF090515MZ 64 POPCNTX6 21.5 - 18.5 +8/=27/-5 53.75%
Komodo 9.01 64-bit_X6 - Stockfish 080515b 64 SSE42X6 23.5 - 16.5 +9/=29/-2 58.75%
Komodo 9.01 64-bit_X6 - Stockfish 090515 64 POPCNTX6 18.5 - 21.5 +7/=23/-10 46.25%
Komodo 9.01 64-bit_X6 - Stockfish 230415b 64 SSE42_ 16.0 – 24.0 +6/=20/-14 40.00%

Time Control 2 + 2

Komodo 9.01 64-bit_X6 - SF090515MZ 64 POPCNTX6 18.0 - 22.0 +5/=26/-9 45.00%
Komodo 9.01 64-bit_X6 - Stockfish 080515b 64 SSE42X6 18.5 - 21.5 +5/=27/-8 46.25%
Komodo 9.01 64-bit_X6 - Stockfish 090515 64 POPCNTX6 19.0 - 21.0 +8/=22/-10 47.50%
Komodo 9.01 64-bit_X6 - Stockfish 230415b 64 SSE42_ 18.0 - 22.0 +4/=28/-8 45.00%

TOTAL 6 CORES
Komodo 9.01 64-bit_X6 - SF090515MZ 64 POPCNTX6 39.5 – 40.5 49.37%
Komodo 9.01 64-bit_X6 - Stockfish 080515b 64 SSE42X6 42.0 – 38.0 52.50%
Komodo 9.01 64-bit_X6 - Stockfish 090515 64 POPCNTX6 37.5 – 42.5 46.87%
Komodo 9.01 64-bit_X6 - Stockfish 230415b 64 SSE42_ 34.0 – 46.0 42.50%

320 GAMES:
http://www.mediafire.com/download/nsnnx ... 0Games.pgn


8 real cores
Ponder: Off
GUI: Fritz 14
Book: Perfect 2014t – Limit 8 moves. Sedat –
No tablebases. No RTB used.
Hash 1024
Relative Speed: 44.92
Knodes per Second: 21.562

Time Control 4 + 0

Komodo 9.01 64-bit-X8 - SF090515MZ 64 AVXx8 18.5 - 21.5 +6/=25/-9 46.25%
Komodo 9.01 64-bit-X8 - Stockfish 080515b 64 SSE42X8 19.0 - 21.0 +4/=30/-6 47.50%
Komodo 9.01 64-bit-X8 - Stockfish 090515 64 BMI2X8 19.0 - 21.0 +5/=28/-7 47.50%
Komodo 9.01 64-bit-X8 - Stockfish 230415b 64 SSE42 16.5 – 23.5 +6/=21/-13 41.25%

Time Control 2+2

Komodo 9.01 64-bit-X8 - SF090515MZ 64 AVXx8 20.5 - 19.5 +4/=33/-3 51.25%
Komodo 9.01 64-bit-X8 - Stockfish 080515b 64 SSE42X8 22.5 - 17.5 +8/=29/-3 56.25%
Komodo 9.01 64-bit-X8 - Stockfish 090515 64 BMI2X8 20.0 - 20.0 +6/=28/-6 50.00%
Komodo 9.01 64-bit-X8 - Stockfish 230415b 64 SSE42_ 19.0 - 21.0 +5/=28/-7 47.50%

TOTAL 8 CORES=
Komodo 9.01 64-bit-X8 - SF090515MZ 64 AVXx8 39.0 – 41.0 48.75%
Komodo 9.01 64-bit-X8 - Stockfish 080515b 64 SSE42X8 41.5 – 38.5 51.97%
Komodo 9.01 64-bit-X8 - Stockfish 090515 64 BMI2X8 39.0 – 41.0 48.75%
komodo 9.01 64-bit-X8 - Stockfish 230415b 64 SSE42_ 35.5 – 44.5 44.37%

320 Games=
http://www.mediafire.com/download/48m18 ... 0Games.pgn

TOTAL 6 AND 8 CORES AFTER 640 GAMES:

Komodo 9.01 64-bit - SF090515MZ 64 78.5 – 81.5 49.06%
Komodo 9.01 64-bit - Stockfish 080515b 64 SSE42 83.5 – 76.5 52.19%
Komodo 9.01 64-bit - Stockfish 090515 64 76.5 – 83.5 47.81%
Komodo 9.01 64-bit- - Stockfish 230415b 64 SSE42 69.5 – 90.5 43.44%

Segmenting by Time Control:

Sudden Death 4+0 = 47.66%
Incremental 2+2= 48.59%

As a reference, the performance of Komodo 9.01 against Stockfish 6 after 480 games was 49.69%

Regards,

Tom.

[shrapnel]

Hi Tom
I may be wrong, but it seems to me that Stockfish is actually scaling better than Komodo 9.01 going from 6 to 8 Cores !
This is very interesting for me, as my CPU has 8 Cores.