Komodo 9.3 has been released

[lkaufman]

Komodo 9.3 was released this morning. As usual, it is free for subscribers, and at 20% discount for anyone who bought Komodo 8 or later version. Available at komodochess.com.
Komodo 9.3 is slightly upgraded from the version now close to winning the TCEC final against Stockfish. The biggest difference is that it will use about 10% more time on the first few moves out of book, since it was pretty clear watching TCEC that Komodo keeps too much time in reserve for the endgame. Komodo 9.3 will still usually move quicker than Stockfish, but the difference will not be as great or as obvious. Komodo 9.3 is also slightly upgraded from the versions that performed well giving material handicaps to Grandmasters Movsesian, Lenderman, and Hess in serious matches.
Estimated elo gain from version 9.2 is about 15, both on one core and on many. Each of the steps since version 9.0 has been about 15 elo, so the total gain since Komodo 9 is about 45 elo, and the total gain since Komodo 8 is approaching one hundred elo.
As usual the gains come from all areas. Selectivity has been modified significantly, new eval terms were added, many weights re-tuned, time manangement improved, contempt improved, multiipv improved, MP changes which help on certain machines, free speedups, etc.

[beram]

Thx Larry,

My testing against the strong SF 121115IP has begun
btw. what do you think is best contempt setting for K9.3 against Stockfish?

grts Bram

[MikeB]

Komodo 9.3 was released this morning. As usual, it is free for subscribers, and at 20% discount for anyone who bought Komodo 8 or later version. Available at komodochess.com.
Komodo 9.3 is slightly upgraded from the version now close to winning the TCEC final against Stockfish. The biggest difference is that it will use about 10% more time on the first few moves out of book, since it was pretty clear watching TCEC that Komodo keeps too much time in reserve for the endgame. Komodo 9.3 will still usually move quicker than Stockfish, but the difference will not be as great or as obvious. Komodo 9.3 is also slightly upgraded from the versions that performed well giving material handicaps to Grandmasters Movsesian, Lenderman, and Hess in serious matches.
Estimated elo gain from version 9.2 is about 15, both on one core and on many. Each of the steps since version 9.0 has been about 15 elo, so the total gain since Komodo 9 is about 45 elo, and the total gain since Komodo 8 is approaching one hundred elo.
As usual the gains come from all areas. Selectivity has been modified significantly, new eval terms were added, many weights re-tuned, time manangement improved, contempt improved, multiipv improved, MP changes which help on certain machines, free speedups, etc.

Thank you Larry, Mark and team. Someone on the K-team ( I forget who, it doesn't matter) had mentioned that there is an internal benchmark builti-in Komodo - is that included in the release and is it accebisble by users?

The normal "bench" or "benchmark" does not appear to do anything. It would be interesting to a Komodo's benchmark run on different platforms and hardware configurations.

Congrats on K's excellent performance in the Super Final TCEC - clearly exceeded everyone's expectations ( except for maybe the K team ).

[schack]

Any reason the files are much smaller than previous versions?

[lkaufman]

Thx Larry,

My testing against the strong SF 121115IP has begun
btw. what do you think is best contempt setting for K9.3 against Stockfish?

grts Bram

Against a recent SF dev. version I would expect best results with contempt set to zero. The default is probably fine against anything else (except SF derivatives).

[Jesse Gersenson]

Any reason the files are much smaller than previous versions?

Yes, it's so we don't go over our bandwidth limit.

[MikeB]

Micro Bullet Results:

Hardware : 2010 Mac Pro 12 Core , x5690, OSX 10.1 , 48GB Ram

Settings:

TC 10 sec inc 0.5 sec
Single Core , 64MB Hash
EGTB Enabled
Ponder Off

Code: Select all

ResultSet-EloRating>details
   1 Stockfish 110315    21 1500.0 (820.0 : 680.0)
                            500.0 (249.0 : 251.0) Stockfish 112715    19
                            500.0 (278.5 : 221.5) Komodo 9.3         -12
                            500.0 (292.5 : 207.5) Komodo 9.2         -28
   2 Stockfish 112715    19 1500.0 (814.5 : 685.5)
                            500.0 (251.0 : 249.0) Stockfish 110315    21
                            500.0 (274.5 : 225.5) Komodo 9.3         -12
                            500.0 (289.0 : 211.0) Komodo 9.2         -28
   3 Komodo 9.3         -12 1500.0 (710.0 : 790.0)
                            500.0 (221.5 : 278.5) Stockfish 110315    21
                            500.0 (225.5 : 274.5) Stockfish 112715    19
                            500.0 (263.0 : 237.0) Komodo 9.2         -28
   4 Komodo 9.2         -28 1500.0 (655.5 : 844.5)
                            500.0 (207.5 : 292.5) Stockfish 110315    21
                            500.0 (211.0 : 289.0) Stockfish 112715    19
                            500.0 (237.0 : 263.0) Komodo 9.3         -12
ResultSet-EloRating>ratings
Rank Name               Elo    +    - games score oppo. draws 
   1 Stockfish 110315    21   13   13  1500   55%    -7   59% 
   2 Stockfish 112715    19   13   13  1500   54%    -6   59% 
   3 Komodo 9.3         -12   14   14  1500   47%     4   56% 
   4 Komodo 9.2         -28   14   14  1500   44%     9   56% 

Not a big fan of Micro Bullet, just wanted to set a baseline for future versions. Nice ELO gain for K. Wanted to run the SF version used in TCEC and the most current. 2 ELO difference here with this number of games is not statistically significant.

[MikeB]

Micro Bullet Results:

Hardware : 2010 Mac Pro 12 Core , x5690, OSX 10.1 , 48GB Ram

Settings:

TC 10 sec inc 0.5 sec
Single Core , 64MB Hash
EGTB Enabled
Ponder Off

Code: Select all

ResultSet-EloRating>details
   1 Stockfish 110315    21 1500.0 (820.0 : 680.0)
                            500.0 (249.0 : 251.0) Stockfish 112715    19
                            500.0 (278.5 : 221.5) Komodo 9.3         -12
                            500.0 (292.5 : 207.5) Komodo 9.2         -28
   2 Stockfish 112715    19 1500.0 (814.5 : 685.5)
                            500.0 (251.0 : 249.0) Stockfish 110315    21
                            500.0 (274.5 : 225.5) Komodo 9.3         -12
                            500.0 (289.0 : 211.0) Komodo 9.2         -28
   3 Komodo 9.3         -12 1500.0 (710.0 : 790.0)
                            500.0 (221.5 : 278.5) Stockfish 110315    21
                            500.0 (225.5 : 274.5) Stockfish 112715    19
                            500.0 (263.0 : 237.0) Komodo 9.2         -28
   4 Komodo 9.2         -28 1500.0 (655.5 : 844.5)
                            500.0 (207.5 : 292.5) Stockfish 110315    21
                            500.0 (211.0 : 289.0) Stockfish 112715    19
                            500.0 (237.0 : 263.0) Komodo 9.3         -12
ResultSet-EloRating>ratings
Rank Name               Elo    +    - games score oppo. draws 
   1 Stockfish 110315    21   13   13  1500   55%    -7   59% 
   2 Stockfish 112715    19   13   13  1500   54%    -6   59% 
   3 Komodo 9.3         -12   14   14  1500   47%     4   56% 
   4 Komodo 9.2         -28   14   14  1500   44%     9   56% 

Not a big fan of Micro Bullet, just wanted to set a baseline for future versions. Nice ELO gain for K. Wanted to run the SF version used in TCEC and the most current. 2 ELO difference here with this number of games is not statistically significant.

Rerunning with same parameters using the TCEC openings (games 1 thru 90)

This is the first run:

Code: Select all

ResultSet-EloRating>details
   1 Stockfish 110315    13  90.0 ( 49.0 :  41.0)
                             90.0 ( 49.0 :  41.0) Komodo 9.3         -13
   2 Komodo 9.3         -13  90.0 ( 41.0 :  49.0)
                             90.0 ( 41.0 :  49.0) Stockfish 110315    13
ResultSet-EloRating>ratings
Rank Name               Elo    +    - games score oppo. draws 
   1 Stockfish 110315    13   54   54    90   54%   -13   58% 
   2 Komodo 9.3         -13   54   54    90   46%    13   58% 
ResultSet-EloRating>

More or less in line with the 750 opening suite used above. Will run it 11 times (990 games in total)

[Laskos]

Micro Bullet Results:

Hardware : 2010 Mac Pro 12 Core , x5690, OSX 10.1 , 48GB Ram

Settings:

TC 10 sec inc 0.5 sec
Single Core , 64MB Hash
EGTB Enabled
Ponder Off

Code: Select all

ResultSet-EloRating>details
   1 Stockfish 110315    21 1500.0 (820.0 : 680.0)
                            500.0 (249.0 : 251.0) Stockfish 112715    19
                            500.0 (278.5 : 221.5) Komodo 9.3         -12
                            500.0 (292.5 : 207.5) Komodo 9.2         -28
   2 Stockfish 112715    19 1500.0 (814.5 : 685.5)
                            500.0 (251.0 : 249.0) Stockfish 110315    21
                            500.0 (274.5 : 225.5) Komodo 9.3         -12
                            500.0 (289.0 : 211.0) Komodo 9.2         -28
   3 Komodo 9.3         -12 1500.0 (710.0 : 790.0)
                            500.0 (221.5 : 278.5) Stockfish 110315    21
                            500.0 (225.5 : 274.5) Stockfish 112715    19
                            500.0 (263.0 : 237.0) Komodo 9.2         -28
   4 Komodo 9.2         -28 1500.0 (655.5 : 844.5)
                            500.0 (207.5 : 292.5) Stockfish 110315    21
                            500.0 (211.0 : 289.0) Stockfish 112715    19
                            500.0 (237.0 : 263.0) Komodo 9.3         -12
ResultSet-EloRating>ratings
Rank Name               Elo    +    - games score oppo. draws 
   1 Stockfish 110315    21   13   13  1500   55%    -7   59% 
   2 Stockfish 112715    19   13   13  1500   54%    -6   59% 
   3 Komodo 9.3         -12   14   14  1500   47%     4   56% 
   4 Komodo 9.2         -28   14   14  1500   44%     9   56% 

Not a big fan of Micro Bullet, just wanted to set a baseline for future versions. Nice ELO gain for K. Wanted to run the SF version used in TCEC and the most current. 2 ELO difference here with this number of games is not statistically significant.

At ultra-fast Stockfish is clearly better. Being unable to run many games at longer time controls on several cores, I tested with STS15 at 0.2s per position 1 thread (roughly Fishtest TC) and then to 20s per position 4 threads (rapid), to see the trend and the scaling. Komodo seems to scale the best to longer TC and several threads, and in fact for the first time be the best on STS15 in these conditions:

0.2s/position using 1 core:

Code: Select all

Houdini 4             761/1000
Stockfish 251115      755/1000
Komodo 9.3            748/1000

20s/position using 4 cores:

Code: Select all

Komodo 9.3            953/1000
Stockfish 251115      947/1000
Houdini 4             921/1000

Houdini seems to scale the worst.

[MikeB]

Micro Bullet Results:

Hardware : 2010 Mac Pro 12 Core , x5690, OSX 10.1 , 48GB Ram

Settings:

TC 10 sec inc 0.5 sec
Single Core , 64MB Hash
EGTB Enabled
Ponder Off

Code: Select all

ResultSet-EloRating>details
   1 Stockfish 110315    21 1500.0 (820.0 : 680.0)
                            500.0 (249.0 : 251.0) Stockfish 112715    19
                            500.0 (278.5 : 221.5) Komodo 9.3         -12
                            500.0 (292.5 : 207.5) Komodo 9.2         -28
   2 Stockfish 112715    19 1500.0 (814.5 : 685.5)
                            500.0 (251.0 : 249.0) Stockfish 110315    21
                            500.0 (274.5 : 225.5) Komodo 9.3         -12
                            500.0 (289.0 : 211.0) Komodo 9.2         -28
   3 Komodo 9.3         -12 1500.0 (710.0 : 790.0)
                            500.0 (221.5 : 278.5) Stockfish 110315    21
                            500.0 (225.5 : 274.5) Stockfish 112715    19
                            500.0 (263.0 : 237.0) Komodo 9.2         -28
   4 Komodo 9.2         -28 1500.0 (655.5 : 844.5)
                            500.0 (207.5 : 292.5) Stockfish 110315    21
                            500.0 (211.0 : 289.0) Stockfish 112715    19
                            500.0 (237.0 : 263.0) Komodo 9.3         -12
ResultSet-EloRating>ratings
Rank Name               Elo    +    - games score oppo. draws 
   1 Stockfish 110315    21   13   13  1500   55%    -7   59% 
   2 Stockfish 112715    19   13   13  1500   54%    -6   59% 
   3 Komodo 9.3         -12   14   14  1500   47%     4   56% 
   4 Komodo 9.2         -28   14   14  1500   44%     9   56% 

Not a big fan of Micro Bullet, just wanted to set a baseline for future versions. Nice ELO gain for K. Wanted to run the SF version used in TCEC and the most current. 2 ELO difference here with this number of games is not statistically significant.

Rerunning with same parameters using the TCEC openings (games 1 thru 90)

This is the first run:

Code: Select all

ResultSet-EloRating>details
   1 Stockfish 110315    13  90.0 ( 49.0 :  41.0)
                             90.0 ( 49.0 :  41.0) Komodo 9.3         -13
   2 Komodo 9.3         -13  90.0 ( 41.0 :  49.0)
                             90.0 ( 41.0 :  49.0) Stockfish 110315    13
ResultSet-EloRating>ratings
Rank Name               Elo    +    - games score oppo. draws 
   1 Stockfish 110315    13   54   54    90   54%   -13   58% 
   2 Komodo 9.3         -13   54   54    90   46%    13   58% 
ResultSet-EloRating>

More or less in line with the 750 opening suite used above. Will run it 11 times (990 games in total)

Code: Select all

Update - 900 completed , using the TECE openings 01-90 10x.

ResultSet-EloRating>details
   1 Stockfish 110315    13 900.0 (489.5 : 410.5)
                            900.0 (489.5 : 410.5) Komodo 9.3         -13
   2 Komodo 9.3         -13 900.0 (410.5 : 489.5)
                            900.0 (410.5 : 489.5) Stockfish 110315    13
ResultSet-EloRating>ratings
Rank Name               Elo    +    - games score oppo. draws 
   1 Stockfish 110315    13   18   18   900   54%   -13   53% 
   2 Komodo 9.3         -13   18   18   900   46%    13   53% 
ResultSet-EloRating>

ELO differential remains the same , although tighter than using random 750 position opening suite. Interesting at super fast micro controls, draw % is way down from the TCEC Super Final and actually less than the 750 random 750 position opening suite. I thought the openings used in TCEC Super Final were very well chosen and relevant.