Komodo 9.3 has been released

MikeB · Post by **MikeB** » Sat Nov 28, 2015 8:04 am

Laskos wrote:
MikeB wrote:Micro Bullet Results:

Hardware : 2010 Mac Pro 12 Core , x5690, OSX 10.1 , 48GB Ram

Settings:

TC 10 sec inc 0.5 sec
Single Core , 64MB Hash
EGTB Enabled
Ponder Off
Code: Select all
ResultSet-EloRating>details
   1 Stockfish 110315    21 1500.0 (820.0 : 680.0)
                            500.0 (249.0 : 251.0) Stockfish 112715    19
                            500.0 (278.5 : 221.5) Komodo 9.3         -12
                            500.0 (292.5 : 207.5) Komodo 9.2         -28
   2 Stockfish 112715    19 1500.0 (814.5 : 685.5)
                            500.0 (251.0 : 249.0) Stockfish 110315    21
                            500.0 (274.5 : 225.5) Komodo 9.3         -12
                            500.0 (289.0 : 211.0) Komodo 9.2         -28
   3 Komodo 9.3         -12 1500.0 (710.0 : 790.0)
                            500.0 (221.5 : 278.5) Stockfish 110315    21
                            500.0 (225.5 : 274.5) Stockfish 112715    19
                            500.0 (263.0 : 237.0) Komodo 9.2         -28
   4 Komodo 9.2         -28 1500.0 (655.5 : 844.5)
                            500.0 (207.5 : 292.5) Stockfish 110315    21
                            500.0 (211.0 : 289.0) Stockfish 112715    19
                            500.0 (237.0 : 263.0) Komodo 9.3         -12
ResultSet-EloRating>ratings
Rank Name               Elo    +    - games score oppo. draws 
   1 Stockfish 110315    21   13   13  1500   55%    -7   59% 
   2 Stockfish 112715    19   13   13  1500   54%    -6   59% 
   3 Komodo 9.3         -12   14   14  1500   47%     4   56% 
   4 Komodo 9.2         -28   14   14  1500   44%     9   56% 
Not a big fan of Micro Bullet, just wanted to set a baseline for future versions. Nice ELO gain for K. Wanted to run the SF version used in TCEC and the most current. 2 ELO difference here with this number of games is not statistically significant.
At ultra-fast Stockfish is clearly better. Being unable to run many games at longer time controls on several cores, I tested with STS15 at 0.2s per position 1 thread (roughly Fishtest TC) and then to 20s per position 4 threads (rapid), to see the trend and the scaling. Komodo seems to scale the best to longer TC and several threads, and in fact for the first time be the best on STS15 in these conditions:

0.2s/position using 1 core:
Code: Select all
Houdini 4             761/1000
Stockfish 251115      755/1000
Komodo 9.3            748/1000
20s/position using 4 cores:
Code: Select all
Komodo 9.3            953/1000
Stockfish 251115      947/1000
Houdini 4             921/1000
Houdini seems to scale the worst.

That's exactly where I was headed. Next run, which will take about 10 hours, is to replay the first 90 games (openings) of TCEC using 4CPU per side. Looks like TC 5/5 will give me a complete game in about 20 minutes and I can run three 4CPU games at once on the 12 Core Mac Pro. After that, I will go to 12CPU with a TC of 20/20. That will take five or six days to run. And of course, we already know what the 24CPU results look like.

MikeB · Post by **MikeB** » Sat Nov 28, 2015 7:57 pm

After 50 games, following the Super Final TCEC openings tc 2.5/2.5 ( about equal to CCRl 40/4), Draw rate is up and given more time and more CPUs, Komodo starts showing its strength.

Hardware : 2010 Mac Pro 12 Core , x5690, OSX 10.1 , 48GB Ram

Settings:

TC 2:30m inc 2.5 sec
4CPU , 64MB Hash
EGTB Enabled
Ponder Off

Code: Select all

ResultSet-EloRating>details
   1 Komodo 9.3           4  50.0 ( 26.0 :  24.0)
                             50.0 ( 26.0 :  24.0) Stockfish 110315    -4
   2 Stockfish 110315    -4  50.0 ( 24.0 :  26.0)
                             50.0 ( 24.0 :  26.0) Komodo 9.3           4
ResultSet-EloRating>ratings
Rank Name               Elo    +    - games score oppo. draws 
   1 Komodo 9.3           4   69   69    50   52%    -4   72% 
   2 Stockfish 110315    -4   69   69    50   48%     4   72% 
ResultSet-EloRating>

Was watching play and saw this one very interesting game.
(Note, This is following the TCEC , but xboard started with SF being white first - so this follows TCEC #50 game for the first 8 moves,)
[pgn][Event "Computer Chess Game"]
[Site "Mac-Pro.local"]
[Date "2015.11.28"]
[Round "49"]
[White "Stockfish 110315"]
[Black "Komodo 9.3"]
[Result "1-0"]
[TimeControl "150+2"]
[Annotator "9. -0.05 9... -0.09"]

1. Nf3
{ book }
1... d5
{ book }
2. c4
{ book }
2... c6
{ book }
3. g3
{
book }
3... Nf6
{ book }
4. Bg2
{ book }
4... dxc4
{ book }
5. O-O
{ book }
5... Nbd7
{ book }
6. Qc2
{ book }
6... Nb6
{ book }
7. a4
{ book }
7... a5
{ book }
8. Na3 Be6 9. Ne5 {-0.05/22 4} g6 {-0.09/22 9} 10. Naxc4 {+0.12/23 4} Nfd5
{+0.00/23 7} 11. b3 {+0.15/25 18} Nb4 {+0.00/24 12} 12. Qb2 {+0.23/26 7}
Bg7 {-0.03/23 5} 13. Nxc6 {+0.25/24 0.6} Bxb2 {-0.07/25 4} 14. Nxd8
{+0.30/25 2.9} Bxa1 {-0.04/25 3} 15. Nxe6 {+0.29/25 3} fxe6 {+0.00/25 2.5}
16. Nxb6 {+0.33/25 3} Ra6 {-0.03/26 13} 17. Nc4 {+0.32/25 3} Nd5
{-0.10/24 1.9} 18. e3 {+0.26/24 3} O-O {-0.03/25 6} 19. Ba3 {+0.37/25 10}
Bg7 {+0.00/23 2.4} 20. Rc1 {+0.40/21 0.5} Rc8 {-0.13/24 6} 21. Bf3
{+0.34/24 13} Rac6 {+0.00/22 4} 22. d4 {+0.31/24 7} Kf7 {+0.00/25 2.6} 23.
Kg2 {+0.43/24 5} R6c7 {+0.00/26 3} 24. Bc5 {+0.43/25 1.0} b6 {-0.14/30 20}
25. Ba3 {+0.43/26 11} h5 {-0.22/24 8} 26. Be4 {+0.36/24 3} Bh8 {+0.00/27 4}
27. Bb2 {+0.36/25 11} Bf6 {-0.14/29 36} 28. Bf3 {+0.34/23 3} Nb4
{+0.00/31 6} 29. Be2 {+0.33/24 3} Bg7 {+0.00/31 5} 30. f4 {+0.29/23 21} Rc6
{-0.07/27 13} 31. Ba3 {+0.41/24 6} Nd5 {-0.06/21 1.6} 32. Kf2 {+0.59/26 7}
Bf6 {-0.14/22 3} 33. Bf3 {+0.57/24 1.3} R6c7 {-0.26/21 2.8} 34. Ke2
{+0.62/24 5} Bh8 {-0.51/22 8} 35. h3 {+0.79/20 2.7} Kg7 {-0.42/23 2.1} 36.
g4 {+0.68/21 6} hxg4 {-0.58/23 3} 37. hxg4 {+0.65/24 4} Rb8 {-0.56/25 2.1}
38. g5 {+0.65/26 9} Kf8 {-0.38/20 2.8} 39. Kd3 {+0.64/23 0.8} Rcc8
{-0.23/23 1.7} 40. Bg4 {+0.72/24 2.6} Rc6 {-0.31/23 2.8} 41. Ke4
{+0.58/23 2.6} Rd8 {-0.27/27 6} 42. Rh1 {+0.58/25 4} Bg7 {-0.23/23 2.1} 43.
Bb2 {+0.65/24 1.3} Kf7 {-0.35/26 7} 44. Rc1 {+0.52/25 14} Rc7 {-0.35/23 5}
45. Kf3 {+0.46/23 4} Rb7 {-0.25/22 3} 46. Ba3 {+0.49/20 5} Ra7 {-0.18/22 4}
47. Ke4 {+0.77/24 7} Rh8 {-0.15/22 1.7} 48. Ne5+ {+0.96/22 3} Bxe5
{-0.60/21 1.0} 49. Kxe5 {+1.06/22 0.4} Nc7 {-0.87/22 2.5} 50. d5
{+1.12/22 2.8} exd5 {-0.97/24 3} 51. Bb2 {+1.15/21 0.4} Rh4 {-0.97/26 5}
52. Bd7 {+1.18/26 8} Rh8 {-1.10/24 2.0} 53. Bd4 {+1.47/23 2.9} Rb7
{-1.16/25 4} 54. Bg4 {+1.44/19 0.4} b5 {-1.14/24 2.8} 55. Rxc7 {+2.05/25 6}
Rxc7 {-1.18/18 0.8} 56. Be6+ {+2.15/21 0.5} Ke8 {-1.53/21 2.3} 57. axb5
{+2.22/24 3} Rc1 {-1.69/23 3} 58. b6 {+2.65/25 6} Kd8 {-1.70/23 1.2} 59.
Bxd5 {+2.66/20 0.3} Kd7 {-1.67/21 1.7} 60. Be6+ {+3.13/22 2.7} Kd8
{-1.76/23 1.5} 61. Ke4 {+4.13/24 4} Rh7 {-2.12/24 2.9} 62. b7 {+5.92/24 3}
Kc7 {-3.64/24 1.1} 63. Bd5 {+6.11/21 0.7} Kb8 {-6.87/26 10} 64. Be5+
{+7.09/21 2.8} Rc7 {-6.87/22 0.8} 65. f5 {+7.21/18 0.3} Rh4+ {-7.40/23 2.4}
66. Bf4 {+8.12/23 3} e6 {-10.12/24 10} 67. Bxe6 {+10.37/22 6} Kxb7
{-11.26/21 2.3} 68. fxg6 {+11.08/21 2.4} Rc5 {-13.05/22 4} 69. Bd5+
{+15.57/21 4} Kb6 {-13.34/24 0.8} 70. g7 {+41.86/22 2.6} Rc8
{-14.79/24 2.0} 71. g8=Q {+47.70/27 1.1} Rxg8 {-250.00/28 1.4} 72. Bxg8
{+53.12/30 4} Rh1 {-250.00/33 1.4} 73. g6 {+54.67/27 2.5} Rg1
{-250.00/34 1.6} 74. Kf5 {+64.09/29 4} Kc5 {-250.00/31 1.0} 75. Be6
{+123.42/29 4} Kb4 {-250.00/31 1.7} 76. Kf6 {+123.45/31 1.6} Kb5
{-250.00/31 2.8} 77. g7 {+123.47/34 1.7} a4 {-250.00/32 0.9} 78. bxa4+
{+123.50/35 2.4} Ka5 {-250.00/37 1.0} 79. g8=Q {+123.50/43 1.7} Rxg8
{-250.00/49 0.8} 80. Bxg8 {+123.51/63 2.4} Kb4 {-250.00/99 0.1} 81. Bd6+
{+123.52/62 1.6} Kxa4 {-250.00/99 0.2} 82. Bc4 {+99.85/40 2.9} Ka5
{+0.00/5 0.2} 83. Bc5 {+99.87/42 0.3} Ka4 {+0.00/5 0.1} 84. e4
{+99.89/68 2.1} Ka5 {+0.00/5 0.1} 85. e5 {+99.91/106 2.0} Ka4 {+0.00/5 0.1}
86. e6 {+99.93/127 0.1} Ka5 {+0.00/5 0.1} 87. e7 {+99.95/127 0.1} Ka4
{+0.00/5 0.1} 88. e8=Q+ {+99.97/127 0.1} Ka5 {+0.00/5 0.1} 89. Qa8#
{+99.99/127 0.1}
1-0[/pgn]

Most interesting move played:

[d] 7r/1rn1pk2/6p1/pp1pK1P1/P2B1PB1/1P2P3/8/2R5 w - - 0 55

Rxc7! having lost the exchange earlier, SF now gives up the exchange to go with BB vs RR in the endgame and SF plays this ending masterfully. The bishops skewered everything in sight.

MikeB · Post by **MikeB** » Sat Nov 28, 2015 8:52 pm

[quote="MikeB"]After 50 games, following the Super Final TCEC openings tc 2.5/2.5 ( about equal to CCRl 40/4), Draw rate is up and given more time and more CPUs, Komodo starts showing its strength.

Hardware : 2010 Mac Pro 12 Core , x5690, OSX 10.1 , 48GB Ram

Settings:

TC 2:30m inc 2.5 sec
4CPU , 64MB Hash
EGTB Enabled
Ponder Off

Code: Select all

ResultSet-EloRating>details
   1 Komodo 9.3           4  50.0 ( 26.0 :  24.0)
                             50.0 ( 26.0 :  24.0) Stockfish 110315    -4
   2 Stockfish 110315    -4  50.0 ( 24.0 :  26.0)
                             50.0 ( 24.0 :  26.0) Komodo 9.3           4
ResultSet-EloRating>ratings
Rank Name               Elo    +    - games score oppo. draws 
   1 Komodo 9.3           4   69   69    50   52%    -4   72% 
   2 Stockfish 110315    -4   69   69    50   48%     4   72% 
ResultSet-EloRating>

UPDATE after 67 games - this is the match that everyone was expecting:

Code: Select all

ResultSet>rResultSet-EloRating>details
   1 Komodo 9.3           1  67.0 ( 34.0 :  33.0)
                             67.0 ( 34.0 :  33.0) Stockfish 110315    -1
   2 Stockfish 110315    -1  67.0 ( 33.0 :  34.0)
                             67.0 ( 33.0 :  34.0) Komodo 9.3           1
ResultSet-EloRating>ratings
Rank Name               Elo    +    - games score oppo. draws 
   1 Komodo 9.3           1   60   60    67   51%    -1   72% 
   2 Stockfish 110315    -1   60   60    67   49%     1   72%

So whatever happened to SF in the Super Final - appears more related to SMP (24 Threads) and the much longer time control than anything else. Short time controls and 4CPUs are not an issue at all.

Norm Pollock · Post by **Norm Pollock** » Sat Nov 28, 2015 9:52 pm

MikeB wrote:
MikeB wrote:After 50 games, following the Super Final TCEC openings tc 2.5/2.5 ( about equal to CCRl 40/4), Draw rate is up and given more time and more CPUs, Komodo starts showing its strength.

Hardware : 2010 Mac Pro 12 Core , x5690, OSX 10.1 , 48GB Ram

Settings:

TC 2:30m inc 2.5 sec
4CPU , 64MB Hash
EGTB Enabled
Ponder Off
Code: Select all
ResultSet-EloRating>details
   1 Komodo 9.3           4  50.0 ( 26.0 :  24.0)
                             50.0 ( 26.0 :  24.0) Stockfish 110315    -4
   2 Stockfish 110315    -4  50.0 ( 24.0 :  26.0)
                             50.0 ( 24.0 :  26.0) Komodo 9.3           4
ResultSet-EloRating>ratings
Rank Name               Elo    +    - games score oppo. draws 
   1 Komodo 9.3           4   69   69    50   52%    -4   72% 
   2 Stockfish 110315    -4   69   69    50   48%     4   72% 
ResultSet-EloRating>
UPDATE after 67 games - this is the match that everyone was expecting:
Code: Select all
ResultSet>rResultSet-EloRating>details
   1 Komodo 9.3           1  67.0 ( 34.0 :  33.0)
                             67.0 ( 34.0 :  33.0) Stockfish 110315    -1
   2 Stockfish 110315    -1  67.0 ( 33.0 :  34.0)
                             67.0 ( 33.0 :  34.0) Komodo 9.3           1
ResultSet-EloRating>ratings
Rank Name               Elo    +    - games score oppo. draws 
   1 Komodo 9.3           1   60   60    67   51%    -1   72% 
   2 Stockfish 110315    -1   60   60    67   49%     1   72% 
So whatever happened to SF in the Super Final - appears more related to SMP (24 Threads) and the much longer time control than anything else. Short time controls and 4CPUs are not an issue at all.
Would it be possible to just print the number of games, how many each engine won, and how many were drawn. All the above statistics are very confusing to me. I'm sure the data I seek is buried in there somewheres, but I don't have the time, energy or expertise to find it. What ever happened to simplicity?

MikeB · Post by **MikeB** » Sat Nov 28, 2015 10:54 pm

MikeB wrote:
MikeB wrote:After 50 games, following the Super Final TCEC openings tc 2.5/2.5 ( about equal to CCRl 40/4), Draw rate is up and given more time and more CPUs, Komodo starts showing its strength.

Hardware : 2010 Mac Pro 12 Core , x5690, OSX 10.1 , 48GB Ram

Settings:

TC 2:30m inc 2.5 sec
4CPU , 64MB Hash
EGTB Enabled
Ponder Off
Code: Select all
ResultSet-EloRating>details
   1 Komodo 9.3           4  50.0 ( 26.0 :  24.0)
                             50.0 ( 26.0 :  24.0) Stockfish 110315    -4
   2 Stockfish 110315    -4  50.0 ( 24.0 :  26.0)
                             50.0 ( 24.0 :  26.0) Komodo 9.3           4
ResultSet-EloRating>ratings
Rank Name               Elo    +    - games score oppo. draws 
   1 Komodo 9.3           4   69   69    50   52%    -4   72% 
   2 Stockfish 110315    -4   69   69    50   48%     4   72% 
ResultSet-EloRating>
UPDATE after 67 games - this is the match that everyone was expecting:
Code: Select all
ResultSet>rResultSet-EloRating>details
   1 Komodo 9.3           1  67.0 ( 34.0 :  33.0)
                             67.0 ( 34.0 :  33.0) Stockfish 110315    -1
   2 Stockfish 110315    -1  67.0 ( 33.0 :  34.0)
                             67.0 ( 33.0 :  34.0) Komodo 9.3           1
ResultSet-EloRating>ratings
Rank Name               Elo    +    - games score oppo. draws 
   1 Komodo 9.3           1   60   60    67   51%    -1   72% 
   2 Stockfish 110315    -1   60   60    67   49%     1   72% 
So whatever happened to SF in the Super Final - appears more related to SMP (24 Threads) and the much longer time control than anything else. Short time controls and 4CPUs are not an issue at all.
After 90 games, mirroring the TCEC openings , it also mirrors the +6 score.
Code: Select all
ResultSet-EloRating>details
   1 Komodo 9.3           8  90.0 ( 48.0 :  42.0)
                             90.0 ( 48.0 :  42.0) Stockfish 110315    -8
   2 Stockfish 110315    -8  90.0 ( 42.0 :  48.0)
                             90.0 ( 42.0 :  48.0) Komodo 9.3           8
ResultSet-EloRating>ratings
Rank Name               Elo    +    - games score oppo. draws 
   1 Komodo 9.3           8   53   53    90   53%    -8   69% 
   2 Stockfish 110315    -8   53   53    90   47%     8   69% 
ResultSet-EloRating>

cma6 · Post by **cma6** » Sun Nov 29, 2015 2:41 am

LK, I have noticed changes in default parameters in K 9.3. Are the changes strongly recommended because the engine has improved; or should one stick with parameters used in K 9.2?
New parameters:
Contempt 10 (from -10)
King Safety 60 (from 65)
Selectivity 131 (from 80)

You mentioned that vs. a recent SF, Contempt of "0" would be recommended. What about the other two parameters above?
My questions are focused on long time controls, e.g., correspondence chess? And thanks for your team's superbly creative work!

lkaufman · Post by **lkaufman** » Sun Nov 29, 2015 6:26 am

cma6 wrote:LK, I have noticed changes in default parameters in K 9.3. Are the changes strongly recommended because the engine has improved; or should one stick with parameters used in K 9.2?
New parameters:
Contempt 10 (from -10)
King Safety 60 (from 65)
Selectivity 131 (from 80)

The values you give in parens are not the defaults from 9.2, perhaps from an older version?

You mentioned that vs. a recent SF, Contempt of "0" would be recommended. What about the other two parameters above?
My questions are focused on long time controls, e.g., correspondence chess? And thanks for your team's superbly creative work!

Contempt is always set to zero in analysis (infinite) mode. For game play, the default is probably best regardless of time limit against engines rated 100 to 200 below Komodo. It was lowered from 15 to 10 partly due to changes in how it works but also due to various tests that indicated that 15 was a bit too aggressive against engines in that rating range. King safety increase was due to tuning and is what we believe to be best at any reasonable time control. There are slight indications that higher values might be okay at longer time controls but this is not nearly proven. The large increase in selectivity was due to some changes in pruning rules and to tuning. We used 100 in TCEC to be conservative even though our tests already suggested a higher value, but with more testing we are fairly confident in the higher value, give or take a few. Here too there is some indication that higher values might work well at longer time limits, but again the data is much too limited to recommend doing so. If anyone wants to run some reasonably slow games (maybe something like 30' + 10" on each core) I would suggest matching a value like 150 or so against the default to see if it's really stronger. I would do it myself but we have higher priority tests to run.

JohnS · Post by **JohnS** » Sun Nov 29, 2015 11:53 am

lkaufman wrote:Komodo 9.3 was released this morning. As usual, it is free for subscribers, and at 20% discount for anyone who bought Komodo 8 or later version. Available at komodochess.com.
Komodo 9.3 is slightly upgraded from the version now close to winning the TCEC final against Stockfish. The biggest difference is that it will use about 10% more time on the first few moves out of book, since it was pretty clear watching TCEC that Komodo keeps too much time in reserve for the endgame. Komodo 9.3 will still usually move quicker than Stockfish, but the difference will not be as great or as obvious. Komodo 9.3 is also slightly upgraded from the versions that performed well giving material handicaps to Grandmasters Movsesian, Lenderman, and Hess in serious matches.
Estimated elo gain from version 9.2 is about 15, both on one core and on many. Each of the steps since version 9.0 has been about 15 elo, so the total gain since Komodo 9 is about 45 elo, and the total gain since Komodo 8 is approaching one hundred elo.
As usual the gains come from all areas. Selectivity has been modified significantly, new eval terms were added, many weights re-tuned, time manangement improved, contempt improved, multiipv improved, MP changes which help on certain machines, free speedups, etc.

Larry

Will those of us who bought the Chessbase version of K9 be able to upgrade to version 9.3?

Thanks.

cma6 · Post by **cma6** » Sun Nov 29, 2015 4:00 pm

LK, your detailed reply is greatly appreciated.

lkaufman · Post by **lkaufman** » Sun Nov 29, 2015 5:06 pm

JohnS wrote:
lkaufman wrote:Komodo 9.3 was released this morning. As usual, it is free for subscribers, and at 20% discount for anyone who bought Komodo 8 or later version. Available at komodochess.com.
Komodo 9.3 is slightly upgraded from the version now close to winning the TCEC final against Stockfish. The biggest difference is that it will use about 10% more time on the first few moves out of book, since it was pretty clear watching TCEC that Komodo keeps too much time in reserve for the endgame. Komodo 9.3 will still usually move quicker than Stockfish, but the difference will not be as great or as obvious. Komodo 9.3 is also slightly upgraded from the versions that performed well giving material handicaps to Grandmasters Movsesian, Lenderman, and Hess in serious matches.
Estimated elo gain from version 9.2 is about 15, both on one core and on many. Each of the steps since version 9.0 has been about 15 elo, so the total gain since Komodo 9 is about 45 elo, and the total gain since Komodo 8 is approaching one hundred elo.
As usual the gains come from all areas. Selectivity has been modified significantly, new eval terms were added, many weights re-tuned, time manangement improved, contempt improved, multiipv improved, MP changes which help on certain machines, free speedups, etc.
Larry

Will those of us who bought the Chessbase version of K9 be able to upgrade to version 9.3?

Thanks.

ChessBase does not plan to offer any upgrades of Komodo but will offer Komodo 10 when it comes out next year. Of course you can buy 9.3 from us and install it in your ChessBase gui. We don't give the $12 discount to Chessbase customers as our system won't recognize them, and anyway we don't want to appear to be trying to take customers from ChessBase. Those who want the ChessBase gui and book but also always want the latest version should buy initially from CB and then after a few months buy a subscription from us.

Komodo 9.3 has been released

Re: Komodo 9.3 has been released

Re: Komodo 9.3 has been released

Re: Komodo 9.3 has been released

Re: Komodo 9.3 has been released

Re: Komodo 9.3 has been released

Recommended K 9.3 parameters

Re: Recommended K 9.3 parameters

Re: Komodo 9.3 has been released

Re: Recommended K 9.3 parameters

Re: Komodo 9.3 has been released