Komodo 5 release now available!

[carldaman]

If i am wrong i will apologize with Don and larry but i think this version has (if) a very little improvement compared to Komodo 4, i would say around 10 elo.

If it is so it is a very sad day for all computer chess lovers and the triumph of Robert Houdart.

Best Regards

That seems a bit hasty. There are quite a few issues with IPON that could work against Komodo (ponder on, AMD processors, bayselo, and the fast time control itself). Why not wait till it's tested by CCRL and others, and then draw a better conclusion?

Regards,
Carl

[Alibaba]

Hi,

after waiting for 10 hours i finally found the download link in a spam filter...so if any problems, then first check your spam filters guys...

Thanks for the new Komodo. I will soon start with long time controls match vs top engines....

Regards
Andi

[Rebel]

If i am wrong i will apologize with Don and larry but i think this version has (if) a very little improvement compared to Komodo 4, i would say around 10 elo.

If it is so it is a very sad day for all computer chess lovers and the triumph of Robert Houdart.

Best Regards

We have MANY tester who have verified that Komodo 5 is much stronger than Komodo 4. This is based on thousands of games - and our own testing is showing over 30 ELO at 2+2 on very fast hardware - roughly equivalent to the IPON time control.

So I trust our 30,000 game sample much more than the 700 game sample - although I admit this is disappointing so far.

4448.00 and 4452.00 are very close to Komodo 5 -hardly any differences and Komodo 5 is just a better compiled version of these.

So let's wait and see what happens.

Agree!

Everyone with a quad and interested in modern testing (thousands of games) once should do the following experiment: Match program X vs program Y of about equal strength in 4 threads using a fast time control with Arena or cutechess-cli. Each thread uses its own predefined opening set of (say) 500 games so in the end you are playing 2 x 500 = 1000 x 4 = 4000 games.

Then watch the score development and behaviour of the 4 threads. For example, when the 4000 games are finished you won't typically get:

X1 vs Y1 = 500 - 500
X2 vs Y2 = 510 - 490
X3 vs Y3 = 490 - 510
X4 vs Y4 = 500 - 500

or something similar, instead it's far more likely you get something like this:

X1 vs Y1 = 550 - 450
X2 vs Y2 = 530 - 470
X3 vs Y3 = 460 - 540
X4 vs Y4 = 460 - 540

Which is the same end result.

Now imagine you only played 1000 games with X1 and its corresponding opening set you would conclude X1 is a lot better than Y1 and vice versa for X2.

Randomness is a monster and you beat it by volume.

[MM]

If i am wrong i will apologize with Don and larry but i think this version has (if) a very little improvement compared to Komodo 4, i would say around 10 elo.

If it is so it is a very sad day for all computer chess lovers and the triumph of Robert Houdart.

Best Regards

We have MANY tester who have verified that Komodo 5 is much stronger than Komodo 4. This is based on thousands of games - and our own testing is showing over 30 ELO at 2+2 on very fast hardware - roughly equivalent to the IPON time control.

So I trust our 30,000 game sample much more than the 700 game sample - although I admit this is disappointing so far.


So let's wait and see what happens.

Code: Select all

Rank Name                       Elo      +      -    games   score   oppo.   draws 
   1 Komodo 5 64bit           3042.1   10.8   10.8    3097   56.8%  2995.2   43.8% 
   2 Komodo 4452.00 64 bit    3031.2    5.8    5.8   10793   55.2%  2995.4   44.4% 
   3 Komodo 4448.00 64 bit    3030.9   12.7   12.7    2251   56.5%  2985.8   43.4% 
   4 Houdini 1.5a x64         3025.9    7.3    7.3    6935   50.0%  3025.9   41.2% 
   5 Komodo 4445.00 64 bit    3024.3   10.4   10.4    3392   54.8%  2991.5   43.8% 
   6 Komodo 4 SSE4            3006.4   10.5   10.5    3381   52.1%  2991.7   40.2% 
   7 Critter 1.4 64-bit SSE4  3000.0    6.4    6.4    8696   46.3%  3025.4   45.2% 
   8 Stockfish 2.2.2 JA       2960.8    6.5    6.5    8677   40.5%  3026.7   44.0% 

Yes, Don, i am bit disappointing too. I never thought that you claimed a fake improvement and i don't claim i'm right. It was just an impression. If it is wrong i will apologize.

Best Regards

[MM]

Since we are currently showing +36 elo for the actual release version over Komodo 4 after 3000 games by our network of (mostly) independent testers, against Houdini, Critter, and SF, if the IPON results end up way below this we need to find out why. The most likely suspect would be that we are doing something wrong relating to pondering, since we test with Ponder off and IPON (as the name implies) tests with Ponder ON. Another possibility is that the relatively large base time (relative to increment) in IPON (they use 5' + 3", we use 2' + 2") is unfavorable for K5. The AMD platform of IPON could also be an issue, we optimize on Intel. We'll have more info once the CCRL and CEGT tests come in for comparison.

This sounds to me as very logical and moderate comment. I fully agree.

Best Regards

[MM]

If i am wrong i will apologize with Don and larry but i think this version has (if) a very little improvement compared to Komodo 4, i would say around 10 elo.

If it is so it is a very sad day for all computer chess lovers and the triumph of Robert Houdart.

Best Regards

That seems a bit hasty. There are quite a few issues with IPON that could work against Komodo (ponder on, AMD processors, bayselo, and the fast time control itself). Why not wait till it's tested by CCRL and others, and then draw a better conclusion?

Regards,
Carl

Hi Carl,

yes it is possible that it was a bit hasty but it was anyway based on over 710 games of ipon chess and some games of mine.

It wouldn't be the 1st time that the results of ipon are different from the expectations of the authors. That's why i wrote my feeling (my fear). But it was clear that it was an impression, i didn't claim anything definite (of course).

Thank you

Best Regards

[melajara]

Since we are currently showing +36 elo for the actual release version over Komodo 4 after 3000 games by our network of (mostly) independent testers, against Houdini, Critter, and SF, if the IPON results end up way below this we need to find out why. The most likely suspect would be that we are doing something wrong relating to pondering, since we test with Ponder off and IPON (as the name implies) tests with Ponder ON. Another possibility is that the relatively large base time (relative to increment) in IPON (they use 5' + 3", we use 2' + 2") is unfavorable for K5. The AMD platform of IPON could also be an issue, we optimize on Intel. We'll have more info once the CCRL and CEGT tests come in for comparison.

Currently, one can see from the IPON snapshot that there are 2 relatively weak programs (probably you never used in your internal testing) dragging down Komodo, Hannibal and MinkoChess, albeit the situation is improving with the sample size

Komodo 5

Komodo 5 - Houdini 2.0 STD (3028) 42.0 - 47.0 47.19% Perf=3009
Komodo 5 - Critter 1.6a (2973) 50.5 - 38.5 56.74% Perf=3020
Komodo 5 - Stockfish 2.2.2 JA (2966) 46.0 - 41.0 52.87% Perf=2985
Komodo 5 - Deep Rybka 4.1 (2956) 50.5 - 38.5 56.74% Perf=3003
Komodo 5 - Naum 4.2 (2834) 69.0 - 19.0 78.41% Perf=3058
Komodo 5 - Deep Shredder 12 (2800) 69.5 - 19.5 78.09% Perf=3020
Komodo 5 - Gull 1.2 (2796) 71.5 - 16.5 81.25% Perf=3050
Komodo 5 - Hannibal 1.2 (2792) 63.5 - 25.5 71.35% Perf=2950
Komodo 5 - Deep Sjeng c't 2010 32b (2791) 68.5 - 20.5 76.97% Perf=3000
Komodo 5 - Spike 1.4 32b (2782) 70.5 - 17.5 80.11% Perf=3024
Komodo 5 - spark-1.0 (2764) 75.5 - 12.5 85.80% Perf=3076
Komodo 5 - Protector 1.4.0 (2755) 75.0 - 14.0 84.27% Perf=3046
Komodo 5 - Deep Junior 13.3 (2744) 74.0 - 15.0 83.15% Perf=3021
Komodo 5 - HIARCS 13.2 MP 32b (2743) 75.5 - 13.5 84.83% Perf=3042
Komodo 5 - Quazar 0.4 (2732) 73.5 - 14.5 83.52% Perf=3013
Komodo 5 - Zappa Mexico II (2699) 78.5 - 9.5 89.20% Perf=3065
Komodo 5 - MinkoChess 1.3 (2691) 73.0 - 15.0 82.95% Perf=2965
1126.5 - 377.5 74.90% Perf=3003


1504 out of 2550 games played
Level: 5 Minutes/Game + 3 Seconds/Move

[MM]

Testing Komodo at 1'+0'' against Houdini i keep on experiencing a lot of time losses. I thought it was only a problem in 10 seconds per game but it isn't.

I know that generally sudden death games are not considered too much when the time control is relatively short but i wonder if would't it good a fix about that.

Thank you

Best Regards

[MM]

Since we are currently showing +36 elo for the actual release version over Komodo 4 after 3000 games by our network of (mostly) independent testers, against Houdini, Critter, and SF, if the IPON results end up way below this we need to find out why. The most likely suspect would be that we are doing something wrong relating to pondering, since we test with Ponder off and IPON (as the name implies) tests with Ponder ON. Another possibility is that the relatively large base time (relative to increment) in IPON (they use 5' + 3", we use 2' + 2") is unfavorable for K5. The AMD platform of IPON could also be an issue, we optimize on Intel. We'll have more info once the CCRL and CEGT tests come in for comparison.

Currently, one can see from the IPON snapshot that there are 2 relatively weak programs (probably you never used in your internal testing) dragging down Komodo, Hannibal and MinkoChess, albeit the situation is improving with the sample size

Komodo 5

Komodo 5 - Houdini 2.0 STD (3028) 42.0 - 47.0 47.19% Perf=3009
Komodo 5 - Critter 1.6a (2973) 50.5 - 38.5 56.74% Perf=3020
Komodo 5 - Stockfish 2.2.2 JA (2966) 46.0 - 41.0 52.87% Perf=2985
Komodo 5 - Deep Rybka 4.1 (2956) 50.5 - 38.5 56.74% Perf=3003
Komodo 5 - Naum 4.2 (2834) 69.0 - 19.0 78.41% Perf=3058
Komodo 5 - Deep Shredder 12 (2800) 69.5 - 19.5 78.09% Perf=3020
Komodo 5 - Gull 1.2 (2796) 71.5 - 16.5 81.25% Perf=3050
Komodo 5 - Hannibal 1.2 (2792) 63.5 - 25.5 71.35% Perf=2950
Komodo 5 - Deep Sjeng c't 2010 32b (2791) 68.5 - 20.5 76.97% Perf=3000
Komodo 5 - Spike 1.4 32b (2782) 70.5 - 17.5 80.11% Perf=3024
Komodo 5 - spark-1.0 (2764) 75.5 - 12.5 85.80% Perf=3076
Komodo 5 - Protector 1.4.0 (2755) 75.0 - 14.0 84.27% Perf=3046
Komodo 5 - Deep Junior 13.3 (2744) 74.0 - 15.0 83.15% Perf=3021
Komodo 5 - HIARCS 13.2 MP 32b (2743) 75.5 - 13.5 84.83% Perf=3042
Komodo 5 - Quazar 0.4 (2732) 73.5 - 14.5 83.52% Perf=3013
Komodo 5 - Zappa Mexico II (2699) 78.5 - 9.5 89.20% Perf=3065
Komodo 5 - MinkoChess 1.3 (2691) 73.0 - 15.0 82.95% Perf=2965
1126.5 - 377.5 74.90% Perf=3003


1504 out of 2550 games played
Level: 5 Minutes/Game + 3 Seconds/Move

You are probably right but what impresses me more is the results against Houdini (negative) and Stockfish (not impressive), especially Houdini.

I was pretty sure that in a face to face match against Houdini, Komodo would have won a similar match but it seems that it is not so, at least for now. I'm very surprised about that. Perhaps it depends by the compile (?).

Best Regards

[TimoK]

Thank you Don, for the release!

I've started a new series of testing quite similar to my former "Clash of the Titans" match at longer TCs:

Komodo vs.
- Houdini: http://www.team-oh.de/livegames2/
- Stockfish: http://www.team-oh.de/livegames3/
- Critter: http://www.team-oh.de/livegames4/
- Rybka: http://www.team-oh.de/livegames5/

All engines use 1 core (AMD Phenom II @ 3GHz), 512MB Hash, 120m+3s, Ponder=On, 30 Openings (Noomen Test Suite 2012) have to be played with both colors.

Let's see how the new Komodo 5 fights the others compared to my first test: http://www.team-oh.de/Computerschach/Clash.htm

Best regards
Timo

P.S.: I know that the test sample is of no statistical relevance (only 4 x 60 games), but it should give us a first glance of how Komodo 5 behaves at longer TCs against top opponents.

EDIT: I forgot to mention that it is possible to download the games using "games.pgn" added to the above links. E.g. http://www.team-oh.de/livegames2/games.pgn downloads the games from the Komodo vs. Houdini match.