Stockfish vs Komodo 9.4

[lkaufman]

Komodo once again tops all the major rating lists (CCRL, CEGT, IPON, and Frank's list) yet falls a bit short in direct matches with SF. Some of this is due to Contempt, which hurts in direct matches with SF, but this doesn't appear to be the full explanation. I think that it is because SF searches slightly deeper than Komodo, which is important in direct matchups, but against other engines it is not so significant, and Komodo does better due to better eval. But that's hard to prove.

Could it be that Stockfish wins because it is just stronger than Komodo?

No, because this wouldn't explain why Komodo consistently scores better against other engines to get higher ratings everywhere. Just look at the CEGT 5' + 3" list! In human play, if A has a huge score against B but a consistently lower rating, most would say that B is stronger. Why is it different for engines?

A really pathetic answer
In human chess if A wins from B in WC final than A is champion, no matter if it is rated 10 or 20 ELO lower

When Kramnik won the match for WC he became WC, but Kasparov remained number one on rating list and simply used the title "World Number 1". I think if you took a poll among strong chessplayers as to who was the stronger player, a big majority would have voted for Kasparov. Botvinnik and Topalov both won WC in RR events; even Carlsen, who benefits from the current system, favors RR WC. There is such a thing as chess style, and sometimes one player just doesn't do well against another even though he is widely perceived as the stronger player.

[Frank Quisinsky]

Or ...
Kasparow playing vs. Gulko only.

Never Kasparow are under the TOP-10 in times Kasparow are the clear best player the World saw at this time.

[Frank Quisinsky]

Gulko are for me the best chess player. Ivantchuk have to often bad times, but it's possible that Ivantchuk is the best chess player in his good times.

Have to wrote that.
And ... absolutely ... Gulko and Ivantchuk ... maybe Shirow and of course Tal are the Players with the most interesting playing style. I like Miles too and Komodo is great



Best
Frank

PS: the same situation in human chess, compare to computer chess. The World is looking to Fischer, Karpow and Kasparow and can't see the others, maybe much better players. Yes, Kasparow was a World Champion and plays many games for us. Indeed very very strong player ... but others, like Gulko never have a chance in the system at this times.

[Graham Banks]

...There is not much between them is too gently put, 52-56% is substantial
And if you are sportive you admit that...

In my 8 core matches, Stockfish has had the edge so far.
Too early to say from my end whether the same applies to 1 core or 4 cores head to head.

[JJJ]

Graham showed a result of 50/50 , so did Clemens in direct encounter and contempt 0.

And we can add that Komodo score betters against weakest opponent than Stockfish.

So for me, Komodo is still the best engine so far with a nice margin, no matter what's your result shows in your condition of direct encounter.

Everyone is free to think what's he wants about this. But stop your judgment Bram, we all know you prefer Stockfish and hates Komodo.

[Tdunbug]

I am wondering if specific CPU, DRAM and Motherboard combos along with open books chosen play a role in how these rating tests play out?

[lkaufman]

I am wondering if specific CPU, DRAM and Motherboard combos along with open books chosen play a role in how these rating tests play out?

These things do play a role, but in my opinion not a very big one, just a few elo either way usually. Time limit and number of cores are bigger factors. Stockfish does get a few elo extra on new machines with their bmi2 compile; we didn't see enough benefit for Komodo to bother doing this. I remember AMD vs Intel was an issue when Houdini was the target, but now that it is relegated to history I don't think that matters much anymore. Choice of opponents is another factor; a tester could artificially inflate or deflate the rating of a certain engine by clever choices of opponent and perhaps also book. But I don't believe any of the testers actually do this, they all seem honest to me.

[beram]

In fact, you should even wait for Komodo 9.42 as far I understood.

Well, i have allready 150 games now at same TC 3m2s and same other conditions
In fact, sofar Komodo 9.42 is doing worse
55,3% for Stockfish +33 =100 -17

After 200 games 55,75% for SF 150316 against Komodo 9.42(with contempt=0)
+44 =135 -21, with 67,5% drawpercentage

Code: Select all

SF 150316 - Komodo 9.4(ct=0), Blitz 3m+2s
                                    
1   Stockfish 150316 64 BMI2   +40  +44/=135/-21 55.75%  111.5/200
2   Komodo 9.42 64-bit         -40  +21/=135/-44 44.25%   88.5/200

At same time I have allready 50 games on AMD1090T 4core each at TC15m10s SF020316 against Komodo 9.42(with contempt=0)
52% for SF 020316
+9 =34 -7, with 68% drawpercentage
26 games before with Komodo 9.4 at same PC +5 =17 -4 for SF 020316

I let it run for another 50 games to see the outcome and have a comparison over 100 games between SF020316 vs K9.3 and K9.42
SF020316 against Komodo 9.3 scored 58% at same TC 15m10s 4cores

Outcome for SF 020316 against Komodo 9.42 53% win for SF 020316 with 68% drawpercentage
Meaning a 5% improvement for Komodo 9.42 in a 100 game match over Komodo 9.3 under same match conditions

Code: Select all

SF 020316 - Komodo 9.42(ct=0), Rapid 15m 10s
                               
1   Stockfish 020316 64   +21  +19/=68/-13 53.00%   53.0/100
2   Komodo 9.42 64-bit    -21  +13/=68/-19 47.00%   47.0/100

Latest results:
SF270316 gets 55,75% in 200 games against Komodo 9.42(with contempt=0) on my I5 same TC, same testbook of 25 lines as ever
Is same result as SF150316 gets but with higher drawpercentage of almost 70%

End of august and septembre 2015 with same testbook same TC, results between best SF and best Komodo9.2 at the time, results where about equal around 50%
[disclaimer for Larry, this doesnt necessary means that Komodo comes second now]

Code: Select all

SF 270316 - Komodo 9.42(ct=0), Blitz 3m+2s   
                                 
1   Stockfish 270316 64 BMI2   +40  +42/=139/-19 55.75%  111.5/200
2   Komodo 9.42 64-bit         -40  +19/=139/-42 44.25%   88.5/200

Code: Select all

SF 290815 – Komodo 9.2, Blitz 3m+2s	i5 4200M @2500Mhz, 2 cpu (Fritzmark 5,1)                             
1   Komodo 9.2 64-bit           +3  +25/=51/-24 50.50%   50.5/100
2   Stockfish 290815 64 BMI2    -3  +24/=51/-25 49.50%   49.5/100

SF 290815- Komodo 9.2, Blitz 3m+2s	AMD 1090T @3200Mhz, 4cpu (Fritzmark 15,6)                         
1   Komodo 9.2 64-bit    +19/=63/-18 50.50%   50.5/100
2   Stockfish 290815 64  +18/=63/-19 49.50%   49.5/100

[MikeB]

A really pathetic answer
In human chess if A wins from B in WC final than A is champion, no matter if it is rated 10 or 20 ELO lower

This is pathetic. You insult someone just because you don't like their answer. The truth when opponents in any sport are evenly match and they play until one side wins, someone will the champion. But the champion is not always the best player, the best team - whatever , they were just best on that day. These short matches do not prove anything as to where one is better than the other. It may take hundreds of thousands of games - and even then , there is a chance that you hit the outlier and you may not be correct. Show a little more respect towards Larry and cut the shit.

[beram]

Komodo once again tops all the major rating lists (CCRL, CEGT, IPON, and Frank's list) yet falls a bit short in direct matches with SF. Some of this is due to Contempt, which hurts in direct matches with SF, but this doesn't appear to be the full explanation. I think that it is because SF searches slightly deeper than Komodo, which is important in direct matchups, but against other engines it is not so significant, and Komodo does better due to better eval. But that's hard to prove.

Komodo doesnt top all of the lists, at CCRL 40/4 and at CEGT with 1 cpu Stockfish is better on both 40/4 as at 40/20 And perhaps surprisingly, the difference at CEGT is even greater at the longer TC 40/20 where SF7(!) scores 55% against K9.42
The reason that K wins more against very weak opponents is because of its clever contempt setting and not due to better eval.
When it comes to matches between Komodo and Stockfish, contempt=0 against Stockfish doesnt prevent it from loosing nor helps at all, as me and others have prooven in numerous matches, which you can find at this and other fora
There isnt yet one single match between K.942 and SF7 or latest SFdev to be found on the common lists where Komodo won. The best is 40/40 4cpu with just few games and CEGT5m3s where on both it is an even outcome
All the other matches have figures with average around 53-55% better for SF.
The list from Tom Casanovas gives the most realistic figure, only the four best engines play and that list shows an ELO difference of 26 points
http://www.talkchess.com/forum/viewtopi ... 56&t=56274