Komodo 9.3 vs Stockfish

Discussion of computer chess matches and engine tournaments.

Moderators: hgm, Rebel, chrisw

Roger Brown
Posts: 782
Joined: Wed Mar 08, 2006 9:22 pm

Re: Komodo 9.3 vs Stockfish

Post by Roger Brown »

Modern Times wrote:Seems that on the hardware that 99% of us can afford and run, Stockfish is stronger in game-play. But on TCEC like hardware, with long time control, the tables are turned. For analysis purposes that makes Komodo the weapon of choice, but if you are serious about things you'll use at least one other engine as well.

Hello Ray,

A question:

As a serious tester for years now, which would get your vote?

The one strongest on "normal" hardware or the strongest on "ultimate" hardware?

Later.
Roger Brown
Posts: 782
Joined: Wed Mar 08, 2006 9:22 pm

Re: Komodo 9.3 vs Stockfish

Post by Roger Brown »

WuShock wrote:
Blitz games ? Gimme a break !
Everyone knows that Komodo can only beat Stockfish in 30 Minutes games and above.
Anything shorter, and Stockfish will win always.
Pointless testing.

Gimme a break .......... cut out your nasty comments .......

Thank you , that is all ..........
Hello Tom,

I had a look.

Childish. Immature. Lacking perspective. Certainly.

Nasty? Not really.

Later.
beram
Posts: 1187
Joined: Wed Jan 06, 2010 3:11 pm

Re: Komodo 9.3 vs Stockfish

Post by beram »

Modern Times wrote:Seems that on the hardware that 99% of us can afford and run, Stockfish is stronger in game-play. But on TCEC like hardware, with long time control, the tables are turned. For analysis purposes that makes Komodo the weapon of choice, but if you are serious about things you'll use at least one other engine as well.
Agreed Ray, although 2 or 3 % better is just marginally whether for Stockfish or Komodo in TCEC final
I think that up to 8 cores latest SF is better and above, because of (still) better scaling with more cores above 8, Komodo gets the upperhand

I haven't noticed any difference in win-loss ratio or ELO going from blitz 3m2s to LTC 30m20s between same opponents in my own testing, nor did I notice such on the CEGT, CCRL or FastGM lists, just to mention some

grts Bram
APassionForCriminalJustic
Posts: 417
Joined: Sat May 24, 2014 9:16 am

Re: Komodo 9.3 vs Stockfish

Post by APassionForCriminalJustic »

beram wrote:
Modern Times wrote:Seems that on the hardware that 99% of us can afford and run, Stockfish is stronger in game-play. But on TCEC like hardware, with long time control, the tables are turned. For analysis purposes that makes Komodo the weapon of choice, but if you are serious about things you'll use at least one other engine as well.
Agreed Ray, although 2 or 3 % better is just marginally whether for Stockfish or Komodo in TCEC final
I think that up to 8 cores latest SF is better and above, because of (still) better scaling with more cores above 8, Komodo gets the upperhand

I haven't noticed any difference in win-loss ratio or ELO going from blitz 3m2s to LTC 30m20s between same opponents in my own testing, nor did I notice such on the CEGT, CCRL or FastGM lists, just to mention some

grts Bram
I'm quite sure that it has been mentioned and agreed many times that Komodo also has a better evaluation function. To simply say it is just better scaling is sort of a slap in the face to both Mark and Larry. I happen to believe that Stockfish also scales very well at VLTC, so I'm not entirely convinced that Komodo can be that much better in that department. Stockfish uses a ton of time which is absolutely the pivotal thing when playing very long games. There was never any reason to assume that it played worse at VLTC; as long as time is used the engines should shine.

Komodo 9.2 absolutely destroys Stockfish 6 from the rating lists that I've seen. Thus I can't see SF development being that much better than K9.3 on mediocre hardware...
beram
Posts: 1187
Joined: Wed Jan 06, 2010 3:11 pm

Re: Komodo 9.3 vs Stockfish

Post by beram »

beram wrote:
beram wrote:
beram wrote:First test with Komodo 9.3 is done with one of the strongest latest Stockfish compiles SF 121115IP (Ipman compile)
SF came out as a winner but +20 - 16 is pretty close I would say
Privat testbook of 25 positions 2 runs 100games
Both engines with standard setting, 512 Mb Hash, ponder off

Code: Select all

SF 121115IP - Komodo 9.3 Blitz 3m2s     i5 4200M @2500Mhz, 2 cpu (Fritzmark 5,1) 
1   SF 121115IP 64 BMI2   +14  +20/=64/-16 52.00%   52.0/100
2   Komodo 9.3 64-bit     -14  +16/=64/-20 48.00%   48.0/100
earlier result of Komodo 9.2(ct=0) against SF 121115 was a 60% win and a record for a SFdev against Komodo 9.2(ct=0) under these conditions
(http://www.talkchess.com/forum/viewtopi ... 71&t=57871)
Second test with Komodo 9.3 with contempt=0 setting
51% for SF 121115IP

Code: Select all

SF 121115IP - Komodo 9.3(ct=0) Blitz 3m2s     i5 4200M @2500Mhz, 2 cpu (Fritzmark 5,1)
1   SF 121115IP 64 BMI2    +7  +21/=60/-19 51.00%   51.0/100
2   Komodo 9.3 64-bit      -7  +19/=60/-21 49.00%   49.0/100
And a third test on my AMD X6 1090T with 4 cores and same openingsbook
gives a 53% win for SF 121115IP with a high 72% draw percentage

Code: Select all

SF 121115IP - Komodo 9.3(ct=0), Blitz 3m2s
                             
1   SF 121115IP 64      +21  +17/=72/-11 53.00%   53.0/100
2   Komodo 9.3 64-bit   -21  +11/=72/-17 47.00%   47.0/100
Fourth cycle with Komodo 9.3 default on my AMD X6 1090T, just very small win for SF121115IP 51,5% with 57% drawpercentage

Code: Select all

SF 121115IP - Komodo 9.3, Blitz 3m2s
                             
1   SF 121115IP 64      +10  +23/=57/-20 51.50%   51.5/100
2   Komodo 9.3 64-bit   -10  +20/=57/-23 48.50%   48.5/100
Modern Times
Posts: 3546
Joined: Thu Jun 07, 2012 11:02 pm

Re: Komodo 9.3 vs Stockfish

Post by Modern Times »

Roger Brown wrote:
Hello Ray,

A question:

As a serious tester for years now, which would get your vote?

The one strongest on "normal" hardware or the strongest on "ultimate" hardware?

Later.

Well, I'm a very weak chess player myself, so either one is overkill for my purposes. The level of play for both is way over my head.

In both cases, leave it overnight on your 4,6 or 8 core home machine, with multipv perhaps and you'll get an extremely high quality analysis from either of them.
beram
Posts: 1187
Joined: Wed Jan 06, 2010 3:11 pm

Re: Komodo 9.3 vs Stockfish

Post by beram »

beram wrote:
beram wrote:
beram wrote:
beram wrote:First test with Komodo 9.3 is done with one of the strongest latest Stockfish compiles SF 121115IP (Ipman compile)
SF came out as a winner but +20 - 16 is pretty close I would say
Privat testbook of 25 positions 2 runs 100games
Both engines with standard setting, 512 Mb Hash, ponder off

Code: Select all

SF 121115IP - Komodo 9.3 Blitz 3m2s     i5 4200M @2500Mhz, 2 cpu (Fritzmark 5,1) 
1   SF 121115IP 64 BMI2   +14  +20/=64/-16 52.00%   52.0/100
2   Komodo 9.3 64-bit     -14  +16/=64/-20 48.00%   48.0/100
earlier result of Komodo 9.2(ct=0) against SF 121115 was a 60% win and a record for a SFdev against Komodo 9.2(ct=0) under these conditions
(http://www.talkchess.com/forum/viewtopi ... 71&t=57871)
Second test with Komodo 9.3 with contempt=0 setting
51% for SF 121115IP

Code: Select all

SF 121115IP - Komodo 9.3(ct=0) Blitz 3m2s     i5 4200M @2500Mhz, 2 cpu (Fritzmark 5,1)
1   SF 121115IP 64 BMI2    +7  +21/=60/-19 51.00%   51.0/100
2   Komodo 9.3 64-bit      -7  +19/=60/-21 49.00%   49.0/100
And a third test on my AMD X6 1090T with 4 cores and same openingsbook
gives a 53% win for SF 121115IP with a high 72% draw percentage

Code: Select all

SF 121115IP - Komodo 9.3(ct=0), Blitz 3m2s
                             
1   SF 121115IP 64      +21  +17/=72/-11 53.00%   53.0/100
2   Komodo 9.3 64-bit   -21  +11/=72/-17 47.00%   47.0/100
Fourth cycle with Komodo 9.3 default on my AMD X6 1090T, just very small win for SF121115IP 51,5% with 57% drawpercentage

Code: Select all

SF 121115IP - Komodo 9.3, Blitz 3m2s
                             
1   SF 121115IP 64      +10  +23/=57/-20 51.50%   51.5/100
2   Komodo 9.3 64-bit   -10  +20/=57/-23 48.50%   48.5/100
Fifth cycle the latest SF 021215 against Komodo 9.3 (default) on my AMD X6 1090T
52% +24 -20 =56

Code: Select all

SF 021215 - Komodo 9.3, Blitz 3m+2s
                         
1   Stockfish 021215 64  +24/=56/-20 52.00%   52.0/100
2   Komodo 9.3 64-bit    +20/=56/-24 48.00%   48.0/100
beram
Posts: 1187
Joined: Wed Jan 06, 2010 3:11 pm

Re: Komodo 9.3 vs Stockfish

Post by beram »

beram wrote:
beram wrote:
beram wrote:
beram wrote:
beram wrote:First test with Komodo 9.3 is done with one of the strongest latest Stockfish compiles SF 121115IP (Ipman compile)
SF came out as a winner but +20 - 16 is pretty close I would say
Privat testbook of 25 positions 2 runs 100games
Both engines with standard setting, 512 Mb Hash, ponder off

Code: Select all

SF 121115IP - Komodo 9.3 Blitz 3m2s     i5 4200M @2500Mhz, 2 cpu (Fritzmark 5,1) 
1   SF 121115IP 64 BMI2   +14  +20/=64/-16 52.00%   52.0/100
2   Komodo 9.3 64-bit     -14  +16/=64/-20 48.00%   48.0/100
earlier result of Komodo 9.2(ct=0) against SF 121115 was a 60% win and a record for a SFdev against Komodo 9.2(ct=0) under these conditions
(http://www.talkchess.com/forum/viewtopi ... 71&t=57871)
Second test with Komodo 9.3 with contempt=0 setting
51% for SF 121115IP

Code: Select all

SF 121115IP - Komodo 9.3(ct=0) Blitz 3m2s     i5 4200M @2500Mhz, 2 cpu (Fritzmark 5,1)
1   SF 121115IP 64 BMI2    +7  +21/=60/-19 51.00%   51.0/100
2   Komodo 9.3 64-bit      -7  +19/=60/-21 49.00%   49.0/100
And a third test on my AMD X6 1090T with 4 cores and same openingsbook
gives a 53% win for SF 121115IP with a high 72% draw percentage

Code: Select all

SF 121115IP - Komodo 9.3(ct=0), Blitz 3m2s
                             
1   SF 121115IP 64      +21  +17/=72/-11 53.00%   53.0/100
2   Komodo 9.3 64-bit   -21  +11/=72/-17 47.00%   47.0/100
Fourth cycle with Komodo 9.3 default on my AMD X6 1090T, just very small win for SF121115IP 51,5% with 57% drawpercentage

Code: Select all

SF 121115IP - Komodo 9.3, Blitz 3m2s
                             
1   SF 121115IP 64      +10  +23/=57/-20 51.50%   51.5/100
2   Komodo 9.3 64-bit   -10  +20/=57/-23 48.50%   48.5/100
Fifth cycle the latest SF 021215 against Komodo 9.3 (default) on my AMD X6 1090T
52% +24 -20 =56

Code: Select all

SF 021215 - Komodo 9.3, Blitz 3m+2s

              
1   Stockfish 021215 64  +24/=56/-20 52.00%   52.0/100
2   Komodo 9.3 64-bit    +20/=56/-24 48.00%   48.0/100
6th cycle the latest SF 021215 against Komodo 9.3 (ct=0) on my AMD X6 1090T
This time +21 -12 =67 far more draws but lesser wins for Komodo with contempt set to zero

Code: Select all

SF 021215 - Komodo 9.3, Blitz 3m+2s
                         
1   Stockfish 021215 64  +21/=67/-12 54.50%   54.5/100
2   Komodo 9.3 64-bit    +12/=67/-21 45.50%   45.5/100
beram
Posts: 1187
Joined: Wed Jan 06, 2010 3:11 pm

Re: Komodo 9.3 vs Stockfish

Post by beram »

New test Komodo 9.3 (default) against latest SF 051215 on my i5
This time 53% with low draw percentage

Code: Select all

SF 051215 - Komodo 9.3 Blitz 3m2s     i5 4200M @2500Mhz, 2 cpu (Fritzmark 5,1)                                    
1   Stockfish 051215 64 BMI2   +21  +27/=52/-21 53.00%   53.0/100
2   Komodo 9.3 64-bit          -21  +21/=52/-27 47.00%   47.0/100
S.Taylor
Posts: 8514
Joined: Thu Mar 09, 2006 3:25 am
Location: Jerusalem Israel

Re: Komodo 9.3 vs Stockfish

Post by S.Taylor »

3 main uses of computerchess:
1).To challenge yourself against. Well, Komodo should be the same as sf at ANY TC.

2).To use for analysis, correspondence etc. Well, we already agree that Komodo is now top.

3).For comp vs comp shorter TC's. No normal consumer should buy it for that purpose. It is anti-intellectual to do so. It's like an irrelevant curiosity, That's fine, but not more than that.

I can't think of any other important value of chess engines.