If komodo is 2900 elo human rating at 1 sec / move, it could be the challenge I d like to see
2900 is close to the highest elo of Magnus Carslen. Could be interesting to see strong GM ( around 2700 ) play some against it.
Nakamura vs. Komodo
Moderators: hgm, Rebel, chrisw
-
- Posts: 12038
- Joined: Mon Jul 07, 2008 10:50 pm
Re: Round 3 fight now
if nakamura had had normal time controls, while komodo keeps his, do you think he would have done better ?Laskos wrote:Thanks Larry and Mark for this match. By now you are able pick the right handicaps in order to have balanced matches, as the last matches had shown. People are getting pretty good too at predicting the result, 2.5/4 for Komodo was the central value in the poll answers.lkaufman wrote:Well, the handicap obviously depends on the opponent. I don't know when or if we'll have another chance to play a 2800 level opponent. I'll try to think of some new ideas for matches with "ordinary" GMs.JJJ wrote:Nakamura played well. So, now we know that draw is best thing to expect at these handicap, will you change them Larry ?
What is the currently elo of komodo based only with his game against human ? Can you determine it ?
I plan to calculate Komodo's performance rating at each handicap so far. The question is how much to add to those numbers for the handicap. Based on both my own tests and those of Kai, we have some idea, but it's not very exact. Anyway, I'll post something on this once I have the time.
Naka played very well, congratulations.
The overall performance of Komodo versus Naka is probably somewhere around 3200 ELO. Also, this performance is much more stable than computer ratings. An improvement of 200 ELO points of Komodo on CCRL will be reflected only in some minor increase of the balanced handicap against top human. This "compression" of improvement comes from the dilation of the handicap with the strength, engine improves not much faster than the same handicap increases too in its ELO value with the strength improvement of the engine. Probably Naka can draw at f7 + c7 odds even against the perfect engine, because there are only 1000-1200 more ELO points to Komodo, and the value of the handicap against top human increases too to huge value at this level of engine strength.
Other ways of giving odds seem less spectacular. Fixed depth is meaningless, a 1992 Mephisto is probably not far away from Komodo at fixed depth (in fact I don't know which is stronger). Time odds: not intuitive for most chess players, not clear in its magnitude even to many CC specialists. That Russian GM commentary on twitch was very interesting, at one time he asked something like: "do computers really need time to think? I don't see any difference in the level of playing whether it thinks for 1 second or 1 minute". Time odds would surely not impress him. At 1 second/move Komodo is roughly 2900 or so in human ratings, at 1 minute/move maybe 3100, both beyond the usual human play. Maybe "taking back" is something new to consider, as humans, even top ones, are particularly vulnerable to some "stupid" blunders, which ruin almost all good intentions.
Thanks again for this entertainment!
PS What was the value of the contempt in these games?
-
- Posts: 565
- Joined: Thu Nov 13, 2014 12:03 pm
Re: Round 3 fight now
That seems plausible, if not optimistic.lkaufman wrote:Komodo at four ply would probably be something like 1800 elo or so, I would guess.
I played a few games at 3 moves (6 plies) with the Mac OS simple chess engine (Sjeng-based?). I should be around 2000-2200 ELO and at those settings I could beat it easily. Moving the setting to 3 sec/move totally turns the tables though.. lol.
-
- Posts: 10948
- Joined: Wed Jul 26, 2006 10:21 pm
- Full name: Kai Laskos
Re: Round 3 fight now
Yes, definitely so, although might not be clear in 1-2 games. Komodo can play too in 2 hours, doesn't matter too much.duncan wrote:if nakamura had had normal time controls, while komodo keeps his, do you think he would have done better ?Laskos wrote:Thanks Larry and Mark for this match. By now you are able pick the right handicaps in order to have balanced matches, as the last matches had shown. People are getting pretty good too at predicting the result, 2.5/4 for Komodo was the central value in the poll answers.lkaufman wrote:Well, the handicap obviously depends on the opponent. I don't know when or if we'll have another chance to play a 2800 level opponent. I'll try to think of some new ideas for matches with "ordinary" GMs.JJJ wrote:Nakamura played well. So, now we know that draw is best thing to expect at these handicap, will you change them Larry ?
What is the currently elo of komodo based only with his game against human ? Can you determine it ?
I plan to calculate Komodo's performance rating at each handicap so far. The question is how much to add to those numbers for the handicap. Based on both my own tests and those of Kai, we have some idea, but it's not very exact. Anyway, I'll post something on this once I have the time.
Naka played very well, congratulations.
The overall performance of Komodo versus Naka is probably somewhere around 3200 ELO. Also, this performance is much more stable than computer ratings. An improvement of 200 ELO points of Komodo on CCRL will be reflected only in some minor increase of the balanced handicap against top human. This "compression" of improvement comes from the dilation of the handicap with the strength, engine improves not much faster than the same handicap increases too in its ELO value with the strength improvement of the engine. Probably Naka can draw at f7 + c7 odds even against the perfect engine, because there are only 1000-1200 more ELO points to Komodo, and the value of the handicap against top human increases too to huge value at this level of engine strength.
Other ways of giving odds seem less spectacular. Fixed depth is meaningless, a 1992 Mephisto is probably not far away from Komodo at fixed depth (in fact I don't know which is stronger). Time odds: not intuitive for most chess players, not clear in its magnitude even to many CC specialists. That Russian GM commentary on twitch was very interesting, at one time he asked something like: "do computers really need time to think? I don't see any difference in the level of playing whether it thinks for 1 second or 1 minute". Time odds would surely not impress him. At 1 second/move Komodo is roughly 2900 or so in human ratings, at 1 minute/move maybe 3100, both beyond the usual human play. Maybe "taking back" is something new to consider, as humans, even top ones, are particularly vulnerable to some "stupid" blunders, which ruin almost all good intentions.
Thanks again for this entertainment!
PS What was the value of the contempt in these games?
-
- Posts: 10948
- Joined: Wed Jul 26, 2006 10:21 pm
- Full name: Kai Laskos
Re: Round 3 fight now
A strong GM would probably lose against 24 core machine at 1 s/move. Better play against a smartphone at 1 s/move. Do you think the chess community would be impressed by such time-odds match? I don't think so.JJJ wrote:If komodo is 2900 elo human rating at 1 sec / move, it could be the challenge I d like to see
2900 is close to the highest elo of Magnus Carslen. Could be interesting to see strong GM ( around 2700 ) play some against it.
-
- Posts: 12038
- Joined: Mon Jul 07, 2008 10:50 pm
Re: Round 3 fight now
so do you think it will be possible in the future for kommodo to give pawn and move odds and beat the world champion under normal game conditions. would 200-300 extra elo be sufficient ?Laskos wrote:Yes, definitely so, although might not be clear in 1-2 games. Komodo can play too in 2 hours, doesn't matter too much.duncan wrote:if nakamura had had normal time controls, while komodo keeps his, do you think he would have done better ?Laskos wrote:Thanks Larry and Mark for this match. By now you are able pick the right handicaps in order to have balanced matches, as the last matches had shown. People are getting pretty good too at predicting the result, 2.5/4 for Komodo was the central value in the poll answers.lkaufman wrote:Well, the handicap obviously depends on the opponent. I don't know when or if we'll have another chance to play a 2800 level opponent. I'll try to think of some new ideas for matches with "ordinary" GMs.JJJ wrote:Nakamura played well. So, now we know that draw is best thing to expect at these handicap, will you change them Larry ?
What is the currently elo of komodo based only with his game against human ? Can you determine it ?
I plan to calculate Komodo's performance rating at each handicap so far. The question is how much to add to those numbers for the handicap. Based on both my own tests and those of Kai, we have some idea, but it's not very exact. Anyway, I'll post something on this once I have the time.
Naka played very well, congratulations.
The overall performance of Komodo versus Naka is probably somewhere around 3200 ELO. Also, this performance is much more stable than computer ratings. An improvement of 200 ELO points of Komodo on CCRL will be reflected only in some minor increase of the balanced handicap against top human. This "compression" of improvement comes from the dilation of the handicap with the strength, engine improves not much faster than the same handicap increases too in its ELO value with the strength improvement of the engine. Probably Naka can draw at f7 + c7 odds even against the perfect engine, because there are only 1000-1200 more ELO points to Komodo, and the value of the handicap against top human increases too to huge value at this level of engine strength.
Other ways of giving odds seem less spectacular. Fixed depth is meaningless, a 1992 Mephisto is probably not far away from Komodo at fixed depth (in fact I don't know which is stronger). Time odds: not intuitive for most chess players, not clear in its magnitude even to many CC specialists. That Russian GM commentary on twitch was very interesting, at one time he asked something like: "do computers really need time to think? I don't see any difference in the level of playing whether it thinks for 1 second or 1 minute". Time odds would surely not impress him. At 1 second/move Komodo is roughly 2900 or so in human ratings, at 1 minute/move maybe 3100, both beyond the usual human play. Maybe "taking back" is something new to consider, as humans, even top ones, are particularly vulnerable to some "stupid" blunders, which ruin almost all good intentions.
Thanks again for this entertainment!
PS What was the value of the contempt in these games?
-
- Posts: 5960
- Joined: Sun Jan 10, 2010 6:15 am
- Location: Maryland USA
Re: Round 3 fight now
40Laskos wrote:Thanks Larry and Mark for this match. By now you are able pick the right handicaps in order to have balanced matches, as the last matches had shown. People are getting pretty good too at predicting the result, 2.5/4 for Komodo was the central value in the poll answers.lkaufman wrote:Well, the handicap obviously depends on the opponent. I don't know when or if we'll have another chance to play a 2800 level opponent. I'll try to think of some new ideas for matches with "ordinary" GMs.JJJ wrote:Nakamura played well. So, now we know that draw is best thing to expect at these handicap, will you change them Larry ?
What is the currently elo of komodo based only with his game against human ? Can you determine it ?
I plan to calculate Komodo's performance rating at each handicap so far. The question is how much to add to those numbers for the handicap. Based on both my own tests and those of Kai, we have some idea, but it's not very exact. Anyway, I'll post something on this once I have the time.
Naka played very well, congratulations.
The overall performance of Komodo versus Naka is probably somewhere around 3200 ELO. Also, this performance is much more stable than computer ratings. An improvement of 200 ELO points of Komodo on CCRL will be reflected only in some minor increase of the balanced handicap against top human. This "compression" of improvement comes from the dilation of the handicap with the strength, engine improves not much faster than the same handicap increases too in its ELO value with the strength improvement of the engine. Probably Naka can draw at f7 + c7 odds even against the perfect engine, because there are only 1000-1200 more ELO points to Komodo, and the value of the handicap against top human increases too to huge value at this level of engine strength.
Other ways of giving odds seem less spectacular. Fixed depth is meaningless, a 1992 Mephisto is probably not far away from Komodo at fixed depth (in fact I don't know which is stronger). Time odds: not intuitive for most chess players, not clear in its magnitude even to many CC specialists. That Russian GM commentary on twitch was very interesting, at one time he asked something like: "do computers really need time to think? I don't see any difference in the level of playing whether it thinks for 1 second or 1 minute". Time odds would surely not impress him. At 1 second/move Komodo is roughly 2900 or so in human ratings, at 1 minute/move maybe 3100, both beyond the usual human play. Maybe "taking back" is something new to consider, as humans, even top ones, are particularly vulnerable to some "stupid" blunders, which ruin almost all good intentions.
Thanks again for this entertainment!
PS What was the value of the contempt in these games?
Komodo rules!
-
- Posts: 5960
- Joined: Sun Jan 10, 2010 6:15 am
- Location: Maryland USA
Re: Round 3 fight now
Well, it's far from clear who would win a match tomorrow at f7 odds between Nakamura and Komodo at 45' + 15", since they only played one game and it was a draw. Carlsen might be a bit tougher, and doubling the time limit would favor him, but I think in a few years the top engine would win such a match, though most of the games would be draws.duncan wrote:so do you think it will be possible in the future for kommodo to give pawn and move odds and beat the world champion under normal game conditions. would 200-300 extra elo be sufficient ?Laskos wrote:Yes, definitely so, although might not be clear in 1-2 games. Komodo can play too in 2 hours, doesn't matter too much.duncan wrote:if nakamura had had normal time controls, while komodo keeps his, do you think he would have done better ?Laskos wrote:Thanks Larry and Mark for this match. By now you are able pick the right handicaps in order to have balanced matches, as the last matches had shown. People are getting pretty good too at predicting the result, 2.5/4 for Komodo was the central value in the poll answers.lkaufman wrote:Well, the handicap obviously depends on the opponent. I don't know when or if we'll have another chance to play a 2800 level opponent. I'll try to think of some new ideas for matches with "ordinary" GMs.JJJ wrote:Nakamura played well. So, now we know that draw is best thing to expect at these handicap, will you change them Larry ?
What is the currently elo of komodo based only with his game against human ? Can you determine it ?
I plan to calculate Komodo's performance rating at each handicap so far. The question is how much to add to those numbers for the handicap. Based on both my own tests and those of Kai, we have some idea, but it's not very exact. Anyway, I'll post something on this once I have the time.
Naka played very well, congratulations.
The overall performance of Komodo versus Naka is probably somewhere around 3200 ELO. Also, this performance is much more stable than computer ratings. An improvement of 200 ELO points of Komodo on CCRL will be reflected only in some minor increase of the balanced handicap against top human. This "compression" of improvement comes from the dilation of the handicap with the strength, engine improves not much faster than the same handicap increases too in its ELO value with the strength improvement of the engine. Probably Naka can draw at f7 + c7 odds even against the perfect engine, because there are only 1000-1200 more ELO points to Komodo, and the value of the handicap against top human increases too to huge value at this level of engine strength.
Other ways of giving odds seem less spectacular. Fixed depth is meaningless, a 1992 Mephisto is probably not far away from Komodo at fixed depth (in fact I don't know which is stronger). Time odds: not intuitive for most chess players, not clear in its magnitude even to many CC specialists. That Russian GM commentary on twitch was very interesting, at one time he asked something like: "do computers really need time to think? I don't see any difference in the level of playing whether it thinks for 1 second or 1 minute". Time odds would surely not impress him. At 1 second/move Komodo is roughly 2900 or so in human ratings, at 1 minute/move maybe 3100, both beyond the usual human play. Maybe "taking back" is something new to consider, as humans, even top ones, are particularly vulnerable to some "stupid" blunders, which ruin almost all good intentions.
Thanks again for this entertainment!
PS What was the value of the contempt in these games?
Komodo rules!
-
- Posts: 10948
- Joined: Wed Jul 26, 2006 10:21 pm
- Full name: Kai Laskos
Re: Round 3 fight now
To beat the champion with say 70-80% probability (to call it consistent beating) at regular TC with f7 odds would require probably more, maybe 400-600 extra CCRL ELO. I am not sure how Komodo stands today on this handicap against a 2800 GM, and it depends also on individual GM too.duncan wrote:
so do you think it will be possible in the future for kommodo to give pawn and move odds and beat the world champion under normal game conditions. would 200-300 extra elo be sufficient ?
-
- Posts: 10948
- Joined: Wed Jul 26, 2006 10:21 pm
- Full name: Kai Laskos
Re: Nakamura vs. Komodo
I simulated the top GM too, the situation is the same, humans would better choose material handicap instead of a positional one for the same nominal value in centipawns of handicap. I can show the results, although it is lengthy:duncan wrote:was the human a super grand-master?Laskos wrote:I remember when simulating these handicaps that the rule of thumb would be that for human against computer "material odds beat positional odds" (for the same nominal value of odds).
The performance of Machine (Komodo LTC) versus Human (Komodo STC) from regular starting positions (openings 2moves_v1.pgn):
Code: Select all
Score of Machine vs Human: 1532 - 29 - 439 [0.876] 2000
ELO difference: 339
Finished match
Positional:
[D]rnbqkbnr/pppppppp/8/8/4PP2/5N2/PPPP2PP/RNBQKB1R w KQkq -
Code: Select all
Engine: Komodo 9.3 64-bit (4096 MB)
by Don Dailey, Larry Kaufman, Mark Lefler
29.01 3:16 +0.92 1.c4 c5 2.d4 cxd4 3.Nxd4 e6 4.Nc3 Nc6
5.Be3 Nh6 6.Be2 Be7 7.O-O O-O 8.Qd3 f6
9.Rad1 b6 10.a3 Nf7 11.b4 Nxd4
12.Bxd4 Bb7 13.Be3 d6 14.Nb5 (804.547.930) 4090
30.01 3:42 +0.90 1.c4 e6 2.Nc3 c5 3.d4 cxd4 4.Nxd4 Nc6
5.Be3 Nh6 6.Be2 Be7 7.O-O O-O 8.Qd3 f6
9.Rad1 b6 10.a3 Nf7 11.b4 Nxd4
12.Bxd4 Bb7 13.Be3 d6 14.Rd2 (909.886.536) 4093
31.01 5:37 +0.97++ 1.c4 e6 2.Nc3 (1.380.858.073) 4092
31.01 6:01 +0.90-- 1.c4 e6 (1.477.978.172) 4088
31.01 6:31 +0.91 1.c4 e6 2.Nc3 c5 3.d4 cxd4 4.Nxd4 Nc6
5.Be3 Nh6 6.Be2 Be7 7.O-O O-O 8.Qd3 f6
9.a3 b6 10.Rad1 Nf7 11.b4 Nxd4
12.Bxd4 Bb7 13.Be3 d6 14.Nb5 (1.603.084.173) 4092
32.01 8:06 +0.98++ 1.c4 e6 2.Nc3 (1.991.904.940) 4092
[D]rnbqkbnr/p1pppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq -
Code: Select all
Engine: Komodo 9.3 64-bit (4096 MB)
by Don Dailey, Larry Kaufman, Mark Lefler
28.01 3:16 +1.01 1.Nf3 Nf6 2.d4 Ba6 3.Nbd2 e6 4.c4 Bb7
5.e3 c5 6.Bd3 d6 7.O-O Nbd7 8.Qc2 h6
9.b3 Be7 10.Bb2 O-O 11.h3 Rb8 12.Ne4 Qc7
13.Nc3 a6 14.Qd2 (756.606.512) 3844
29.01 4:03 +0.99 1.Nf3 Nf6 2.d4 Ba6 3.Nbd2 e6 4.c4 Bb7
5.e3 c5 6.Bd3 Be7 7.O-O O-O 8.b3 d6
9.h3 Nbd7 10.Qc2 h6 11.Bb2 a5 12.Bc3 Qc7
13.Rfe1 Rab8 14.Ne4 (938.374.833) 3856
30.01 5:37 +0.96 1.Nf3 Nf6 2.d4 Ba6 3.Nbd2 e6 4.c4 Bb7
5.e3 Be7 6.Bd3 c5 7.O-O O-O 8.b3 d6
9.h3 a5 10.Bb2 Nbd7 11.Qc2 h6 12.Ne4 Nxe4
13.Bxe4 Bxe4 14.Qxe4 (1.305.455.827) 3870
30.02 8:17 +1.01 1.d4 Nf6 2.Nf3 Ba6 3.Nbd2 e6 4.c4 Bb7
5.e3 Be7 6.Bd3 c5 7.O-O O-O 8.b3 d6
9.Bb2 Nbd7 10.h3 h6 11.Qc2 Qc7
12.Ne4 Nxe4 13.Bxe4 Bxe4 14.Qxe4 (1.952.013.201) 3924
31.01 11:38 +0.95 1.d4 Nf6 2.Nf3 e6 3.c4 Bb7 4.a3 Be7
5.Nc3 Ne4 6.Nxe4 Bxe4 7.e3 O-O 8.Bd3 d5
9.O-O Nd7 10.b4 c6 11.c5 h6 12.a4 a6
13.Bb2 Qb8 14.Bc3 (2.791.841.185) 3995
Performance of Human versus Machine positional odds:
Code: Select all
Program Score % Av.Op. Elo + - Draws
1 Machine : 1484.5/2000 74.2 -92 92 13 13 34.8 %
2 Human : 515.5/2000 25.8 92 -92 13 13 34.8 %
Performance of Human versus Machine material odds:
Code: Select all
Program Score % Av.Op. Elo + - Draws
1 Machine : 1319.5/2000 66.0 -58 58 12 11 43.9 %
2 Human : 680.5/2000 34.0 58 -58 11 12 43.9 %
For the same nominal value (centipawns), human gains about 40% more from material odds than from positional odds. Therefore the handicap in game 4 was 70cp (instead of apparent 100cp by the eval) in terms of material, the smallest of the 4 games Naka played. I don't think he lost accidentally the fourth game, in fact I predicted that he will have problems in this game.