Nakamura vs. Komodo

[lucasart]

Game 4
[pgn][Event "Live Chess"]
[Site "Chess.com"]
[Date "2016.01.07"]
[White "Hikaru"]
[Black "KomodoChess"]
[Result "0-1"]
[WhiteElo "2787"]
[BlackElo "3368"]
[TimeControl "45|15"]
[SetUp "1"]
[FEN "rnbqkbnr/pppppppp/8/8/3PP3/5N2/PPP2PPP/RNBQKB1R w KQkq - 0 1"]
[Termination "KomodoChess won by resignation"]

1.c4 d6 2.Nc3 g6 3.Qb3 c6 4.Be2 Qb6 5.Qc2 Bg7 6.O-O Nf6 7.h3 O-O 8.Be3 Qc7 9.Rac1 Nbd7 10.Rfd1 b6 11.Qd2 Bb7 12.b3 Rad8 13.Bh6 e5 14.Bxg7 Kxg7 15.d5 a5 16.Qe3 Ra8 17.Ne1 Nc5 18.Nd3 Qe7 19.Nb2 Nfd7 20.Nca4 f5 21.Nxc5 bxc5 22.f3 Nf6 23.g4 Rf7 24.Rf1 Raf8 25.Kh2 fxg4 26.fxg4 h5 27.gxh5 Nxh5 28.Rxf7+ Qxf7 29.Rf1 Nf4 30.Bg4 cxd5 31.exd5 Qc7 32.Qg3 Rf6 33.Rf2 Bc8 34.Nd1 Bxg4 35.Qxg4 Qf7 36.Qh4 Rf5 37.Ne3 Rh5 38.Qg4 Qe7 39.Rf3 Rg5 40.Qc8 Ne2 41.Qe6 Qxe6 42.dxe6 Nf4 43.Rg3 Rh5 44.Nd5 Nxe6 45.Nc3 Nd4 46.Ne4 Nf5 47.Rd3 Rh4 48.Nxd6 Rf4 49.Nxf5+ gxf5 50.Rd5 e4 51.Rxc5 Kf6 52.Rc8 Rf2+ 53.Kg3 Rf3+ 54.Kg2 Ke5 55.h4 Kf4 56.h5 e3 57.h6 Rg3+ 58.Kh2 Rg6 {Nakamura resigns} 0-1
[/pgn]

It's interesting to note that:
* humans are totally incapable of capitalizing on dynamic advantages, compared to computers. out of the 4 move advantage, Naka got absolutly nothing out of the opening, and Komodo easily equalized in a few moves.
* it is in the endgame that Naka got destroyed. yet, we hear so much crap about how humans are supposedly superior to computers in the endgame...

[APassionForCriminalJustic]

Game 4
[pgn][Event "Live Chess"]
[Site "Chess.com"]
[Date "2016.01.07"]
[White "Hikaru"]
[Black "KomodoChess"]
[Result "0-1"]
[WhiteElo "2787"]
[BlackElo "3368"]
[TimeControl "45|15"]
[SetUp "1"]
[FEN "rnbqkbnr/pppppppp/8/8/3PP3/5N2/PPP2PPP/RNBQKB1R w KQkq - 0 1"]
[Termination "KomodoChess won by resignation"]

1.c4 d6 2.Nc3 g6 3.Qb3 c6 4.Be2 Qb6 5.Qc2 Bg7 6.O-O Nf6 7.h3 O-O 8.Be3 Qc7 9.Rac1 Nbd7 10.Rfd1 b6 11.Qd2 Bb7 12.b3 Rad8 13.Bh6 e5 14.Bxg7 Kxg7 15.d5 a5 16.Qe3 Ra8 17.Ne1 Nc5 18.Nd3 Qe7 19.Nb2 Nfd7 20.Nca4 f5 21.Nxc5 bxc5 22.f3 Nf6 23.g4 Rf7 24.Rf1 Raf8 25.Kh2 fxg4 26.fxg4 h5 27.gxh5 Nxh5 28.Rxf7+ Qxf7 29.Rf1 Nf4 30.Bg4 cxd5 31.exd5 Qc7 32.Qg3 Rf6 33.Rf2 Bc8 34.Nd1 Bxg4 35.Qxg4 Qf7 36.Qh4 Rf5 37.Ne3 Rh5 38.Qg4 Qe7 39.Rf3 Rg5 40.Qc8 Ne2 41.Qe6 Qxe6 42.dxe6 Nf4 43.Rg3 Rh5 44.Nd5 Nxe6 45.Nc3 Nd4 46.Ne4 Nf5 47.Rd3 Rh4 48.Nxd6 Rf4 49.Nxf5+ gxf5 50.Rd5 e4 51.Rxc5 Kf6 52.Rc8 Rf2+ 53.Kg3 Rf3+ 54.Kg2 Ke5 55.h4 Kf4 56.h5 e3 57.h6 Rg3+ 58.Kh2 Rg6 {Nakamura resigns} 0-1
[/pgn]

It's interesting to note that:
* humans are totally incapable of capitalizing on dynamic advantages, compared to computers. out of the 4 move advantage, Naka got absolutly nothing out of the opening, and Komodo easily equalized in a few moves.
* it is in the endgame that Naka got destroyed. yet, we hear so much crap about how humans are supposedly superior to computers in the endgame...

I've heard that same crap from some of the noobs out there who know absolutely nothing about how strong computers really are. Engines today, particularly Stockfish and Komodo, play the endgame in ways in which a human being could not even dream of. The complicated endgame positions that I've seen Komodo win on my 28 core are out of this world. Literally the endgames are so complex that one or two slightly inferior moves made by the winning side could then find themselves losing or drawing that same position. Engines simply see far too deep and play far too accurately that a player like naka is merely an ant on Komodo's shirt.

From what I continue to see over and over again is Komodo's extraordinary ability to win positions in some of the most subtle, yet complex ways imaginable.

That fourth and final game was great. Look at what happened when Naka had to play the computer without any sort of material advantage. He got spanked. These handicap matches give some of these masters false hope in my opinion. In the end, the truth still stands. Real human versus machine matches are over.

[M ANSARI]

I think today there are very few areas where a top engine is weak. Maybe you can still find some position where the horizon effect will make the engine play poorly ... but that horizon is a lot further today than it was a few years ago. In a real game such positions would probably be very rare as a classical game against a human would be long over before they ever reached such positions. Engines used to be weak in openings ... but that is no longer the case and many of the top GM's actually use engines to try and get an opening advantage. 4 moves and white is a pretty big handicap when you consider that most GM's consider having white alone is already a big advantage and that black starts off the game trying to equalize.

Would be interesting to see how Komodo would do against SF7 if it is given that kind of advantage (or vice versa).

[Laskos]

Would be interesting to see how Komodo would do against SF7 if it is given that kind of advantage (or vice versa).

I ran some quick tests using fast games at depth=11 for both Komodo and Stockfish. It just happens that in these conditions on 4 threads the engines are very close in strength playing from normal opening positions. Matches of 1,000 games from the following opening position:
[D]rnbqkbnr/pppppppp/8/8/4PP2/5N2/PPPP2PP/RNBQKB1R w KQkq -

Komodo overperforms as White against Stockfish by 166 ELO points.
Stockfish overperforms as White against Komodo by 110 ELO points.

Komodo seems to like more White here (positional advantage) than Stockfish, by a solid 50%. It can be said that Komodo uses better the positional advantage here. At some not that fast games, the value of this White opening position is probably higher than 166 ELO points.

[Uri Blass]

so do you think it will be possible in the future for kommodo to give pawn and move odds and beat the world champion under normal game conditions. would 200-300 extra elo be sufficient ?

To beat the champion with say 70-80% probability (to call it consistent beating) at regular TC with f7 odds would require probably more, maybe 400-600 extra CCRL ELO. I am not sure how Komodo stands today on this handicap against a 2800 GM, and it depends also on individual GM too.

I am not sure if it is possible to beat the best players with f7 odd assuming they prepare by constructing a special opening book with 100,000 lines or more than it and memorize the book.

Assuming the human prepare then I think that the weight of the ability to memorize many lines is relatively more important than normal chess.

[Vinvin]

Just curious : how much money did Ikaru win for this 1.5/4 ?

[thekingman]

Amazing and long awaited match, thanks so much for organizing this, Larry!

I, like many others, now find myself wondering "What next?" There are a few different possible directions to move from here. In general, I think Komodo has more or less conquered all the most interesting material handicaps, and these matches should move in a different direction. Here are a few of the possibilities as I see them, and a few thoughts on each:

1. A match entirely of f7 and move. Komodo has struggled to create chances in several of its recent handicap games at these odds, but has never lost and has won several. So I think it's fair to say Komodo has a reasonable handle on this handicap.

2. A larger material handicap, such as two pawns or knight for pawn. I am not a huge fan of these matches, as they do not much resemble real chess. The human simply needs to trade and not make any gross errors, and the endgame will be an easy win. Not to mention that many people do not recognize the magnitude of the difference between one pawn and two pawns, and would not be excited by this.

3. Initiative odds, such as the 4 moves given to Nakamura, or with black Kd5, 1..f6 2...Kf7, etc. These represent more dynamic handicaps, where the human, although starting at a big advantage, needs to accurately attack and cannot just simplify. It highlights both the attacking skills of the human and the defensive skills of Komodo, and any information gleaned in improving defense against strong attacks like this seems to be quite generally useful for Komodo. This is my favorite type of match played so far, and I would love to see more of it going forward.

4. Draw odds. A draw counts as a win for the human. A chance for Komodo to display and improve attacking skills against weaker opponents. A resourceful opponent may find ways to highlight any weaknesses (understanding/opening blocked positions, etc), which could again be generally useful for development. The games would greatly resemble normal chess, which would make it more enjoyable for spectators. Balance would be important, of course - I would start with giving the human the white pieces in every game, letting them consult any and all databases (including whatever opening ones they want, and endgame tablebases), and no opening book for the computer. I would LOVE to see this played, and I imagine many others would as well.

Edit: GM Kaufman surely needs no reminding of this, as he was involved, but Rybka won a match at draw odds 6-2 against GM Joel Benjamin in 2008 (http://chessok.com/?p=21973). Maybe with a stronger human opponent and database access it could still be balanced, but I suspect the human needs something more, like black forfeiting castling rights.

[lkaufman]

Just curious : how much money did Ikaru win for this 1.5/4 ?

Since chess.com is involved, I'm not at liberty to say what his base fee was, but he got a bonus of $200 per draw and $600 per win, so $600 bonus.

[duncan]

Just curious : how much money did Ikaru win for this 1.5/4 ?

Since chess.com is involved, I'm not at liberty to say what his base fee was, but he got a bonus of $200 per draw and $600 per win, so $600 bonus.

your coffers maybe somewhat depleted . super grandmasters are not cheap.
If I wanted to help you out and challenge komodo. if I win I get nothing, If komodo wins it gets $500.

my problem is I am weak at chess 1300-1350 elo. so for the odds to be 50-50 for komodo to win, it will need a certain handicap. what would it take to be an even match. komodo plays game in an attosecond. ?

[lkaufman]

Amazing and long awaited match, thanks so much for organizing this, Larry!

I, like many others, now find myself wondering "What next?" There are a few different possible directions to move from here. In general, I think Komodo has more or less conquered all the most interesting material handicaps, and these matches should move in a different direction. Here are a few of the possibilities as I see them, and a few thoughts on each:

1. A match entirely of f7 and move. Komodo has struggled to create chances in several of its recent handicap games at these odds, but has never lost and has won several. So I think it's fair to say Komodo has a reasonable handle on this handicap.

2. A larger material handicap, such as two pawns or knight for pawn. I am not a huge fan of these matches, as they do not much resemble real chess. The human simply needs to trade and not make any gross errors, and the endgame will be an easy win. Not to mention that many people do not recognize the magnitude of the difference between one pawn and two pawns, and would not be excited by this.

3. Initiative odds, such as the 4 moves given to Nakamura, or with black Kd5, 1..f6 2...Kf7, etc. These represent more dynamic handicaps, where the human, although starting at a big advantage, needs to accurately attack and cannot just simplify. It highlights both the attacking skills of the human and the defensive skills of Komodo, and any information gleaned in improving defense against strong attacks like this seems to be quite generally useful for Komodo. This is my favorite type of match played so far, and I would love to see more of it going forward.

4. Draw odds. A draw counts as a win for the human. A chance for Komodo to display and improve attacking skills against weaker opponents. A resourceful opponent may find ways to highlight any weaknesses (understanding/opening blocked positions, etc), which could again be generally useful for development. The games would greatly resemble normal chess, which would make it more enjoyable for spectators. Balance would be important, of course - I would start with giving the human the white pieces in every game, letting them consult any and all databases (including whatever opening ones they want, and endgame tablebases), and no opening book for the computer. I would LOVE to see this played, and I imagine many others would as well.

Edit: GM Kaufman surely needs no reminding of this, as he was involved, but Rybka won a match at draw odds 6-2 against GM Joel Benjamin in 2008 (http://chessok.com/?p=21973). Maybe with a stronger human opponent and database access it could still be balanced, but I suspect the human needs something more, like black forfeiting castling rights.

Actually I too would like to see such a match (draw odds plus White odds plus no (or just two or three move for variety) book for Komodo, and I like your idea of database access for the human as well. But forfeiting castling rights throws the humans' book advantage out the window, as Black will play strangely right from the start. The practical problem is that we would need a 2800 level player to have any chance, and even if Nakamura wanted to try this he is probably a poor choice for draw odds, since playing for a draw is hardly his thing. If we can only get a 2600-2700 GM, we would have to throw in huge time odds and no ponder, or perhaps run on a cellphone. Without time odds the math says that even Carlsen would have no chance, but that may not be true. It may be that Komodo as Black will simply duplicate some known equalizing line from which it is too simple to make a draw. We could avoid this by for example making Komodo open with 1...a6 or 1...h6 or such, but perhaps White can use his advantage to create a position from which no progress can be made. We just don't know. Probably at some point we'll do such a match or matches, especially if chess.com or some other sponsor backs the idea.