Next GM vs. Komodo handicap match

[APassionForCriminalJustic]

The next in our series of handicap matches between Komodo and Grandmasters is set for Tuesday March 22 and Wednesday March 23 on chess.com. Games at 11 a.m. and 5 p.m. Eastern Daylight Time each day. The opponent is three-time U.S. Champion Joel Benjamin, FIDE 2544, USCF 2627, born in 1964. He has played two matches with Rybka nearly a decade ago and was part of the IBM Deep Blue team that beat Kasparov two decades ago, so he is quite familiar with playing against engines. He will be commenting on the games live in the same manner as Robert Hess and Simon Williams did. Time limit will be raised to 45' and 30" to allow for time spent commenting and for operator time on my end.
Handicaps will be as follows: One game at pawn (f7) and two moves, one game at rook for knight (a8 for b1, R moved to replace knight on b1) with Benjamin getting White. So far these were used in the Hess and Mikhalevski matches. One game at five move handicap (within first four ranks), one more than given to Nakamura but one less than given to FM Larry Gilden. The fourth game will be at queen (d1) for two bishop handicap, the first time we will play without a queen in an official game! I think Joel really ought to win this one; I think it falls somewhere in between pawn and two moves and knight odds, but I don't think it will be a trivial win for him. the actual order of the four games hasn't been decided yet. Komodo will be a version very close to 9.4, running on my 24 core machine.

Just a reminder the first game is supposed to start around 11:00 am ET. The cover the match with live video here:

chess.com/tv

Enjoy!

Benjamin got absolutely destroyed in that first game...

[simon]

[pgn]
[Event "Benjamin vs Komodo odds match"]
[Site "Chess.com"]
[Date "2016.03.22"]
[Round "1"]
[White "Benjamin, Joel"]
[Black "Komodo"]
[Result "0-1"]
[EventDate "2016.03.22"]
[FEN "rnbqkbnr/pppppppp/8/8/3PPP2/2N5/PPP3PP/R1BQKBNR w KQkq - 0 1"]
[SetUp "1"]
[Termination "Komodo won by resignation"]
[TimeControl "2700+30"]
[Variant "Odds Chess"]
[WhiteElo "2544"]

1.Nf3 { [%clk 0:45:27] } 1...e6 { [%clk 0:45:04] } 2.Bd3 { [%clk 0:45:43] } 2...Be7 { [%clk 0:43:57] } 3.O-O { [%clk 0:46:01] } 3...d5 { [%clk 0:42:58] } 4.Qe1 { [%clk 0:44:22] } 4...dxe4 { [%clk 0:42:04] } 5.Nxe4 { [%clk 0:44:46] } 5...Nf6 { [%clk 0:41:39] } 6.Kh1 { [%clk 0:43:32] } 6...Nxe4 { [%clk 0:39:55] } 7.Bxe4 { [%clk 0:43:00] } 7...O-O { [%clk 0:38:43] } 8.Bd2 { [%clk 0:41:39] } 8...Nd7 { [%clk 0:38:27] } 9.c4 { [%clk 0:41:39] } 9...Nf6 { [%clk 0:37:40] } 10.Bc2 { [%clk 0:41:58] } 10...b5 { [%clk 0:36:23] } 11.cxb5 { [%clk 0:40:23] } 11...Bb7 { [%clk 0:36:30] } 12.Bb4 { [%clk 0:35:54] } 12...Bxb4 { [%clk 0:35:45] } 13.Qxb4 { [%clk 0:36:20] } 13...g6 { [%clk 0:33:59] } 14.Rac1 { [%clk 0:34:39] } 14...a6 { [%clk 0:32:42] } 15.bxa6 { [%clk 0:34:08] } 15...Bxa6 { [%clk 0:33:03] } 16.Rfe1 { [%clk 0:34:02] } 16...Qd6 { [%clk 0:32:19] } 17.Qxd6 { [%clk 0:30:53] } 17...cxd6 { [%clk 0:32:41] } 18.Bb3 { [%clk 0:30:21] } 18...Bb7 { [%clk 0:32:05] } 19.Nd2 { [%clk 0:24:47] } 19...Bd5 { [%clk 0:31:01] } 20.Nc4 { [%clk 0:23:44] } 20...Ra6 { [%clk 0:30:39] } 21.Kg1 { [%clk 0:23:54] } 21...Kg7 { [%clk 0:30:21] } 22.g3 { [%clk 0:22:39] } 22...Rb8 { [%clk 0:28:23] } 23.Rc3 { [%clk 0:21:18] } 23...Ne4 { [%clk 0:28:28] } 24.Rce3 { [%clk 0:19:46] } 24...Rc8 { [%clk 0:26:59] } 25.Na3 { [%clk 0:18:49] } 25...Bxb3 { [%clk 0:27:17] } 26.Rxb3 { [%clk 0:18:49] } 26...d5 { [%clk 0:26:57] } 27.Kg2 { [%clk 0:17:56] } 27...Ra4 { [%clk 0:25:52] } 28.Rd1 { [%clk 0:17:59] } 28...h5 { [%clk 0:25:47] } 29.Rb6 { [%clk 0:17:18] } 29...h4 { [%clk 0:25:29] } 30.Rb3 { [%clk 0:14:11] } 30...f5 { [%clk 0:24:59] } 31.Kf3 { [%clk 0:13:58] } 31...Kf6 { [%clk 0:23:31] } 32.Rbd3 { [%clk 0:13:13] } 32...Rb8 { [%clk 0:21:34] } 33.Rb3 { [%clk 0:13:23] } 33...Rh8 { [%clk 0:21:21] } 34.Kg2 { [%clk 0:11:24] } 34...Ra7 { [%clk 0:20:38] } 35.gxh4 { [%clk 0:11:21] } 35...Rxh4 { [%clk 0:20:24] } 36.Rf3 { [%clk 0:11:38] } 36...g5 { [%clk 0:19:47] } 37.fxg5+ { [%clk 0:11:46] } 37...Nxg5 { [%clk 0:19:51] } 38.Rfd3 { [%clk 0:11:12] } 38...Rg7 { [%clk 0:19:05] } 39.Rg3 { [%clk 0:06:53] } 39...Rgh7 { [%clk 0:19:02] } 40.h3 { [%clk 0:03:22] } 40...Nxh3 { [%clk 0:18:20] } 41.Kf1 { [%clk 0:01:56] } 41...f4 { [%clk 0:14:53] } 42.Rg8 { [%clk 0:01:47] } 42...Ng5 { [%clk 0:14:20] } 43.Ke1 { [%clk 0:01:17] } 43...Ne4 { [%clk 0:14:28] } 44.Rg1 { [%clk 0:00:56] } 44...f3 { [%clk 0:14:40] } 0-1
[/pgn]

[whereagles]

Well, that was pretty impressive... humans clearly struggle to find the best continuation against underdeveloped engines.

From what I see, a slow, consolidating strategy does nothing but slowly throwing away the initiative. I wonder if a frontal assault would do better

[simon]

[pgn]
[Event "Benjamin vs Komodo odds match"]
[Site "Chess.com"]
[Date "2016.03.22"]
[Round "2"]
[White "Benjamin, Joel"]
[Black "Komodo"]
[Result "1/2-1/2"]
[EventDate "2016.03.22"]
[FEN "1nbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/1RBQKBNR w Kk - 0 1"]
[SetUp "1"]
[Termination "Game drawn by insufficient material"]
[TimeControl "2700+30"]
[Variant "Odds Chess"]
[WhiteElo "2544"]

1.Nf3 { [%clk 0:45:26] } 1...Nf6 { [%clk 0:45:18] } 2.g3 { [%clk 0:45:38] } 2...e6 { [%clk 0:43:24] } 3.Bg2 { [%clk 0:45:59] } 3...c5 { [%clk 0:39:05] } 4.c4 { [%clk 0:43:29] } 4...Be7 { [%clk 0:37:53] } 5.O-O { [%clk 0:43:48] } 5...Qc7 { [%clk 0:35:45] } 6.b3 { [%clk 0:42:33] } 6...d6 { [%clk 0:34:16] } 7.d4 { [%clk 0:42:43] } 7...cxd4 { [%clk 0:33:38] } 8.Nxd4 { [%clk 0:43:02] } 8...a6 { [%clk 0:32:44] } 9.Qd2 { [%clk 0:42:26] } 9...h6 { [%clk 0:30:58] } 10.Bb2 { [%clk 0:41:33] } 10...O-O { [%clk 0:31:03] } 11.Rfd1 { [%clk 0:40:04] } 11...b6 { [%clk 0:30:09] } 12.Qe3 { [%clk 0:34:12] } 12...Bb7 { [%clk 0:28:25] } 13.Bxb7 { [%clk 0:34:24] } 13...Qxb7 { [%clk 0:28:46] } 14.Qf3 { [%clk 0:34:49] } 14...Qc7 { [%clk 0:27:45] } 15.Rbc1 { [%clk 0:34:14] } 15...Re8 { [%clk 0:25:53] } 16.Nc2 { [%clk 0:33:15] } 16...Nfd7 { [%clk 0:25:10] } 17.Ne3 { [%clk 0:31:40] } 17...Bg5 { [%clk 0:23:39] } 18.h4 { [%clk 0:31:06] } 18...Be7 { [%clk 0:23:28] } 19.Ng4 { [%clk 0:30:12] } 19...Bf8 { [%clk 0:23:17] } 20.Rd2 { [%clk 0:27:50] } 20...Nc6 { [%clk 0:19:19] } 21.Rcd1 { [%clk 0:27:25] } 21...Rc8 { [%clk 0:18:57] } 22.Qf4 { [%clk 0:23:07] } 22...Nc5 { [%clk 0:18:08] } 23.Rxd6 { [%clk 0:21:39] } 23...Bxd6 { [%clk 0:17:03] } 24.Qxd6 { [%clk 0:20:37] } 24...Qxd6 { [%clk 0:17:23] } 25.Rxd6 { [%clk 0:21:04] } 25...b5 { [%clk 0:15:32] } 26.Ne5 { [%clk 0:19:25] } 26...Nxe5 { [%clk 0:15:09] } 27.Bxe5 { [%clk 0:19:51] } 27...bxc4 { [%clk 0:14:58] } 28.bxc4 { [%clk 0:19:58] } 28...f6 { [%clk 0:14:02] } 29.Bd4 { [%clk 0:20:10] } 29...a5 { [%clk 0:14:06] } 30.Bxc5 { [%clk 0:17:49] } 30...Rxc5 { [%clk 0:14:11] } 31.Rxe6 { [%clk 0:18:17] } 31...Rxc4 { [%clk 0:13:58] } 32.Re3 { [%clk 0:18:27] } 32...Kf7 { [%clk 0:12:49] } 33.h5 { [%clk 0:13:44] } 33...Rc5 { [%clk 0:12:44] } 34.g4 { [%clk 0:13:45] } 34...f5 { [%clk 0:13:05] } 35.Rf3 { [%clk 0:12:54] } 35...Ke6 { [%clk 0:13:26] } 36.gxf5+ { [%clk 0:11:25] } 36...Rxf5 { [%clk 0:13:45] } 37.Rh3 { [%clk 0:11:21] } 37...Kf6 { [%clk 0:13:49] } 38.Kg2 { [%clk 0:10:57] } 38...Rg5+ { [%clk 0:14:09] } 39.Kf3 { [%clk 0:10:32] } 39...Rc5 { [%clk 0:14:12] } 40.e3 { [%clk 0:08:46] } 40...Rc4 { [%clk 0:13:59] } 41.Rh1 { [%clk 0:08:13] } 41...Ra4 { [%clk 0:13:47] } 42.Rb1 { [%clk 0:08:17] } 42...Rxa2 { [%clk 0:13:39] } 43.Rb6+ { [%clk 0:08:42] } 43...Kg5 { [%clk 0:13:42] } 44.Rg6+ { [%clk 0:08:54] } 44...Kxh5 { [%clk 0:14:02] } 45.Rxg7 { [%clk 0:09:22] } 45...a4 { [%clk 0:14:10] } 46.Kg3 { [%clk 0:09:21] } 46...Ra1 { [%clk 0:14:17] } 47.Ra7 { [%clk 0:09:39] } 47...Kg6 { [%clk 0:13:43] } 48.Ra6+ { [%clk 0:09:40] } 48...Kf5 { [%clk 0:13:28] } 49.Rxh6 { [%clk 0:09:11] } 49...Ke4 { [%clk 0:13:44] } 50.Re6+ { [%clk 0:07:44] } 50...Kd3 { [%clk 0:13:38] } 51.Ra6 { [%clk 0:08:06] } 51...a3 { [%clk 0:13:52] } 52.Kf4 { [%clk 0:08:14] } 52...Kc3 { [%clk 0:14:05] } 53.e4 { [%clk 0:08:17] } 53...Kb2 { [%clk 0:14:26] } 54.e5 { [%clk 0:07:23] } 54...Rf1 { [%clk 0:14:03] } 55.e6 { [%clk 0:07:10] } 55...a2 { [%clk 0:13:27] } 56.Ke5 { [%clk 0:07:10] } 56...a1=Q { [%clk 0:12:39] } 57.Rxa1 { [%clk 0:07:36] } 57...Rxa1 { [%clk 0:12:55] } 58.e7 { [%clk 0:06:12] } 58...Kc3 { [%clk 0:13:11] } 59.Kf6 { [%clk 0:05:35] } 59...Re1 { [%clk 0:13:25] } 60.Kf7 { [%clk 0:05:54] } 60...Kd3 { [%clk 0:13:44] } 61.f4 { [%clk 0:06:05] } 61...Rxe7+ { [%clk 0:14:00] } 62.Kxe7 { [%clk 0:06:28] } 62...Ke4 { [%clk 0:14:18] } 63.Ke6 { [%clk 0:06:57] } 63...Kxf4 { [%clk 0:14:40] } 1/2-1/2
[/pgn]

[whereagles]

Now, that's better. Joel had some good chances.

[simon]

[pgn]
[Event "Benjamin vs Komodo odds match"]
[Site "Chess.com"]
[Date "2016.03.23"]
[Round "3"]
[White "Benjamin, Joel"]
[Black "Komodo"]
[Result "1/2-1/2"]
[EventDate "2016.03.22"]
[FEN "rnbqkbnr/ppppp1pp/8/8/3PP3/8/PPP2PPP/RNBQKBNR b KQkq - 0 1"]
[SetUp "1"]
[Termination "Game drawn by agreement"]
[TimeControl "2700+30"]
[Variant "Odds Chess"]
[WhiteElo "2544"]

1...e6 { [%clk 0:45:27] } 2.Bd3 { [%clk 0:45:17] } 2...Nc6 { [%clk 0:45:42] } 3.Nf3 { [%clk 0:44:38] } 3...d5 { [%clk 0:44:46] } 4.e5 { [%clk 0:40:49] } 4...Nb4 { [%clk 0:43:11] } 5.Ng5 { [%clk 0:39:39] } 5...Nxd3+ { [%clk 0:43:27] } 6.Qxd3 { [%clk 0:40:01] } 6...Ne7 { [%clk 0:41:16] } 7.Nxh7 { [%clk 0:39:12] } 7...Nf5 { [%clk 0:40:58] } 8.Nxf8 { [%clk 0:38:54] } 8...Rxf8 { [%clk 0:39:50] } 9.h4 { [%clk 0:27:10] } 9...c5 { [%clk 0:39:35] } 10.dxc5 { [%clk 0:27:01] } 10...b6 { [%clk 0:39:46] } 11.c6 { [%clk 0:20:29] } 11...Qc7 { [%clk 0:37:59] } 12.Nc3 { [%clk 0:20:20] } 12...Qxe5+ { [%clk 0:37:01] } 13.Qe2 { [%clk 0:20:24] } 13...Qxe2+ { [%clk 0:36:25] } 14.Nxe2 { [%clk 0:20:41] } 14...Ba6 { [%clk 0:35:34] } 15.c3 { [%clk 0:20:29] } 15...Rc8 { [%clk 0:35:04] } 16.g4 { [%clk 0:19:39] } 16...Ne7 { [%clk 0:34:38] } 17.Nd4 { [%clk 0:19:39] } 17...Nxc6 { [%clk 0:34:49] } 18.Nxe6 { [%clk 0:19:34] } 18...Rf7 { [%clk 0:34:56] } 19.Be3 { [%clk 0:18:06] } 19...Re7 { [%clk 0:34:17] } 20.Nd4 { [%clk 0:16:15] } 20...Nxd4 { [%clk 0:34:29] } 21.cxd4 { [%clk 0:16:42] } 21...Rc2 { [%clk 0:34:14] } 22.b3 { [%clk 0:11:09] } 22...Be2 { [%clk 0:33:56] } 23.Rg1 { [%clk 0:09:49] } 23...Re4 { [%clk 0:34:07] } 24.Rg3 { [%clk 0:08:40] } 24...Bxg4 { [%clk 0:32:50] } 25.Kf1 { [%clk 0:08:57] } 25...g6 { [%clk 0:31:36] } 26.Re1 { [%clk 0:07:46] } 26...Bf5 { [%clk 0:31:49] } 27.Re2 { [%clk 0:06:32] } 27...Rxh4 { [%clk 0:32:10] } 28.Rxc2 { [%clk 0:06:39] } 28...Bxc2 { [%clk 0:32:16] } 29.Rg1 { [%clk 0:06:30] } 29...Bb1 { [%clk 0:31:40] } 30.a3 { [%clk 0:06:41] } 30...Bc2 { [%clk 0:29:52] } 31.b4 { [%clk 0:06:54] } 31...Bd3+ { [%clk 0:28:42] } 32.Ke1 { [%clk 0:07:20] } 32...Kf7 { [%clk 0:28:08] } 33.Kd2 { [%clk 0:07:46] } 33...Bc4 { [%clk 0:25:57] } 34.Kc3 { [%clk 0:08:01] } 34...Bb5 { [%clk 0:24:18] } 35.Kb3 { [%clk 0:07:52] } 35...Bc4+ { [%clk 0:22:01] } 36.Kc3 { [%clk 0:08:17] } 36...b5 { [%clk 0:11:10] } 37.Kd2 { [%clk 0:07:31] } 37...Rh3 { [%clk 0:10:05] } 38.Kc3 { [%clk 0:07:25] } 38...Kf6 { [%clk 0:09:19] } 39.Kd2 { [%clk 0:06:47] } 39...Kg7 { [%clk 0:09:06] } 40.Kc3 { [%clk 0:07:00] } 40...Kf7 { [%clk 0:08:47] } 41.Kc2 { [%clk 0:06:51] } 41...Rf3 { [%clk 0:07:38] } 42.Rg3 { [%clk 0:06:04] } 42...Rxg3 { [%clk 0:07:11] } 43.fxg3 { [%clk 0:06:30] } 43...Kf6 { [%clk 0:07:22] } 44.Bf4 { [%clk 0:06:49] } 44...Kf5 { [%clk 0:07:05] } 45.Bb8 { [%clk 0:07:02] } 45...a6 { [%clk 0:06:50] } 46.Kd2 { [%clk 0:07:09] } 46...Ke4 { [%clk 0:06:56] } 47.Be5 { [%clk 0:07:33] } 1/2-1/2
[/pgn]

[APassionForCriminalJustic]

Now, that's better. Joel had some good chances.

From a grandmaster's perspective, having any of these handicaps is a rather large chance\winning opportunity in and of itself. These matches are pretty much a testament that Komodo is simply far too strong for mere human players of any level.

[simon]

[pgn]
[Event "Benjamin vs Komodo odds match"]
[Site "Chess.com"]
[Date "2016.03.23"]
[Round "4"]
[White "Komodo"]
[Black "Benjamin, Joel"]
[Result "1/2-1/2"]
[BlackElo "2544"]
[EventDate "2016.03.22"]
[FEN "rn1qk1nr/pppppppp/8/8/8/8/PPPPPPPP/RNB1KBNR w KQkq - 0 1"]
[SetUp "1"]
[Termination "Game drawn by agreement"]
[TimeControl "2700+30"]
[Variant "Odds Chess"]

1.d4 { [%clk 0:45:12] } 1...Nf6 { [%clk 0:45:24] } 2.e3 { [%clk 0:45:21] } 2...Nc6 { [%clk 0:45:15] } 3.Nf3 { [%clk 0:44:37] } 3...O-O { [%clk 0:45:14] } 4.a3 { [%clk 0:44:14] } 4...d6 { [%clk 0:45:29] } 5.Be2 { [%clk 0:44:06] } 5...e5 { [%clk 0:45:53] } 6.O-O { [%clk 0:40:44] } 6...exd4 { [%clk 0:44:20] } 7.exd4 { [%clk 0:40:15] } 7...Re8 { [%clk 0:44:36] } 8.Nc3 { [%clk 0:37:58] } 8...Qd7 { [%clk 0:41:18] } 9.Be3 { [%clk 0:38:12] } 9...Ne7 { [%clk 0:41:08] } 10.Bd3 { [%clk 0:37:55] } 10...Ned5 { [%clk 0:41:03] } 11.Nxd5 { [%clk 0:37:29] } 11...Nxd5 { [%clk 0:41:29] } 12.Bd2 { [%clk 0:37:09] } 12...h6 { [%clk 0:38:57] } 13.c4 { [%clk 0:36:44] } 13...Nf6 { [%clk 0:39:13] } 14.d5 { [%clk 0:36:25] } 14...Ne4 { [%clk 0:38:05] } 15.Be3 { [%clk 0:36:00] } 15...a5 { [%clk 0:35:26] } 16.Nd4 { [%clk 0:34:54] } 16...a4 { [%clk 0:34:23] } 17.Rab1 { [%clk 0:31:09] } 17...Re7 { [%clk 0:31:16] } 18.Bc2 { [%clk 0:28:46] } 18...Rae8 { [%clk 0:28:15] } 19.h3 { [%clk 0:27:22] } 19...Nc5 { [%clk 0:22:49] } 20.Bd2 { [%clk 0:26:29] } 20...Ne4 { [%clk 0:13:35] } 21.Bf4 { [%clk 0:25:47] } 21...Nc5 { [%clk 0:13:33] } 22.Bd2 { [%clk 0:24:48] } 22...Re2 { [%clk 0:13:15] } 23.Bf5 { [%clk 0:23:59] } 23...Qxf5 { [%clk 0:12:27] } 24.Nxf5 { [%clk 0:23:33] } 24...Rxd2 { [%clk 0:12:54] } 25.Ne3 { [%clk 0:17:39] } 25...Re4 { [%clk 0:13:20] } 26.Rfd1 { [%clk 0:17:05] } 26...Red4 { [%clk 0:13:35] } 27.Rbc1 { [%clk 0:12:57] } 27...g6 { [%clk 0:11:46] } 28.Rxd2 { [%clk 0:13:10] } 28...Rxd2 { [%clk 0:12:12] } 29.Rc2 { [%clk 0:13:26] } 29...Rxc2 { [%clk 0:12:02] } 30.Nxc2 { [%clk 0:13:44] } 30...Nd3 { [%clk 0:12:25] } 31.Kf1 { [%clk 0:13:57] } 31...Nxb2 { [%clk 0:12:43] } 32.Ne3 { [%clk 0:13:43] } 32...f5 { [%clk 0:10:02] } 33.g3 { [%clk 0:13:38] } 33...Nd3 { [%clk 0:10:27] } 34.Ke2 { [%clk 0:13:38] } 34...Nc5 { [%clk 0:10:31] } 35.Nd1 { [%clk 0:13:03] } 35...Kf7 { [%clk 0:10:29] } 36.Ke3 { [%clk 0:13:15] } 36...c6 { [%clk 0:10:36] } 37.dxc6 { [%clk 0:12:53] } 37...bxc6 { [%clk 0:11:02] } 38.Nc3 { [%clk 0:12:25] } 38...Ke6 { [%clk 0:11:15] } 39.Ne2 { [%clk 0:12:15] } 39...Kd7 { [%clk 0:09:37] } 40.Nf4 { [%clk 0:12:09] } 40...g5 { [%clk 0:09:59] } 41.Ne2 { [%clk 0:12:26] } 41...Nb3 { [%clk 0:07:32] } 42.Nc3 { [%clk 0:11:12] } 42...Nc5 { [%clk 0:07:53] } 43.Ne2 { [%clk 0:11:30] } 43...Nb3 { [%clk 0:07:29] } 44.Nc3 { [%clk 0:11:50] } 44...Nc5 { [%clk 0:07:55] } 1/2-1/2
[/pgn]

[Nordlandia]

What contempt did Komodo use in game 4?

Maybe contempt for white enabled.

[lkaufman]

What contempt did Komodo use in game 4?

Maybe contempt for white enabled.

Contempt was 60 in game 1, 75 in games 2 and 3, 100 in game 4. There is no reason to use contempt for White when the engine is playing a game (although it won't hurt if you set the sign of Contempt correctly); its purpose is to use when analyzing so that switching colors won't cause problems.