some Notes from LearningLemming
Posted: Tue Oct 30, 2007 6:21 pm
LearningLemming has not changed much in the last few months, just a few bug fixes and such, but made some big leaps forward before that so I was pleased to see a solid performace. It has an extremely selective search, usually searching as deep or deeper than Rybka on 4 processors or Crafty on 8 processors despite using only 1 processor. To do this, LearningLemming is throwing away a lot of tactically relevant moves, so despite reaching great depths, it can still get blown away tactically by programs with much more shallow, broad searches.
Round 1. White vs. Telepath. Its results may not show it, but Telepath has been improving a lot. In a Grunfeld Lemming delayed castling a bit to encourage Telepath to develop awkwardly. Telepath attempted to refute this, sacrificing the exchange for a pawn and an attack. The position looked unpleasant for LearningLemming, but then Telepath exchaged queens losing its attack, and allowed an easy endgame win for Lemming.
Round 2. Black vs. Now. Now is one of my favorite programs, in part because the author tries lots of crazy things and procrastinates going SMP while still getting pretty good results, an approach I am beginning to emulate. We played a Ruy Lopez, both sides playing solid chess and going into an endgame that maybe was slightly better for Now. Lemming outplayed Now in the endgame however, when Now let its pieces get clogged up in order to get a passed pawn that wasn't going anywhere. This was a good win for Lemming, and I was glad to see Now do well in the rest of the tournament finishing 5th behind only Rybka, Crafty, LearningLemming, and Naum.
Round 3. White vs. Tinker. Tinker is a frequent sparring partner for Lemming on ICC, so I knew that while Lemming was probably a favorite, anything could happen. We played a Nimzo with Qc2, and went to a dynamic middlegame with LearningLemming having hanging pawns on the b and c files. For a while it looks like all the queenside pawns might trade off for a drawish position, but Lemming managed to create an advanced passed pawn that tied up blacks position, and maneuver into a win.
Round 4. Black vs. Crafty. On ICC I generally do decently against crafty...when it is running on Bob's old laptop. After the mighty 8 processor version logged in the night before the tournament and crushed LearningLemming in a couple blitz games though, I expected a quick loss. Crafty was out of book by move 2, and Lemming out of its learn file by move 3, so we got to enjoy watching both engines play the opening on their own. Generally, this is good for LearningLemming, since it has a bit more opening knowledge than some engines since it has no Grandmaster theory to draw on (it makes up its own openings). So it has to try to understand how to play / defend gambits, the importance of development in open vs. closed positions, when to delay castling and when to get safe, and other concepts whose importance are probably more important during the first ten moves than the next ten.
All that said, Crafty got out of the opening with a nice edge I think. LearningLemming sacked a pawn early which Crafty returned for a position where LearningLemming had no safe place to castle. It was one of those positions, however, that while better for white, had long term plusses for black should it go to endgame. I expected Crafty to find some crushing combo to pry out a dubious black king, but when it did not happen Lemming was able to shut down pawn breaks against its king, whip up a couple threats to win a pawn and trade into a superior endgame. A very happy win, and while I would not call it undeserved, it is very clear that 8 processor crafty is by far superior to 1 processor LearningLemming and any rematch would very likely have a different result.
Round 5. White vs. Rybka. There are a lot of Rybka copies running on ICC, so LearningLemming has a lot of experience in this matchup...enough to know that a draw or even a win was not impossible. The most likely result, however, was a loss, and that is what happened. This game was surprisingly interesting, though. Lemming does a lot of opening experiments to populate its learn file, and chose this game to try an odd one. It tried an early h4 in a Queens Indian which dropped the jaws of most human watchers, and proceded to commit to keeping its king in the center and leave both its bishops enprise while attacking the center and kingside vigorously. Apparently Rybka thought its play was somewhat more sound than it looked, and decided to sacrfice its queen for a rook and a knight to blunt the attack and activate all its pieces. At this point Rybka thought the position was even, leaving most of us humans pretty confused. Lemming immediately made a very ugly maneuver with its king, however, which resulted in locking its rook out of the game, and then got rolled. A loss, but the kind of interesting loss I don't mind. At least one of the human watchers swore they were going to develop LearningLemming's opening novelty into a new opening system, but it seemed likely that drugs and/or alcohol was involved in the decision.
Round 6. Black vs. Naum. This was the kind of loss I do mind. Not that I do not mind losing to Naum, which is a strong program, but the method was annoying. The learnfile went about 13 moves into the game, and ended in a position that was just a dead loss. While LearningLemming will try some odd opening experiments, the loss of massive material in just a few ply should not be possible. Since I never reset the learn file, I guess it is possible the position was added by a very buggy version of Lemming..or that there is some bug in the learning process itself. In any case, Lemming never had a chance, being completely lost before even thinking about a single move. I felt a little inhibited about whining about it too much, however, since Naum lost its first round game due to a technical glitch which caused it to forfeit on move one.
Anway, overall a fun tournament, and its nice to see that its still possible for us single processor engines to occasionaly upset big-iron monsters. Tournaments like this are particularly valuable for us private engine authors that are not on rating lists and such so we can get a little feedback on our engines and chat with more famous public/professional engine authors in a forum where they can't escape our inane chatter.
-Sam
Round 1. White vs. Telepath. Its results may not show it, but Telepath has been improving a lot. In a Grunfeld Lemming delayed castling a bit to encourage Telepath to develop awkwardly. Telepath attempted to refute this, sacrificing the exchange for a pawn and an attack. The position looked unpleasant for LearningLemming, but then Telepath exchaged queens losing its attack, and allowed an easy endgame win for Lemming.
Round 2. Black vs. Now. Now is one of my favorite programs, in part because the author tries lots of crazy things and procrastinates going SMP while still getting pretty good results, an approach I am beginning to emulate. We played a Ruy Lopez, both sides playing solid chess and going into an endgame that maybe was slightly better for Now. Lemming outplayed Now in the endgame however, when Now let its pieces get clogged up in order to get a passed pawn that wasn't going anywhere. This was a good win for Lemming, and I was glad to see Now do well in the rest of the tournament finishing 5th behind only Rybka, Crafty, LearningLemming, and Naum.
Round 3. White vs. Tinker. Tinker is a frequent sparring partner for Lemming on ICC, so I knew that while Lemming was probably a favorite, anything could happen. We played a Nimzo with Qc2, and went to a dynamic middlegame with LearningLemming having hanging pawns on the b and c files. For a while it looks like all the queenside pawns might trade off for a drawish position, but Lemming managed to create an advanced passed pawn that tied up blacks position, and maneuver into a win.
Round 4. Black vs. Crafty. On ICC I generally do decently against crafty...when it is running on Bob's old laptop. After the mighty 8 processor version logged in the night before the tournament and crushed LearningLemming in a couple blitz games though, I expected a quick loss. Crafty was out of book by move 2, and Lemming out of its learn file by move 3, so we got to enjoy watching both engines play the opening on their own. Generally, this is good for LearningLemming, since it has a bit more opening knowledge than some engines since it has no Grandmaster theory to draw on (it makes up its own openings). So it has to try to understand how to play / defend gambits, the importance of development in open vs. closed positions, when to delay castling and when to get safe, and other concepts whose importance are probably more important during the first ten moves than the next ten.
All that said, Crafty got out of the opening with a nice edge I think. LearningLemming sacked a pawn early which Crafty returned for a position where LearningLemming had no safe place to castle. It was one of those positions, however, that while better for white, had long term plusses for black should it go to endgame. I expected Crafty to find some crushing combo to pry out a dubious black king, but when it did not happen Lemming was able to shut down pawn breaks against its king, whip up a couple threats to win a pawn and trade into a superior endgame. A very happy win, and while I would not call it undeserved, it is very clear that 8 processor crafty is by far superior to 1 processor LearningLemming and any rematch would very likely have a different result.
Round 5. White vs. Rybka. There are a lot of Rybka copies running on ICC, so LearningLemming has a lot of experience in this matchup...enough to know that a draw or even a win was not impossible. The most likely result, however, was a loss, and that is what happened. This game was surprisingly interesting, though. Lemming does a lot of opening experiments to populate its learn file, and chose this game to try an odd one. It tried an early h4 in a Queens Indian which dropped the jaws of most human watchers, and proceded to commit to keeping its king in the center and leave both its bishops enprise while attacking the center and kingside vigorously. Apparently Rybka thought its play was somewhat more sound than it looked, and decided to sacrfice its queen for a rook and a knight to blunt the attack and activate all its pieces. At this point Rybka thought the position was even, leaving most of us humans pretty confused. Lemming immediately made a very ugly maneuver with its king, however, which resulted in locking its rook out of the game, and then got rolled. A loss, but the kind of interesting loss I don't mind. At least one of the human watchers swore they were going to develop LearningLemming's opening novelty into a new opening system, but it seemed likely that drugs and/or alcohol was involved in the decision.
Round 6. Black vs. Naum. This was the kind of loss I do mind. Not that I do not mind losing to Naum, which is a strong program, but the method was annoying. The learnfile went about 13 moves into the game, and ended in a position that was just a dead loss. While LearningLemming will try some odd opening experiments, the loss of massive material in just a few ply should not be possible. Since I never reset the learn file, I guess it is possible the position was added by a very buggy version of Lemming..or that there is some bug in the learning process itself. In any case, Lemming never had a chance, being completely lost before even thinking about a single move. I felt a little inhibited about whining about it too much, however, since Naum lost its first round game due to a technical glitch which caused it to forfeit on move one.
Anway, overall a fun tournament, and its nice to see that its still possible for us single processor engines to occasionaly upset big-iron monsters. Tournaments like this are particularly valuable for us private engine authors that are not on rating lists and such so we can get a little feedback on our engines and chat with more famous public/professional engine authors in a forum where they can't escape our inane chatter.
-Sam