learning in computer chess

[irvstein1]

does anyone here know a program that has the best example of leaning in chess or are they all the same ? does anyone here have any untried ideas on the subject , im trying to create a learning chess program that gets better as it plays . i have a few good ideas but im always interested in something new .

[BubbaTough]

Why don't you tell us your ideas, and we can tell you if they have already been tried .

The most common methods are variants of the system used by Crafty. It memorizes some positions, rewards good openings, punished bad, etc.. Nothing outside the area of improving opening lines really.

Assuming you are already familiar with that stuff, the next thing you might want to look at is the learning implemented in Romi. It has had good success at improving its play from specific positions...and can learn to beat superior programs. I am not sure where the best description is of it, but some helpful person here can probably dig up some links. Here is the last link I had to a discussion of it but it seems to be broken now:

http://wbforum.vpittlik.org/viewtopic.php?t=4835

A few of us have tried a number of other methods, and may not even remember them all without some prompting, so posting an idea is really the best way to get feedback on it. For example I seem to remember Dr. Bob having done a fair amount of hill climbing stuff to try tweaking eval functions which you probably won't find published anywhere because it did not turn out very well.

My own program (LearningLemming) can learn from its games and games of others in a variety of ways, most of which are currently turned off...because they don't work very well. In other words, a number of people have tried a number of things, but it is probably up to you to throw out some ideas for feedback to get the ball rolling.

-Sam

[Michael Sherwin]

The learning in RomiChess is based on two common learning methods:

1.) a reward/punishment system, based on Pavlov's dog experiments, but modified for the computer

2.) imitation, based on the observed behavior; "monkey see, monkey do".

A tree of all Romi's (and/or imported) games is stored on disk that is read into the hash tables before each search (unless a book move is played) in which a score is kept for each position. After a game these scores are modified. If Romi wins with a small bonus/malus (Romi's moves/opponents moves) and a small malus/bonus if Romi looses. So, even bad moves by Romi get a bonus if Romi wins and good moves by Romi get a malus if she looses. As the bonus/malus is small it can take a few games to cause Romi to choose a different move, however, over time Romi does find better moves. Even hard to find sacrifices that are the only way to win.

Also, Romi keeps win/loss data and will play back winning lines as though they are book. Romi will create her own book from scratch with no previous book loaded. Therefore, if Rybka plays white against Romi and wins the next game Romi with white will play back Rybka's moves. Rybka will beat Romi again and Romi will play those moves next time as black. Eventually Romi will learn how to beat Rybka with Rybka's own book. Just 1.) above would work all by itself, but adding 2.) makes it more interesting.

Either 1.) or 2.) or both can be enabled or disabled.

[oysteijo]

KnightCap used a TD(lambda) learning algorithm in it's evaluation. Take a look at that.