marcelk wrote:Don wrote:
TD also uses the result of the game as a training signal. A checkmate score is in the future and all positions leading to checkmate tend to be high scoring, so even if you have NO knowledge whatsoever of how weights should be set the program very quickly discovers that queens are better than anything else, and so on.
I can see where you're going. I should indeed revisit TD. I always had the somewhat poor (IMO..) results of KnightCap in mind, but maybe KnightCap's problem was the number of games and not the method.
How do you tune in Komodo may I ask?
Tuning in Komodo is done tediously and manually. We propose a change and run 50,000 games to verify it. I have little faith in "fast" automated tuning because we have found many evaluation improvements that only increase the ELO by 1 or 2 ELO and it takes 50,000 games to have much confidence that the change is really an improvement. But we can end up with 20 or 30 ELO from a dozen of these tweaks of the weights because the add up.
In such cases the training signal from 100 games or even 1000 is pretty weak.
By the way, if you are using any sort of population based algorithms, such as GA or PBIL and you need to find the strongest player in a population of N players, a very simple and relatively efficient method is a knockout tournament (or elimination.) No method is guaranteed to give you the best player, but K/O is very resource friendly compared to playing in a round robin for instance because it is based on the tried and true divide and conquer computer science technique. Each match should consider of many games if the players are relatively closely matched but in general K/O ensures that you spend very little time testing weaker players (since they are eliminated early.)
One characteristic of GA's is that you don't necessarly want the BEST individual in a population but you want better players to be selected with higher probability. A K/O tournament would be excellent for doing such a thing as the strongest are not guaranteed to win and even a weaker play can have a good run. For classic GA you could run such a tournament and select the 2 finalists for breeding.
For a learning algorithm setting up such a tournament is trivial and recursive and players can be generated on the fly as needed. A perfect fit. Here is an example:
Code: Select all
Player tourney( int round )
{
Player a;
player b;
if (round == 1) {
a = generateNewPlayer();
b = generateNewPlayer();
} else {
a = tourney( round - 1 );
b = tourney( round - 1 );
}
return( winner_of_match_between( a, b ) );
}
// generate a 6 round tournament (64 players) and return the winner
Player strong = tourney(6);
I think the code is correct but the point is how simple it is. generateNewPlayer() of course does what you need it to do in order to generate a new player from scratch, perhaps applying GA, or SA against previous winners.
