Idea for Automatic Calibration of Evaluation Function...

[Steve Maughan]

Sadly it would seem that I all to often find myself too busy for computer chess programming - so I'm going to share this idea with the hope that someone else may find it of value.

I've been giving some thought to how one may be able to automatically calibrate an evaluation function. I also noticed that Stockfish has a random evaluation term that can be used to simulate human play. This triggered an idea based on simulated annealing (SA), a probabilistic optimization algorithm. For those who haven't come across SA, the algorithm basically randomly varies the elements that can be optimized, keeping the best solution, and slowly decreases the amount of randomness from the current best solution as time goes by and the system "cools".

So how about this for a evaluation annealing algorithm. Create ten versions of an engine each with a different set of evaluation coefficients. Let them play against one another. After each game, if an engine wins, decrease the randomness that you apply to adjust its coefficients. If it repeatedly wins then the randomness will approach zero. After an engine loses, increase the randomness of the variation in coefficients and bias the change to come closer to the coefficients of the engine that beat it (there are a number of ways to do this). After "many" games the coefficients "should" converge on a good set of values.

Any comments?

Naturally there are a zillion ways to implement and play around with how the coefficients are adjusted after each game.

Best regards,

Steve

[hgm]

A nice idea, if you can afford a billion games, or so. Bob just showed that being off a full Pawn on the Queen value only costs about 10 Elo points. Most eval terms are likely to be much smaller than 100 cP.

[BubbaTough]

A nice idea, if you can afford a billion games, or so. Bob just showed that being off a full Pawn on the Queen value only costs about 10 Elo points. Most eval terms are likely to be much smaller than 100 cP.

Just to nitpick a little...you should not generalize too much from Bob's finding there as he cherry-picked that value. I am sure being off by 100cp will have a HUGE ELO effect on most eval terms. It also seems clear at this point that automated tuning of the eval function is actually worth quite a lot of ELO in general, more than traditionally suspected, say 10 years ago.

-Sam

[MattieShoes]

Self-play is tricky. You're optimizing the engine to beat itself, which doesn't always translated to beating other players...

[smcracraft]

Would like to see a repeat of the Knightcap experiments
on (F)ICS. Auto-tuning of eval... in a context of external competitors
chosen for being approximately the same current rating.

Has that experiment been repeated?

Would like to see it repeated with all the positional term coefficients
started at zero, all the piece weights started at a pawn, etc. After
the physical piece values "settle", let the positional terms start
changing in tandem.

Use whatever multi-coefficient reinforcement learning/regression/
annealing/etc. you like.

[Dann Corbit]

Read this thread:
http://www.talkchess.com/forum/viewtopi ... 47&t=31667

The technique presented looks far more effective than td-lamda and td-leaf.

[MattieShoes]

I don't recall what knightcap used initially but I do seem to recall knightcap in the 1300s on FICS when it was starting (back when the average was closer to 1600). Or maybe I'm thinking of another engine. It was comical to watch though -- the learning algorithm figured out that a side that is never put in check also never loses, so it'd sac material to push being checked over the horizon.

[Gian-Carlo Pascutto]

Any comments?

It's called PBIL and has been discussed at length here.