Evaluation tuning and constraints

[gaard]

I am completely revamping my evaluation for Cadie. During my last tune a lot of parameters walked all over the place. When I fix one or more of the parameters, say pawn middle game value, they all start to look more reasonable. My question is what other parameters may make sense to constrain. The obvious are P < N <= B < R < Q. I am looking for other constraints that should also work more often than not. For example, current tuning says the rook and queen go up in value towards the end game much more proportionally than the knight and bishop relative to their respective middle game values. Piece values are just examples. Another might be winning the exchange, versus a passed pawn in the end game. Essentially, any combination of two or more material or positional values. If I can start with better constraints, I can tune faster and more likely avoid local minima.

[JoAnnP38]

How big is your data set and how long does it take you to tune? TBH I don't constrain any of my values because I think I would probably only screw things up. Here are my current piece values as most recently tuned:

Code: Select all

/* opening piece values */
95, 370, 395, 555, 1315, 0,

/* end game piece values */
100, 325, 315, 580, 1025, 0,

The one thing I do though to help speed things along is to set the increment size for each evaluation term. Pieces for instance increment by +/-5 and my game phase boundaries (I realize most people don't seem to let the tuner set these, but I do) I have an increment of +/-50. As far as avoiding local minimums I suppose it depends on what technique you are using for optimization. I still use the simple local search I first saw described by the Texel tuning method. However, I have made some changes to allow the tuner to better "escape" local minimums. One of these is to iterate through the evaluation terms randomly on every iteration. This gives an equal chance for each term to affect the overall error and you avoid trying to change the same terms in the same order each iteration. I have not had any trouble at all with local minimums and I break from iteration based on either a fixed number of iterations or when my error is accurate to within N decimal places (currently 7) whichever comes first. Good luck. This is one of the most interesting areas of chess engine programming.

[gaard]

How big is your data set and how long does it take you to tune? TBH I don't constrain any of my values because I think I would probably only screw things up. Here are my current piece values as most recently tuned:

Code: Select all

/* opening piece values */
95, 370, 395, 555, 1315, 0,

/* end game piece values */
100, 325, 315, 580, 1025, 0,

The one thing I do though to help speed things along is to set the increment size for each evaluation term. Pieces for instance increment by +/-5 and my game phase boundaries (I realize most people don't seem to let the tuner set these, but I do) I have an increment of +/-50. As far as avoiding local minimums I suppose it depends on what technique you are using for optimization. I still use the simple local search I first saw described by the Texel tuning method. However, I have made some changes to allow the tuner to better "escape" local minimums. One of these is to iterate through the evaluation terms randomly on every iteration. This gives an equal chance for each term to affect the overall error and you avoid trying to change the same terms in the same order each iteration. I have not had any trouble at all with local minimums and I break from iteration based on either a fixed number of iterations or when my error is accurate to within N decimal places (currently 7) whichever comes first. Good luck. This is one of the most interesting areas of chess engine programming.

My data set is about 10M positions, courtesy of Andrew Grant. A full tune takes about 20 hours before parameters begin the settle, sometimes at their bounded constraints when constraints are defined. Regarding your tuned piece values, I have no idea how a bishop would lose value relative to a knight transitioning from the opening to the endgame. I have only ever seen the opposite. I can only assume you have some other factors at work that must be overcompensating.

[JoAnnP38]

WRT piece values, it's all about the probability of a piece appear in the end-game and its effect on minimizing error. Training involves no "evaluation" of "factors" beyond the data and fitting an equation.

My only advice from a lot of tunings (I use between 12-14M positions btw) is to not set any boundaries, collect quality data and let the learning take the lead. I think trying to assume where things should be, you will be robbing your engine of potential advantages from learning.

[jdart]

There's an argument for not constraining, but when I was doing eval tuning, I constrained all parameters, although within fairly broad ranges.
Some were tuned to zero values and then I removed them.