Non-linear eval terms

Discussion of chess software programming and technical issues.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Post Reply
Posts: 778
Joined: Sat Jul 01, 2006 5:11 am

Non-linear eval terms

Post by jwes » Fri Jan 29, 2016 7:31 am

How do you determine if eval term or terms are non-linear and what function would best represent them?

User avatar
Posts: 2929
Joined: Fri Jan 21, 2011 11:42 pm
Location: NL

Re: Non-linear eval terms

Post by Evert » Fri Jan 29, 2016 8:22 am

jwes wrote:How do you determine if eval term or terms are non-linear and what function would best represent them?
I'm not entirely sure what you mean.

If the evaluation function is

Code: Select all

Eval = sum eval_term_i
then the evaluation function is linear in each evaluation term. If the evaluation term has the form

Code: Select all

eval_term_i = weight_i * feature_i
then it is linear in terms of its features (say, "knight on e5" for a PST term). If it has the form

Code: Select all

eval_term_i = weight_ijk feature_ijk (implied sum over indices j and k)
then it is multi-linear - which can be re-written in terms of different linear evaluation terms.
If it has the form

Code: Select all

eval_term_i = weight_ij feature_ik feature jk
then it is quadratic in its features, but still linear in its evaluation weights.

A function f(x) is linear if f(lambda x) = lambda f(x).

Now, if you meant to ask: how do I know which evaluation terms should be linear and which shouldn't - you don't. That's one of the things you have to experiment with. There's also no absolute rule that says what sort of functional form terms should take. There are a few pointers though. For instance, what is more valuable: passed pawns on c4 and f6, or passed pawns on c5 and f5? A term that is linear in the distance to promotion squares considers these equal. A term that is quadratic in the distance considers the first preferable (which I think is what most programs have). Similar for pressure on the king: it becomes much stronger as pieces are added. Note that these things are typically non-linear in the feature they measure (attack pressure against the king, distance to promotion) but not in the evaluation weight.

As for what form you should use - a question is whether it really matters that much. I personally like logistic-like functions because they saturate as their argument approaches infinity (as opposed to blowing up, as quadratic functions would). Of course chess boards are pretty small, so I suspect that you can approximate any more complicated function on it quite reasonably by a multi-nomial with only a few terms.

Posts: 6419
Joined: Mon May 27, 2013 8:31 am

Re: Non-linear eval terms

Post by Henk » Fri Jan 29, 2016 9:15 am

In neural networks they use Sigmoid which looks like an arctangent.
Or maybe search on radial basis functions.

Posts: 3956
Joined: Fri Mar 10, 2006 4:23 am

Re: Non-linear eval terms

Post by jdart » Fri Jan 29, 2016 5:07 pm

Many engines have used non-linear formulas to derive eval terms, for example this can be done for passed pawns. One reason to do this is that you have fewer eval terms to tune. So for example if the passed pawn formula is quadratic, you can get away with 3 terms (the coefficients) instead of 6 terms (each possible passed pawn rank).

But if you are using an automatic tuning method based on linear weights, as I do now, it is actually simpler and better to just have a lot of independent terms each of which is triggered by a unique feature. It costs practically no more to tune a large set vs. a small set of parameters using gradient descent (as opposed to say, using something like CLOP, where tuning large parameter sets is hard).

I now have very few non-linear terms. The main ones remaining are related to king cover and king safety. The scores for king cover and king attacks interact and combine in a fairly complex non-linear way. I have tuned the parameters for this using various methods but can't readily use the automated tuner for it.


Post Reply