Non-linear eval terms

[jwes]

How do you determine if eval term or terms are non-linear and what function would best represent them?

[Evert]

How do you determine if eval term or terms are non-linear and what function would best represent them?

I'm not entirely sure what you mean.

If the evaluation function is

Code: Select all

Eval = sum eval_term_i

then the evaluation function is linear in each evaluation term. If the evaluation term has the form

Code: Select all

eval_term_i = weight_i * feature_i

then it is linear in terms of its features (say, "knight on e5" for a PST term). If it has the form

Code: Select all

eval_term_i = weight_ijk feature_ijk (implied sum over indices j and k)

then it is multi-linear - which can be re-written in terms of different linear evaluation terms.
If it has the form

Code: Select all

eval_term_i = weight_ij feature_ik feature jk

then it is quadratic in its features, but still linear in its evaluation weights.

A function f(x) is linear if f(lambda x) = lambda f(x).

Now, if you meant to ask: how do I know which evaluation terms should be linear and which shouldn't - you don't. That's one of the things you have to experiment with. There's also no absolute rule that says what sort of functional form terms should take. There are a few pointers though. For instance, what is more valuable: passed pawns on c4 and f6, or passed pawns on c5 and f5? A term that is linear in the distance to promotion squares considers these equal. A term that is quadratic in the distance considers the first preferable (which I think is what most programs have). Similar for pressure on the king: it becomes much stronger as pieces are added. Note that these things are typically non-linear in the feature they measure (attack pressure against the king, distance to promotion) but not in the evaluation weight.

As for what form you should use - a question is whether it really matters that much. I personally like logistic-like functions because they saturate as their argument approaches infinity (as opposed to blowing up, as quadratic functions would). Of course chess boards are pretty small, so I suspect that you can approximate any more complicated function on it quite reasonably by a multi-nomial with only a few terms.

[Henk]

In neural networks they use Sigmoid which looks like an arctangent.
Or maybe search on radial basis functions.

[jdart]

Many engines have used non-linear formulas to derive eval terms, for example this can be done for passed pawns. One reason to do this is that you have fewer eval terms to tune. So for example if the passed pawn formula is quadratic, you can get away with 3 terms (the coefficients) instead of 6 terms (each possible passed pawn rank).

But if you are using an automatic tuning method based on linear weights, as I do now, it is actually simpler and better to just have a lot of independent terms each of which is triggered by a unique feature. It costs practically no more to tune a large set vs. a small set of parameters using gradient descent (as opposed to say, using something like CLOP, where tuning large parameter sets is hard).

I now have very few non-linear terms. The main ones remaining are related to king cover and king safety. The scores for king cover and king attacks interact and combine in a fairly complex non-linear way. I have tuned the parameters for this using various methods but can't readily use the automated tuner for it.

--Jon