Problems with eval function

[Kempelen]

I completed the move generator and a basic search algoritm. Now I am writting my eval function and I am having more problems that I expected.
I am in the proccess of "calibrate" the eval function. I mean, I am now assigning values to different parameters, like isolated pawn, kninght in the rim, pawn in center, etc. The problem I am having is that if I weight one of them, it looks like the others are subestimated. If I re-calibrate those, them it looks like others loose some kind of value.
This is more difficult that I supposed. Maybe all will have to have the same value....
I imagine it is all a matter of experimentation, but, are there any rule / tip / advise I can follow to make a good calibration?
How do you assign those values?

[Harald Johnsen]

You don't need to give scores to features that do not influence your engine.

If your engine do not loose pawns because they are doubled then there is no need to even evaluate doubled pawns. If it starts to loose the doubled pawn or play defensive moves then give a score for doubled pawns. If it continues to create double pawns (for no reason) then increase this score. If it starts to play strange moves to not have doubled pawn, decrease the score.
You can not throw number and think that it will work, you need to verify them.

Some features are long term or permanent, others not. This is the hard part. Will you allow the creation of a double pawn so that you can put a rook on an open file ?

HJ.

[Kempelen]

You can not throw number and think that it will work, you need to verify them.

I think I didn't express well. What I mean is that when I verify a value is not corretly tuned, and then try to calibrate it, then I see other parameters that I had tuned re-calibrated, so values that I thought was correct now are affected and lose its "initial meaning".

I don't know if I express well, my english is regular, what I say is samething like when I tune a value, is not that you need to calibrate it, but calibrate in accordance with the rest of the values. In this way, when you insert a new parameter, the rest need same kind of adjustement "to fit" with the new one.

[hgm]

How do you know if something is 'correctly tuned' or not?

[PK]

Hi,

Your description seems to indicate that You have done a lot of major eval changes without actually testing them one by one. as a result, You probably create a lot of not very obvous interactions which hamper your developement now. the real problem is that even *good* changes may cause wrong results in some class of positions - and You need to see and understand a substantial amount of test games to deal with the side-effects.

a simple example: right now I'm doing some experiments with tscp eval function. the original thing has a wrong ratio between the value of a minor piece and a (passed) pawn, and as a result tends to sac a minor for two pawns, only to get outplayed very quickly. so the first correction was simple: to raise bishop and knight value a little bit, which prevented also stupid exchanges of B+N for R+p. the score went up immediately, but in the 100 games played to confirm it, 2 or 3 featured a loss after exchanging a queen for a couple of pieces.

another example: tscp has a very high penalty for leaving d/e pawns unmoved. after I added simple mobility evaluation (which again was an overall gain), I saw a program play the following opening as white:

1.c4 b6 2.d4 Bb7 3.e4?! Bxe4 4.Nc3

By now You probably see what happened: black gets -40 for a pawn on d7, another -40 for a pawn on e7, the mobility of both white bishops changed from -5 to +5... and we already have a compensation for a pawn.

This is not to say that it is bad for TSCP to have -40 penalty for a pawn on d7. This penalty simply replaces mobility assessement in the opening position, discourages playing ..e5 and ..Bd6 (which probably deserves a separate clause in eval function) etc. Adding those useful features one has to remember that something else took care for them - and modify that "something else" accordingly.

I don't claim to be an expert on evaluation functions (perhaps I'll venture to do it when my program will evaluate Pirc defense as -30 and Alekhine as -40, and not as under -100 centipawns), but my advice is as follows: do one change at a time. look at the games to trace negative side-effects of positive changes. create a collection of stupid positions where side-effects occur. write a function which displays detailed information about what and how is evaluated. play blitz games against Your program (some at Your full strength, some with silly openings or deliberate positional errors) to get a feel of its style.

and enjoy - trying to make Your program intelligent is more fun than trying to make it faster.

[Kempelen]

Hi Pawel.
This is the kind of side effect I am having. I test my engine playing with it or examining analysis in possitions. As you say "I correct them". I did this work, but the problem I having is that when I correct a side effect, then other side effect previously tuned take effect.... so the process is samethink like a never ending story. Is a little frustrating adjust a value you previously tuned.

Maybe my problem is I need more experimentation ..... I am seeing this is one of the most difficult task I am facing.

[Kempelen]

How do you know if something is 'correctly tuned' or not?

If the new weakness introduced as side effect compensate the new feature.
I mean, if I encourage bishop in diagonal, they can take effect for a pawn sacrifice, it is not enought.

I am concluding my problem is I need a lot of experimentation and observation to tune the eval function in a right way

[bob]

I completed the move generator and a basic search algoritm. Now I am writting my eval function and I am having more problems that I expected.
I am in the proccess of "calibrate" the eval function. I mean, I am now assigning values to different parameters, like isolated pawn, kninght in the rim, pawn in center, etc. The problem I am having is that if I weight one of them, it looks like the others are subestimated. If I re-calibrate those, them it looks like others loose some kind of value.
This is more difficult that I supposed. Maybe all will have to have the same value....
I imagine it is all a matter of experimentation, but, are there any rule / tip / advise I can follow to make a good calibration?
How do you assign those values?

This is a very difficult problem, for exactly the reason you mentioned. The most important thing is to recognize the important features of a position. It is more common that you only have the choice to create advantage A, or advantage B. Less commonly you will have a choice and have to choose between them, and that's where it gets interesting.

This is far more "art" than "science" once you reach this point... and there is no quick and easy solution.

[Kempelen]

This is far more "art" than "science" once you reach this point... and there is no quick and easy solution.

I have noticed it. That's true and look funny too: science and art . Anyway I am thinking in a way to do things in order and with same kind of "control", always where I can, of course.

Thx.

[Ron Murawski]

I test my engine playing with it or examining analysis in possitions. As you say "I correct them".

I've tried tuning values based on test positions. It doesn't work. Not for me, anyway. The only thing that works for me are the results my engine gets in actual games played against other engines.

Ron