Evaluation Tuning

[Ferdy]

Another observation is, that the result values have the property that (mgValue+egValue)/2 are close to a realistic value.

Something like the ff, is how I treat my passer bonus. And I believe every parameter has its own patterns.
As material of my opponent is close to zero, only then that its eval received a bigger bonus.

[Desperado]

Hi,

i just wanted to point out, that i use a standard form of phase computation.
You can also choose Fruit computation which is described on chessprogramming.wikispaces.com.

The result values of the regression algorithm described at the beginning of the thread are the problem.

Think of a single value for the queen which may be 800, but if i measure values depending on phase i do not get values like [mg,eg] / [750,850] but
it looks like [200,1400] for example.

So, it doesn't matter what i do, the computed result values are somehow crazy. I simply do not know how to change the algorithm and avoid these kind of drifts.

It does not matter if i prepare the dataset or use search results instead of static evaluation or, or, or...

The interpolation function has the simple form of

Code: Select all

(mg * phase + eg * (MAXPHASE-phase)) / MAXPHASE

thx

[Desperado]

Another note: As described above, phase values like [200,1400] which
have the property (200 + 1400) / 2 == the single point value of 800.

This is an arbitrary example, but finally the regression algorithm will prefer strongly one phase.
My guess was that the reason is an unequal distribution of the phases, so having implicit different weights.
Well, forcing an equal distribution of the game phases does not help too.
Using different kind of phase computations(weights,interpolation formular) doesn't improve anything at all. And so on...

[Ferdy]

Hi,

i just wanted to point out, that i use a standard form of phase computation.

Is this what you call standard where you set Q, R, B and N to 1?

Code: Select all

int Evaluation::getMaterialPhase(int tid,Position* pos) 
{ 
    const int PHASE_P  = 0; 
    const int PHASE_N  = 1; 
    const int PHASE_B  = 1; 
    const int PHASE_R  = 1; 
    const int PHASE_Q  = 1;

[...]

[Desperado]

Hi,

i just wanted to point out, that i use a standard form of phase computation.

Is this what you call standard where you set Q, R, B and N to 1?

Code: Select all

int Evaluation::getMaterialPhase(int tid,Position* pos) 
{ 
    const int PHASE_P  = 0; 
    const int PHASE_N  = 1; 
    const int PHASE_B  = 1; 
    const int PHASE_R  = 1; 
    const int PHASE_Q  = 1;

[...]

Hi Ferdy,

you can ignore the weights and use the complete "Fruit formular/computation". It won't change anything.
The "algorithm" that is used to compute phase values is standard, including the interpolation function.
I do not use any kind of "special" algorithm for phase computing. That is all, i wanted to say.

[Ferdy]

The result values of the regression algorithm described at the beginning of the thread are the problem.

Think of a single value for the queen which may be 800, but if i measure values depending on phase i do not get values like [mg,eg] / [750,850] but
it looks like [200,1400] for example.

So, it doesn't matter what i do, the computed result values are somehow crazy. I simply do not know how to change the algorithm and avoid these kind of drifts.

In my param tuning, I start with the default engine value. Say
1. Tune queen value
2. Get default value say 999
3. Add 2, that would become 1001
4. Do the fitness
5. If not good Subtract 999 by 2, that would become 997
6. Do the fitness
But even though I increment and decrement the param as the tuning continues,
I do not allow the algo to exceed the param min and max values of every parameter. I respect the engine setting like the ff.

Code: Select all

option name QueenValueEg type spin default 999 min 950 max 1200

The minimum of this param is 950 and the maximum is 1200.
Once the param reaches 1200, the tuner will not tune it above 1200.
Do you allow your tuner to tune to any values?

[Desperado]

The result values of the regression algorithm described at the beginning of the thread are the problem.

Think of a single value for the queen which may be 800, but if i measure values depending on phase i do not get values like [mg,eg] / [750,850] but
it looks like [200,1400] for example.

So, it doesn't matter what i do, the computed result values are somehow crazy. I simply do not know how to change the algorithm and avoid these kind of drifts.

In my param tuning, I start with the default engine value. Say
1. Tune queen value
2. Get default value say 999
3. Add 2, that would become 1001
4. Do the fitness
5. If not good Subtract 999 by 2, that would become 997
6. Do the fitness
But even though I increment and decrement the param as the tuning continues,
I do not allow the algo to exceed the param min and max values of every parameter. I respect the engine setting like the ff.

Code: Select all

option name QueenValueEg type spin default 999 min 950 max 1200

The minimum of this param is 950 and the maximum is 1200.
Once the param reaches 1200, the tuner will not tune it above 1200.
Do you allow your tuner to tune to any values?

There is no restriction and until now the algorithm was able to work without any. But of course, that may be a solution to get values that are usable. On the other hand i would like to explore the reason more closely.

Mentioned this... well, the fitness function returns winning probabilities and it is pretty obvious that mg advantages/values are more exhalable.
Therefore the algorithm just reports the optimal values based on the minimized error of winning probabilities. I may dislike these numbers, but based on the fitness function they seem to be a fact (simply measured).

Now the question is, if a "standard" game phase computation / interpolation is a good solution if the optimal values would be like [200,1400] for a queen. Without any doubt, such values in a standard engine will lead to a disaster.

I would prefer a general solution, that does not depend on restrictions on the values, but on a adaption for phase computing or more general weighted evaluations.

[Ferdy]

Mentioned this... well, the fitness function returns winning probabilities and it is pretty obvious that mg advantages/values are more exhalable.

It is the mg values that matters and not the winning probabilities. Winning probabilities is the more exhalable.

[Ferdy]

I would prefer a general solution, that does not depend on restrictions on the values, but on a adaption for phase computing or more general weighted evaluations.

That would be good indeed for positional factors but for piece values I have some doubts.

[Desperado]

Mentioned this... well, the fitness function returns winning probabilities and it is pretty obvious that mg advantages/values are more exhalable.

It is the mg values that matters and not the winning probabilities. Winning probabilities is the more exhalable.

I am not sure what you mean.

Code: Select all

double Database::getFitness(uint32_t id,Position* pos)
{
    // Evaluate current game position
    double score = getWhitePositionScore(id,pos);
           score = Misc::sigmoid(score,400);

    // Game result 1.0/0.5/0.0
    double result = getGameResult(id);

    // Fitness logic
    return 1.0 - pow(result-score,2);
}

Only the error of the computed score expressed as winning probability will be minimized. The probability function maps the evaluation score, that's all.
Of course the mg score influences the straightforward output of the evaluation function but finally the probability will be different too.