Hi!
I tried to use the Texel Tuning Method.
I use the zurichess quiet.epd file. I tried with another epd file with same result.
I remove all pruning from Qsearch and disable all hash table.
Eval: only material
I changed all piece value to: 0 (zero)..
Texel increment was 1; 10; 20; 50; 100
eg. with 10 i get the following results:
Final pawn value: 180
Final knight value: -20
Final bishop value: 140
Final rook value: 160
Final queen value: 0
Does anyone know what's wrong? Is Texel Method suitable for such tuning?
The source code can be found here:
https://github.com/tomitank/tomitankChe ... /tuning.js
Best Regards,
Tamás
Texel Tuning
Moderators: hgm, Rebel, chrisw
-
- Posts: 276
- Joined: Sat Mar 04, 2017 12:24 pm
- Location: Hungary
-
- Posts: 4833
- Joined: Sun Aug 10, 2008 3:15 pm
- Location: Philippines
Re: Texel Tuning
It seems like your sigmoid is wrong.
function Sigmoid(K, S) {
return 1.0 / (1.0 + Math.pow((-K * S / 400), 10));
}
Should be,
pow(base, exp)
function Sigmoid(K, S) {
return 1.0 / (1.0 + Math.pow((-K * S / 400), 10));
}
Should be,
Code: Select all
return 1.0 / (1.0 + Math.pow(10, (-K * S / 400)));
-
- Posts: 276
- Joined: Sat Mar 04, 2017 12:24 pm
- Location: Hungary
Re: Texel Tuning
Thank you very much!!!Ferdy wrote: ↑Wed Jul 04, 2018 12:20 am It seems like your sigmoid is wrong.
function Sigmoid(K, S) {
return 1.0 / (1.0 + Math.pow((-K * S / 400), 10));
}
Should be,pow(base, exp)Code: Select all
return 1.0 / (1.0 + Math.pow(10, (-K * S / 400)));
I've been looking for this for two days..
-
- Posts: 276
- Joined: Sat Mar 04, 2017 12:24 pm
- Location: Hungary
Re: Texel Tuning
Other question:
I tried do while statement for better results, but i getting different scores.
This happened about the new "K" value..
Is there any rule for determining the "final" K value ?
To get a similar scaling. (eg: pawn value = 100)
Example for do while statement:
2. question:
Did you tried all parameters tuning always to min error and just then tuning the next parameter?
Example:
The result is different.
Best Regards,
Tamás
I tried do while statement for better results, but i getting different scores.
This happened about the new "K" value..
Is there any rule for determining the "final" K value ?
To get a similar scaling. (eg: pawn value = 100)
Example for do while statement:
Code: Select all
for (var i = 0; i < numFens; i++) { // inicializalas..
var trimed = $.trim(positions[i]);
var result = trimed.substr(-5, 3);
results[i] = result == '1-0' ? 1 : result == '0-1' ? 0 : 0.5;
}
do
{
for (var i = 0; i < numFens; i++) {
tuneEvals[i] = tuning_evaluation(positions[i]);
}
K = compute_optimal_k();
console.log('Optimal K: '+K);
}
while (run_texel_tuning());
function run_texel_tuning() {
var best_error = total_eval_error(K); // alap hiba
var improved = true;
var changed = false;
var iteration = 1;
while (improved) {
improved = false;
console.log('-= Tuning iteration: '+(iteration++)+' =-');
for (var i = 0; i < params.length; i++) {
params[i].value++;
var this_error = tune_error(i);
if (this_error < best_error) {
best_error = this_error;
improved = true;
changed = true;
} else {
params[i].value -= 2;
var this_error = tune_error(i);
if (this_error < best_error) {
best_error = this_error;
improved = true;
changed = true;
} else {
params[i].value++;
}
}
console.log(params[i].name+': '+params[i].value);
}
}
return changed;
}
2. question:
Did you tried all parameters tuning always to min error and just then tuning the next parameter?
Example:
Code: Select all
while (improved) {
improved = false;
console.log('-= Tuning iteration: '+(iteration++)+' =-');
for (var i = 0; i < params.length; i++) {
var success = false;
var stepmul = 0;
while (true)
{
stepmul++;
params[i].value += 1 * stepmul;
var this_error = tune_error(i); // vizsgalat..
if (this_error < best_error) { // kisebb hiba
best_error = this_error;
improved = true;
success = true;
} else {
params[i].value -= 1 * stepmul;
if (stepmul == 1)
break;
else stepmul = 0;
}
}
if (!success) // masik irany..
{
stepmul = 0;
while (true)
{
stepmul++;
params[i].value -= 1 * stepmul;
var this_error = tune_error(i); // vizsgalat..
if (this_error < best_error) { // kisebb hiba
best_error = this_error;
improved = true;
} else {
params[i].value += 1 * stepmul;
if (stepmul == 1)
break;
else stepmul = 0;
}
}
}
console.log(params[i].name+': '+params[i].value);
}
}
Best Regards,
Tamás
-
- Posts: 855
- Joined: Sun May 23, 2010 1:32 pm
Re: Texel Tuning
I don't understand your algorithm.... you have to calculate 1 time in your life a K that minimize the error of E=1N∑i=1N(Ri−Sigmoid(qi))2 (taken from chessprogramming)
You can create a program that does this task and then stop. Calculating K should not be done while tuning. You only have to calculate it one time and then alwayis use the same value forever.
K is only used to match your internal evalutation scaling to proability of win.
you can even do somethink like that to find the best k for your program:
thius code will give an output like that:
10: 37
20: 37
30: 35
40: 34
50: 36
60: 38
.....
....
99980: 79
99990: 80
so you choose k = 40 and insert it in you texel tuning algorithm.
You can create a program that does this task and then stop. Calculating K should not be done while tuning. You only have to calculate it one time and then alwayis use the same value forever.
K is only used to match your internal evalutation scaling to proability of win.
you can even do somethink like that to find the best k for your program:
Code: Select all
for ( int k = 10; k < 100000; k+=10)
{
res = calculate_sum_of_error(k);
printf("%d: %d\n", k, res );
}
10: 37
20: 37
30: 35
40: 34
50: 36
60: 38
.....
....
99980: 79
99990: 80
so you choose k = 40 and insert it in you texel tuning algorithm.
-
- Posts: 276
- Joined: Sat Mar 04, 2017 12:24 pm
- Location: Hungary
Re: Texel Tuning
Hi!elcabesa wrote: ↑Fri Jul 06, 2018 10:33 pm I don't understand your algorithm.... you have to calculate 1 time in your life a K that minimize the error of E=1N∑i=1N(Ri−Sigmoid(qi))2 (taken from chessprogramming)
You can create a program that does this task and then stop. Calculating K should not be done while tuning. You only have to calculate it one time and then alwayis use the same value forever.
K is only used to match your internal evalutation scaling to proability of win.
After the tuning you can computing new K value...I do this until the tuning is stopped. (nothing changed)
This may not be a good idea, so I'm interested..
Yes, I use a similar one.elcabesa wrote: ↑Fri Jul 06, 2018 10:33 pm you can even do somethink like that to find the best k for your program:
Code: Select all
for ( int k = 10; k < 100000; k+=10) { res = calculate_sum_of_error(k); printf("%d: %d\n", k, res ); }
Here is my actual code:
https://github.com/tomitank/tomitankChe ... /tuning.js
But if you start all the material values from zero, it does not reach normal values.
eg: (100, 300, 300, 500, 900)
this solution helps...get closer to it
Is it possible with Texel Method at all? (reach the normal piece value from zero)
-
- Posts: 4052
- Joined: Thu May 15, 2008 9:57 pm
- Location: Berlin, Germany
- Full name: Sven Schüle
Re: Texel Tuning
You restrict the set of training positions to those with an eval between -600 and +600 (function good_tuning_value()). But you do not decide this once in the beginning but every time you visit a position. So the decision always depends on the current parameter values. This may cause some instability: positions are sometimes included (if eval fits the interval) and sometimes excluded (if it doesn't fit). Therefore you might get a wrong comparison of errors due to different position sets being compared, so your algorithm could fail to find the correct minimal error.tomitank wrote: ↑Fri Jul 06, 2018 11:13 pm Here is my actual code:
https://github.com/tomitank/tomitankChe ... /tuning.js
I suggest to let good_tuning_value() always return true, and see if that helps. Filtering of positions should be done outside the tuning process, which will also improve overall performance as a side effect (by avoiding useless eval/qsearch calls for positions which you exclude anyway).
Sven Schüle (engine author: Jumbo, KnockOut, Surprise)
-
- Posts: 4833
- Joined: Sun Aug 10, 2008 3:15 pm
- Location: Philippines
Re: Texel Tuning
Yes it is possible, tried it even using 20k training pos, stopped it after some iterations with queen still increasing. I use an increment or steps of
[+5, -5, +10, -10], K=0.65.
-
- Posts: 276
- Joined: Sat Mar 04, 2017 12:24 pm
- Location: Hungary
Re: Texel Tuning
Thanks, the graph is very good!
I believe that increment is also important. Sometimes it happens, that at half of the normal value, the tuning stops.
Did you calculated earlier the currently "K" (0.65) for 20k positions?
Or did you used the "k optimizer" function again (with zero piece values), at the beginning of this "tuning"?
(My "K" = 1.5 with zero pieces value and 1.7 with "normal" values)
If I good understand, then I must be still use the K = 1.7 value for the current training set, even if the next K has changed (after the tuning). So the current scaling does not change.
Short:
When should i "save" (and never change with current training set) the value of K?
1 .: Before each tuning method (eg tuned parameters already exist)
2 .: Once. (eg: all parameters are zero)
Thanks your help!
Tamás
-
- Posts: 276
- Joined: Sat Mar 04, 2017 12:24 pm
- Location: Hungary
Re: Texel Tuning
Thanks, I'll try it!Sven wrote: ↑Sat Jul 07, 2018 12:35 pmYou restrict the set of training positions to those with an eval between -600 and +600 (function good_tuning_value()). But you do not decide this once in the beginning but every time you visit a position. So the decision always depends on the current parameter values. This may cause some instability: positions are sometimes included (if eval fits the interval) and sometimes excluded (if it doesn't fit). Therefore you might get a wrong comparison of errors due to different position sets being compared, so your algorithm could fail to find the correct minimal error.tomitank wrote: ↑Fri Jul 06, 2018 11:13 pm Here is my actual code:
https://github.com/tomitank/tomitankChe ... /tuning.js
I suggest to let good_tuning_value() always return true, and see if that helps. Filtering of positions should be done outside the tuning process, which will also improve overall performance as a side effect (by avoiding useless eval/qsearch calls for positions which you exclude anyway).