tuning for the uninformed
Moderators: hgm, Harvey Williamson, bob
Re: tuning for the uninformed
I don't know how to get an estimate of the value of a position without using other engines.
Maybe use your own engine and research it on shallow depth. But who will say that that gives a right estimate if your current evaluation is bad.
You can not evaluate every test position manually for that is too much work.
Maybe use your own engine and research it on shallow depth. But who will say that that gives a right estimate if your current evaluation is bad.
You can not evaluate every test position manually for that is too much work.

 Posts: 111
 Joined: Sat Jan 28, 2017 12:29 pm
 Location: The Netherlands
Re: tuning for the uninformed
quietlabeled.epd contains the outcome of every positionHenk wrote:I don't know how to get an estimate of the value of a position without using other engines.
Maybe use your own engine and research it on shallow depth. But who will say that that gives a right estimate if your current evaluation is bad.
You can not evaluate every test position manually for that is too much work.
Re: tuning for the uninformed
Don't understand. What is quietlabeled.epd?sandermvdb wrote:quietlabeled.epd contains the outcome of every positionHenk wrote:I don't know how to get an estimate of the value of a position without using other engines.
Maybe use your own engine and research it on shallow depth. But who will say that that gives a right estimate if your current evaluation is bad.
You can not evaluate every test position manually for that is too much work.

 Posts: 111
 Joined: Sat Jan 28, 2017 12:29 pm
 Location: The Netherlands
Re: tuning for the uninformed
Sorry, that is one of the testsets by Alexandru Mosoi, the author of Zurichess. It conains quiet positions including the outcome of the game.Henk wrote:Don't understand. What is quietlabeled.epd?sandermvdb wrote:quietlabeled.epd contains the outcome of every positionHenk wrote: Maybe use your own engine and research it on shallow depth. But who will say that that gives a right estimate if your current evaluation is bad.
You can not evaluate every test position manually for that is too much work.
Re: tuning for the uninformed
What do you mean by that?sandermvdb wrote:The basic idea is pretty simple: calculate the error of the evaluation when it is compared to the actual outcome of the positions.
Do you mean the following:
 fen as input
 calc move with an eval val
 calc eval of the move that should've been moved
 compare these two (how? percentual difference? or what?)

 Posts: 111
 Joined: Sat Jan 28, 2017 12:29 pm
 Location: The Netherlands
Re: tuning for the uninformed
No. I mean:flok wrote:What do you mean by that?sandermvdb wrote:The basic idea is pretty simple: calculate the error of the evaluation when it is compared to the actual outcome of the positions.
Do you mean the following:
 fen as input
 calc move with an eval val
 calc eval of the move that should've been moved
 compare these two (how? percentual difference? or what?)
 fen as input
 calculate evaluation
 calculate error: compare evaluation score with the actual outcome. If the evaluation calculates that white has a big advantage but black wins > big error. The exact formula is described on the cpw. This is my (pseudo) implementation, where K=1.3:
Code: Select all
public double calculateTotalError() {
double totalError = 0;
for (Entry<String, Double> entry : fens.entrySet()) { // fens contains all positions, including the outcome
ChessBoard cb = new ChessBoard(entry.getKey());
totalError += Math.pow(entry.getValue()  calculateSigmoid(Eval.calculateScore(cb)), 2);
}
totalError /= fens.size();
return totalError;
}
public double calculateSigmoid(int score) {
return 1 / (1 + Math.pow(10, 1.3 * score / 400));
}

 Posts: 272
 Joined: Wed Aug 24, 2016 7:49 pm
Re: tuning for the uninformed
This is the local search algorithm but I would assume that it is better to run some gradient based algorithm first (Maybe gradient descent or gaussnewton). Then if the error doesn't change by much anymore I would switch to local search.sandermvdb wrote:The basic idea is pretty simple: calculate the error of the evaluation when it is compared to the actual outcome of the positions. Lower a particular evaluation parameter and check if the error has improved, if not, higher the parameter, if again not improved, keep the original value. Do this for all parameters until you have reached the lowest error.flok wrote:If anyone is willing to explain the Texel tuning method tht would be great!
Sofar I understand I have to let it play (well, run QS + eval on FENs) millions of games and then do something with the evaluationvalue. But what? I don't understand the wiki explanation.
I am going to implement texel tuning this week and see what I get
Re: tuning for the uninformed
All does not work if search space has a great many local optima and only very few global optima that you are interested in. But simulated annealing taking too long.

 Posts: 272
 Joined: Wed Aug 24, 2016 7:49 pm
Re: tuning for the uninformed
Local search and any other practical algorithm to minimize the error will end up in a local optimum.Henk wrote:All does not work if search space has a great many local optima and only very few global optima that you are interested in. But simulated annealing taking too long.
Re: tuning for the uninformed
Wasn't it that if it optimizes enough parameters you won't get trapped in a local optimum. I can't remember.CheckersGuy wrote:Local search and any other practical algorithm to minimize the error will end up in a local optimum.Henk wrote:All does not work if search space has a great many local optima and only very few global optima that you are interested in. But simulated annealing taking too long.