Texel tuning speed

[Joost Buijs]

I use plain GD, SGD seems to be sensitive to the order of the training positions, so I never tried. My error function calls the evaluator (with pawn hash disabled) and not the quiescence search, of course I tried both but the difference in final tuning is negligible.

When I start with reasonable values the number of iterations through the GD needed to reach good convergence usually lies between 100-200. Tuning ~400 evaluation terms with 7.5 million positions takes roughly 2 hrs on my 4GHz. 6950X using 20 threads (with hyperthreading).

There are still a number of things that could be improved to increase tuning speed, e.g. each iteration I read and parse the training positions from disk, reading them once and storing them in binary form in memory would already increase speed.

[Joost Buijs]

I meant of course that I read and parse the examples from disk for each call to the error function and not for each iteration. Windows probably caches the examples in memory, but parsing them takes quite some time.

[mar]

I don't remember the times but IIRC eval cache helps a ton. Of course you want to parallelize as well.
Texel tuning is very fast because you don't have to play actual games.

[xr_a_y]

Thanks for the advices. I'll first try a simple GD then look at this very interesting Andrew's method.

[Joost Buijs]

I don't remember the times but IIRC eval cache helps a ton. Of course you want to parallelize as well.
Texel tuning is very fast because you don't have to play actual games.

I think using eval cache when tuning is an error, each time you modify a term and call the evaluator you will get the cached value instead of the new value. The same holds for quiescence, if you use TT pruning in quiescence you have to disable it.

Texel tuning works better than I expected, but you really need a high number of training samples covering all terms you want to tune otherwise you'll get strange results.

[Ronald]

I used the "quiet-labeled.epd" set created by Zurichess for Texel tuning, which contains 750.000 quiet positions. Because they are quiet you don't need to call quiescence but you can call the eval function directly. This saves a lot of time.

[Joost Buijs]

I used the "quiet-labeled.epd" set created by Zurichess for Texel tuning, which contains 750.000 quiet positions. Because they are quiet you don't need to call quiescence but you can call the eval function directly. This saves a lot of time.

When I started with Texel tuning I used the same set, recently I generated a larger (quiet) set from the many computer games I collected over the years, this in the hope that these positions have a wider spread because they are played by many different engines with different opening lines. On the other hand you only want to have games played (or at least have the outcome analyzed) by the strongest engines, so I don't know if it is any better.

[Ronald]

I would expect that a correct outcome of the position is important for the tuning result, and if you use the outcome of those games which are probably played with different rated engines the "true" outcome may differ more often from the game outcome and thus create error. There's only one way to find out however.., did you already make a comparison ?

[Joost Buijs]

I would expect that a correct outcome of the position is important for the tuning result, and if you use the outcome of those games which are probably played with different rated engines the "true" outcome may differ more often from the game outcome and thus create error. There's only one way to find out however.., did you already make a comparison ?

No, I didn't make a comparison yet, when I can find some time for it I will.

Somehow I have the feeling that with enough games statistics will make up for the lower quality (or uncertain outcome) of the games played by weaker engines.

Most of the time I spend working on a new version of my engine which is atm basic search with material and psq only. Right now I'm busy implementing YBW using C++11 threads, in the past I always used Windows threads, things like mutexes, locks and condition variables behave somewhat differently now. Maybe lazy SMP would be a better choice, at least 10 times easier to implement.

[mar]

I think using eval cache when tuning is an error, each time you modify a term and call the evaluator you will get the cached value instead of the new value. The same holds for quiescence, if you use TT pruning in quiescence you have to disable it.

Not at all, you clear it before each iteration, so you modify your parameter(s), then run through the set of positions with eval cache enabled.