Page 1 of 12
txt: automated chess engine tuning
Posted: Wed Mar 18, 2015 12:13 pm
by brtzsnr
Hi, all!
I wrote a small framework to do automated chess tuning based on Texel's Tuning method (
https://chessprogramming.wikispaces.com ... ing+Method). You can find the source code and instructions how to use it here
https://bitbucket.org/brtzsnr/txt.
I'm still experimenting with it, so I cannot yet report any success. Nevertheless, feel free to experiment with txt and if you find it useful, please consider contributing.
Regards,
Re: txt: automated chess engine tuning
Posted: Wed Mar 18, 2015 11:57 pm
by Engin
Great !
is there a compiled exe file for windows allready ?
i have allready in my Tornado the texel method but i am interesting if your method is much easier to use or if its really working right.
the texel method are working only for my material score, but if i want to tune some other values like pawn its give a wrong sign of the value, i don't know why is this happen.
Re: txt: automated chess engine tuning
Posted: Thu Mar 19, 2015 12:18 pm
by brtzsnr
Engin wrote:
is there a compiled exe file for windows allready ?
I can't provide an exe because you have to write a small program to configure which variables to tune. See the instructions. It's like a configuration file, but it is much easier to write and allows you to do some more complicated stuff.
Code: Select all
package main
import "bitbucket.org/brtzsnr/txt/txt"
func main() {
opts := make(txt.Options)
opts["EndGameMaterial.BishopPairBonus"] = txt.NewIntegerRange(10, 100)
opts["MidGameMaterial.BishopPairBonus"] = txt.NewIntegerRange(10, 100)
txt.Run("setvalue {{.Name}} {{.Value}}\n", opts)
}
Re: txt: automated chess engine tuning
Posted: Thu Mar 19, 2015 8:05 pm
by jdart
I am giving up on this method after spending quite some time experimenting with it.
See
http://arasanchess.org/blog.shtml.
--Jon
Re: txt: automated chess engine tuning
Posted: Thu Mar 19, 2015 8:18 pm
by Engin
yes, agree with you, in this case the tuning method is unlogical, some values become very high or low values that i can not understand how this can be, so better tuning with own logical brain. Maybe i have an bug anywhere, i have checked everywhere and did not found it.
Re: txt: automated chess engine tuning
Posted: Thu Mar 19, 2015 8:36 pm
by Engin
i said i have allready implemented the texel method with some help from Martin Sedlak and Peter Österlund it self in my Tornado, so i am not need your code to run. just i was interested if that would be really work. it is not easy because it make differences between wich PGN games you are using CC games, master games or self played games from own engine. everytime i started i get other strange values and K is each time different. Peter checked after i send him my code that i have many doubled positions in there, so i have remove repeated position out too. of course there are no mates and i save only after 10 book moves in my filtered EPD file. What i am not understanding is that is work for material values good and why its not working for my pawns where i give them negative values and its giving me positive values what is nonsense. for example doubled pawns = -0.10cp or isolated pawns = -0.20cp i have and i am sure that all other programmers giving penalty values for double, isolated, backward pawns too. the final best value it say give me some kind of +1.5 cp what is incredible nonsense
Re: txt: automated chess engine tuning
Posted: Thu Mar 19, 2015 9:10 pm
by brtzsnr
I tried generating ~100.000 positions from stockfish games and optimized a few values. Initial results after 500 games were promising (10-40ELO increase) but after 2000 games all elo increase disappeared.
Now I'm trying the same with 900.000 positions from 12.000 self played games.
I removed all duplicate positions (so this should remove most of the openings), but I did not remove positions close to mate. I couldn't find how to do that with pgn-extract. Maybe somebody can suggest a way to cleanup more of the positions?
I know that most engine authors have difficulty tuning more than a few values, and I'm hoping that txt will fill some of the void. I need, however, feed back of more experienced programmers
Re: txt: automated chess engine tuning
Posted: Thu Mar 19, 2015 10:09 pm
by Ferdy
brtzsnr wrote:
I removed all duplicate positions (so this should remove most of the openings), but I did not remove positions close to mate. I couldn't find how to do that with pgn-extract. Maybe somebody can suggest a way to cleanup more of the positions?
I have this window's executable where it will analyze a fen positions in a file with uci engine, say sf6 and any position with search score within a given window say [-50/+3000] will be saved. Just let the engine analyze at 50ms/position for example.
I use that window to generate my training sets. If interested I will upload it.
Re: txt: automated chess engine tuning
Posted: Thu Mar 19, 2015 11:13 pm
by brtzsnr
I wonder why you chose such an asymmetric score window [-50/+3000]. Is there a good reason behind it?
Re: txt: automated chess engine tuning
Posted: Thu Mar 19, 2015 11:52 pm
by michiguel
Ferdy wrote:brtzsnr wrote:
I removed all duplicate positions (so this should remove most of the openings), but I did not remove positions close to mate. I couldn't find how to do that with pgn-extract. Maybe somebody can suggest a way to cleanup more of the positions?
I have this window's executable where it will analyze a fen positions in a file with uci engine, say sf6 and any position with search score within a given window say [-50/+3000] will be saved. Just let the engine analyze at 50ms/position for example.
I use that window to generate my training sets. If interested I will upload it.
Gaviota uses ALL positions that are quiet (no captures in the next 6 plies of the game) from engines games. I have like 3,000,000 positions. I do not think that being selective is a good idea. The whole thing is probabilistic, so the more the merrier. It should not be only games from top engines! that would completely defeat the purpose and narrow the landscape.
Miguel
Miguel