The idea is not to train my eval on positions that are too inferior, say less than -50, it might begin to like it . If the score is too big on the other hand, there is no point in tuning anymore, it is already winning, your K may vary here. With that window, the positions there are from won games. I also don't use positions from drawn games, and don't train my eval on drawn positions either. Only black to play and win and white to play and win positions.brtzsnr wrote:I wonder why you chose such an asymmetric score window [-50/+3000]. Is there a good reason behind it?
txt: automated chess engine tuning
Moderators: hgm, Rebel, chrisw
-
- Posts: 4833
- Joined: Sun Aug 10, 2008 3:15 pm
- Location: Philippines
Re: txt: automated chess engine tuning
-
- Posts: 4833
- Joined: Sun Aug 10, 2008 3:15 pm
- Location: Philippines
Re: txt: automated chess engine tuning
Impressive quiet criteria, I will try to collect something like that too .michiguel wrote:Gaviota uses ALL positions that are quiet (no captures in the next 6 plies of the game) from engines games. I have like 3,000,000 positions. I do not think that being selective is a good idea. The whole thing is probabilistic, so the more the merrier. It should not be only games from top engines! that would completely defeat the purpose and narrow the landscape.Ferdy wrote:I have this window's executable where it will analyze a fen positions in a file with uci engine, say sf6 and any position with search score within a given window say [-50/+3000] will be saved. Just let the engine analyze at 50ms/position for example.brtzsnr wrote: I removed all duplicate positions (so this should remove most of the openings), but I did not remove positions close to mate. I couldn't find how to do that with pgn-extract. Maybe somebody can suggest a way to cleanup more of the positions?
I use that window to generate my training sets. If interested I will upload it.
Miguel
Miguel
-
- Posts: 433
- Joined: Fri Jan 16, 2015 4:02 pm
Re: txt: automated chess engine tuning
Why do you search only _very_ quiet moves?michiguel wrote: Gaviota uses ALL positions that are quiet (no captures in the next 6 plies of the game) from engines games. I have like 3,000,000 positions. I do not think that being selective is a good idea. The whole thing is probabilistic, so the more the merrier. It should not be only games from top engines! that would completely defeat the purpose and narrow the landscape.
-
- Posts: 937
- Joined: Fri Mar 10, 2006 4:29 pm
- Location: Germany
Re: txt: automated chess engine tuning
Very interesting.brtzsnr wrote:Hi, all!
I wrote a small framework to do automated chess tuning based on Texel's Tuning method (https://chessprogramming.wikispaces.com ... ing+Method). You can find the source code and instructions how to use it here https://bitbucket.org/brtzsnr/txt.
I'm still experimenting with it, so I cannot yet report any success. Nevertheless, feel free to experiment with txt and if you find it useful, please consider contributing.
Regards,
How does a epd line need to look like?
I already have a huge bunch of fens but obviously not with the correct syntax.
P. S. extract.sh only gives errors.
Jörg Oster
-
- Posts: 6401
- Joined: Thu Mar 09, 2006 8:30 pm
- Location: Chicago, Illinois, USA
Re: txt: automated chess engine tuning
Because the Gaviota system uses eval, not qsearch. It is not quiet moves, it is quiet positions, in which you are not caught in the middle of a recapture.brtzsnr wrote:Why do you search only _very_ quiet moves?michiguel wrote: Gaviota uses ALL positions that are quiet (no captures in the next 6 plies of the game) from engines games. I have like 3,000,000 positions. I do not think that being selective is a good idea. The whole thing is probabilistic, so the more the merrier. It should not be only games from top engines! that would completely defeat the purpose and narrow the landscape.
Miguel
-
- Posts: 6401
- Joined: Thu Mar 09, 2006 8:30 pm
- Location: Chicago, Illinois, USA
Re: txt: automated chess engine tuning
I do not restrict it. Even if winning, Gaviota needs to know it is winning by far, so it will score it heavily. It needs to know the value of a Queen advantage, for instance.Ferdy wrote:The idea is not to train my eval on positions that are too inferior, say less than -50, it might begin to like it . If the score is too big on the other hand, there is no point in tuning anymore, it is already winning, your K may vary here. With that window, the positions there are from won games. I also don't use positions from drawn games, and don't train my eval on drawn positions either. Only black to play and win and white to play and win positions.brtzsnr wrote:I wonder why you chose such an asymmetric score window [-50/+3000]. Is there a good reason behind it?
Miguel
-
- Posts: 433
- Joined: Fri Jan 16, 2015 4:02 pm
Re: txt: automated chess engine tuning
I'll post an epd line when I get home.Joerg Oster wrote:P. S. extract.sh only gives errors.
For the errors please file a bug with a sample .pgn and include the error message too. I know that pgn-extract complains for cutechess produced .pgn files because the FENs are incorrect because they lack halfmove clock and fullmove number. Nevertheless, epds are still produced.
-
- Posts: 433
- Joined: Fri Jan 16, 2015 4:02 pm
Re: txt: automated chess engine tuning
Why only evaluate on quiet position? Is it to remove the effect of search?michiguel wrote:Because the Gaviota system uses eval, not qsearch. It is not quiet moves, it is quiet positions, in which you are not caught in the middle of a recapture.brtzsnr wrote:Why do you search only _very_ quiet moves?michiguel wrote: Gaviota uses ALL positions that are quiet (no captures in the next 6 plies of the game) from engines games. I have like 3,000,000 positions. I do not think that being selective is a good idea. The whole thing is probabilistic, so the more the merrier. It should not be only games from top engines! that would completely defeat the purpose and narrow the landscape.
Miguel
-
- Posts: 6401
- Joined: Thu Mar 09, 2006 8:30 pm
- Location: Chicago, Illinois, USA
Re: txt: automated chess engine tuning
The static evaluation is only valid for a quiescent position by definition. If you evaluate the position statically after e4 e5/ Nf3 Nc6/ Bb5 a6/ Bxc6 you will get that black is up by a piece, which is highly inaccurate.brtzsnr wrote:Why only evaluate on quiet position? Is it to remove the effect of search?michiguel wrote:Because the Gaviota system uses eval, not qsearch. It is not quiet moves, it is quiet positions, in which you are not caught in the middle of a recapture.brtzsnr wrote:Why do you search only _very_ quiet moves?michiguel wrote: Gaviota uses ALL positions that are quiet (no captures in the next 6 plies of the game) from engines games. I have like 3,000,000 positions. I do not think that being selective is a good idea. The whole thing is probabilistic, so the more the merrier. It should not be only games from top engines! that would completely defeat the purpose and narrow the landscape.
Miguel
Miguel
PS: Texel system uses qsearch, then quiet positions are not needed. Gaviota system uses eval, then quiet positions are needed.
-
- Posts: 931
- Joined: Tue Mar 09, 2010 3:46 pm
- Location: New York
- Full name: Álvaro Begué (RuyDos)
Re: txt: automated chess engine tuning
[Insignificant correction: You probably meant "white is up by a piece".]michiguel wrote:The static evaluation is only valid for a quiescent position by definition. If you evaluate the position statically after e4 e5/ Nf3 Nc6/ Bb5 a6/ Bxc6 you will get that black is up by a piece, which is highly inaccurate.brtzsnr wrote:Why only evaluate on quiet position? Is it to remove the effect of search?michiguel wrote:Because the Gaviota system uses eval, not qsearch. It is not quiet moves, it is quiet positions, in which you are not caught in the middle of a recapture.brtzsnr wrote:Why do you search only _very_ quiet moves?michiguel wrote: Gaviota uses ALL positions that are quiet (no captures in the next 6 plies of the game) from engines games. I have like 3,000,000 positions. I do not think that being selective is a good idea. The whole thing is probabilistic, so the more the merrier. It should not be only games from top engines! that would completely defeat the purpose and narrow the landscape.
Miguel
Miguel
PS: Texel system uses qsearch, then quiet positions are not needed. Gaviota system uses eval, then quiet positions are needed.
One possible problem with requiring no captures in the next six plies is that it will not work for some of the more tactical terms in the evaluation function, like pins or hanging pieces, because those are generally resolved quickly and the resolution involves captures. This would also be a problem for king safety evaluation, since strong attacks rarely have six plies without captures.