JChess wrote:Hello, very very interesting games. You are to be commended for your hard work on this project. Is there any possibility that you will make this latest version of your engine available for download on your web site?
Thank you for your time and keep up the great work!!
I will release the current version very soon. I just need to organize the release, make sure it compiles well in windows (I am currently developing it in LInux) and put in the web page. I removed a bug that was annoying me last week. I think I reached a point where a release may be worth it. It is apparently ~200 elo points stronger than the previous version and I think it has an interesting style.
michiguel wrote:I have ~6 million of relatively quiescent positions from comp-comp games. Of course, I know the outcome of those games and the result from the perspective of the "side to move" (0, 0.5, or 1). Just for this experiment, I modified the evaluation of gaviota to give me probability to win for the side to move (from 0 to 1). Then, I fit the eval parameters to minimize the difference between the probability predicted and the real result. This was the intermediate step to test other ideas. For instance, I intend to refit the parameters to the score that Gaviota will obtain after a short search with the parameters that I just got (and more iterations could follow).
There's a bit of a reinforcement learning flavour to what you are doing. When you minimize the difference between the probability predicted and the real result, it may make sense to penalize early (but non-book) positions less than positions closer to the end of the game.
michiguel wrote:Still, with this method I got back some parameters that look fishy (70 cp for bishop pair seems too high).
I see what you mean, though they frequently can be worth half a pawn in the middlegame, and may be decisive in the endgame.
michiguel wrote:I tested in the past other things but they did not work. For instance, I tried to fit it to the evaluation given by stronger engines like Crafty and Yace (which have a command "score" and are stronger than Gaviota); but the parameters obtained were not good at all. Maybe the idea was flawed, maybe I did not have enough positions (~30 k). I don't know.
Perhaps your engine does not have a superset of the concepts of these engines, or your engine interprets the occurance of some features differently from these engines? Perhaps the 30k positions did not give balanced coverage of the domain space?
Dave
Most likely. It is difficult to fit elaborated king safety evaluations with primitive ones.
I also observed that there were positions in which the evaluations from YACE and Crafty were *VERY* different. Some positions one engine gave +7.00 and the other +3.00. etc. Those positions may have poluted the set.
JChess wrote:Hello, very very interesting games. You are to be commended for your hard work on this project. Is there any possibility that you will make this latest version of your engine available for download on your web site?
Thank you for your time and keep up the great work!!
I will release the current version very soon. I just need to organize the release, make sure it compiles well in windows (I am currently developing it in LInux) and put in the web page. I removed a bug that was annoying me last week. I think I reached a point where a release may be worth it. It is apparently ~200 elo points stronger than the previous version and I think it has an interesting style.
Miguel
Great news Miguel
Waiting eagerly regards,
Dr.D
_No one can hit as hard as life.But it ain’t about how hard you can hit.It’s about how hard you can get hit and keep moving forward.How much you can take and keep moving forward….
michiguel wrote:I also observed that there were positions in which the evaluations from YACE and Crafty were *VERY* different. Some positions one engine gave +7.00 and the other +3.00. etc. Those positions may have poluted the set.
michiguel wrote:I also observed that there were positions in which the evaluations from YACE and Crafty were *VERY* different. Some positions one engine gave +7.00 and the other +3.00. etc. Those positions may have poluted the set.
Miguel
Those evaluations are almost identical.
Dave
Yes, if I treat them as I do now as "probability to win" (both will get relatively closer values). But if I fit the evaluation score in centipawns, those positions will end up having a stronger weight than roughly equal positions.
In regression, one of the assumptions is that the "variability" or "error"of all the data points are the same. At least in practice, they should be close. In our case, with positions that are +7.00 pawns, the program will try to fit it with the same error than positions that are +0.10 pawns. But, I will prefer parameters that will give me an evaluation that simulate the score of those two positions with values +5.00 and +0.11 rather than one that gives me +6.70 and +0.40. Those type of positions may be few, but they will have a strong influence in polluting the set. If the error or variability they have is 10 times higher than the regular positions, their influence will be 100-fold stronger than they should (when we try to minimize the square of the differences between the data and the fitting equation). I do not think that the problem is related to chess values, but related to how the assumptions for regression work. There are ways to prevent this (giving weights according to different things) but I have not tried. Using probability to win should make this problem much less problematic, IMHO.
michiguel wrote:I also observed that there were positions in which the evaluations from YACE and Crafty were *VERY* different. Some positions one engine gave +7.00 and the other +3.00. etc. Those positions may have poluted the set.
Miguel
Those evaluations are almost identical.
Dave
Yes, if I treat them as I do now as "probability to win" (both will get relatively closer values). But if I fit the evaluation score in centipawns, those positions will end up having a stronger weight than roughly equal positions.
In regression, one of the assumptions is that the "variability" or "error"of all the data points are the same. At least in practice, they should be close. In our case, with positions that are +7.00 pawns, the program will try to fit it with the same error than positions that are +0.10 pawns. But, I will prefer parameters that will give me an evaluation that simulate the score of those two positions with values +5.00 and +0.11 rather than one that gives me +6.70 and +0.40. Those type of positions may be few, but they will have a strong influence in polluting the set. If the error or variability they have is 10 times higher than the regular positions, their influence will be 100-fold stronger than they should (when we try to minimize the square of the differences between the data and the fitting equation). I do not think that the problem is related to chess values, but related to how the assumptions for regression work. There are ways to prevent this (giving weights according to different things) but I have not tried. Using probability to win should make this problem much less problematic, IMHO.
Miguel
You could use measure concordance ordinally instead of using linear regression. ;-) (Of course, manipulating probabilities is also possible.)
