op12no2 wrote:This is a lot of fun. I downloaded a wodge of Stockfish games from CCRL and used Ferdinands script to generate ~65k EPDs, which reduced to ~62k unique positions. I compared the eval to my engine to find the biggest difference:-
[d]3rq3/p4r1k/bpnppnp1/1N2b1B1/1PP1PNP1/P6Q/6B1/3R1RK1 b - -
My eval is an order of magnitude less !! than SFs eval of 683 which is based largely on the white attack of the black king position. Seems a huge value.
Now doing some tuning by getting a least squares fit across these 62k positions which is both interesting and pretty quick...
One variation I tried is to calculate error this way.
if(my_eval > sf_eval) error = 0
else error = sqrt[(my_eval - sf_eval)^2]
The idea is we consider sf eval as best, if there is a score greater
than that we just consider that as as good as sf eval, making the error zero. I got a slight improvement doing this way rather than always getting
the square of the difference. Other than that I use qsearch value and compare it with the score of the move from Frank's commented pgn files.
Positions from drawn games are not included and positions with move score > 500 cp are also not included.
i did the folowing steps to post my result:
1) calculate corrective factor that minimize th error, VajoletEval= K*StockfihEval+Q, ( q is near 0, K is something between 0.9 and 1.1)
2) remove from the epd all the position where the material difference is very big (e.g. >5)
Ferdy wrote:Other than that I use qsearch value and compare it with the score of the move from Frank's commented pgn files.
I've been wondering if there is mileage in doing this with the goal of discovering a set of simple orthogonal 'phenotypes' not necessarily based on current human knowledge. e.g. more generic pawn structures rather than passed, backwards etc, which are human based and while essential to bootstrap engine strength, are not necessarily end points. Not thinking of my engine! just in general.
Put another way, the best modern engines are better than the best humans, so their chess is not necessarily 'beautiful' as measured in human terms, therefore they may benefit from new root knowledge - but is it discoverable? I guess I'm wondering if search can provide emergence that can be back-calculated to new structural knowledge; kindasortaish.
Why is Total Evaluation 0.07? Shouldn't it be between -4.62 and -5.27? And closer to -5.27 since this is an endgame position?
ETA: I just noticed the ---'s. Does that mean Stockfish is ignoring that component of the evaluation, i.e. material balance is irrelevant in this position?
I think it would be nice to print something like "evaluated by endgame knowledge", to better clarify the fact that:
* what you see in the eval breakdown is the normal eval calculation
* in the end it gets overwritten by a special KBPK eval knowledge function
Theory and practice sometimes clash. And when that happens, theory loses. Every single time.