With the help of a tool written by Ferdinand Mosca, I plotted the evolution of eval of Komodo eval starting with positions at certain move number, tracked for 10 moves (not more, because they disperse too much for longer series of moves and the average becomes a bit meaningless). The analyzed games were 3000 Komodo self-games at 240''+2.4''. To read this plot: say a worm-like line begins at move 20 with an eval of 1.17. It is expected the eval reaches a value of 1.56 at move 30, and all intermediate values are plotted too (well, the lines are worm-like because of the noise). Eval in Chess as a dynamic system has three fixed points attractors: Win, Draw and Loss.We empirically know that say an eval of above +2.00 will go towards Win pretty much independently of move number, and the eval of +0.10 at move 70 is probably a Draw. Using game statistics function by eval (another script by Ferdinand), I also colored the background in two different shades: darker one - game tends to a Draw, lighter one - game tends to a Win.
The second plot is the fitted function - vector field. The fit is not hard, as the vector field of worm-like lines is optically close to {x, y^2} with a small adjustment to account for a tendency for Draws at small evals. The arrows fit my worms pretty closely. Here too, darker region is a flow towards mostly draws, lighter one - towards mostly wins.
The flow of Komodo eval
Moderators: hgm, Rebel, chrisw
-
- Posts: 10948
- Joined: Wed Jul 26, 2006 10:21 pm
- Full name: Kai Laskos
-
- Posts: 12542
- Joined: Wed Mar 08, 2006 8:57 pm
- Location: Redmond, WA USA
Re: The flow of Komodo eval
If the score is above 70 cp on the 10th move, you are trapped in the vector field of doom.
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.
-
- Posts: 46
- Joined: Sun Nov 30, 2014 12:06 am
Re: The flow of Komodo eval
Thanks, it is very interesting and informative!
I wonder if translating centipawn evals into win-draw-loss transitions is actually more trustworthy than into expected scores via a logistic function.
So, when I see a player has made a move whose eval drops from 0.96 to -1.02 on move 15th, I just enter 2 for two transitions, as a won game turned into a lost game?
Could you give the formula, so I can check it myself on my data?
I wonder if translating centipawn evals into win-draw-loss transitions is actually more trustworthy than into expected scores via a logistic function.
So, when I see a player has made a move whose eval drops from 0.96 to -1.02 on move 15th, I just enter 2 for two transitions, as a won game turned into a lost game?
Could you give the formula, so I can check it myself on my data?
-
- Posts: 10948
- Joined: Wed Jul 26, 2006 10:21 pm
- Full name: Kai Laskos
Re: The flow of Komodo eval
Formula for vector field lines? The win/draw boundary is close to be one of them. To solve y[x] for the vector field {x, y^2} we have the simple differential equationnimh wrote:Thanks, it is very interesting and informative!
I wonder if translating centipawn evals into win-draw-loss transitions is actually more trustworthy than into expected scores via a logistic function.
So, when I see a player has made a move whose eval drops from 0.96 to -1.02 on move 15th, I just enter 2 for two transitions, as a won game turned into a lost game?
Could you give the formula, so I can check it myself on my data?
y'[x] * x = y[x]^2
y[move(0)] = eval(0)
getting
y[x] = eval(0) / (1 + eval(0)*log[move(0)] - eval(0)*log[x])
For example:
move(0) = 20 (we start at move 20).
eval(0) = 0.7 (we start with an eval of 0.7)
We get from the above formula eval at move 30 (or y[30]) to be equal to 0.977. But keep in mind that this is a pretty rough approximation, as I didn't keep in this solution the small additional term in the vector field needed to fit it better to my "worms", differential equation doesn't solve in closed form including it.
-
- Posts: 46
- Joined: Sun Nov 30, 2014 12:06 am
Re: The flow of Komodo eval
Thanks, but what I had in mind is a formula that returns 0.5 when below, and 1.0 when above the boundary.
-
- Posts: 10948
- Joined: Wed Jul 26, 2006 10:21 pm
- Full name: Kai Laskos
Re: The flow of Komodo eval
But that's easy to translate:nimh wrote:Thanks, but what I had in mind is a formula that returns 0.5 when below, and 1.0 when above the boundary.
having
y[x] = eval(0) / (1 + eval(0)*log[move(0)] - eval(0)*log[x])
and picking the initial point on the boundary (say 30, 0.75), we get
y_b[x] on boundary = 0.75 / (1+0.75*log[30]-0.75*log[x])
x is the variable move number
And the needed formula is
0.5 + 0.5 * theta(y[x] - y_b[x])
-
- Posts: 1346
- Joined: Sat Apr 19, 2014 1:47 pm
Re: The flow of Komodo eval
How strong are these value against his rival and slighty better Stockfish ?
-
- Posts: 10948
- Joined: Wed Jul 26, 2006 10:21 pm
- Full name: Kai Laskos
Re: The flow of Komodo eval
I managed a good fit solvable in closed form. Here is how the fit to my worms looks like, only the small eval values need a correction, but introducing it makes the differential equation unsolvable (well, numerically it's solvable for given boundary conditions), at least by Mathematica.Laskos wrote:But that's easy to translate:nimh wrote:Thanks, but what I had in mind is a formula that returns 0.5 when below, and 1.0 when above the boundary.
having
y[x] = eval(0) / (1 + eval(0)*log[move(0)] - eval(0)*log[x])
and picking the initial point on the boundary (say 30, 0.75), we get
y_b[x] on boundary = 0.75 / (1+0.75*log[30]-0.75*log[x])
x is the variable move number
And the needed formula is
0.5 + 0.5 * theta(y[x] - y_b[x])
The vector field is:
{x, 0.1*x^(1/2)*y^2}
The differential equation for the flow lines is:
y'[x] = 0.1*x^(-1/2)*y[x]^2
y[move(0)] = eval(0)
The solution to this (better) fit is even simpler than before:
y[x] = 5*eval(0)/{5 - eval0*sqrt[x] + eval0*sqrt[move(0)]}
From this family of flow lines, I picked the a particular boundary condition for the Win/Draw border line, and the border equation is:
y_b[x] = 5*0.75/(5 + 0.75*sqrt(30) - 0.75*sqrt(x))
x is the variable move number.
You probably need the same:
f[x] = 0.5 + 0.5 * theta(y[x] - y_b[x])
Keep in mind that the border is soft, there is no phase-transition between Wins and Draw, the border is simply showing where the probability of Draw is equal to probability of Win.
-
- Posts: 10948
- Joined: Wed Jul 26, 2006 10:21 pm
- Full name: Kai Laskos
Re: The flow of Komodo eval
If SF is sensibly better, you probably need to shift a bit the whole graph up on vertical axis, so what was before (self-games) "eval" takes say value "eval + 0.1" against SF.JJJ wrote:How strong are these value against his rival and slighty better Stockfish ?
-
- Posts: 46
- Joined: Sun Nov 30, 2014 12:06 am
Re: The flow of Komodo eval
I've never been interested in maths, I'm weak at it.