The flow of Komodo eval

Discussion of anything and everything relating to chess playing software and machines.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
User avatar
Laskos
Posts: 9527
Joined: Wed Jul 26, 2006 8:21 pm
Full name: Kai Laskos

The flow of Komodo eval

Post by Laskos » Thu May 19, 2016 5:38 am

With the help of a tool written by Ferdinand Mosca, I plotted the evolution of eval of Komodo eval starting with positions at certain move number, tracked for 10 moves (not more, because they disperse too much for longer series of moves and the average becomes a bit meaningless). The analyzed games were 3000 Komodo self-games at 240''+2.4''. To read this plot: say a worm-like line begins at move 20 with an eval of 1.17. It is expected the eval reaches a value of 1.56 at move 30, and all intermediate values are plotted too (well, the lines are worm-like because of the noise). Eval in Chess as a dynamic system has three fixed points attractors: Win, Draw and Loss.We empirically know that say an eval of above +2.00 will go towards Win pretty much independently of move number, and the eval of +0.10 at move 70 is probably a Draw. Using game statistics function by eval (another script by Ferdinand), I also colored the background in two different shades: darker one - game tends to a Draw, lighter one - game tends to a Win.

The second plot is the fitted function - vector field. The fit is not hard, as the vector field of worm-like lines is optically close to {x, y^2} with a small adjustment to account for a tendency for Draws at small evals. The arrows fit my worms pretty closely. Here too, darker region is a flow towards mostly draws, lighter one - towards mostly wins.

Image


Image

Dann Corbit
Posts: 10205
Joined: Wed Mar 08, 2006 7:57 pm
Location: Redmond, WA USA
Contact:

Re: The flow of Komodo eval

Post by Dann Corbit » Thu May 19, 2016 6:56 pm

If the score is above 70 cp on the 10th move, you are trapped in the vector field of doom.
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.

nimh
Posts: 46
Joined: Sat Nov 29, 2014 11:06 pm
Contact:

Re: The flow of Komodo eval

Post by nimh » Thu May 19, 2016 10:01 pm

Thanks, it is very interesting and informative!

I wonder if translating centipawn evals into win-draw-loss transitions is actually more trustworthy than into expected scores via a logistic function.

So, when I see a player has made a move whose eval drops from 0.96 to -1.02 on move 15th, I just enter 2 for two transitions, as a won game turned into a lost game?

Could you give the formula, so I can check it myself on my data?

User avatar
Laskos
Posts: 9527
Joined: Wed Jul 26, 2006 8:21 pm
Full name: Kai Laskos

Re: The flow of Komodo eval

Post by Laskos » Fri May 20, 2016 12:24 am

nimh wrote:Thanks, it is very interesting and informative!

I wonder if translating centipawn evals into win-draw-loss transitions is actually more trustworthy than into expected scores via a logistic function.

So, when I see a player has made a move whose eval drops from 0.96 to -1.02 on move 15th, I just enter 2 for two transitions, as a won game turned into a lost game?

Could you give the formula, so I can check it myself on my data?
Formula for vector field lines? The win/draw boundary is close to be one of them. To solve y[x] for the vector field {x, y^2} we have the simple differential equation

y'[x] * x = y[x]^2
y[move(0)] = eval(0)

getting

y[x] = eval(0) / (1 + eval(0)*log[move(0)] - eval(0)*log[x])

For example:
move(0) = 20 (we start at move 20).
eval(0) = 0.7 (we start with an eval of 0.7)

We get from the above formula eval at move 30 (or y[30]) to be equal to 0.977. But keep in mind that this is a pretty rough approximation, as I didn't keep in this solution the small additional term in the vector field needed to fit it better to my "worms", differential equation doesn't solve in closed form including it.

nimh
Posts: 46
Joined: Sat Nov 29, 2014 11:06 pm
Contact:

Re: The flow of Komodo eval

Post by nimh » Fri May 20, 2016 8:28 am

Thanks, but what I had in mind is a formula that returns 0.5 when below, and 1.0 when above the boundary.

User avatar
Laskos
Posts: 9527
Joined: Wed Jul 26, 2006 8:21 pm
Full name: Kai Laskos

Re: The flow of Komodo eval

Post by Laskos » Fri May 20, 2016 8:46 am

nimh wrote:Thanks, but what I had in mind is a formula that returns 0.5 when below, and 1.0 when above the boundary.
But that's easy to translate:

having

y[x] = eval(0) / (1 + eval(0)*log[move(0)] - eval(0)*log[x])

and picking the initial point on the boundary (say 30, 0.75), we get

y_b[x] on boundary = 0.75 / (1+0.75*log[30]-0.75*log[x])
x is the variable move number

And the needed formula is

0.5 + 0.5 * theta(y[x] - y_b[x])

JJJ
Posts: 1287
Joined: Sat Apr 19, 2014 11:47 am

Re: The flow of Komodo eval

Post by JJJ » Fri May 20, 2016 10:55 am

How strong are these value against his rival and slighty better Stockfish ?

User avatar
Laskos
Posts: 9527
Joined: Wed Jul 26, 2006 8:21 pm
Full name: Kai Laskos

Re: The flow of Komodo eval

Post by Laskos » Fri May 20, 2016 11:16 am

Laskos wrote:
nimh wrote:Thanks, but what I had in mind is a formula that returns 0.5 when below, and 1.0 when above the boundary.
But that's easy to translate:

having

y[x] = eval(0) / (1 + eval(0)*log[move(0)] - eval(0)*log[x])

and picking the initial point on the boundary (say 30, 0.75), we get

y_b[x] on boundary = 0.75 / (1+0.75*log[30]-0.75*log[x])
x is the variable move number

And the needed formula is

0.5 + 0.5 * theta(y[x] - y_b[x])
I managed a good fit solvable in closed form. Here is how the fit to my worms looks like, only the small eval values need a correction, but introducing it makes the differential equation unsolvable (well, numerically it's solvable for given boundary conditions), at least by Mathematica.

Image

The vector field is:
{x, 0.1*x^(1/2)*y^2}

The differential equation for the flow lines is:
y'[x] = 0.1*x^(-1/2)*y[x]^2
y[move(0)] = eval(0)

The solution to this (better) fit is even simpler than before:
y[x] = 5*eval(0)/{5 - eval0*sqrt[x] + eval0*sqrt[move(0)]}

From this family of flow lines, I picked the a particular boundary condition for the Win/Draw border line, and the border equation is:

y_b[x] = 5*0.75/(5 + 0.75*sqrt(30) - 0.75*sqrt(x))
x is the variable move number.

You probably need the same:
f[x] = 0.5 + 0.5 * theta(y[x] - y_b[x])

Keep in mind that the border is soft, there is no phase-transition between Wins and Draw, the border is simply showing where the probability of Draw is equal to probability of Win.

User avatar
Laskos
Posts: 9527
Joined: Wed Jul 26, 2006 8:21 pm
Full name: Kai Laskos

Re: The flow of Komodo eval

Post by Laskos » Fri May 20, 2016 11:22 am

JJJ wrote:How strong are these value against his rival and slighty better Stockfish ?
If SF is sensibly better, you probably need to shift a bit the whole graph up on vertical axis, so what was before (self-games) "eval" takes say value "eval + 0.1" against SF.

nimh
Posts: 46
Joined: Sat Nov 29, 2014 11:06 pm
Contact:

Re: The flow of Komodo eval

Post by nimh » Fri May 20, 2016 5:05 pm

I've never been interested in maths, I'm weak at it. :)

Post Reply