The future of chess and elo ratings

[Michel]

can you repeat for drawelo=240?

I did. Balanced is still best according the model I am using although the difference is small.

When I have time I will redo the computation for Davidson. Do you know what is a sensible value for d/sqrt(w*l) ? I have no experience with Davidson.

[Laskos]

can you repeat for drawelo=240?

I did. Balanced is still best according the model I am using although the difference is small.

When I have time I will redo the computation for Davidson. Do you know what is a sensible value for d/sqrt(w*l) ? I have no experience with Davidson.

Say 6 would be fine. I am not sure about the model, you seem to use drawelo for both draw model and unbalanced positions model, and in high values of drawelo, where the model becomes dubious.

[Michel]

can you repeat for drawelo=240?

I did. Balanced is still best according the model I am using although the difference is small.

When I have time I will redo the computation for Davidson. Do you know what is a sensible value for d/sqrt(w*l) ? I have no experience with Davidson.

Say 6 would be fine. I am not sure about the model, you seem to use drawelo for both draw model and unbalanced positions model, and in high values of drawelo, where the model becomes dubious.

Thanks! I'll try 6 then.

I am just using the standard BE model. The "Advantage" parameter which is supposed to measure the white advantage in the opening position is now used to measure the unbalancedness of the positions that are being used.

[Michel]

With Davidson it doesn't seem to make any difference. The resolution of balanced versus unbalanced is indistinguishable.

Code: Select all

bb=log(10)/400
theta=6
var("un")
L(x)=1/(1+exp(-bb*x))
w1_=L(x+un)
l1_=L(-x-un)
d1_=theta*sqrt(w1_*l1_)
s1=w1_+l1_+d1_
w1=w1_/s1
l1=l1_/s1

w2_=L(x-un)
l2_=L(-x+un)
d2_=theta*sqrt(w2_*l2_)
s2=w2_+l2_+d2_
w2=w2_/s2
l2=l2_/s2

avw=(w1+w2)/2
avl=(l1+l2)/2
d=avw-avl
s=sqrt(avw*(1-avw)+avl*(1-avl)+2*avl*avw)
res=d/s
A=plot(res(un=0),[x,-200,200],color="blue")
B=plot(res(un=50),[x,-200,200],color="yellow")
C=plot(res(un=100),[x,-200,200],color="red")
D=plot(res(un=200),[x,-200,200],color="green")
show(A+B+C+D)

[Laskos]

With Davidson it doesn't seem to make any difference. The resolution of balanced versus unbalanced is indistinguishable.

Code: Select all

bb=log(10)/400
theta=6
var("un")
L(x)=1/(1+exp(-bb*x))
w1_=L(x+un)
l1_=L(-x-un)
d1_=theta*sqrt(w1_*l1_)
s1=w1_+l1_+d1_
w1=w1_/s1
l1=l1_/s1

w2_=L(x-un)
l2_=L(-x+un)
d2_=theta*sqrt(w2_*l2_)
s2=w2_+l2_+d2_
w2=w2_/s2
l2=l2_/s2

avw=(w1+w2)/2
avl=(l1+l2)/2
d=avw-avl
s=sqrt(avw*(1-avw)+avl*(1-avl)+2*avl*avw)
res=d/s
A=plot(res(un=0),[x,-200,200],color="blue")
B=plot(res(un=50),[x,-200,200],color="yellow")
C=plot(res(un=100),[x,-200,200],color="red")
D=plot(res(un=200),[x,-200,200],color="green")
show(A+B+C+D)

Thanks!
Even if I trust more Davidson and even if it is correctly describing the draw model, the opening positions model is still drawelo based here. As these things seem to be model dependent, I took a database of 30,000 games of closely related recent Stockfishes, and performed manipulations with the python tool provided several months ago by Ferdinand. The initial openings are from 2moves_v2.pgn used in SF testing framework.

Code: Select all

Balanced openings:

Summary:
                 players      min      max     Gcnt     Wcnt     Lcnt     Dcnt
   perf
              Stockfish1     0.00     0.40    25819     6080     5392    14347
 51.33%
              Stockfish2     0.00     0.40    26283     6956     4981    14346
 53.76%

Total real games: 30000
Eval window: 0.00 to 0.40
Divisor: 1
Move range: 6 to 8
Elpased time: 0.64m



Unbalanced openings:

Summary:
                 players      min      max     Gcnt     Wcnt     Lcnt     Dcnt
   perf
              Stockfish1     1.20     1.40      957      501      121      335
 69.85%
              Stockfish2     1.20     1.40     1174      670       97      407
 74.40%

Total real games: 30000
Eval window: 1.20 to 1.40
Divisor: 1
Move range: 6 to 8
Elpased time: 0.63m
---------------------------------------------------------------------------------


Balanced: w-l = 2.43%
sigma = sqrt(w*(1-w)+l*(1-l)+2*w*l) = 0.668
(w-l)/sigma = 3.64


Unbalanced: w-l = 4.55% 
sigma = sqrt(w*(1-w)+l*(1-l)+2*w*l) = 0.675
(w-l)/sigma = 6.74

Empiric result shows a clear improvement in (w-l)/sigma of _unbalanced_ positions compared to balanced ones (6.74 versus 3.64).

[Michel]

I can confirm your findings although I do not quite understand your numbers. Perhaps I made an arithmetic mistake since I did it quickly before going to work. Here is what I get (with hopefully self explanatory notations)

Code: Select all

W1=11061
D1=28693
L1=12348
N1=52102
w1=W1/N1=0.212295113431346
l1=L1/N1=0.236996660396914
w1-l1=-0.0247015469655675
sigma(w1-l1)=sqrt((1-w1)*w1+(1-l1)*l1+2*w1*l1)=0.669837000624605
(w1-l1)/sigma=-0.0368769520682405

W2=598
D2=742
L2=791
N2=2131
w2=W2/N2=0.280619427498827
l2=L2/N2=0.371187236039418
w2-l2=-0.0905678085405913
sigma(w2-l2)=0.802249422308548
(w2-l2)/sigma=-0.112892332511719

At least my computation also suggests the second option is better (I checked that the difference is actually significant).

So it seems that assigning elo to a position is incorrect. This is weird since what else could you do?

[Michel]

Actually now I must confess I do not understand your data. I assumed that in the matrix

Code: Select all

   players      min      max     Gcnt     Wcnt     Lcnt     Dcnt 
Stockfish1     0.00     0.40    25819     6080     5392    14347
Stockfish2     0.00     0.40    26283     6956     4981    14346 

you were giving the games where the bias was respectively in [0.00,0.40] and [-0.40,0.00] for SF1. But now I see that this is impossible since the total number of games is only 30,000.

However if it concerns the games where the bias is in the interval [-0.40,0.40] then we should have wins(SF1)=losses(SF2) which is not satisfied.

Can you clarify?

[Laskos]

Actually now I must confess I do not understand your data. I assumed that in the matrix

Code: Select all

   players      min      max     Gcnt     Wcnt     Lcnt     Dcnt 
Stockfish1     0.00     0.40    25819     6080     5392    14347
Stockfish2     0.00     0.40    26283     6956     4981    14346 

you were giving the games where the bias was respectively in [0.00,0.40] and [-0.40,0.00] for SF1. But now I see that this is impossible since the total number of games is only 30,000.

However if it concerns the games where the bias is in the interval [-0.40,0.40] then we should have wins(SF1)=losses(SF2) which is not satisfied.

Can you clarify?

Actually, Gcnt shows how many times a position with SF1 eval (respectively SF2 eval) in the interval [0.00,0.40] occurred in 30,000 "real" games between moves [6,8]. If the eval of [0.00,0.40] occurs at each of the moves 6,7,8 of a game, it will count inside Gcnt all 3 times as 3 independent positions. Evals of [-0.40,00] do not appear at all. Then, for each picked position inside Gcnt it assigns the result of that game where this position occurred.

[Michel]

Evals of [-0.40,00] do not appear at all.

Ok. I understand. I was assuming the eval was provided by a referee engine to avoid bias. But what you are doing might be ok.

But if you are really talking consecutive positions from the same game and counting them as different games then there are too may unknowns I think to do a valid statistical analysis. For one thing the computation I did to check that the result is significant would no longer be valid.

The (theoretical) statistical analysis I did is for an eng1-eng2 match with the unbalancedness of the end of book position being +-A with equal probability. (*)

If we want to refute the theoretical result we should do it under those conditions.

(*) The reason for these conditions is that the analysis can be tested empirically without reference to any elo model as the outcome of the match is controlled by a trinomial distribution and under the null hypothesis we have w=l. +-A is not so important but care should be taking that the average bias is zero from the POV of eng1, for example by assigning sides randomly. Replaying games with different colors is also ok (with the caveat that has already been discussed).

[jhellis3]

I don't really buy the premise of the OP.

1) Engines still have a long, long way to go yet.

2) Humans still play the game (without consulting the engines).

3) There are many, many paths to all three results.


Exhibit A: Magnus Carlsen - has made his way to World Champion and Number 1 Rating often using "sub-optimal" moves in the opening. He has stated he wants to just "play chess", and I would say thus far has proven very capable at achieving exactly that.

The only problem I can foresee is a savant with a photographic memory who is capable of memorizing billions of computer evals and is also GM strength in natural chess playing ability. That seems pretty unlikely, but even if someone like that were to come along.... fair play to them.