Is e4 significantly better than d4?

[N Konstantakis]

The reason for the higher percentage of e4 is that it includes more unique and complex positions, than d4 and others which have a lot of transpositions to similar structures. In the way that the NN is trained this helped it patternize more, hence strategically explore more the d4 tree and figure how to equalize. Also the weakest area of Leela is tactics and e4 is definitely more tactical. So it makes a lot of sense that in Leela vs Leela games e4 performs better. As Leela plays imperfect chess it is inevitable the the amount of imperfection is not equally spread among openings. d4 c4 and Nf3 lead to very similar positions, while e4 gets very rare transpositions.

Now lets consider about what could be objectively better. I have a question, lets suppose that we have perfect information on 2 lines, and the first one leads to a forced draw which requires 10 perfect moves in a row from the defending side. The 2nd line on the other hand could go on for 100 moves but in every position any move of the defending side can draw except one. Can we decide on which line is objectively better? I'd say they are objectively equal. But what about vs imperfect entities like Leela and SF? Just from this information I would choose SF to defend the 1st line and Leela to defend the 2nd one.

In this way stating that e4 is objectively better is a stretch. We can safely say that the statistics indicate that e4 is the best move for Leela vs Leela games of the used NN.

[Ovyron]

Thanks for the explanation N Konstantakis. I think it supports my posture that what chess moves are best depends on the opponent, and that it makes no sense to talk about what is the best move in isolation, which is often done in analysis of positions at Talkchess (which wants to assume "best against perfect play" even though "perfect play" can't be achieved).

[Daniel Shawul]

The performance of d2d4 decreases further to 53.60% (while that previous of e2e4 increased to 55.11% in the previous very long search). To observe that all these values are VERY slowly moving, there are no any jumps like those with regular AB engines. So, e4 performs about 40% better than d4, taking the draw as the baseline. All other opening moves are worse than both. The difference IS significant.

Daniel pointed to the statistic of outcomes being more useful than an eval from the root with hundreds of millions of nodes, as shown above in the opening post. I performed this statistic of outcomes of Lc0 net 42850 at 440 nodes versus Lc0 net 42850 at 400 nodes, to have some engine "diversity". The games were from the 5 most promising White 1-ply openings. The noise in the opening was the following:

Code: Select all

      {                                 
        "name" : "Temperature",          
        "value" : "1.00"          
      },
      {                                 
        "name" : "TempDecayMoves",          
        "value" : "20"          
      },
      {                                 
        "name" : "TempCutoffMove",  
        "value" : 10          
      }                                   

The outcomes do not depend much on the shape of the noise, but this decreasing temperature seemed the best option.

2*16,000 + 3*8,000 = 56,000 games were played. The level of play is somewhere GM level at tournament time control. The statistic of outcomes for these 5 openings is the following:

Openings_5.jpg


The results for d4 and e4 for the statistic of outcomes is remarkably similar to the analysis to hundreds of millions of nodes using a different net quoted above. Not that stretched and only a 30% difference between d4 and e4, but within error margins of the long analysis result.

I consider this statistic of outcomes more relevant than that of human games outcomes, where preparation counts a lot, as well as psychological factors as "surprise" and such (for example, "left-handedness", as explained in another post). And the results come somehow naturally, unlike the results in databases of human games, which look a bit weird.

I am agnostic of the opening theory, but I am not entirely sure it's very relevant. What I see here, or it seems to me, is that in GM level play, aside subjective factors (preparation, psychology, tricks), objectively and "inhumanely", e4 is the best opening for White.

Thanks for running the test! Since a NN evaluation is much more powerful than a hand-written eval and can capture result of simulation games better, i think searching from the root position with mcts or not is a viable approach. However, with hand-written eval() that was not trained from multiple self play games and given the limited number of eval parameters, I suspect the simulation method to capture long term effects more than root search method.

Daniel