Super bullet chess: LC0 network 40445 (4000 selfplay elo) vs Stockfish 8

[Hai]

Super bullet chess: LC0 network 40445 (4000 selfplay elo) vs Stockfish 8

The 20 possible opening moves
40 games
Ponder on
LC0 with RTX 2080 Ti 32 kn/s
Stockfish with 4 cores @ 4.2 ghz
Time control: 1 min per game + 1 second per move.

Result:
LC0 21.5 points
Stockfish 18.5 points
+13 =17 -10
Winning percentage = 53.75%

= LC0 is 26 elo stronger than Stockfish.

http://www.mediafire.com/file/es5ad1kog ... 8.pgn/file

[yanquis1972]

...with a serious hardware advantage & in 40 games

[Nay Lin Tun]

...with a serious hardware advantage & in 40 games

If you check the progress of Leela, people usually test with lower rated AB engines(or intentionally make them weaker) and then gradually test against stronger engines!
Why?
If you play these new nets vs full strength SF dev, SF may win 75-80% of the game. But that is not the point, the point is "it is easier to notice the progress when you compare milestones with known rated similar strength". Just an extreme example, If you ask "Carlsen to play vs SF, SF will beat 99% of games, and if you ask Naka to play against SF, SF may still beat 99% games, then it become more difficult to see who is better between Carlsen and Naka. But if you ask them to play against 2800 rated engine, you may see the difference.

[yanquis1972]

yes, but my point is there's no point of comparison. it's +26 elo in that sample, with that particular configuration. what's test10? test30? it's clearly an excellent result for a young net but to say test40 is already ahead of SF8 (in comparison to any previous benchmark) is simply not true.

[Nay Lin Tun]

yes, but my point is there's no point of comparison. it's +26 elo in that sample, with that particular configuration. what's test10? test30? it's clearly an excellent result for a young net but to say test40 is already ahead of SF8 (in comparison to any previous benchmark) is simply not true.

As everyone has different hardware setting, there is no realistic elo comparison .
E.g, In TCEC, SF could probably be +70 above leela but in cccc SF could be +30. And in his home hardware, it may be the reverse!
However, there is Leela ratio (comparing speed of SF vs Leela) to standardize hardware benefits. (Tcec leela ratio is about 0.6 and ccc leela ratio is about 1.0, Leela ratio= speed of leela in testing vs AB engines in comparison with SF 8 vs A0 match)