Dann Corbit wrote: ↑Sun Oct 18, 2020 4:41 am
That is quite a leap, +34b Elo better than the previous top score
I found it surprising, but I think that the "advanced scoring Armageddon" ("ASA") scheme used here can introduce a very large bias compared to standard elo. From S. Pohl website (
https://www.sp-cc.de/nbsc-armageddon-openings.htm):
My Advanced Armageddon scoring is:
Win for white = 1 point for white
Draw = 1 point for black
Win for black = 2 points for black (!!!)
When white looses a game, it is recorded as two losses. Plus the standard Armageddon "a draw with white is a loss". It is getting far from regular chess scoring scheme. i.e., assume that Engine A and Engine B play a reference engine, Engine C (like Stockfish in this case, with elo = 0):
Engine A vs Engine C:
Engine A score as white: +1,-0,=4 --> 1/5
Engine A score as black: +2,-2,=1 --> 5/7 (the "draw as black" win, plus the two wins as black that count for four wins, so 2 extra games)
"Advanced scoring Armageddon" total --> 6/12 (elo = 0).
Standard chess total (+3,-2,=5) --> 5.5/10 (about + 40 elo).
There is a downward bias of 40 elo here.
Engine B vs Engine C:
Engine B score as white: +2,-2,=1 --> 2/7 (two extra games as white losses count twice)
Engine B score as black: +0,-0,=5 --> 5/5
"Advanced scoring Armageddon" total --> 7/12 (about + 65 elo).
Standard chess total (+2,-2,=6) --> 5/10 (elo = 0)
There is an upward bias of 65 elo here.
The gap between the two rating systems is over 100 elo in this example, and Engine B is better than Engine A with ASA, whereas in standard chess Engine A is superior. The bias can be important and unsystematic. This might explain the unexpected leap in elo.
As for J92 nets, it seems that they are generally improving. The author test them at fast time control (
https://github.com/jhorthos/lczero-trai ... a-Training ), with thousands of games. The nets are a few elo apart from each other:
Code: Select all
1 lc0.net.J92-270 : 20.4 5.1 8000 53 2132 1678 4190 52 59
2 lc0.net.J92-190 : 19.5 5.3 8000 53 2069 1635 4296 54 83
3 lc0.net.J92-210 : 15.8 5.4 8000 52 2007 1655 4338 54 67
4 lc0.net.J92B-205 : 14.2 5.3 8000 52 2026 1710 4264 53 51
5 lc0.net.J92-240 : 14.1 5.2 8000 52 1984 1670 4346 54 55
6 lc0.net.J92-180 : 13.6 5.2 8000 52 1990 1686 4324 54 56
7 lc0.net.J92-220 : 13.1 5.3 8000 52 2031 1739 4230 53 66
8 lc0.net.J92-145 : 11.5 5.4 8000 52 1963 1706 4331 54 72
9 lc0.net.J92-160 : 9.2 5.3 8000 51 1995 1789 4216 53 58
10 lc0.net.J92-130 : 8.5 5.3 8000 51 1987 1798 4215 53 60