500games tests of all SVnnue nets released so far

[pohl4711]

Here quick 500-games tests of all S.V. nets released so far versus Stockfish 200717 bmi2.
The Elo-numbers are meaningless (I chose 3300 base value for ORDO), the score in percent is important.

Code: Select all

Hert250 openings, 60''+0.5'' singlethread (average game-duration: 3 minutes). 
500 games each net vs SF 200717 bmi2
Binary: nodchip 200719 bmi2, Intel Haswell CPU, 256MB Hash

sv200730_1934          : 3328 500 (+208,=248,- 44), 66.4 %
sv200729_0912          : 3329 500 (+199,=264,- 37), 66.2 %
sv200729_1910          : 3323 500 (+201,=256,- 43), 65.8 %
sv200728_0633          : 3326 500 (+198,=257,- 45), 65.3 %
sv200801_1209          : 3333 500 (+190,=272,- 38), 65.2 %
sv200728_1104          : 3325 500 (+185,=281,- 34), 65.1 %
sv200731_0821          : 3316 500 (+192,=264,- 44), 64.8 %
sv200729_1218          : 3318 500 (+193,=262,- 45), 64.8 %
sv200801_1515          : 3329 500 (+187,=273,- 40), 64.7 %
sv200731_0429          : 3313 500 (+193,=259,- 48), 64.5 %
sv200728_1817          : 3315 500 (+188,=268,- 44), 64.4 %
sv200728_2138          : 3315 500 (+181,=281,- 38), 64.3 %
sv200729_0629          : 3314 500 (+184,=274,- 42), 64.2 %
sv200731_0631          : 3310 500 (+187,=267,- 46), 64.1 %
sv200730_2333          : 3309 500 (+179,=281,- 40), 63.9 %
sv200729_0335          : 3311 500 (+189,=261,- 50), 63.9 %
sv200727_2151          : 3315 500 (+189,=261,- 50), 63.9 %
sv200729_1743          : 3309 500 (+181,=274,- 45), 63.6 %
sv200724_0123          : 3326 500 (+175,=286,- 39), 63.6 %
sv200731_0111          : 3306 500 (+172,=291,- 37), 63.5 %
sv200729_0109          : 3308 500 (+186,=263,- 51), 63.5 %
sv200727_1540          : 3311 500 (+178,=277,- 45), 63.3 %
sv200729_1500          : 3306 500 (+183,=266,- 51), 63.2 %
sv200728_0207          : 3309 500 (+177,=276,- 47), 63.0 %
sv200722_1621exp       : 3321 500 (+174,=281,- 45), 62.9 %
sv200731_1607          : 3314 500 (+171,=285,- 44), 62.7 %
sv200722_2115exp       : 3324 500 (+168,=289,- 43), 62.5 %
sv200722_1130exp       : 3317 500 (+172,=280,- 48), 62.4 %
sv200729_2214          : 3297 500 (+168,=287,- 45), 62.3 %
sv200727_0928          : 3342 500 (+167,=289,- 44), 62.3 %
sv200723_0511          : 3322 500 (+167,=288,- 45), 62.2 %
sv200728_1442          : 3300 500 (+166,=286,- 48), 61.8 %
sv200724_0640          : 3320 500 (+165,=286,- 49), 61.6 %
sv200722_2141          : 3355 500 (+177,=262,- 61), 61.6 %
sv200731_0252          : 3291 500 (+170,=275,- 55), 61.5 %
sv200727_0332          : 3335 500 (+176,=262,- 62), 61.4 %
sv200724_0650exp       : 3318 500 (+171,=271,- 58), 61.3 %
sv200724_1240          : 3317 500 (+174,=264,- 62), 61.2 %
sv200725_1313          : 3325 500 (+167,=276,- 57), 61.0 %
sv200723_1334exp       : 3307 500 (+159,=292,- 49), 61.0 %
sv200726_0504          : 3362 500 (+176,=256,- 68), 60.8 %
sv200725_2051          : 3359 500 (+156,=292,- 52), 60.4 %
sv200723_1134          : 3311 500 (+155,=294,- 51), 60.4 %
sv200724_2244exp       : 3320 500 (+156,=291,- 53), 60.3 %
sv200726_1135          : 3357 500 (+166,=269,- 65), 60.1 %
sv200724_1215exp       : 3308 500 (+153,=294,- 53), 60.0 %
sv200723_1844          : 3300 500 (+157,=286,- 57), 60.0 %
sv200725_0545          : 3314 500 (+155,=285,- 60), 59.5 %
sv200725_2237          : 3351 500 (+152,=289,- 59), 59.3 %
SF GitHub 200725       : 3348 500 (+137,=315,- 48), 58.9 %
sv200724_1732          : 3300 500 (+159,=271,- 70), 58.9 %
sv200724_2344          : 3309 500 (+150,=288,- 62), 58.8 %
sv200724_2343          : 3308 500 (+154,=278,- 68), 58.6 %
SF Github 200728       : 3262 500 (+121,=330,- 49), 57.2 %
gk200627               : 3305 500 (+118,=311,- 71), 54.7 %
StockFiNN CCC          : 3276 500 (+115,=311,- 74), 54.1 %
xXH4CK3RXx-net-1       : 3269 500 (+ 93,=336,- 71), 52.2 %
josh 384cr             : 3269 500 (+ 87,=322,- 91), 49.6 %
Lizardfish 0.3         : 3192 500 (+ 41,=288,-171), 37.0 %
Lizardfish 0.2         : 3148 500 (+ 36,=242,-222), 31.4 %

[Dann Corbit]

Thank you for this, it is very useful.
So far as I know, you are the only one performing this experiment.

[pohl4711]

Thank you for this, it is very useful.
So far as I know, you are the only one performing this experiment.

Thank you.

I added the pre-test result-list to my website.
https://www.sp-cc.de

On the main page, in the website-news on top, you find a link:
SF nnue pre-test results can be seen here

If anybody has new, promising strong nets, please contact me (with a download-link to the net, if possible), I will do a pre-test with that net as fast as possible and add the result to that small list.

[Jouni]

SF framework has started testing now:

ELO: 81.44 +-7.1 (95%) LOS: 100.0%
Total: 5026 W: 1948 L: 791 D: 2287
Ptnml(0-2): 64, 348, 863, 843, 395

WOW! And next also NCM?

[pohl4711]

SF framework has started testing now:

ELO: 81.44 +-7.1 (95%) LOS: 100.0%
Total: 5026 W: 1948 L: 791 D: 2287
Ptnml(0-2): 64, 348, 863, 843, 395

WOW! And next also NCM?

The new default-net, used here in the SF framework, is a S.Vieri net, as far as I know. But I dont know, which net exactly. Could anybody help me here?
And, at the moment, the "official" SFnnue compiles on abrok are measureable slower, than the latest nodchip-compile. But if the nodchip-compiles will not be updated anymore (correct?), they will be outdated very soon in the future. Then we will have to use the "official" SFnnue compiles on abrok. Hopefully, they will get faster until this point.

[Jouni]

Note this is STC vs dev version. So clearly over +110 vs SF11. The net version is explained there, but not easily understand .

[pohl4711]

Note this is STC vs dev version. So clearly over +110 vs SF11. The net version is explained there, but not easily understand .

I asked S.Vieri on discord. He told me, it is the net 200801_1515. Not a good choice IMHO. One of the weakest nets of the last days of development in my tests.