Sergio Vieri second net is out

[Laskos]

2 more nets, no progress, 59% apparently is a hard nut to crack.

Code: Select all

https://github.com/nodchip/Stockfish
https://www.comp.nus.edu.sg/~sergio-v/nnue/

Stockfish 11        Nodes/second   : 1.664.010
SF-NNUE (popcount)  Nodes/second   :   796.674

SF-NNUE (popcount) vs Stockfish 11
henk-2706     54.7%   2020-07-19
sergio-1432   56.1%   2020-07-21 15:56
sergio-1907   58.9%   2020-07-21 19:14
sergio-2323   58.4%   2020-07-21 23:23
sergio-0359   58.8%   2020-07-22 03:59
sergio-0944   58.9%   2020-07-22 09:44
sergio-1153   58.3%   2020-07-22 11:53

Here I have the cumulative improvement of Sergio's nets after 6 nets:

6s + 0.1s

Code: Select all

Score of Sergio_22.07_0944 vs Sergio_20.07_1017: 1399 - 1137 - 1464  [0.533] 4000

...      Sergio_22.07_0944 playing White: 737 - 528 - 735  [0.552] 2000
...      Sergio_22.07_0944 playing Black: 662 - 609 - 729  [0.513] 2000
...      White vs Black: 1346 - 1190 - 1464  [0.519] 4000

Elo difference: 22.8 +/- 8.6, LOS: 100.0 %, DrawRatio: 36.6 %
Finished match

I get about 4 Elo points per net, if the improvement is linear (it is probably not).

Two more nets and the difference is now above 30 Elo points since the first net.

Code: Select all

Score of Sergio_22.07_2141 vs Sergio_20.07_1017: 1434 - 1064 - 1502  [0.546] 4000

...      Sergio_22.07_2141 playing White: 771 - 489 - 740  [0.571] 2000
...      Sergio_22.07_2141 playing Black: 663 - 575 - 762  [0.522] 2000
...      White vs Black: 1346 - 1152 - 1502  [0.524] 4000

Elo difference: 32.2 +/- 8.5, LOS: 100.0 %, DrawRatio: 37.5 %
Finished match

Seems to never stop improving, about 4 Elo points every net.

[MikeB]

+1 and a shoutout to Sergio Vieri, make it a plus +1000!

You are doing great work Sergio

We are now picking up about 8 Elo every 12-15 hours or in self-play - that is mind boggling

Code: Select all

Score of Stockfish-XI-NN 0722-2141 vs Stockfish-XI-NN 0722-0944: 327 - 297 - 553 [0.513]
Elo difference: 8.9 +/- 14.4, LOS: 88.5 %, DrawRatio: 47.0 %

1177 of 6000 games finished.

[MikeB]

+1 and a shoutout to Sergio Vieri, make it a plus +1000!

You are doing great work Sergio

We are now picking up about 8 Elo every 12-15 hours or in self-play - that is mind boggling

Code: Select all

Score of Stockfish-XI-NN 0722-2141 vs Stockfish-XI-NN 0722-0944: 327 - 297 - 553 [0.513]
Elo difference: 8.9 +/- 14.4, LOS: 88.5 %, DrawRatio: 47.0 %

1177 of 6000 games finished.

Final results after 6000 games ...

Code: Select all

Rank Name                       Rating         +    -     #                W    L    D    W%    =%   OppR
---------------------------------------------------------------------------------------------------------
   1 Stockfish-XI-NN 0722-2141   3103   0.0    6    6  6000 3043.0  50.7 1526 1440 3034  25.4  50.6  3097
   2 Stockfish-XI-NN 0722-0944   3097   5.0    6    6  6000 2957.0  49.3 1440 1526 3034  24.0  50.6  3103
---------------------------------------------------------------------------------------------------------

[cdani]

Stockfish Dev 1706 vs sergio 20200722-1517 200 + 2:

Code: Select all

   # PLAYER                       : RATING  ERROR   POINTS  PLAYED    (%)
   1 stnnuesergio20200722-1517    : 2886.5   10.8    160.5     274   58.6%
   2 st200617                     : 2825.5   10.8    113.5     274   41.4%

Games: https://1drv.ms/u/s!Aj8wlSmr_Hq3irYFs2R ... g?e=BTECXZ

[Rebel]

The last (2210) finally 59%, +63 elo above the official stockfish 11.

Code: Select all

SF-NNUE (popcount) vs Stockfish 11
henk-2706     54.7%   2020-07-19
sergio-1432   56.1%   2020-07-21 15:56
sergio-1907   58.9%   2020-07-21 19:14
sergio-2323   58.4%   2020-07-21 23:23
sergio-0359   58.8%   2020-07-22 03:59
sergio-0944   58.9%   2020-07-22 09:44
sergio-1153   58.3%   2020-07-22 11:53
sergio-1807   58.0%   2020-07-22 18:07
sergio-2210   59.0%   2020-07-22 22:10

[nabildanial]

The last (2210) finally 59%, +63 elo above the official stockfish 11.

Code: Select all

SF-NNUE (popcount) vs Stockfish 11
henk-2706     54.7%   2020-07-19
sergio-1432   56.1%   2020-07-21 15:56
sergio-1907   58.9%   2020-07-21 19:14
sergio-2323   58.4%   2020-07-21 23:23
sergio-0359   58.8%   2020-07-22 03:59
sergio-0944   58.9%   2020-07-22 09:44
sergio-1153   58.3%   2020-07-22 11:53
sergio-1807   58.0%   2020-07-22 18:07
sergio-2210   59.0%   2020-07-22 22:10

Holy cow. Is this real?

[Laskos]

The last (2210) finally 59%, +63 elo above the official stockfish 11.

Code: Select all

SF-NNUE (popcount) vs Stockfish 11
henk-2706     54.7%   2020-07-19
sergio-1432   56.1%   2020-07-21 15:56
sergio-1907   58.9%   2020-07-21 19:14
sergio-2323   58.4%   2020-07-21 23:23
sergio-0359   58.8%   2020-07-22 03:59
sergio-0944   58.9%   2020-07-22 09:44
sergio-1153   58.3%   2020-07-22 11:53
sergio-1807   58.0%   2020-07-22 18:07
sergio-2210   59.0%   2020-07-22 22:10

I get a strong regression for 20200723-1134.bin

[Laskos]

The last (2210) finally 59%, +63 elo above the official stockfish 11.

Code: Select all

SF-NNUE (popcount) vs Stockfish 11
henk-2706     54.7%   2020-07-19
sergio-1432   56.1%   2020-07-21 15:56
sergio-1907   58.9%   2020-07-21 19:14
sergio-2323   58.4%   2020-07-21 23:23
sergio-0359   58.8%   2020-07-22 03:59
sergio-0944   58.9%   2020-07-22 09:44
sergio-1153   58.3%   2020-07-22 11:53
sergio-1807   58.0%   2020-07-22 18:07
sergio-2210   59.0%   2020-07-22 22:10

Holy cow. Is this real?

It's real, I am getting even better results against SF_dev with the best Sergio nets like SF Sergio 22.07_2141 and fast GK binary.

Code: Select all

Games Completed = 1000 of 1000 (Avg game length = 23.177 sec)
Settings = RR/128MB/6000ms+100ms/M 700cp for 3 moves, D 120 moves/EPD:C:\LittleBlitzer\3M_08_10.epd(395)
Time = 5950 sec elapsed, 0 sec remaining
 1.  SF Sergio 22.07_2141     	607.0/1000	441-227-332  	(L: m=0 t=0 i=0 a=227)	(D: r=211 i=46 f=28 s=3 a=44)	(tpm=185.4 d=16.59 nps=958244)
 2.  SF_dev                   	393.0/1000	227-441-332  	(L: m=0 t=0 i=0 a=441)	(D: r=211 i=46 f=28 s=3 a=44)	(tpm=187.7 d=18.08 nps=1717547)

60.7%
75.5 +/- 9.8 Elo points difference compared to SF_dev.

[Zenmastur]

It's real, I am getting even better results against SF_dev with the best Sergio nets like SF Sergio 22.07_2141 and fast GK binary.

Specifically, which binary are you talking about?

Regards,

Zenmastur

[Laskos]

It's real, I am getting even better results against SF_dev with the best Sergio nets like SF Sergio 22.07_2141 and fast GK binary.

Specifically, which binary are you talking about?

Regards,

Zenmastur

GK binary here:
http://talkchess.com/forum3/viewtopic.php?f=2&t=74484