Sergio Vieri second net is out

Discussion of anything and everything relating to chess playing software and machines.

Moderators: Harvey Williamson, bob, hgm

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
User avatar
Laskos
Posts: 10779
Joined: Wed Jul 26, 2006 8:21 pm
Full name: Kai Laskos

Re: Sergio Vieri second net is out

Post by Laskos » Wed Jul 22, 2020 2:51 pm

Laskos wrote:
Wed Jul 22, 2020 8:57 am
Rebel wrote:
Wed Jul 22, 2020 8:44 am
2 more nets, no progress, 59% apparently is a hard nut to crack.

Code: Select all

https://github.com/nodchip/Stockfish
https://www.comp.nus.edu.sg/~sergio-v/nnue/

Stockfish 11        Nodes/second   : 1.664.010
SF-NNUE (popcount)  Nodes/second   :   796.674

SF-NNUE (popcount) vs Stockfish 11
henk-2706     54.7%   2020-07-19
sergio-1432   56.1%   2020-07-21 15:56
sergio-1907   58.9%   2020-07-21 19:14
sergio-2323   58.4%   2020-07-21 23:23
sergio-0359   58.8%   2020-07-22 03:59
sergio-0944   58.9%   2020-07-22 09:44
sergio-1153   58.3%   2020-07-22 11:53
Here I have the cumulative improvement of Sergio's nets after 6 nets:

6s + 0.1s

Code: Select all

Score of Sergio_22.07_0944 vs Sergio_20.07_1017: 1399 - 1137 - 1464  [0.533] 4000

...      Sergio_22.07_0944 playing White: 737 - 528 - 735  [0.552] 2000
...      Sergio_22.07_0944 playing Black: 662 - 609 - 729  [0.513] 2000
...      White vs Black: 1346 - 1190 - 1464  [0.519] 4000

Elo difference: 22.8 +/- 8.6, LOS: 100.0 %, DrawRatio: 36.6 %
Finished match
I get about 4 Elo points per net, if the improvement is linear (it is probably not).
Two more nets and the difference is now above 30 Elo points since the first net.

Code: Select all

Score of Sergio_22.07_2141 vs Sergio_20.07_1017: 1434 - 1064 - 1502  [0.546] 4000

...      Sergio_22.07_2141 playing White: 771 - 489 - 740  [0.571] 2000
...      Sergio_22.07_2141 playing Black: 663 - 575 - 762  [0.522] 2000
...      White vs Black: 1346 - 1152 - 1502  [0.524] 4000

Elo difference: 32.2 +/- 8.5, LOS: 100.0 %, DrawRatio: 37.5 %
Finished match
Seems to never stop improving, about 4 Elo points every net.

User avatar
MikeB
Posts: 4391
Joined: Thu Mar 09, 2006 5:34 am
Location: Pen Argyl, Pennsylvania

Re: Sergio Vieri second net is out

Post by MikeB » Wed Jul 22, 2020 2:59 pm

+1 and a shoutout to Sergio Vieri, make it a plus +1000! :D

You are doing great work Sergio :!:

We are now picking up about 8 Elo every 12-15 hours or in self-play - that is mind boggling :!:

Code: Select all

Score of Stockfish-XI-NN 0722-2141 vs Stockfish-XI-NN 0722-0944: 327 - 297 - 553 [0.513]
Elo difference: 8.9 +/- 14.4, LOS: 88.5 %, DrawRatio: 47.0 %

1177 of 6000 games finished.
Image

User avatar
MikeB
Posts: 4391
Joined: Thu Mar 09, 2006 5:34 am
Location: Pen Argyl, Pennsylvania

Re: Sergio Vieri second net is out

Post by MikeB » Wed Jul 22, 2020 3:55 pm

MikeB wrote:
Wed Jul 22, 2020 2:59 pm
+1 and a shoutout to Sergio Vieri, make it a plus +1000! :D

You are doing great work Sergio :!:

We are now picking up about 8 Elo every 12-15 hours or in self-play - that is mind boggling :!:

Code: Select all

Score of Stockfish-XI-NN 0722-2141 vs Stockfish-XI-NN 0722-0944: 327 - 297 - 553 [0.513]
Elo difference: 8.9 +/- 14.4, LOS: 88.5 %, DrawRatio: 47.0 %

1177 of 6000 games finished.
Final results after 6000 games ...

Code: Select all

Rank Name                       Rating         +    -     #                W    L    D    W%    =%   OppR
---------------------------------------------------------------------------------------------------------
   1 Stockfish-XI-NN 0722-2141   3103   0.0    6    6  6000 3043.0  50.7 1526 1440 3034  25.4  50.6  3097
   2 Stockfish-XI-NN 0722-0944   3097   5.0    6    6  6000 2957.0  49.3 1440 1526 3034  24.0  50.6  3103
---------------------------------------------------------------------------------------------------------
Image

User avatar
cdani
Posts: 2175
Joined: Sat Jan 18, 2014 9:24 am
Location: Andorra
Contact:

Re: Sergio Vieri second net is out

Post by cdani » Wed Jul 22, 2020 4:38 pm

Stockfish Dev 1706 vs sergio 20200722-1517 200 + 2:

Code: Select all

   # PLAYER                       : RATING  ERROR   POINTS  PLAYED    (%)
   1 stnnuesergio20200722-1517    : 2886.5   10.8    160.5     274   58.6%
   2 st200617                     : 2825.5   10.8    113.5     274   41.4%
Games: https://1drv.ms/u/s!Aj8wlSmr_Hq3irYFs2R ... g?e=BTECXZ

User avatar
Rebel
Posts: 5651
Joined: Thu Aug 18, 2011 10:04 am

Re: Sergio Vieri second net is out

Post by Rebel » Wed Jul 22, 2020 7:36 pm

The last (2210) finally 59%, +63 elo above the official stockfish 11.

Code: Select all

SF-NNUE (popcount) vs Stockfish 11
henk-2706     54.7%   2020-07-19
sergio-1432   56.1%   2020-07-21 15:56
sergio-1907   58.9%   2020-07-21 19:14
sergio-2323   58.4%   2020-07-21 23:23
sergio-0359   58.8%   2020-07-22 03:59
sergio-0944   58.9%   2020-07-22 09:44
sergio-1153   58.3%   2020-07-22 11:53
sergio-1807   58.0%   2020-07-22 18:07
sergio-2210   59.0%   2020-07-22 22:10
90% of coding is debugging, the other 10% is writing bugs.

nabildanial
Posts: 124
Joined: Thu Jun 05, 2014 3:29 am
Location: Malaysia

Re: Sergio Vieri second net is out

Post by nabildanial » Thu Jul 23, 2020 2:27 am

Rebel wrote:
Wed Jul 22, 2020 7:36 pm
The last (2210) finally 59%, +63 elo above the official stockfish 11.

Code: Select all

SF-NNUE (popcount) vs Stockfish 11
henk-2706     54.7%   2020-07-19
sergio-1432   56.1%   2020-07-21 15:56
sergio-1907   58.9%   2020-07-21 19:14
sergio-2323   58.4%   2020-07-21 23:23
sergio-0359   58.8%   2020-07-22 03:59
sergio-0944   58.9%   2020-07-22 09:44
sergio-1153   58.3%   2020-07-22 11:53
sergio-1807   58.0%   2020-07-22 18:07
sergio-2210   59.0%   2020-07-22 22:10
Holy cow. Is this real?

User avatar
Laskos
Posts: 10779
Joined: Wed Jul 26, 2006 8:21 pm
Full name: Kai Laskos

Re: Sergio Vieri second net is out

Post by Laskos » Thu Jul 23, 2020 3:59 am

Rebel wrote:
Wed Jul 22, 2020 7:36 pm
The last (2210) finally 59%, +63 elo above the official stockfish 11.

Code: Select all

SF-NNUE (popcount) vs Stockfish 11
henk-2706     54.7%   2020-07-19
sergio-1432   56.1%   2020-07-21 15:56
sergio-1907   58.9%   2020-07-21 19:14
sergio-2323   58.4%   2020-07-21 23:23
sergio-0359   58.8%   2020-07-22 03:59
sergio-0944   58.9%   2020-07-22 09:44
sergio-1153   58.3%   2020-07-22 11:53
sergio-1807   58.0%   2020-07-22 18:07
sergio-2210   59.0%   2020-07-22 22:10
I get a strong regression for 20200723-1134.bin

User avatar
Laskos
Posts: 10779
Joined: Wed Jul 26, 2006 8:21 pm
Full name: Kai Laskos

Re: Sergio Vieri second net is out

Post by Laskos » Thu Jul 23, 2020 6:13 am

nabildanial wrote:
Thu Jul 23, 2020 2:27 am
Rebel wrote:
Wed Jul 22, 2020 7:36 pm
The last (2210) finally 59%, +63 elo above the official stockfish 11.

Code: Select all

SF-NNUE (popcount) vs Stockfish 11
henk-2706     54.7%   2020-07-19
sergio-1432   56.1%   2020-07-21 15:56
sergio-1907   58.9%   2020-07-21 19:14
sergio-2323   58.4%   2020-07-21 23:23
sergio-0359   58.8%   2020-07-22 03:59
sergio-0944   58.9%   2020-07-22 09:44
sergio-1153   58.3%   2020-07-22 11:53
sergio-1807   58.0%   2020-07-22 18:07
sergio-2210   59.0%   2020-07-22 22:10
Holy cow. Is this real?
It's real, I am getting even better results against SF_dev with the best Sergio nets like SF Sergio 22.07_2141 and fast GK binary.

Code: Select all

Games Completed = 1000 of 1000 (Avg game length = 23.177 sec)
Settings = RR/128MB/6000ms+100ms/M 700cp for 3 moves, D 120 moves/EPD:C:\LittleBlitzer\3M_08_10.epd(395)
Time = 5950 sec elapsed, 0 sec remaining
 1.  SF Sergio 22.07_2141     	607.0/1000	441-227-332  	(L: m=0 t=0 i=0 a=227)	(D: r=211 i=46 f=28 s=3 a=44)	(tpm=185.4 d=16.59 nps=958244)
 2.  SF_dev                   	393.0/1000	227-441-332  	(L: m=0 t=0 i=0 a=441)	(D: r=211 i=46 f=28 s=3 a=44)	(tpm=187.7 d=18.08 nps=1717547)


60.7%
75.5 +/- 9.8 Elo points difference compared to SF_dev.

Zenmastur
Posts: 912
Joined: Sat May 31, 2014 6:28 am

Re: Sergio Vieri second net is out

Post by Zenmastur » Thu Jul 23, 2020 6:20 am

Laskos wrote:
Thu Jul 23, 2020 6:13 am
It's real, I am getting even better results against SF_dev with the best Sergio nets like SF Sergio 22.07_2141 and fast GK binary.
Specifically, which binary are you talking about?

Regards,

Zenmastur
Only 2 defining forces have ever offered to die for you.....Jesus Christ and the American Soldier. One died for your soul, the other for your freedom.

User avatar
Laskos
Posts: 10779
Joined: Wed Jul 26, 2006 8:21 pm
Full name: Kai Laskos

Re: Sergio Vieri second net is out

Post by Laskos » Thu Jul 23, 2020 6:32 am

Zenmastur wrote:
Thu Jul 23, 2020 6:20 am
Laskos wrote:
Thu Jul 23, 2020 6:13 am
It's real, I am getting even better results against SF_dev with the best Sergio nets like SF Sergio 22.07_2141 and fast GK binary.
Specifically, which binary are you talking about?

Regards,

Zenmastur
GK binary here:
viewtopic.php?f=2&t=74484

Post Reply