Discussion of anything and everything relating to chess playing software and machines.
Moderators: hgm , Rebel , chrisw
Laskos
Posts: 10948 Joined: Wed Jul 26, 2006 10:21 pm
Full name: Kai Laskos
Post
by Laskos » Wed Jul 22, 2020 4:51 pm
Laskos wrote: ↑ Wed Jul 22, 2020 10:57 am
Rebel wrote: ↑ Wed Jul 22, 2020 10:44 am
2 more nets, no progress, 59% apparently is a hard nut to crack.
Code: Select all
https://github.com/nodchip/Stockfish
https://www.comp.nus.edu.sg/~sergio-v/nnue/
Stockfish 11 Nodes/second : 1.664.010
SF-NNUE (popcount) Nodes/second : 796.674
SF-NNUE (popcount) vs Stockfish 11
henk-2706 54.7% 2020-07-19
sergio-1432 56.1% 2020-07-21 15:56
sergio-1907 58.9% 2020-07-21 19:14
sergio-2323 58.4% 2020-07-21 23:23
sergio-0359 58.8% 2020-07-22 03:59
sergio-0944 58.9% 2020-07-22 09:44
sergio-1153 58.3% 2020-07-22 11:53
Here I have the cumulative improvement of Sergio's nets after 6 nets:
6s + 0.1s
Code: Select all
Score of Sergio_22.07_0944 vs Sergio_20.07_1017: 1399 - 1137 - 1464 [0.533] 4000
... Sergio_22.07_0944 playing White: 737 - 528 - 735 [0.552] 2000
... Sergio_22.07_0944 playing Black: 662 - 609 - 729 [0.513] 2000
... White vs Black: 1346 - 1190 - 1464 [0.519] 4000
Elo difference: 22.8 +/- 8.6, LOS: 100.0 %, DrawRatio: 36.6 %
Finished match
I get about 4 Elo points per net, if the improvement is linear (it is probably not).
Two more nets and the difference is now above 30 Elo points since the first net.
Code: Select all
Score of Sergio_22.07_2141 vs Sergio_20.07_1017: 1434 - 1064 - 1502 [0.546] 4000
... Sergio_22.07_2141 playing White: 771 - 489 - 740 [0.571] 2000
... Sergio_22.07_2141 playing Black: 663 - 575 - 762 [0.522] 2000
... White vs Black: 1346 - 1152 - 1502 [0.524] 4000
Elo difference: 32.2 +/- 8.5, LOS: 100.0 %, DrawRatio: 37.5 %
Finished match
Seems to never stop improving, about 4 Elo points every net.
MikeB
Posts: 4889 Joined: Thu Mar 09, 2006 6:34 am
Location: Pen Argyl, Pennsylvania
Post
by MikeB » Wed Jul 22, 2020 4:59 pm
+1 and a shoutout to Sergio Vieri, make it a plus +1000!
You are doing great work Sergio
We are now picking up about 8 Elo every 12-15 hours or in self-play - that is mind boggling
Code: Select all
Score of Stockfish-XI-NN 0722-2141 vs Stockfish-XI-NN 0722-0944: 327 - 297 - 553 [0.513]
Elo difference: 8.9 +/- 14.4, LOS: 88.5 %, DrawRatio: 47.0 %
1177 of 6000 games finished.
MikeB
Posts: 4889 Joined: Thu Mar 09, 2006 6:34 am
Location: Pen Argyl, Pennsylvania
Post
by MikeB » Wed Jul 22, 2020 5:55 pm
MikeB wrote: ↑ Wed Jul 22, 2020 4:59 pm
+1 and a shoutout to Sergio Vieri, make it a plus +1000!
You are doing great work Sergio
We are now picking up about 8 Elo every 12-15 hours or in self-play - that is mind boggling
Code: Select all
Score of Stockfish-XI-NN 0722-2141 vs Stockfish-XI-NN 0722-0944: 327 - 297 - 553 [0.513]
Elo difference: 8.9 +/- 14.4, LOS: 88.5 %, DrawRatio: 47.0 %
1177 of 6000 games finished.
Final results after 6000 games ...
Code: Select all
Rank Name Rating + - # W L D W% =% OppR
---------------------------------------------------------------------------------------------------------
1 Stockfish-XI-NN 0722-2141 3103 0.0 6 6 6000 3043.0 50.7 1526 1440 3034 25.4 50.6 3097
2 Stockfish-XI-NN 0722-0944 3097 5.0 6 6 6000 2957.0 49.3 1440 1526 3034 24.0 50.6 3103
---------------------------------------------------------------------------------------------------------
cdani
Posts: 2204 Joined: Sat Jan 18, 2014 10:24 am
Location: Andorra
Post
by cdani » Wed Jul 22, 2020 6:38 pm
Stockfish Dev 1706 vs sergio 20200722-1517 200 + 2:
Code: Select all
# PLAYER : RATING ERROR POINTS PLAYED (%)
1 stnnuesergio20200722-1517 : 2886.5 10.8 160.5 274 58.6%
2 st200617 : 2825.5 10.8 113.5 274 41.4%
Games:
https://1drv.ms/u/s!Aj8wlSmr_Hq3irYFs2R ... g?e=BTECXZ
Rebel
Posts: 6993 Joined: Thu Aug 18, 2011 12:04 pm
Post
by Rebel » Wed Jul 22, 2020 9:36 pm
The last (2210) finally 59%, +63 elo above the official stockfish 11.
Code: Select all
SF-NNUE (popcount) vs Stockfish 11
henk-2706 54.7% 2020-07-19
sergio-1432 56.1% 2020-07-21 15:56
sergio-1907 58.9% 2020-07-21 19:14
sergio-2323 58.4% 2020-07-21 23:23
sergio-0359 58.8% 2020-07-22 03:59
sergio-0944 58.9% 2020-07-22 09:44
sergio-1153 58.3% 2020-07-22 11:53
sergio-1807 58.0% 2020-07-22 18:07
sergio-2210 59.0% 2020-07-22 22:10
90% of coding is debugging, the other 10% is writing bugs.
nabildanial
Posts: 126 Joined: Thu Jun 05, 2014 5:29 am
Location: Malaysia
Post
by nabildanial » Thu Jul 23, 2020 4:27 am
Rebel wrote: ↑ Wed Jul 22, 2020 9:36 pm
The last (2210) finally 59%, +63 elo above the official stockfish 11.
Code: Select all
SF-NNUE (popcount) vs Stockfish 11
henk-2706 54.7% 2020-07-19
sergio-1432 56.1% 2020-07-21 15:56
sergio-1907 58.9% 2020-07-21 19:14
sergio-2323 58.4% 2020-07-21 23:23
sergio-0359 58.8% 2020-07-22 03:59
sergio-0944 58.9% 2020-07-22 09:44
sergio-1153 58.3% 2020-07-22 11:53
sergio-1807 58.0% 2020-07-22 18:07
sergio-2210 59.0% 2020-07-22 22:10
Holy cow. Is this real?
Laskos
Posts: 10948 Joined: Wed Jul 26, 2006 10:21 pm
Full name: Kai Laskos
Post
by Laskos » Thu Jul 23, 2020 5:59 am
Rebel wrote: ↑ Wed Jul 22, 2020 9:36 pm
The last (2210) finally 59%, +63 elo above the official stockfish 11.
Code: Select all
SF-NNUE (popcount) vs Stockfish 11
henk-2706 54.7% 2020-07-19
sergio-1432 56.1% 2020-07-21 15:56
sergio-1907 58.9% 2020-07-21 19:14
sergio-2323 58.4% 2020-07-21 23:23
sergio-0359 58.8% 2020-07-22 03:59
sergio-0944 58.9% 2020-07-22 09:44
sergio-1153 58.3% 2020-07-22 11:53
sergio-1807 58.0% 2020-07-22 18:07
sergio-2210 59.0% 2020-07-22 22:10
I get a strong regression for 20200723-1134.bin
Laskos
Posts: 10948 Joined: Wed Jul 26, 2006 10:21 pm
Full name: Kai Laskos
Post
by Laskos » Thu Jul 23, 2020 8:13 am
nabildanial wrote: ↑ Thu Jul 23, 2020 4:27 am
Rebel wrote: ↑ Wed Jul 22, 2020 9:36 pm
The last (2210) finally 59%, +63 elo above the official stockfish 11.
Code: Select all
SF-NNUE (popcount) vs Stockfish 11
henk-2706 54.7% 2020-07-19
sergio-1432 56.1% 2020-07-21 15:56
sergio-1907 58.9% 2020-07-21 19:14
sergio-2323 58.4% 2020-07-21 23:23
sergio-0359 58.8% 2020-07-22 03:59
sergio-0944 58.9% 2020-07-22 09:44
sergio-1153 58.3% 2020-07-22 11:53
sergio-1807 58.0% 2020-07-22 18:07
sergio-2210 59.0% 2020-07-22 22:10
Holy cow. Is this real?
It's real, I am getting even better results against SF_dev with the best Sergio nets like SF Sergio 22.07_2141 and fast GK binary.
Code: Select all
Games Completed = 1000 of 1000 (Avg game length = 23.177 sec)
Settings = RR/128MB/6000ms+100ms/M 700cp for 3 moves, D 120 moves/EPD:C:\LittleBlitzer\3M_08_10.epd(395)
Time = 5950 sec elapsed, 0 sec remaining
1. SF Sergio 22.07_2141 607.0/1000 441-227-332 (L: m=0 t=0 i=0 a=227) (D: r=211 i=46 f=28 s=3 a=44) (tpm=185.4 d=16.59 nps=958244)
2. SF_dev 393.0/1000 227-441-332 (L: m=0 t=0 i=0 a=441) (D: r=211 i=46 f=28 s=3 a=44) (tpm=187.7 d=18.08 nps=1717547)
60.7%
75.5 +/- 9.8 Elo points difference compared to SF_dev.
Zenmastur
Posts: 919 Joined: Sat May 31, 2014 8:28 am
Post
by Zenmastur » Thu Jul 23, 2020 8:20 am
Laskos wrote: ↑ Thu Jul 23, 2020 8:13 am
It's real, I am getting even better results against SF_dev with the best Sergio nets like SF Sergio 22.07_2141 and fast GK binary.
Specifically, which binary are you talking about?
Regards,
Zenmastur
Only 2 defining forces have ever offered to die for you.....Jesus Christ and the American Soldier. One died for your soul, the other for your freedom.
Laskos
Posts: 10948 Joined: Wed Jul 26, 2006 10:21 pm
Full name: Kai Laskos
Post
by Laskos » Thu Jul 23, 2020 8:32 am
Zenmastur wrote: ↑ Thu Jul 23, 2020 8:20 am
Laskos wrote: ↑ Thu Jul 23, 2020 8:13 am
It's real, I am getting even better results against SF_dev with the best Sergio nets like SF Sergio 22.07_2141 and fast GK binary.
Specifically, which binary are you talking about?
Regards,
Zenmastur
GK binary here:
http://talkchess.com/forum3/viewtopic.php?f=2&t=74484