Page 1 of 6
Progress of Stockfish in 6 days
Posted: Wed Aug 12, 2020 8:42 pm
by Laskos
It's hilarious, every day SF engine becomes sensibly stronger (and the nets too), so in 6 days I get a 35 Elo points progress of Stockfish at 6s + 0.1s. Really, even if the pace will slow down, I expect another 50 Elo points in one-two months. The binary is also 8-9% faster now compared to 6 days ago, and reaches above 70% speeds of the non-network SF.
Code: Select all
Games Completed = 1000 of 1000 (Avg game length = 23.646 sec)
Settings = Gauntlet/128MB/6000ms+100ms/M 700cp for 3 moves, D 120 moves/EPD:C:\LittleBlitzer\2moves_80_100.epd(1749)
Time = 6057 sec elapsed, 0 sec remaining
1. SF NNUE new 550.5/1000 301-200-499 (L: m=0 t=0 i=0 a=200) (D: r=328 i=94 f=25 s=5 a=47) (tpm=181.0 d=17.79 nps=1355796)
2. SF NNUE 6 days ago 449.5/1000 200-301-499 (L: m=1 t=0 i=0 a=300) (D: r=328 i=94 f=25 s=5 a=47) (tpm=182.8 d=17.53 nps=1248328)
Re: Progress of Stockfish in 6 days
Posted: Wed Aug 12, 2020 9:36 pm
by Jouni
Yes SF NNUE is equal to quadruple your CPU cores for free. Incredible
.
Re: Progress of Stockfish in 6 days
Posted: Wed Aug 12, 2020 11:21 pm
by duncan
Does anyone have the daily/monthly elo Progress of Rybka 3 to 2, and 4 to 3, so we can compare?
Re: Progress of Stockfish in 6 days
Posted: Thu Aug 13, 2020 12:04 am
by Laskos
duncan wrote: ↑Wed Aug 12, 2020 11:21 pm
Does anyone have the daily/monthly elo Progress of Rybka 3 to 2, and 4 to 3, so we can compare?
No, I don't have, but IIRC Rybka 1.0beta to Rybka 2.2 and Rybka 2.3.2 to Rybka 3 were the fastest periods, some 100 Elo points in a year or so each. I don't know about some special faster progress weeks.
Re: Progress of Stockfish in 6 days
Posted: Thu Aug 13, 2020 7:22 am
by lkaufman
Jouni wrote: ↑Wed Aug 12, 2020 9:36 pm
Yes SF NNUE is equal to quadruple your CPU cores for free. Incredible
.
I actually got a result that SFNNUE (a couple days ago) on one thread beat Stockfish 11 on seven threads, at 2' + 1", by 90 to 80! So you may be understating it!
Re: Progress of Stockfish in 6 days
Posted: Thu Aug 13, 2020 7:59 am
by MikeB
Laskos wrote: ↑Wed Aug 12, 2020 8:42 pm
It's hilarious, every day SF engine becomes sensibly stronger (and the nets too), so in 6 days I get a 35 Elo points progress of Stockfish at 6s + 0.1s. Really, even if the pace will slow down, I expect another 50 Elo points in one-two months. The binary is also 8-9% faster now compared to 6 days ago, and reaches above 70% speeds of the non-network SF.
Code: Select all
Games Completed = 1000 of 1000 (Avg game length = 23.646 sec)
Settings = Gauntlet/128MB/6000ms+100ms/M 700cp for 3 moves, D 120 moves/EPD:C:\LittleBlitzer\2moves_80_100.epd(1749)
Time = 6057 sec elapsed, 0 sec remaining
1. SF NNUE new 550.5/1000 301-200-499 (L: m=0 t=0 i=0 a=200) (D: r=328 i=94 f=25 s=5 a=47) (tpm=181.0 d=17.79 nps=1355796)
2. SF NNUE 6 days ago 449.5/1000 200-301-499 (L: m=1 t=0 i=0 a=300) (D: r=328 i=94 f=25 s=5 a=47) (tpm=182.8 d=17.53 nps=1248328)
It's incredible - it's hard to keep up - kudos to the SF development team...
Re: Progress of Stockfish in 6 days
Posted: Thu Aug 13, 2020 8:11 am
by M ANSARI
It is indeed incredible! Who would have thought that evaluation makes such a huge impact! So much for bean counting! I had always thought that a good evaluation was a shortcut to deep calculation, but never would have I imagined it to this level. The progress being made at closing the gap in speed between classic SF and NNUE SF is pretty impressive. I guess this brings AB engines back into pole position!
Re: Progress of Stockfish in 6 days
Posted: Thu Aug 13, 2020 8:12 am
by MikeB
I just kicked off a 30,000 game set for those might be interested,
Code: Select all
PGN File: c:/cluster.mfb/pgn/08130132.pgn
Time Control: Time Control-> base+inc: 25+0.500
Games: 30000
Threads: 1
Hash: 256
Current date : time (EDST)
Date: 08/13/20 : 02:07:35
Projected-> Time: 16h:23m:17s
Run -> Time: 0h:35m:28s
Rank Name Rating Δ + - # Σ Σ% W L D W% =% OppR
---------------------------------------------------------------------------------------------------------
1 cur-dev-stockfish 3526 0.0 22 22 422 230.5 54.6 119 80 223 28.2 52.8 3494
2 Stockfish-XIr4-2257 3524 2.0 22 22 423 230.0 54.4 122 85 216 28.8 51.1 3494
3 Bluefish-2257 3502 22.3 23 23 424 213.5 50.4 107 104 213 25.2 50.2 3500
4 Honey-2257 3494 7.8 23 23 423 207.0 48.9 104 113 206 24.6 48.7 3501
5 Black-Diamond-2257 3453 40.9 23 23 422 176.0 41.7 73 143 206 17.3 48.8 3512
---------------------------------------------------------------------------------------------------------
Δ = delta from the next higher rated opponent
# = number of games played
Σ = total score, 1 point for win, 1/2 point for draw
LOS:
cu St Bl Ho Bl
cur-dev-stockfish 55 95 98 99
Stockfish-XIr4-2257 44 93 98 99
Bluefish-2257 4 6 70 99
Honey-2257 1 1 29 99
Black-Diamond-2257 0 0 0 0
loops/scheduled: 6/200
Dropbox suspended my account for too many public transfers - will post an update in a few hours.
I have sacrificed Elo for Honey and Bluefish to maintain some problem solving capabilities - when you get 100 Elo for free in about 3 weeks, that is an easy choice to make.
Re: Progress of Stockfish in 6 days
Posted: Thu Aug 13, 2020 2:27 pm
by Jouni
My 100 game test SF NNUE 1_core vs SF 11 4_cores game in minute:
Code: Select all
1 Stockfish 120820 NNUE +24 +20/=67/-13 53.50% 53.5/100
2 Stockfish 11 64 BMI2 -24 +13/=67/-20 46.50% 46.5/100
Also new faster compiles seems to be better in testsuites finally!
Re: Progress of Stockfish in 6 days
Posted: Thu Aug 13, 2020 3:33 pm
by MikeB
MikeB wrote: ↑Thu Aug 13, 2020 8:12 am
I just kicked off a 30,000 game set for those might be interested,
Code: Select all
PGN File: c:/cluster.mfb/pgn/08130132.pgn
Time Control: Time Control-> base+inc: 25+0.500
Games: 30000
Threads: 1
Hash: 256
Current date : time (EDST)
Date: 08/13/20 : 02:07:35
Projected-> Time: 16h:23m:17s
Run -> Time: 0h:35m:28s
Rank Name Rating Δ + - # Σ Σ% W L D W% =% OppR
---------------------------------------------------------------------------------------------------------
1 cur-dev-stockfish 3526 0.0 22 22 422 230.5 54.6 119 80 223 28.2 52.8 3494
2 Stockfish-XIr4-2257 3524 2.0 22 22 423 230.0 54.4 122 85 216 28.8 51.1 3494
3 Bluefish-2257 3502 22.3 23 23 424 213.5 50.4 107 104 213 25.2 50.2 3500
4 Honey-2257 3494 7.8 23 23 423 207.0 48.9 104 113 206 24.6 48.7 3501
5 Black-Diamond-2257 3453 40.9 23 23 422 176.0 41.7 73 143 206 17.3 48.8 3512
---------------------------------------------------------------------------------------------------------
Δ = delta from the next higher rated opponent
# = number of games played
Σ = total score, 1 point for win, 1/2 point for draw
LOS:
cu St Bl Ho Bl
cur-dev-stockfish 55 95 98 99
Stockfish-XIr4-2257 44 93 98 99
Bluefish-2257 4 6 70 99
Honey-2257 1 1 29 99
Black-Diamond-2257 0 0 0 0
loops/scheduled: 6/200
Dropbox suspended my account for too many public transfers - will post an update in a few hours.
I have sacrificed Elo for Honey and Bluefish to maintain some problem solving capabilities - when you get 100 Elo for free in about 3 weeks, that is an easy choice to make.
just over the 1/2 way point
Code: Select all
PGN File: c:/cluster.mfb/pgn/08130132.pgn
Time Control: Time Control-> base+inc: 25+0.500
Games: 30000
Threads: 1
Hash: 256
Current date : time (EDST)
Date: 08/13/20 : 09:29:54
Projected-> Time: 16h:23m:17s
Run -> Time: 7h:57m:47s
Rank Name Rating Δ + - # Σ Σ% W L D W% =% OppR
---------------------------------------------------------------------------------------------------------
1 Stockfish-XIr4-2257 3519 0.0 5 5 6032 3218.0 53.3 1419 1015 3598 23.5 59.6 3495
2 cur-dev-stockfish 3514 4.9 5 5 6030 3165.0 52.5 1344 1044 3642 22.3 60.4 3497
3 Bluefish-2257 3507 6.6 5 5 6032 3096.5 51.3 1275 1114 3643 21.1 60.4 3498
4 Honey-2257 3506 1.5 5 5 6032 3078.0 51.0 1237 1113 3682 20.5 61.0 3499
5 Black-Diamond-2257 3454 51.7 5 5 6030 2520.5 41.8 784 1773 3473 13.0 57.6 3511
---------------------------------------------------------------------------------------------------------
Δ = delta from the next higher rated opponent
# = number of games played
Σ = total score, 1 point for win, 1/2 point for draw
LOS:
St cu Bl Ho Bl
Stockfish-XIr4-2257 92 99 99100
cur-dev-stockfish 7 97 99100
Bluefish-2257 0 2 67100
Honey-2257 0 0 32 100
Black-Diamond-2257 0 0 0 0
loops/scheduled: 93/200