Progress of Stockfish in 6 days

Discussion of anything and everything relating to chess playing software and machines.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
User avatar
Laskos
Posts: 10612
Joined: Wed Jul 26, 2006 8:21 pm
Full name: Kai Laskos

Progress of Stockfish in 6 days

Post by Laskos » Wed Aug 12, 2020 6:42 pm

It's hilarious, every day SF engine becomes sensibly stronger (and the nets too), so in 6 days I get a 35 Elo points progress of Stockfish at 6s + 0.1s. Really, even if the pace will slow down, I expect another 50 Elo points in one-two months. The binary is also 8-9% faster now compared to 6 days ago, and reaches above 70% speeds of the non-network SF.

Code: Select all

Games Completed = 1000 of 1000 (Avg game length = 23.646 sec)
Settings = Gauntlet/128MB/6000ms+100ms/M 700cp for 3 moves, D 120 moves/EPD:C:\LittleBlitzer\2moves_80_100.epd(1749)
Time = 6057 sec elapsed, 0 sec remaining
 1.  SF NNUE new                  	550.5/1000	301-200-499  	(L: m=0 t=0 i=0 a=200)	(D: r=328 i=94 f=25 s=5 a=47)	(tpm=181.0 d=17.79 nps=1355796)
 2.  SF NNUE 6 days ago             	449.5/1000	200-301-499  	(L: m=1 t=0 i=0 a=300)	(D: r=328 i=94 f=25 s=5 a=47)	(tpm=182.8 d=17.53 nps=1248328)

Jouni
Posts: 2199
Joined: Wed Mar 08, 2006 7:15 pm

Re: Progress of Stockfish in 6 days

Post by Jouni » Wed Aug 12, 2020 7:36 pm

Yes SF NNUE is equal to quadruple your CPU cores for free. Incredible :!: :!: .
Jouni

duncan
Posts: 11765
Joined: Mon Jul 07, 2008 8:50 pm

Re: Progress of Stockfish in 6 days

Post by duncan » Wed Aug 12, 2020 9:21 pm

Does anyone have the daily/monthly elo Progress of Rybka 3 to 2, and 4 to 3, so we can compare?

User avatar
Laskos
Posts: 10612
Joined: Wed Jul 26, 2006 8:21 pm
Full name: Kai Laskos

Re: Progress of Stockfish in 6 days

Post by Laskos » Wed Aug 12, 2020 10:04 pm

duncan wrote:
Wed Aug 12, 2020 9:21 pm
Does anyone have the daily/monthly elo Progress of Rybka 3 to 2, and 4 to 3, so we can compare?
No, I don't have, but IIRC Rybka 1.0beta to Rybka 2.2 and Rybka 2.3.2 to Rybka 3 were the fastest periods, some 100 Elo points in a year or so each. I don't know about some special faster progress weeks.

lkaufman
Posts: 4324
Joined: Sun Jan 10, 2010 5:15 am
Location: Maryland USA
Contact:

Re: Progress of Stockfish in 6 days

Post by lkaufman » Thu Aug 13, 2020 5:22 am

Jouni wrote:
Wed Aug 12, 2020 7:36 pm
Yes SF NNUE is equal to quadruple your CPU cores for free. Incredible :!: :!: .
I actually got a result that SFNNUE (a couple days ago) on one thread beat Stockfish 11 on seven threads, at 2' + 1", by 90 to 80! So you may be understating it!
Komodo rules!

User avatar
MikeB
Posts: 4358
Joined: Thu Mar 09, 2006 5:34 am
Location: Pen Argyl, Pennsylvania

Re: Progress of Stockfish in 6 days

Post by MikeB » Thu Aug 13, 2020 5:59 am

Laskos wrote:
Wed Aug 12, 2020 6:42 pm
It's hilarious, every day SF engine becomes sensibly stronger (and the nets too), so in 6 days I get a 35 Elo points progress of Stockfish at 6s + 0.1s. Really, even if the pace will slow down, I expect another 50 Elo points in one-two months. The binary is also 8-9% faster now compared to 6 days ago, and reaches above 70% speeds of the non-network SF.

Code: Select all

Games Completed = 1000 of 1000 (Avg game length = 23.646 sec)
Settings = Gauntlet/128MB/6000ms+100ms/M 700cp for 3 moves, D 120 moves/EPD:C:\LittleBlitzer\2moves_80_100.epd(1749)
Time = 6057 sec elapsed, 0 sec remaining
 1.  SF NNUE new                  	550.5/1000	301-200-499  	(L: m=0 t=0 i=0 a=200)	(D: r=328 i=94 f=25 s=5 a=47)	(tpm=181.0 d=17.79 nps=1355796)
 2.  SF NNUE 6 days ago             	449.5/1000	200-301-499  	(L: m=1 t=0 i=0 a=300)	(D: r=328 i=94 f=25 s=5 a=47)	(tpm=182.8 d=17.53 nps=1248328)
It's incredible - it's hard to keep up - kudos to the SF development team...
Image

User avatar
M ANSARI
Posts: 3466
Joined: Thu Mar 16, 2006 6:10 pm

Re: Progress of Stockfish in 6 days

Post by M ANSARI » Thu Aug 13, 2020 6:11 am

It is indeed incredible! Who would have thought that evaluation makes such a huge impact! So much for bean counting! I had always thought that a good evaluation was a shortcut to deep calculation, but never would have I imagined it to this level. The progress being made at closing the gap in speed between classic SF and NNUE SF is pretty impressive. I guess this brings AB engines back into pole position!

User avatar
MikeB
Posts: 4358
Joined: Thu Mar 09, 2006 5:34 am
Location: Pen Argyl, Pennsylvania

Re: Progress of Stockfish in 6 days

Post by MikeB » Thu Aug 13, 2020 6:12 am

I just kicked off a 30,000 game set for those might be interested,

Code: Select all

PGN File: c:/cluster.mfb/pgn/08130132.pgn
Time Control: Time Control-> base+inc: 25+0.500
Games: 30000
Threads: 1
Hash: 256

Current date : time (EDST)
Date: 08/13/20 : 02:07:35

Projected-> Time: 16h:23m:17s
     Run -> Time: 0h:35m:28s

Rank Name                 Rating   Δ     +    -     #     Σ    Σ%     W    L    D   W%    =%   OppR 
---------------------------------------------------------------------------------------------------------

   1 cur-dev-stockfish     3526   0.0   22   22   422  230.5  54.6  119   80  223  28.2  52.8  3494 
   2 Stockfish-XIr4-2257   3524   2.0   22   22   423  230.0  54.4  122   85  216  28.8  51.1  3494 
   3 Bluefish-2257         3502  22.3   23   23   424  213.5  50.4  107  104  213  25.2  50.2  3500 
   4 Honey-2257            3494   7.8   23   23   423  207.0  48.9  104  113  206  24.6  48.7  3501 
   5 Black-Diamond-2257    3453  40.9   23   23   422  176.0  41.7   73  143  206  17.3  48.8  3512 
---------------------------------------------------------------------------------------------------------

  Δ = delta from the next higher rated opponent
  # = number of games played
  Σ = total score, 1 point for win, 1/2 point for draw

LOS:
                     cu St Bl Ho Bl
cur-dev-stockfish       55 95 98 99
Stockfish-XIr4-2257  44    93 98 99
Bluefish-2257         4  6    70 99
Honey-2257            1  1 29    99
Black-Diamond-2257    0  0  0  0   
 
loops/scheduled: 6/200
Dropbox suspended my account for too many public transfers - will post an update in a few hours.
I have sacrificed Elo for Honey and Bluefish to maintain some problem solving capabilities - when you get 100 Elo for free in about 3 weeks, that is an easy choice to make.
Image

Jouni
Posts: 2199
Joined: Wed Mar 08, 2006 7:15 pm

Re: Progress of Stockfish in 6 days

Post by Jouni » Thu Aug 13, 2020 12:27 pm

My 100 game test SF NNUE 1_core vs SF 11 4_cores game in minute:

Code: Select all

                                 
1   Stockfish 120820 NNUE  +24  +20/=67/-13 53.50%   53.5/100
2   Stockfish 11 64 BMI2   -24  +13/=67/-20 46.50%   46.5/100

Also new faster compiles seems to be better in testsuites finally!
Jouni

User avatar
MikeB
Posts: 4358
Joined: Thu Mar 09, 2006 5:34 am
Location: Pen Argyl, Pennsylvania

Re: Progress of Stockfish in 6 days

Post by MikeB » Thu Aug 13, 2020 1:33 pm

MikeB wrote:
Thu Aug 13, 2020 6:12 am
I just kicked off a 30,000 game set for those might be interested,

Code: Select all

PGN File: c:/cluster.mfb/pgn/08130132.pgn
Time Control: Time Control-> base+inc: 25+0.500
Games: 30000
Threads: 1
Hash: 256

Current date : time (EDST)
Date: 08/13/20 : 02:07:35

Projected-> Time: 16h:23m:17s
     Run -> Time: 0h:35m:28s

Rank Name                 Rating   Δ     +    -     #     Σ    Σ%     W    L    D   W%    =%   OppR 
---------------------------------------------------------------------------------------------------------

   1 cur-dev-stockfish     3526   0.0   22   22   422  230.5  54.6  119   80  223  28.2  52.8  3494 
   2 Stockfish-XIr4-2257   3524   2.0   22   22   423  230.0  54.4  122   85  216  28.8  51.1  3494 
   3 Bluefish-2257         3502  22.3   23   23   424  213.5  50.4  107  104  213  25.2  50.2  3500 
   4 Honey-2257            3494   7.8   23   23   423  207.0  48.9  104  113  206  24.6  48.7  3501 
   5 Black-Diamond-2257    3453  40.9   23   23   422  176.0  41.7   73  143  206  17.3  48.8  3512 
---------------------------------------------------------------------------------------------------------

  Δ = delta from the next higher rated opponent
  # = number of games played
  Σ = total score, 1 point for win, 1/2 point for draw

LOS:
                     cu St Bl Ho Bl
cur-dev-stockfish       55 95 98 99
Stockfish-XIr4-2257  44    93 98 99
Bluefish-2257         4  6    70 99
Honey-2257            1  1 29    99
Black-Diamond-2257    0  0  0  0   
 
loops/scheduled: 6/200
Dropbox suspended my account for too many public transfers - will post an update in a few hours.
I have sacrificed Elo for Honey and Bluefish to maintain some problem solving capabilities - when you get 100 Elo for free in about 3 weeks, that is an easy choice to make.
just over the 1/2 way point

Code: Select all

PGN File: c:/cluster.mfb/pgn/08130132.pgn
Time Control: Time Control-> base+inc: 25+0.500
Games: 30000
Threads: 1
Hash: 256

Current date : time (EDST)
Date: 08/13/20 : 09:29:54

Projected-> Time: 16h:23m:17s
     Run -> Time: 7h:57m:47s

Rank Name                 Rating   Δ     +    -     #     Σ    Σ%     W    L    D   W%    =%   OppR 
---------------------------------------------------------------------------------------------------------

   1 Stockfish-XIr4-2257   3519   0.0    5    5  6032 3218.0  53.3 1419 1015 3598  23.5  59.6  3495 
   2 cur-dev-stockfish     3514   4.9    5    5  6030 3165.0  52.5 1344 1044 3642  22.3  60.4  3497 
   3 Bluefish-2257         3507   6.6    5    5  6032 3096.5  51.3 1275 1114 3643  21.1  60.4  3498 
   4 Honey-2257            3506   1.5    5    5  6032 3078.0  51.0 1237 1113 3682  20.5  61.0  3499 
   5 Black-Diamond-2257    3454  51.7    5    5  6030 2520.5  41.8  784 1773 3473  13.0  57.6  3511 
---------------------------------------------------------------------------------------------------------

  Δ = delta from the next higher rated opponent
  # = number of games played
  Σ = total score, 1 point for win, 1/2 point for draw

LOS:
                     St cu Bl Ho Bl
Stockfish-XIr4-2257     92 99 99100
cur-dev-stockfish     7    97 99100
Bluefish-2257         0  2    67100
Honey-2257            0  0 32   100
Black-Diamond-2257    0  0  0  0   
 
loops/scheduled: 93/200
Image

Post Reply