Progress of Stockfish in 6 days

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Rebel, chrisw

duncan
Posts: 12038
Joined: Mon Jul 07, 2008 10:50 pm

Re: Progress of Stockfish in 6 days

Post by duncan »

duncan wrote: Wed Aug 12, 2020 11:21 pm Does anyone have the daily/monthly elo Progress of Rybka 3 to 2, and 4 to 3, so we can compare?
I believe Rybka maximum gain was 72 elo per year for rybka 3.

Rybka 1.0 32-bit, 2787 released may 12, 2005
Rybka 1.0 64-bit, 2824
Rybka 2.1 32-bit, 2862, 75 gain, released Jul 26, 2006, 64 elo /year
Rybka 2.2 64-bit, 2921, 97 gain
Rybka 2.2 64-bit 4CPU, 2973
Rybka 3 32-bit, 2996, 134 gain
Rybka 3 64-bit 4CPU, 3118, 145 gain, released August 6, 2008, 72 elo /year
Rybka 4 64-bit 4CPU, 3147, 29 gain , released May 26, 2010, 16 elo /year

https://www.stmintz.com/ccc/index.php?id=466792
https://www.chessprogramming.org/Rybka#Versions

https://ccrl.chessdom.com/ccrl/4040/cgi ... +opponents

https://www.chessprogramming.org/Rybka_Controversy
JJJ
Posts: 1346
Joined: Sat Apr 19, 2014 1:47 pm

Re: Progress of Stockfish in 6 days

Post by JJJ »

lkaufman wrote: Thu Aug 13, 2020 7:22 am
Jouni wrote: Wed Aug 12, 2020 9:36 pm Yes SF NNUE is equal to quadruple your CPU cores for free. Incredible :!: :!: .
I actually got a result that SFNNUE (a couple days ago) on one thread beat Stockfish 11 on seven threads, at 2' + 1", by 90 to 80! So you may be understating it!
Maybe its time to test Stockfish NNE again at knight handicap and see if it is stronger than any other program !
Jhoravi
Posts: 291
Joined: Wed May 08, 2013 6:49 am

Re: Progress of Stockfish in 6 days

Post by Jhoravi »

JJJ wrote: Thu Aug 13, 2020 4:13 pm
lkaufman wrote: Thu Aug 13, 2020 7:22 am
Jouni wrote: Wed Aug 12, 2020 9:36 pm Yes SF NNUE is equal to quadruple your CPU cores for free. Incredible :!: :!: .
I actually got a result that SFNNUE (a couple days ago) on one thread beat Stockfish 11 on seven threads, at 2' + 1", by 90 to 80! So you may be understating it!
Maybe its time to test Stockfish NNE again at knight handicap and see if it is stronger than any other program !
Unless NNUE is trained to play in losing positions.
Werewolf
Posts: 1795
Joined: Thu Sep 18, 2008 10:24 pm

Re: Progress of Stockfish in 6 days

Post by Werewolf »

Is it now totally clear cut that SF NNUE is stronger than Lc0?

If Lc0 had 2x2080 Ti (say) what hardware would SF NNUE need to equal it?
User avatar
Laskos
Posts: 10948
Joined: Wed Jul 26, 2006 10:21 pm
Full name: Kai Laskos

Re: Progress of Stockfish in 6 days

Post by Laskos »

Werewolf wrote: Thu Aug 13, 2020 5:48 pm Is it now totally clear cut that SF NNUE is stronger than Lc0?

If Lc0 had 2x2080 Ti (say) what hardware would SF NNUE need to equal it?
Probably 16 cores.
lkaufman
Posts: 5960
Joined: Sun Jan 10, 2010 6:15 am
Location: Maryland USA

Re: Progress of Stockfish in 6 days

Post by lkaufman »

JJJ wrote: Thu Aug 13, 2020 4:13 pm
lkaufman wrote: Thu Aug 13, 2020 7:22 am
Jouni wrote: Wed Aug 12, 2020 9:36 pm Yes SF NNUE is equal to quadruple your CPU cores for free. Incredible :!: :!: .
I actually got a result that SFNNUE (a couple days ago) on one thread beat Stockfish 11 on seven threads, at 2' + 1", by 90 to 80! So you may be understating it!
Maybe its time to test Stockfish NNE again at knight handicap and see if it is stronger than any other program !
I've already done some tests with SFNNUE giving handicaps to other engines (and to myself!). Unfortunately it can't be tested at knight odds because it switches to normal SF mode when down (or up) about 3 pawns or more in material+piece location, with a knight counting as about 4 pawns. I ran one test with it giving pawn odds (c2,d2,e2 rotating) to SF 11 and it drew four, lost six. Against weaker engines under CCRL blitz conditions I found that around CCRL 3050 blitz is a fair opponent for it at odds of two pawns and move (remove b7 or c7 and f7 or g7, so four options). I've also tried playing it this way and also at "pawn and 3 moves" (remove f7, play e4 and d4, WTM) and so far I have only losses. In general it seems to be able to give handicaps to engines roughly 100 elo or so higher than those that SF11 can give the same handicaps to.
Komodo rules!
User avatar
Guenther
Posts: 4605
Joined: Wed Oct 01, 2008 6:33 am
Location: Regensburg, Germany
Full name: Guenther Simon

Re: Progress of Stockfish in 6 days

Post by Guenther »

lkaufman wrote: Thu Aug 13, 2020 8:33 pm ...
I've already done some tests with SFNNUE giving handicaps to other engines (and to myself!). Unfortunately it can't be tested at knight odds because it switches to normal SF mode when down (or up) about 3 pawns or more in material+piece location...
You know that there are various compilations/flavours, which have NNUE always enabled?
https://rwbc-chess.de

trollwatch:
Chessqueen + chessica + AlexChess + Eduard + Sylwy
JJJ
Posts: 1346
Joined: Sat Apr 19, 2014 1:47 pm

Re: Progress of Stockfish in 6 days

Post by JJJ »

lkaufman wrote: Thu Aug 13, 2020 8:33 pm
JJJ wrote: Thu Aug 13, 2020 4:13 pm
lkaufman wrote: Thu Aug 13, 2020 7:22 am
Jouni wrote: Wed Aug 12, 2020 9:36 pm Yes SF NNUE is equal to quadruple your CPU cores for free. Incredible :!: :!: .
I actually got a result that SFNNUE (a couple days ago) on one thread beat Stockfish 11 on seven threads, at 2' + 1", by 90 to 80! So you may be understating it!
Maybe its time to test Stockfish NNE again at knight handicap and see if it is stronger than any other program !
I've already done some tests with SFNNUE giving handicaps to other engines (and to myself!). Unfortunately it can't be tested at knight odds because it switches to normal SF mode when down (or up) about 3 pawns or more in material+piece location, with a knight counting as about 4 pawns. I ran one test with it giving pawn odds (c2,d2,e2 rotating) to SF 11 and it drew four, lost six. Against weaker engines under CCRL blitz conditions I found that around CCRL 3050 blitz is a fair opponent for it at odds of two pawns and move (remove b7 or c7 and f7 or g7, so four options). I've also tried playing it this way and also at "pawn and 3 moves" (remove f7, play e4 and d4, WTM) and so far I have only losses. In general it seems to be able to give handicaps to engines roughly 100 elo or so higher than those that SF11 can give the same handicaps to.
too bad. I guess we ll have to wait for Komodo NN to see a better engine for this handicap.
User avatar
Laskos
Posts: 10948
Joined: Wed Jul 26, 2006 10:21 pm
Full name: Kai Laskos

Re: Progress of Stockfish in 6 days

Post by Laskos »

lkaufman wrote: Thu Aug 13, 2020 8:33 pm
JJJ wrote: Thu Aug 13, 2020 4:13 pm
lkaufman wrote: Thu Aug 13, 2020 7:22 am
Jouni wrote: Wed Aug 12, 2020 9:36 pm Yes SF NNUE is equal to quadruple your CPU cores for free. Incredible :!: :!: .
I actually got a result that SFNNUE (a couple days ago) on one thread beat Stockfish 11 on seven threads, at 2' + 1", by 90 to 80! So you may be understating it!
Maybe its time to test Stockfish NNE again at knight handicap and see if it is stronger than any other program !
I've already done some tests with SFNNUE giving handicaps to other engines (and to myself!). Unfortunately it can't be tested at knight odds because it switches to normal SF mode when down (or up) about 3 pawns or more in material+piece location, with a knight counting as about 4 pawns. I ran one test with it giving pawn odds (c2,d2,e2 rotating) to SF 11 and it drew four, lost six. Against weaker engines under CCRL blitz conditions I found that around CCRL 3050 blitz is a fair opponent for it at odds of two pawns and move (remove b7 or c7 and f7 or g7, so four options). I've also tried playing it this way and also at "pawn and 3 moves" (remove f7, play e4 and d4, WTM) and so far I have only losses. In general it seems to be able to give handicaps to engines roughly 100 elo or so higher than those that SF11 can give the same handicaps to.
I had results that NNUE plays badly unfamiliar chess like chess variants, and I assumed that it plays badly handicaps too. Not terribly bad, but underperforming compared to its strength in regular chess.
mwyoung
Posts: 2727
Joined: Wed May 12, 2010 10:00 pm

Re: Progress of Stockfish in 6 days

Post by mwyoung »

lkaufman wrote: Thu Aug 13, 2020 7:22 am
Jouni wrote: Wed Aug 12, 2020 9:36 pm Yes SF NNUE is equal to quadruple your CPU cores for free. Incredible :!: :!: .
I actually got a result that SFNNUE (a couple days ago) on one thread beat Stockfish 11 on seven threads, at 2' + 1", by 90 to 80! So you may be understating it!
I also achieved this kind of results. But I hope to God you don't think it can achieve this results in with more threads, and time. Because it does not, it is only marginally better then the best standard Stockfish, and Lc0. The strongest program yes, crushing the other engines no!
"The worst thing that can happen to a forum is a running wild attacking moderator(HGM) who is not corrected by the community." - Ed Schröder
But my words like silent raindrops fell. And echoed in the wells of silence.