Sergio Vieri second net is out

Discussion of anything and everything relating to chess playing software and machines.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
User avatar
MikeB
Posts: 4358
Joined: Thu Mar 09, 2006 5:34 am
Location: Pen Argyl, Pennsylvania

Re: Sergio Vieri second net is out

Post by MikeB » Sat Jul 25, 2020 5:02 pm

MikeB wrote:
Sat Jul 25, 2020 12:44 pm
JohnS wrote:
Sat Jul 25, 2020 8:06 am
Another quick test of 2344 against H6.03 using Nunn1 openings, G10s+0.2s - result +11 =8 -1.

Here is a crazy sacrifice by SFnnue on move 13. Of course - usual disclaimers and as Mike would say yomv and ymmv.

Nice game

2344 will soon be "my" KOM at least, after 1134 's "lucky" run to stay on top of the heap. yomv and ymmv

two threads here - - not that it matters but these nets seem to test at very stable scaling once you get near 30+.03 or with SMP I will do that tc next with the same two nets to to see how close it it.

Code: Select all

pgn file: c:/cluster.mfb/pgn/2007250307-23441134.pgn
tc/base+inc: 60+0.60
games planned: 4000
Threads: 2
Hash: 256

Current date : time (EDST)
Date: 07/25/20 : 08:42:00

Projected-> Time: 8h:32m:0s
Running  -> Time: 5h:34m:47s

2808 game(s) loaded
Rank Name  Rating   Δ     +    -     #     Σ    Σ%     W    L    D   W%    =%   OppR
---------------------------------------------------------------------------------------------------------

   1 2344   3502   0.0    8    8  2808 1418.5  50.5  644  615 1549  22.9  55.2  3498
   2 1134   3498   3.8    8    8  2808 1389.5  49.5  615  644 1549  21.9  55.2  3502
---------------------------------------------------------------------------------------------------------

  Δ = delta from the next higher rated opponent
  # = number of games played
  Σ = total score, 1 point for win, 1/2 point for draw

LOS:
      23 11
2344     81
1134  18

2808 game(s) loaded

loops/scheduled: 149/228

waiting: 128
  ...seconds remaining:   128
As expected after 2800 games. 2344 finsined on top with a nice indicated gain of 5 Elo perhaps.


Final Results:

Code: Select all

pgn file: c:/cluster.mfb/pgn/2007250307-23441134.pgn
tc/base+inc: 60+0.60
games planned: 4000
Threads: 2
Hash: 256

Current date : time (EDST)
Date: 07/25/20 : 11:37:34

Projected-> Time: 8h:32m:0s
Running  -> Time: 8h:30m:21s

4000 game(s) loaded
Rank Name  Rating   Δ     +    -     #     Σ    Σ%     W    L    D   W%    =%   OppR
---------------------------------------------------------------------------------------------------------

   1 2344   3503   0.0    7    7  4000 2028.0  50.7  928  872 2200  23.2  55.0  3497
   2 1134   3497   5.0    7    7  4000 1972.0  49.3  872  928 2200  21.8  55.0  3503
---------------------------------------------------------------------------------------------------------

  Δ = delta from the next higher rated opponent
  # = number of games played
  Σ = total score, 1 point for win, 1/2 point for draw

LOS:
      23 11
2344     91
1134   8

4000 game(s) loaded

loops/scheduled: 228/228

waiting: 128
  ...seconds remaining:   0

done
4000 games and 8+ hours running of 50 concurrent games at once and the expected completion time falls within two minutes of the projected finish time. If this completion time indicator is off substantially, that usually means something went awry and the games should be checked for anomalies.

Next up is a 30+.3 set with the same nets, 4000 games with single thread to see how well of a proxy this TC is for longer games with multiple threads.

Below is the cutechess-cli output for the match above:

Code: Select all

Score of 2344 vs 1134: 928 - 872 - 2200  [0.507] 4000
...      2344 playing White: 911 - 29 - 1060  [0.721] 2000
...      2344 playing Black: 17 - 843 - 1140  [0.293] 2000
...      White vs Black: 1754 - 46 - 2200  [0.714] 4000
Elo difference: 4.9 +/- 7.2, LOS: 90.7 %, DrawRatio: 55.0 %
Finished match
Image

carldaman
Posts: 1985
Joined: Sat Jun 02, 2012 12:13 am

Re: Sergio Vieri second net is out

Post by carldaman » Sat Jul 25, 2020 5:43 pm

cdani wrote:
Sat Jul 25, 2020 1:56 pm
Someone tested experimental nets? I don't see many comments about them.
viewtopic.php?f=6&t=74518&start=10#p852962

User avatar
MikeB
Posts: 4358
Joined: Thu Mar 09, 2006 5:34 am
Location: Pen Argyl, Pennsylvania

Re: Sergio Vieri second net is out

Post by MikeB » Sat Jul 25, 2020 6:03 pm

carldaman wrote:
Sat Jul 25, 2020 5:43 pm
cdani wrote:
Sat Jul 25, 2020 1:56 pm
Someone tested experimental nets? I don't see many comments about them.
viewtopic.php?f=6&t=74518&start=10#p852962
I have tested a few - everyone was weaker.
Image

chrisw
Posts: 3641
Joined: Tue Apr 03, 2012 2:28 pm

Re: Sergio Vieri second net is out

Post by chrisw » Sat Jul 25, 2020 6:12 pm

MikeB wrote:
Sat Jul 25, 2020 6:03 pm
carldaman wrote:
Sat Jul 25, 2020 5:43 pm
cdani wrote:
Sat Jul 25, 2020 1:56 pm
Someone tested experimental nets? I don't see many comments about them.
viewtopic.php?f=6&t=74518&start=10#p852962
I have tested a few - everyone was weaker.
Yup, that happens when you test lots of things that are more or less the same and put them in some sort of ranked list. There’s one at the top and one at the bottom and the rest in the middle.

User avatar
Rebel
Posts: 5487
Joined: Thu Aug 18, 2011 10:04 am

Re: Sergio Vieri second net is out

Post by Rebel » Sat Jul 25, 2020 6:56 pm

I am giving up for the moment testing Sergio nets, although (just) 2000 games is far from accurate the magic seems to have gone quite quickly after the first releases. I have tested 22 versions. The wait is for the SF team, they have more man and computer power. Can't wait....

Code: Select all

SF-NNUE (popcount) vs Stockfish 11, 2000 games, tc 40m/20s, input 8moves.pgn
henk-2706     54.7%   2020-07-19
sergio-1432   56.1%   2020-07-21 15:56
sergio-1907   58.9%   2020-07-21 19:14
sergio-2323   58.4%   2020-07-21 23:23
sergio-0359   58.8%   2020-07-22 03:59
sergio-0944   58.9%   2020-07-22 09:44
sergio-1153   58.3%   2020-07-22 11:53
sergio-1807   58.0%   2020-07-22 18:07
sergio-2210   59.0%   2020-07-22 22:10
sergio-0511   58.2%   2020-07-23 05:11
sergio-1134   56.5%   2020-07-23 11:34 with 4moves.pgn 58.7%
sergio-1844   57.8%   2020-07-23 18:44 
sergio-1843   58.7%   2020-07-23 18:43 
sergio-0123   57.2%   2020-07-24 01:23 
sergio-0640   56.7%   2020-07-24 06:40 
sergio-1240   57.6%   2020-07-24 12:40
sergio-1224   58.1%   2020-07-24 12:24
sergio-1732   58.7%   2020-07-24 17:32
sergio-2344   57.2%   2020-07-24 23:54
ribbit-0.1    58.7% = 1134 (!!)
sergio-1313   58.5%   2020-07-25 13:13
sergio-2242   57.5%   2020-07-25 22:42
Note that the experimental versions are labeled with the time of day on the webpage.
90% of coding is debugging, the other 10% is writing bugs.

User avatar
Sylwy
Posts: 3417
Joined: Fri Apr 21, 2006 2:19 pm
Location: IASI (Romania) - the historical capital of MOLDOVA

Re: Sergio Vieri second net is out

Post by Sylwy » Sat Jul 25, 2020 6:59 pm

An interesting comparative test (in progress):

https://docs.google.com/spreadsheets/d/ ... 1407938838

User avatar
cdani
Posts: 2172
Joined: Sat Jan 18, 2014 9:24 am
Location: Andorra
Contact:

Re: Sergio Vieri second net is out

Post by cdani » Sat Jul 25, 2020 8:09 pm

Laskos wrote:
Sat Jul 25, 2020 3:19 pm
cdani wrote:
Sat Jul 25, 2020 3:14 pm
20200725-2051.bin seems better than 20200723-0511 after a few hundred games. I publish the test later with more games.
About equal here after 500 games (to 2141).
Is possible that is a tiny bit stronger. 20+0.01

Code: Select all

   # PLAYER                       : RATING  ERROR   POINTS  PLAYED    (%)
   1 stnnuesergio20200725-2051    : 2856.6    2.7   2301.0    4586   50.2%
   2 stnnuesergio20200723-0511    : 2855.4    2.7   2285.0    4586   49.8%

JJJ
Posts: 1316
Joined: Sat Apr 19, 2014 11:47 am

Re: Sergio Vieri second net is out

Post by JJJ » Sat Jul 25, 2020 8:27 pm

Hello, I have trouble understanding how and where to dl Stockfish NNE and his best nets. Also, I d like to run it on arena. Can someone explain this to me ? Or is there a tutorial for this already ?

adnoh
Posts: 17
Joined: Tue Jun 26, 2007 4:31 am
Full name: Charles Wong

Re: Sergio Vieri second net is out

Post by adnoh » Sat Jul 25, 2020 8:54 pm

JJJ wrote:
Sat Jul 25, 2020 8:27 pm
Hello, I have trouble understanding how and where to dl Stockfish NNE and his best nets. Also, I d like to run it on arena. Can someone explain this to me ? Or is there a tutorial for this already ?
To get the SFNNUE executable, go here.
https://github.com/nodchip/Stockfish/releases

The current latest is 2020-07-19. There are a lot of exe in the archive so get the one that is suitable for your CPU but note the naming convention and make sure you get the one that actually uses the network file.

For my fairly old CPU, I am using this.
stockfish.sse42.halfkp_256x2-32-32.profile-nnue.2020-07-19.exe

For the nets, go here and place them in a subdirectory called "eval" (convention only)
https://www.comp.nus.edu.sg/~sergio-v/nnue/

There are many people testing which one is best. I find this one quite good for my conditions.
20200722-2141.bin

In Arena, install it like any other UCI engine and the important part is use the configure engine dialog to point the EvalFile option to where you placed the net.

JohnS
Posts: 183
Joined: Sun Feb 24, 2008 1:08 am

Re: Sergio Vieri second net is out

Post by JohnS » Sun Jul 26, 2020 1:52 am

Now 2344 demolishes Shredder 12 +20 =0 -0. This is getting scary.

Here are two games with thematic attacks for white and black in the Sicilian.




Post Reply