Sergio Vieri second net is out

MikeB · Post by **MikeB** » Sat Jul 25, 2020 7:02 pm

MikeB wrote: ↑Sat Jul 25, 2020 2:44 pm
JohnS wrote: ↑Sat Jul 25, 2020 10:06 am Another quick test of 2344 against H6.03 using Nunn1 openings, G10s+0.2s - result +11 =8 -1.

Here is a crazy sacrifice by SFnnue on move 13. Of course - usual disclaimers and as Mike would say yomv and ymmv.

[pgn][Event "SF-NNUE - Houdini 6.03, Nunn1, G10s + 0.2s"]
[Site "Home"]
[Date "2020.07.25"]
[Round "1"]
[White "Stockfish+NNUE"]
[Black "Houdini 6.03"]
[Result "1-0"]
[TimeControl "10+0.2"]
[Time "17:49:43"]
[Board "9"]
[Termination "adjudication by engines' scores"]
[ECO "D11"]
[Opening "QGD Slav"]

1. d4 d5 2. c4 c6
3. Nf3 {D11: QGD Slav, 3.Nf3} e6 4. cxd5 exd5
5. Nc3 Nf6 6. Bg5 Be7
7. Qc2 Nbd7 8. e3 O-O
9. Bd3 Re8 10. O-O Nf8 {End of opening}
11. h3 {+0.55/21 1.7 1825739} Ne4 {-0.29/16 0.8 1659811} 12. Bf4 {+0.57/15 0.3 340609} f5 {-0.29/15 0.5 1077857}
13. Nxd5 {+2.78/17 0.3 365358} cxd5 {+2.19/14 0.2 541228} 14. Bc7 {-2.63/19 0.6 586658} Qd7 {+2.19/15 0.4 804820}
15. Rfc1 {-2.40/18 0.3 305302} a6 {+2.04/17 1.1 2499173} 16. Qb3 {-2.44/19 0.4 428054} Qe6 {+1.94/17 0.5 1132772}
17. Rc2 {-2.32/17 0.3 291583} b5 {+1.87/18 1.0 2388891} 18. Ne5 {-1.83/17 0.6 596654} Bb7 {+1.99/15 0.3 587827}
19. a4 {-1.68/21 1.1 1189223} b4 {+1.31/16 1.0 2281160} 20. f3 {-1.84/19 0.4 414811} Ng3 {+1.69/15 0.2 526878}
21. Rac1 {-0.68/17 0.4 413539} Qh6 {+0.89/17 1.9 4376992} 22. f4 {+0.00/20 0.2 258037} Ne6 {+0.78/17 0.8 1960946}
23. Rc6 {+0.00/20 0.4 479539} Rec8 {+0.49/17 0.9 2314155} 24. Rb6 {+0.00/20 0.3 363630} Ra7 {+1.47/12 0.2 418161}
25. Kh2 {+0.00/21 0.3 296852} Ne4 {-0.22/17 0.7 1695180} 26. Bxe4 {+1.25/19 0.3 307773} fxe4 {-0.22/16 0.0 637}
27. Qd1 {+1.91/23 1.9 2128183} Re8 {-0.73/19 1.1 2793150} 28. a5 {+3.28/17 0.2 208229} Qf6 {-1.09/18 0.9 2123492}
29. Qa4 {+3.83/21 0.3 350003} Rea8 {-1.21/19 0.4 1173762} 30. f5 {+3.02/23 0.8 895388} Qxf5 {-1.21/18 0.0 2235}
31. Qd7 {+4.07/19 0.2 282314} Bf8 {-1.31/19 0.4 974990} 32. Qxe6+ {+3.94/21 0.5 615758} Qxe6 {-1.20/18 0.3 819661}
33. Rxe6 {+4.06/21 0.3 422568} Rc8 {-1.35/19 0.3 1033236} 34. Kg3 {+4.81/19 0.3 344820} g6 {-1.55/17 0.6 1706853}
35. Rf1 {+4.70/17 0.3 399405} Ba8 {-2.14/17 0.4 952201} 36. Bd6 {+5.59/18 0.3 380173} Bg7 {-2.34/15 0.2 546724}
37. Kh4 {+6.52/17 0.2 270354} Kh8 {-2.27/14 0.2 544657} 38. Kg5 {+7.02/18 0.3 316968} Kg8 {-2.03/14 0.1 320824}
39. g4 {+7.70/18 0.3 413143} b3 {-2.38/15 0.3 742604} 40. h4 {+8.11/19 0.4 558349} Bh8 {-2.84/15 0.2 553155}
41. h5 {+8.56/18 0.3 419325} gxh5 {-2.91/17 0.2 474664} 42. gxh5 {+8.93/18 0.3 455113} Rd8 {-3.47/13 0.2 646262}
43. Ng4 {+9.46/18 0.4 506316} Bg7 {-6.58/15 0.2 586254} 44. Nh6+ {+9.53/16 0.1 155076} Bxh6+ {-6.46/18 0.2 615138}
45. Kxh6 {+11.84/19 0.2 245093} Rad7 {-5.65/16 0.1 312964} 46. Bc5 {+12.79/22 0.2 318832} Rc8 {-8.61/17 0.3 950241}
47. Ref6 {+18.81/29 0.2 309342} Rdd8 {-17.45/21 0.2 743347} 48. Be7 {M+7/41 0.2 353040} Bb7 {M-6/21 0.0 107765}
49. Rg1+ {M+6/51 0.2 326491} Kh8 {M-5/11 0.0 724} 1-0[/pgn]
Nice game

2344 will soon be "my" KOM at least, after 1134 's "lucky" run to stay on top of the heap. yomv and ymmv

two threads here - - not that it matters but these nets seem to test at very stable scaling once you get near 30+.03 or with SMP I will do that tc next with the same two nets to to see how close it it.
Code: Select all
pgn file: c:/cluster.mfb/pgn/2007250307-23441134.pgn
tc/base+inc: 60+0.60
games planned: 4000
Threads: 2
Hash: 256

Current date : time (EDST)
Date: 07/25/20 : 08:42:00

Projected-> Time: 8h:32m:0s
Running  -> Time: 5h:34m:47s

2808 game(s) loaded
Rank Name  Rating   Δ     +    -     #     Σ    Σ%     W    L    D   W%    =%   OppR
---------------------------------------------------------------------------------------------------------

   1 2344   3502   0.0    8    8  2808 1418.5  50.5  644  615 1549  22.9  55.2  3498
   2 1134   3498   3.8    8    8  2808 1389.5  49.5  615  644 1549  21.9  55.2  3502
---------------------------------------------------------------------------------------------------------

  Δ = delta from the next higher rated opponent
  # = number of games played
  Σ = total score, 1 point for win, 1/2 point for draw

LOS:
      23 11
2344     81
1134  18

2808 game(s) loaded

loops/scheduled: 149/228

waiting: 128
  ...seconds remaining:   128

As expected after 2800 games. 2344 finsined on top with a nice indicated gain of 5 Elo perhaps.

Final Results:

Code: Select all

pgn file: c:/cluster.mfb/pgn/2007250307-23441134.pgn
tc/base+inc: 60+0.60
games planned: 4000
Threads: 2
Hash: 256

Current date : time (EDST)
Date: 07/25/20 : 11:37:34

Projected-> Time: 8h:32m:0s
Running  -> Time: 8h:30m:21s

4000 game(s) loaded
Rank Name  Rating   Δ     +    -     #     Σ    Σ%     W    L    D   W%    =%   OppR
---------------------------------------------------------------------------------------------------------

   1 2344   3503   0.0    7    7  4000 2028.0  50.7  928  872 2200  23.2  55.0  3497
   2 1134   3497   5.0    7    7  4000 1972.0  49.3  872  928 2200  21.8  55.0  3503
---------------------------------------------------------------------------------------------------------

  Δ = delta from the next higher rated opponent
  # = number of games played
  Σ = total score, 1 point for win, 1/2 point for draw

LOS:
      23 11
2344     91
1134   8

4000 game(s) loaded

loops/scheduled: 228/228

waiting: 128
  ...seconds remaining:   0

done

4000 games and 8+ hours running of 50 concurrent games at once and the expected completion time falls within two minutes of the projected finish time. If this completion time indicator is off substantially, that usually means something went awry and the games should be checked for anomalies.

Next up is a 30+.3 set with the same nets, 4000 games with single thread to see how well of a proxy this TC is for longer games with multiple threads.

Below is the cutechess-cli output for the match above:

Code: Select all

Score of 2344 vs 1134: 928 - 872 - 2200  [0.507] 4000
...      2344 playing White: 911 - 29 - 1060  [0.721] 2000
...      2344 playing Black: 17 - 843 - 1140  [0.293] 2000
...      White vs Black: 1754 - 46 - 2200  [0.714] 4000
Elo difference: 4.9 +/- 7.2, LOS: 90.7 %, DrawRatio: 55.0 %
Finished match

carldaman · Post by **carldaman** » Sat Jul 25, 2020 7:43 pm

cdani wrote: ↑Sat Jul 25, 2020 3:56 pm Someone tested experimental nets? I don't see many comments about them.

http://talkchess.com/forum3/viewtopic.p ... 10#p852962

MikeB · Post by **MikeB** » Sat Jul 25, 2020 8:03 pm

carldaman wrote: ↑Sat Jul 25, 2020 7:43 pm
cdani wrote: ↑Sat Jul 25, 2020 3:56 pm Someone tested experimental nets? I don't see many comments about them.
http://talkchess.com/forum3/viewtopic.p ... 10#p852962

I have tested a few - everyone was weaker.

chrisw · Post by **chrisw** » Sat Jul 25, 2020 8:12 pm

MikeB wrote: ↑Sat Jul 25, 2020 8:03 pm
carldaman wrote: ↑Sat Jul 25, 2020 7:43 pm
cdani wrote: ↑Sat Jul 25, 2020 3:56 pm Someone tested experimental nets? I don't see many comments about them.
http://talkchess.com/forum3/viewtopic.p ... 10#p852962
I have tested a few - everyone was weaker.

Yup, that happens when you test lots of things that are more or less the same and put them in some sort of ranked list. There’s one at the top and one at the bottom and the rest in the middle.

Rebel · Post by **Rebel** » Sat Jul 25, 2020 8:56 pm

I am giving up for the moment testing Sergio nets, although (just) 2000 games is far from accurate the magic seems to have gone quite quickly after the first releases. I have tested 22 versions. The wait is for the SF team, they have more man and computer power. Can't wait....

Code: Select all

SF-NNUE (popcount) vs Stockfish 11, 2000 games, tc 40m/20s, input 8moves.pgn
henk-2706     54.7%   2020-07-19
sergio-1432   56.1%   2020-07-21 15:56
sergio-1907   58.9%   2020-07-21 19:14
sergio-2323   58.4%   2020-07-21 23:23
sergio-0359   58.8%   2020-07-22 03:59
sergio-0944   58.9%   2020-07-22 09:44
sergio-1153   58.3%   2020-07-22 11:53
sergio-1807   58.0%   2020-07-22 18:07
sergio-2210   59.0%   2020-07-22 22:10
sergio-0511   58.2%   2020-07-23 05:11
sergio-1134   56.5%   2020-07-23 11:34 with 4moves.pgn 58.7%
sergio-1844   57.8%   2020-07-23 18:44 
sergio-1843   58.7%   2020-07-23 18:43 
sergio-0123   57.2%   2020-07-24 01:23 
sergio-0640   56.7%   2020-07-24 06:40 
sergio-1240   57.6%   2020-07-24 12:40
sergio-1224   58.1%   2020-07-24 12:24
sergio-1732   58.7%   2020-07-24 17:32
sergio-2344   57.2%   2020-07-24 23:54
ribbit-0.1    58.7% = 1134 (!!)
sergio-1313   58.5%   2020-07-25 13:13
sergio-2242   57.5%   2020-07-25 22:42

Note that the experimental versions are labeled with the time of day on the webpage.

Sylwy · Post by **Sylwy** » Sat Jul 25, 2020 8:59 pm

An interesting comparative test (in progress):

cdani · Post by **cdani** » Sat Jul 25, 2020 10:09 pm

Laskos wrote: ↑Sat Jul 25, 2020 5:19 pm
cdani wrote: ↑Sat Jul 25, 2020 5:14 pm 20200725-2051.bin seems better than 20200723-0511 after a few hundred games. I publish the test later with more games.
About equal here after 500 games (to 2141).

Is possible that is a tiny bit stronger. 20+0.01

Code: Select all

   # PLAYER                       : RATING  ERROR   POINTS  PLAYED    (%)
   1 stnnuesergio20200725-2051    : 2856.6    2.7   2301.0    4586   50.2%
   2 stnnuesergio20200723-0511    : 2855.4    2.7   2285.0    4586   49.8%

JJJ · Post by **JJJ** » Sat Jul 25, 2020 10:27 pm

Hello, I have trouble understanding how and where to dl Stockfish NNE and his best nets. Also, I d like to run it on arena. Can someone explain this to me ? Or is there a tutorial for this already ?

adnoh · Post by **adnoh** » Sat Jul 25, 2020 10:54 pm

JJJ wrote: ↑Sat Jul 25, 2020 10:27 pm Hello, I have trouble understanding how and where to dl Stockfish NNE and his best nets. Also, I d like to run it on arena. Can someone explain this to me ? Or is there a tutorial for this already ?

To get the SFNNUE executable, go here.
https://github.com/nodchip/Stockfish/releases

The current latest is 2020-07-19. There are a lot of exe in the archive so get the one that is suitable for your CPU but note the naming convention and make sure you get the one that actually uses the network file.

For my fairly old CPU, I am using this.
stockfish.sse42.halfkp_256x2-32-32.profile-nnue.2020-07-19.exe

For the nets, go here and place them in a subdirectory called "eval" (convention only)
https://www.comp.nus.edu.sg/~sergio-v/nnue/

There are many people testing which one is best. I find this one quite good for my conditions.
20200722-2141.bin

In Arena, install it like any other UCI engine and the important part is use the configure engine dialog to point the EvalFile option to where you placed the net.

JohnS · Post by **JohnS** » Sun Jul 26, 2020 3:52 am

Now 2344 demolishes Shredder 12 +20 =0 -0. This is getting scary.

Here are two games with thematic attacks for white and black in the Sicilian.

[pgn][Event "SF-NNUE - Shredder 12, Nunn1, G10s + 0.2s"]
[Site "Home"]
[Date "2020.07.26"]
[Round "1"]
[White "Stockfish+NNUE"]
[Black "Shredder 12"]
[Result "1-0"]
[TimeControl "10+0.2"]
[Time "11:21:02"]
[Board "3"]
[Termination "adjudication by engines' scores"]
[ECO "B89"]
[Opening "Sicilian"]

1. e4 c5 2. Nf3 Nc6
3. d4 cxd4 4. Nxd4 Nf6
5. Nc3 d6 6. Bc4 e6
7. Be3 {B89: Sicilian, Sozin, 7.Be3} a6 8. Qe2 Qc7
9. O-O-O Be7 10. Bb3 O-O {End of opening}
11. g4 {+0.73/20 1.5 1730869} b5 {+0.22/11 0.5 358616} 12. g5 {+0.78/18 0.2 258602} Nxd4 {+0.09/11 0.6 473519}
13. Bxd4 {+1.62/21 2.4 2782003} Nd7 {+0.38/11 0.9 667035} 14. g6 {+0.87/20 0.4 504769} hxg6 {+1.20/11 0.6 431139}
15. h4 {+1.15/19 0.3 373466} Nf6 {+0.72/11 0.8 663427} 16. h5 {+3.13/18 0.2 272207} Nxh5 {-0.91/9 1.3 1015604}
17. Rxh5 {+6.07/17 0.4 474824} gxh5 {-0.52/10 1.2 1024174} 18. Qxh5 {+6.17/18 0.3 354308} f6 {-1.07/9 1.9 1646389}
19. Nd5 {+6.41/16 0.2 273053} Qd7 {-2.65/11 2.2 1924129} 20. Rg1 {+7.10/16 0.3 393924} Qb7 {-3.01/8 0.6 500513}
21. Nf4 {+8.52/18 0.6 714416} d5 {-4.31/8 0.3 247280} 22. exd5 {+8.71/16 0.2 276766} Qc7 {-4.31/7 0.3 212154}
23. Qh6 {+9.01/17 0.3 336780} Bd6 {-4.98/10 0.2 167502} 24. Nh5 {+9.33/16 0.2 285277} Bg3 {-6.29/10 0.2 147664}
25. d6 {+9.73/17 0.2 329550} gxh6 {-6.75/10 0.2 167500} 26. Rxg3+ {+9.98/17 0.3 386582} Qg7 {-6.26/9 0.2 212946}
27. Rxg7+ {+10.46/19 0.3 398407} Kh8 {-9.94/10 0.1 44111} 1-0[/pgn]

[pgn][Event "SF-NNUE - Shredder 12, Nunn1, G10s + 0.2s"]
[Site "Home"]
[Date "2020.07.26"]
[Round "1"]
[White "Shredder 12"]
[Black "Stockfish+NNUE"]
[Result "0-1"]
[TimeControl "10+0.2"]
[Time "11:21:26"]
[Board "4"]
[Termination "adjudication by engines' scores"]
[ECO "B89"]
[Opening "Sicilian"]

1. e4 c5 2. Nf3 Nc6
3. d4 cxd4 4. Nxd4 Nf6
5. Nc3 d6 6. Bc4 e6
7. Be3 {B89: Sicilian, Sozin, 7.Be3} a6 8. Qe2 Qc7
9. O-O-O Be7 10. Bb3 O-O {End of opening}
11. g4 {-0.16/11 0.5 397071} Nxd4 {-0.52/20 2.0 2274057} 12. Rxd4 {-0.22/12 0.3 250950} b5 {-0.61/18 0.2 196167}
13. g5 {-0.21/11 0.5 365590} Nd7 {-0.52/18 0.3 399853} 14. a3 {-0.23/10 0.5 385094} Nc5 {+0.21/16 0.2 279355}
15. Ba2 {-0.08/9 0.4 265902} Rb8 {+0.59/18 0.7 784110} 16. f4 {-0.46/12 6.7 5652681} a5 {+1.94/17 0.3 307332}
17. f5 {-0.14/9 0.2 176918} b4 {+3.48/17 0.3 326674} 18. axb4 {-2.56/9 0.6 485541} axb4 {+3.76/18 0.5 579923}
19. Nb5 {-2.76/8 0.4 339451} Qa5 {+4.06/18 0.4 453591} 20. f6 {-2.86/8 0.4 285342} Qxa2 {+4.46/19 0.4 557965}
21. fxe7 {-1.10/9 0.2 168118} Re8 {+4.56/18 0.3 401500} 22. Nxd6 {-1.19/7 0.2 151911} Rxe7 {+4.92/18 0.3 404609}
23. Rhd1 {-1.65/7 0.2 154231} Ba6 {+5.27/16 0.2 295512} 24. Nc4 {-2.53/7 0.2 147263} Ree8 {+7.40/17 0.3 357731}
25. Bf4 {-2.53/7 0.2 138309} e5 {+7.73/16 0.3 348733} 26. Bxe5 {-3.36/8 0.4 327440} Rbc8 {+7.68/16 0.3 369188}
27. Qg4 {-3.36/7 0.2 111875} b3 {+8.15/16 0.4 500084} 28. c3 {-6.02/7 0.2 204072} Ne6 {+8.37/15 0.4 421937}
29. Rd7 {-7.07/7 0.3 203441} Bxc4 {+8.71/13 0.3 339399} 30. Rb7 {-6.79/7 0.2 113112} Nc5 {+10.06/14 0.4 441740}
31. Re7 {-10.72/7 0.2 148313} 0-1[/pgn]

Sergio Vieri second net is out

Re: Sergio Vieri second net is out

Re: Sergio Vieri second net is out

Re: Sergio Vieri second net is out

Re: Sergio Vieri second net is out

Re: Sergio Vieri second net is out

Re: Sergio Vieri second net is out

Re: Sergio Vieri second net is out

Re: Sergio Vieri second net is out

Re: Sergio Vieri second net is out

Re: Sergio Vieri second net is out