Some amazing 4CPU results for SFNNue

[MMarco]

The sample is small (100 games) but here are the results from Stockfish 110720 results vs different SFNNue nets on 4CPU, 180s + 1s:

Game conditions: 180s + 1s, TCEC 18 openings, 5-men syzygy
Adjudication: resign movecount=5 score=800, draw movenumber=50 movecount=5 score=5
Engine Options: SF: Threads=4, Hash=512. Lc0: cudnn-fp16, nncache=2000000, minibatchsize=64, maxprefetch=28
Hardware: Ryzen-7 3750h, GTX 1660 Ti.
Software: Cutechess GUI 1.0.0, Ordo 1.2.6
Comment: SV 20200723-1134 does an incredible performance!
Games: https://gofile.io/d/zJ7L7i

Code: Select all

     # PLAYER                       :  RATING  ERROR  PLAYED   (%)   CFS    W    D    L   D(%)
     1 SFNNue SV 20200723-1134 4CPU :   103.5   41.8    100   63.00   77   37   52   11  52.00
     2 Lc0 26.1 + LS 15.0           :    79.0   39.7    100   60.00   60   36   48   16  48.00
     3 SFNNue SV 20200721-1907 4CPU :    70.9   42.0    100   59.00   81   33   52   15  52.00
     4 SFNNue 1907 + FiNN 0.3  4CPU :    39.1   46.6    100   55.00   58   30   50   20  50.00
     5 SFNNue 1907 + FiNN 0.2  4CPU :    31.3   41.5    100   54.00   89   24   60   16  60.00
     6 Stockfish 110720        4CPU :     0.0   20.0    500   41.80  ---   78  262  160  52.40

White advantage = 115.89 +/- 11.11
Draw rate (equal opponents) = 61.87 % +/- 2.43

[mehmet123]

Thanks for the test.

What's is Stockfish Ribbit. Is it a hybrid version of Stockfish + Lc0 engine or another thing ?

[MMarco]

Thanks for the test.

What's is Stockfish Ribbit. Is it a hybrid version of Stockfish + Lc0 engine or another thing ?

It was a net posted on SF discord two days ago. But it just been shown to be a renamed 1134 from SV: http://talkchess.com/forum3/viewtopic.p ... 70#p853180

So I edited my message accordingly.

[MMarco]

The sample is small (100 games) but here are the results from Stockfish 110720 results vs different SFNNue nets on 4CPU, 180s + 1s:

Game conditions: 180s + 1s, TCEC 18 openings, 5-men syzygy
Adjudication: resign movecount=5 score=800, draw movenumber=50 movecount=5 score=5
Engine Options: SF: Threads=4, Hash=512. Lc0: cudnn-fp16, nncache=2000000, minibatchsize=64, maxprefetch=28
Hardware: Ryzen-7 3750h, GTX 1660 Ti.
Software: Cutechess GUI 1.0.0, Ordo 1.2.6
Comment: SV 20200723-1134 does an incredible performance!
Games: https://gofile.io/d/zJ7L7i

Code: Select all

     # PLAYER                       :  RATING  ERROR  PLAYED   (%)   CFS    W    D    L   D(%)
     1 SFNNue SV 20200723-1134 4CPU :   103.5   41.8    100   63.00   77   37   52   11  52.00
     2 Lc0 26.1 + LS 15.0           :    79.0   39.7    100   60.00   60   36   48   16  48.00
     3 SFNNue SV 20200721-1907 4CPU :    70.9   42.0    100   59.00   81   33   52   15  52.00
     4 SFNNue 1907 + FiNN 0.3  4CPU :    39.1   46.6    100   55.00   58   30   50   20  50.00
     5 SFNNue 1907 + FiNN 0.2  4CPU :    31.3   41.5    100   54.00   89   24   60   16  60.00
     6 Stockfish 110720        4CPU :     0.0   20.0    500   41.80  ---   78  262  160  52.40

White advantage = 115.89 +/- 11.11
Draw rate (equal opponents) = 61.87 % +/- 2.43

Maybe Bayeselo ratings are more realistic:

Code: Select all

Rank Name                      Elo    +    - games score oppo. draws
   1 SFNNue SV 20200723-1134    76   45   44   100   63%     0   52%
   2 Lc0 26.1 + LS 15.0         60   45   45   100   60%     0   48%
   3 SFNNue SV 20200721-1907    52   45   44   100   59%     0   52%
   4 SFNNue 1907 + FiNN 0.3     29   45   44   100   55%     0   50%
   5 SFNNue 1907 + FiNN 0.2     22   43   43   100   54%     0   60%
   6 Stockfish 110720            0   20   20   500   42%    48   52%

                         SF Lc SF SF SF St
SFNNue SV 20200723-1134     66 73 89 92 99
Lc0 26.1 + LS 15.0       33    57 78 83 98
SFNNue SV 20200721-1907  26 42    72 78 97
SFNNue 1907 + FiNN 0.3   10 21 27    57 85
SFNNue 1907 + FiNN 0.2    7 16 21 42    80
Stockfish 110720          0  1  2 14 19

[Modern Times]

Maybe Bayeselo ratings are more realistic:

Probably so. Most people use Ordo these days, but it really does explode the ratings range in comparison to BayesElo. The math is way beyond my level of comprehension. They are certainly a bit different from each other, neither right or wrong, so I guess you just make your choice.

[zullil]

Game conditions: 180s + 1s, TCEC 18 openings, 5-men syzygy

Where can those TCEC 18 openings be obtained?

[MMarco]

Game conditions: 180s + 1s, TCEC 18 openings, 5-men syzygy

Where can those TCEC 18 openings be obtained?

I got that here: http://blogchess2016.blogspot.com/?m=1

[zullil]

Game conditions: 180s + 1s, TCEC 18 openings, 5-men syzygy

Where can those TCEC 18 openings be obtained?

I got that here: http://blogchess2016.blogspot.com/?m=1

Thank you.