Stockfish NNUE SV Tests

Discussion of computer chess matches and engine tournaments.

Moderators: Harvey Williamson, Dann Corbit, hgm

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Post Reply
mehmet123
Posts: 256
Joined: Sun Jan 26, 2020 9:38 pm
Location: Turkey
Full name: Mehmet Karaman

Stockfish NNUE SV Tests

Post by mehmet123 » Tue Jul 21, 2020 3:02 am

Program Elo + - Games Score Av.Op. Draws

1 Stockfish NNUE SV 200720 : 2412 19 19 360 53.5 % 2388 71.4 %
2 Stockfish 110720 x64 bmi2 : 2388 19 19 360 46.5 % 2412 71.4 %

Individual statistics:

1 Stockfish SV 200720 : 2412 360 (+ 64,=257,- 39), 53.5 %

Stockfish 110720 x64 bmi2 : 360 (+ 64,=257,- 39), 53.5 %

2 Stockfish 110720 x64 bmi2 : 2388 360 (+ 39,=257,- 64), 46.5 %

Stockfish SV 200720 : 360 (+ 39,=257,- 64), 46.5 %



Cutechess Gui, 1 Core (i7 9750h), 1 min + 0.5 sec TC, Balsa 5 Move Opening Book, 16 Mb Hash, Ponder Off

http://www.mediafire.com/file/pztyyaako ... 2.pgn/file

mehmet123
Posts: 256
Joined: Sun Jan 26, 2020 9:38 pm
Location: Turkey
Full name: Mehmet Karaman

Re: Stockfish NNUE SV Tests

Post by mehmet123 » Tue Jul 21, 2020 3:08 am

Program Elo + - Games Score Av.Op. Draws

1 Stockfish NNUE SV 200720 : 2411 21 20 360 53.2 % 2389 67.5 %
2 Stockfish 110720 x64 bmi2 : 2389 20 21 360 46.8 % 2411 67.5 %

Individual statistics:

1 Stockfish NNUE SV 200720 : 2411 360 (+ 70,=243,- 47), 53.2 %

Stockfish 110720 x64 bmi2 : 360 (+ 70,=243,- 47), 53.2 %

2 Stockfish 110720 x64 bmi2 : 2389 360 (+ 47,=243,- 70), 46.8 %

Stockfish NNUE SV 200720 : 360 (+ 47,=243,- 70), 46.8 %


Cutechess Gui, 1 Core (i7 9750h), 1 min + 0.5 sec TC, Balsa 5 Move Opening Book, 128 Mb Hash, Ponder Off

http://www.mediafire.com/file/9ylqyfgi3 ... 3.pgn/file

mehmet123
Posts: 256
Joined: Sun Jan 26, 2020 9:38 pm
Location: Turkey
Full name: Mehmet Karaman

Re: Stockfish NNUE SV Tests

Post by mehmet123 » Tue Jul 21, 2020 3:16 am

I used Sergio Vieri's net (20200720-1017) in these tests.

Download link of Sergio Vieri's nets:
https://www.comp.nus.edu.sg/~sergio-v/nnue/

mehmet123
Posts: 256
Joined: Sun Jan 26, 2020 9:38 pm
Location: Turkey
Full name: Mehmet Karaman

Re: Stockfish NNUE SV Tests

Post by mehmet123 » Tue Jul 21, 2020 6:34 pm

Test of Sergio Vieri's 20200721-1907 Net

Program Elo + - Games Score Av.Op. Draws

1 Stockfish NNUE SV 210720 : 2420 23 23 300 55.7 % 2380 65.3 %
2 Stockfish 170720 x64 bmi2 : 2380 23 23 300 44.3 % 2420 65.3 %

Individual statistics:

1 Stockfish NNUE SV 210720 : 2420 300 (+ 69,=196,- 35), 55.7 %

Stockfish 170720 x64 bmi2 : 300 (+ 69,=196,- 35), 55.7 %

2 Stockfish 170720 x64 bmi2 : 2380 300 (+ 35,=196,- 69), 44.3 %

Stockfish NNUE SV 210720 : 300 (+ 35,=196,- 69), 44.3 %


Cutechess Gui, 1 Core (i7 9750h), 1 min + 0.5 sec TC, Balsa 5 Move Opening Book, 64 Mb Hash, Ponder Off

http://www.mediafire.com/file/ierfyzg7u ... 1.pgn/file

mehmet123
Posts: 256
Joined: Sun Jan 26, 2020 9:38 pm
Location: Turkey
Full name: Mehmet Karaman

Re: Stockfish NNUE SV Tests

Post by mehmet123 » Tue Jul 21, 2020 9:09 pm

Test of Sergio Vieri's 20200721-1907 Net

Program Elo + - Games Score Av.Op. Draws

1 Stockfish NNUE SV 210720 : 2421 20 20 400 55.9 % 2379 66.2 %
2 Stockfish 170720 x64 bmi2 : 2379 20 20 400 44.1 % 2421 66.2 %

Individual statistics:


1 Stockfish NNUE SV 210720 : 2421 400 (+ 91,=265,- 44), 55.9 %

Stockfish 170720 x64 bmi2 : 400 (+ 91,=265,- 44), 55.9 %

2 Stockfish 170720 x64 bmi2 : 2379 400 (+ 44,=265,- 91), 44.1 %

Stockfish NNUE SV 210720 : 400 (+ 44,=265,- 91), 44.1 %


Cutechess Gui, 1 Core (i7 9750h), 30 sec + 0.5 sec TC, Balsa 5 Move Opening Book, 16 Mb Hash, Ponder Off

http://www.mediafire.com/file/7aoefr0an ... 7.pgn/file

mehmet123
Posts: 256
Joined: Sun Jan 26, 2020 9:38 pm
Location: Turkey
Full name: Mehmet Karaman

Re: Stockfish NNUE SV Tests

Post by mehmet123 » Tue Jul 21, 2020 9:15 pm

For now Stockfish SV 20200721-1907 is the strongest chess engine according to my tests.

The performance of Stockfish NNUE Gekehenker 2706 against Stockfish Dev is + 32 elo (1750 games)

The performance of Stockfish NNUE SV 20200721-1907 against Stockfish Dev is + 41 elo (700 games)

mehmet123
Posts: 256
Joined: Sun Jan 26, 2020 9:38 pm
Location: Turkey
Full name: Mehmet Karaman

Re: Stockfish NNUE SV Tests

Post by mehmet123 » Wed Jul 22, 2020 7:36 pm

Test of SV 20200722-2141 Net

Program Elo + - Games Score Av.Op. Draws

1 Stockfish NNUE SV 220720 : 2426 21 21 360 57.5 % 2374 65.6 %
2 Stockfish 170720 x64 bmi2 : 2374 21 21 360 42.5 % 2426 65.6 %

Individual statistics:

1 Stockfish NNUE SV 220720 : 2426 360 (+ 89,=236,- 35), 57.5 %

Stockfish 170720 x64 bmi2 : 360 (+ 89,=236,- 35), 57.5 %

2 Stockfish 170720 x64 bmi2 : 2374 360 (+ 35,=236,- 89), 42.5 %

Stockfish NNUE SV 220720 : 360 (+ 35,=236,- 89), 42.5 %


Cutechess Gui, 1 Core (i7 9750h), 1 min + 0.5 sec TC, Balsa 5 Move Opening Book, 64 Mb Hash, Ponder Off

http://www.mediafire.com/file/7cmqv0my2 ... 0.pgn/file


This is the best test result of Stockish NNUE in my tests (minimum 200 games )

mehmet123
Posts: 256
Joined: Sun Jan 26, 2020 9:38 pm
Location: Turkey
Full name: Mehmet Karaman

Re: Stockfish NNUE SV Tests

Post by mehmet123 » Thu Jul 23, 2020 3:45 pm

Test of SV Experimental-20200722-1130 Net

Program Elo + - Games Score Av.Op. Draws

1 Stockfish NNUE SV Exp 1130 : 2432 19 18 420 59.2 % 2368 67.4 %
2 Stockfish 170720 x64 bmi2 : 2368 18 19 420 40.8 % 2432 67.4 %

Individual statistics:

1 Stockfish NNUE SV Exp 1130 : 2432 420 (+107,=283,- 30), 59.2 %

Stockfish 170720 x64 bmi2 : 420 (+107,=283,- 30), 59.2 %

2 Stockfish 170720 x64 bmi2 : 2368 420 (+ 30,=283,-107), 40.8 %

Stockfish NNUE SV Exp 1130 : 420 (+ 30,=283,-107), 40.8 %


Cutechess Gui, 1 Core (i7 9750h), 30 sec + 0.5 sec TC, Balsa 5 Move Opening Book, 16 Mb Hash, Ponder Off

http://www.mediafire.com/file/8llhb5qui ... 5.pgn/file


According to my tests this is the strongest experimental net of SV. Because of that I decided to test this version against Stocfish Dev.

This is the best test score of Stockfish NNUE against Stockfish Dev in my tests.

mehmet123
Posts: 256
Joined: Sun Jan 26, 2020 9:38 pm
Location: Turkey
Full name: Mehmet Karaman

Re: Stockfish NNUE SV Tests

Post by mehmet123 » Thu Jul 23, 2020 5:54 pm

Test of SV Experimental-20200722-1130 Net

Program Elo + - Games Score Av.Op. Draws

1 Stockfish NNUE SV 1130 : 2426 23 23 275 57.5 % 2374 67.6 %
2 Stockfish 170720 x64 bmi2 : 2374 23 23 275 42.5 % 2426 67.6 %

Individual statistics:

1 Stockfish NNUE SV 1130 : 2426 275 (+ 65,=186,- 24), 57.5 %

Stockfish 170720 x64 bmi2 : 275 (+ 65,=186,- 24), 57.5 %

2 Stockfish 170720 x64 bmi2 : 2374 275 (+ 24,=186,- 65), 42.5 %

Stockfish NNUE SV 1130 : 275 (+ 24,=186,- 65), 42.5 %


Cutechess Gui, 1 Core (i7 9750h), 1 min + 0.5 sec TC, Balsa 5 Move Opening Book, 64 Mb Hash, Ponder Off

http://www.mediafire.com/file/wndete5r4 ... 6.pgn/file


Another great result of Stockfish NNUE (SV Exp 20200722-1130)

mehmet123
Posts: 256
Joined: Sun Jan 26, 2020 9:38 pm
Location: Turkey
Full name: Mehmet Karaman

Re: Stockfish NNUE SV Tests

Post by mehmet123 » Thu Jul 23, 2020 7:58 pm

Test of SV Experimental-20200722-1130 Net

Program Elo + - Games Score Av.Op. Draws

1 Stockfish NNUE SV 1130 : 2427 21 21 330 57.7 % 2373 68.2 %
2 Stockfish 170720 x64 bmi2 : 2373 21 21 330 42.3 % 2427 68.2 %

Individual statistics:

1 Stockfish NNUE SV 1130 : 2427 330 (+ 78,=225,- 27), 57.7 %

Stockfish 170720 x64 bmi2 : 330 (+ 78,=225,- 27), 57.7 %

2 Stockfish 170720 x64 bmi2 : 2373 330 (+ 27,=225,- 78), 42.3 %

Stockfish NNUE SV 1130 : 330 (+ 27,=225,- 78), 42.3 %


Cutechess Gui, 1 Core (i7 9750h), 2 min + 0.5 sec TC, Balsa 5 Move Opening Book, 64 Mb Hash, Ponder Off

http://www.mediafire.com/file/ck4kj0eig ... 7.pgn/file


This is the best test score of Stockfish NNUE against Stockfish Dev in my tests at 2 min + 0.5 sec time control matches.

Post Reply