Stockfish 14 vs Stockfisk 13:
Program Elo + - Games Score Av.Op. Draws
1 Stockfish 14 x64 bmi2 : 2428 11 11 1000 58.1 % 2372 72.7 %
2 Stockfish 13 x64 bmi2 : 2372 11 11 1000 41.9 % 2428 72.7 %
Individual statistics:
1 Stockfish 14 x64 bmi2 : 2428 1000 (+217,=727,- 56), 58.0 %
Stockfish 13 x64 bmi2 : 1000 (+217,=727,- 56), 58.1 %
2 Stockfish 13 x64 bmi2 : 2372 1000 (+ 56,=727,-217), 42.0 %
Stockfish 14 x64 bmi2 : 1000 (+ 56,=727,-217), 41.9 %
Game Conditions: Cutechess Gui, 1 Core (Core-i7 9750h), 10 sec + 0.2 sec TC, Balsa 5 Moves Opening Book, 64 Mb Hash, Ponder Off
https://www.mediafire.com/file/826pqr1y ... 7.pgn/file
Stockfish 14 is very strong at ultra fast time (10 sec + 0.2 sec) control matches.
Stockfish NNUE SV Tests
Moderators: hgm, Rebel, chrisw
-
- Posts: 671
- Joined: Sun Jan 26, 2020 10:38 pm
- Location: Turkey
- Full name: Mehmet Karaman
-
- Posts: 671
- Joined: Sun Jan 26, 2020 10:38 pm
- Location: Turkey
- Full name: Mehmet Karaman
Re: Stockfish NNUE SV Tests
Stockfish 14 vs Stockfisk 13:
Program Elo + - Games Score Av.Op. Draws
1 Stockfish 14 x64 bmi2 : 2417 8 7 1000 54.8 % 2383 87.0 %
2 Stockfish 13 x64 bmi2 : 2383 7 8 1000 45.2 % 2417 87.0 %
Individual statistics:
1 Stockfish 14 x64 bmi2 : 2417 1000 (+113,=870,- 17), 54.8 %
Stockfish 13 x64 bmi2 : 1000 (+113,=870,- 17), 54.8 %
2 Stockfish 13 x64 bmi2 : 2383 1000 (+ 17,=870,-113), 45.2 %
Stockfish 14 x64 bmi2 : 1000 (+ 17,=870,-113), 45.2 %
Game Conditions: Cutechess Gui, 1 Core (Core-i7 9750h), 1 min + 0.5 sec TC, Balsa 5 Moves Opening Book, 128 Mb Hash, Ponder Off
https://www.mediafire.com/file/1m24lq7a ... 8.pgn/file
Program Elo + - Games Score Av.Op. Draws
1 Stockfish 14 x64 bmi2 : 2417 8 7 1000 54.8 % 2383 87.0 %
2 Stockfish 13 x64 bmi2 : 2383 7 8 1000 45.2 % 2417 87.0 %
Individual statistics:
1 Stockfish 14 x64 bmi2 : 2417 1000 (+113,=870,- 17), 54.8 %
Stockfish 13 x64 bmi2 : 1000 (+113,=870,- 17), 54.8 %
2 Stockfish 13 x64 bmi2 : 2383 1000 (+ 17,=870,-113), 45.2 %
Stockfish 14 x64 bmi2 : 1000 (+ 17,=870,-113), 45.2 %
Game Conditions: Cutechess Gui, 1 Core (Core-i7 9750h), 1 min + 0.5 sec TC, Balsa 5 Moves Opening Book, 128 Mb Hash, Ponder Off
https://www.mediafire.com/file/1m24lq7a ... 8.pgn/file
-
- Posts: 671
- Joined: Sun Jan 26, 2020 10:38 pm
- Location: Turkey
- Full name: Mehmet Karaman
Re: Stockfish NNUE SV Tests
ChessMan compile vs Official compile:
Program Elo + - Games Score Av.Op. Draws
1 Stockfish 14 x64 bmi2 C : 2401 7 7 1200 50.3 % 2399 85.9 %
2 Stockfish 14 x64 bmi2 O : 2399 7 7 1200 49.7 % 2401 85.9 %
Individual statistics:
1 Stockfish 14 x64 bmi2 C : 2401 1200 (+ 88,=1031,- 81), 50.3 %
Stockfish 14 x64 bmi2 O : 1200 (+ 88,=1031,- 81), 50.3 %
2 Stockfish 14 x64 bmi2 O : 2399 1200 (+ 81,=1031,- 88), 49.7 %
Stockfish 14 x64 bmi2 C : 1200 (+ 81,=1031,- 88), 49.7 %
Game Conditions: Cutechess Gui, 1 Core (Core-i7 9750h), 10 sec + 0.2 sec TC, Balsa 5 Moves Opening Book, 64 Mb Hash, Ponder Off
Stockfish 14 x64 bmi2 C:ChessMan compile (https://pixeldrain.com/u/cAfodUFh)
Stockfish 14 x64 bmi2 O:Official compile (https://stockfishchess.org/download/windows/)
https://www.mediafire.com/file/ateyd24j ... 9.pgn/file
It would make more sense to do a speed test for Stockfish versions as different versions may be faster on different systems.
Program Elo + - Games Score Av.Op. Draws
1 Stockfish 14 x64 bmi2 C : 2401 7 7 1200 50.3 % 2399 85.9 %
2 Stockfish 14 x64 bmi2 O : 2399 7 7 1200 49.7 % 2401 85.9 %
Individual statistics:
1 Stockfish 14 x64 bmi2 C : 2401 1200 (+ 88,=1031,- 81), 50.3 %
Stockfish 14 x64 bmi2 O : 1200 (+ 88,=1031,- 81), 50.3 %
2 Stockfish 14 x64 bmi2 O : 2399 1200 (+ 81,=1031,- 88), 49.7 %
Stockfish 14 x64 bmi2 C : 1200 (+ 81,=1031,- 88), 49.7 %
Game Conditions: Cutechess Gui, 1 Core (Core-i7 9750h), 10 sec + 0.2 sec TC, Balsa 5 Moves Opening Book, 64 Mb Hash, Ponder Off
Stockfish 14 x64 bmi2 C:ChessMan compile (https://pixeldrain.com/u/cAfodUFh)
Stockfish 14 x64 bmi2 O:Official compile (https://stockfishchess.org/download/windows/)
https://www.mediafire.com/file/ateyd24j ... 9.pgn/file
It would make more sense to do a speed test for Stockfish versions as different versions may be faster on different systems.
-
- Posts: 671
- Joined: Sun Jan 26, 2020 10:38 pm
- Location: Turkey
- Full name: Mehmet Karaman
Re: Stockfish NNUE SV Tests
Koivisto 6.0 vs Stockfish 8:
Program Elo + - Games Score Av.Op. Draws
1 Koivisto 6.0 x64 avx2 : 2411 20 20 500 53.1 % 2389 57.0 %
2 Stockfish 8 x64 bmi2 : 2389 20 20 500 46.9 % 2411 57.0 %
Individual statistics:
1 Koivisto 6.0 x64 avx2 : 2411 500 (+123,=285,- 92), 53.1 %
Stockfish 8 x64 bmi2 : 500 (+123,=285,- 92), 53.1 %
2 Stockfish 8 x64 bmi2 : 2389 500 (+ 92,=285,-123), 46.9 %
Koivisto 6.0 x64 avx2 : 500 (+ 92,=285,-123), 46.9 %
Game Conditions: Cutechess Gui, 1 Core (Core-i7 9750h), 1 min + 0.5 sec TC, Balsa 5 Moves Opening Book, 64 Mb Hash, Ponder Off
https://www.mediafire.com/file/ddvy797i ... 4.pgn/file
Koivisto chess engine is one of the few chess engines to dominate Stockfish 8.
Program Elo + - Games Score Av.Op. Draws
1 Koivisto 6.0 x64 avx2 : 2411 20 20 500 53.1 % 2389 57.0 %
2 Stockfish 8 x64 bmi2 : 2389 20 20 500 46.9 % 2411 57.0 %
Individual statistics:
1 Koivisto 6.0 x64 avx2 : 2411 500 (+123,=285,- 92), 53.1 %
Stockfish 8 x64 bmi2 : 500 (+123,=285,- 92), 53.1 %
2 Stockfish 8 x64 bmi2 : 2389 500 (+ 92,=285,-123), 46.9 %
Koivisto 6.0 x64 avx2 : 500 (+ 92,=285,-123), 46.9 %
Game Conditions: Cutechess Gui, 1 Core (Core-i7 9750h), 1 min + 0.5 sec TC, Balsa 5 Moves Opening Book, 64 Mb Hash, Ponder Off
https://www.mediafire.com/file/ddvy797i ... 4.pgn/file
Koivisto chess engine is one of the few chess engines to dominate Stockfish 8.
-
- Posts: 671
- Joined: Sun Jan 26, 2020 10:38 pm
- Location: Turkey
- Full name: Mehmet Karaman
Re: Stockfish NNUE SV Tests
Koivisto 6 vs Stockfish 8:
Program Elo + - Games Score Av.Op. Draws
1 Koivisto 6.0 x64 avx2 : 2415 21 21 600 54.4 % 2385 45.2 %
2 Stockfish 8 x64 bmi2 : 2385 21 21 600 45.6 % 2415 45.2 %
Individual statistics:
1 Koivisto 6.0 x64 avx2 : 2415 600 (+191,=271,-138), 54.4 %
Stockfish 8 x64 bmi2 : 600 (+191,=271,-138), 54.4 %
2 Stockfish 8 x64 bmi2 : 2385 600 (+138,=271,-191), 45.6 %
Koivisto 6.0 x64 avx2 : 600 (+138,=271,-191), 45.6 %
Game Conditions: Cutechess Gui, 1 Core (Core-i7 9750h), 10 sec + 0.2 sec TC, Balsa 5 Moves Opening Book, 64 Mb Hash, Ponder Off
https://www.mediafire.com/file/xoosj8fy ... 5.pgn/file
Koivisto 6 was able to dominate not only Stockfish 8 but also Houdini 6.
Koivisto vs Houdini:
Program Elo + - Games Score Av.Op. Draws
1 Koivisto 6.0 x64 avx2 : 2402 19 19 500 50.6 % 2398 59.2 %
2 Houdini 6 Pro x64 pext : 2398 19 19 500 49.4 % 2402 59.2 %
Individual statistics:
1 Koivisto 6.0 x64 avx2 : 2402 500 (+105,=296,- 99), 50.6 %
Houdini 6 Pro x64 pext : 500 (+105,=296,- 99), 50.6 %
2 Houdini 6 Pro x64 pext : 2398 500 (+ 99,=296,-105), 49.4 %
Koivisto 6.0 x64 avx2 : 500 (+ 99,=296,-105), 49.4 %
Game Conditions: Cutechess Gui, 1 Core (Core-i7 9750h), 2 min + 0.5 sec TC, Balsa 5 Moves Opening Book, 128 Mb Hash, Ponder Off
https://www.mediafire.com/file/r11q49or ... 7.pgn/file
Program Elo + - Games Score Av.Op. Draws
1 Koivisto 6.0 x64 avx2 : 2415 21 21 600 54.4 % 2385 45.2 %
2 Stockfish 8 x64 bmi2 : 2385 21 21 600 45.6 % 2415 45.2 %
Individual statistics:
1 Koivisto 6.0 x64 avx2 : 2415 600 (+191,=271,-138), 54.4 %
Stockfish 8 x64 bmi2 : 600 (+191,=271,-138), 54.4 %
2 Stockfish 8 x64 bmi2 : 2385 600 (+138,=271,-191), 45.6 %
Koivisto 6.0 x64 avx2 : 600 (+138,=271,-191), 45.6 %
Game Conditions: Cutechess Gui, 1 Core (Core-i7 9750h), 10 sec + 0.2 sec TC, Balsa 5 Moves Opening Book, 64 Mb Hash, Ponder Off
https://www.mediafire.com/file/xoosj8fy ... 5.pgn/file
Koivisto 6 was able to dominate not only Stockfish 8 but also Houdini 6.
Koivisto vs Houdini:
Program Elo + - Games Score Av.Op. Draws
1 Koivisto 6.0 x64 avx2 : 2402 19 19 500 50.6 % 2398 59.2 %
2 Houdini 6 Pro x64 pext : 2398 19 19 500 49.4 % 2402 59.2 %
Individual statistics:
1 Koivisto 6.0 x64 avx2 : 2402 500 (+105,=296,- 99), 50.6 %
Houdini 6 Pro x64 pext : 500 (+105,=296,- 99), 50.6 %
2 Houdini 6 Pro x64 pext : 2398 500 (+ 99,=296,-105), 49.4 %
Koivisto 6.0 x64 avx2 : 500 (+ 99,=296,-105), 49.4 %
Game Conditions: Cutechess Gui, 1 Core (Core-i7 9750h), 2 min + 0.5 sec TC, Balsa 5 Moves Opening Book, 128 Mb Hash, Ponder Off
https://www.mediafire.com/file/r11q49or ... 7.pgn/file
-
- Posts: 671
- Joined: Sun Jan 26, 2020 10:38 pm
- Location: Turkey
- Full name: Mehmet Karaman
Re: Stockfish NNUE SV Tests
Stockfish miniNNUE vs Stockfish Dev:
Program Elo + - Games Score Av.Op. Draws
1 Stockfish 130721 x64 bmi2 : 2401 7 7 700 50.2 % 2399 93.0 %
2 Stockfish miniNNUE 170721 bmi2 : 2400 7 7 700 49.8 % 2401 93.0 %
Individual statistics:
1 Stockfish 130721 x64 bmi2 : 2401 700 (+ 26,=651,- 23), 50.2 %
Stockfish miniNNUE 170721 bmi2 : 700 (+ 26,=651,- 23), 50.2 %
2 Stockfish miniNNUE 170721 bmi2 : 2400 700 (+ 23,=651,- 26), 49.8 %
Stockfish 130721 x64 bmi2 : 700 (+ 23,=651,- 26), 49.8 %
Game Conditions: Cutechess Gui, 1 Core (Core-i7 9750h), 2 min + 0.5 sec TC, Balsa 5 Moves Opening Book, 128 Mb Hash, Ponder Off
https://www.mediafire.com/file/4zgtt4t2 ... 8.pgn/file
The performance of Stockfish miniNNUE against Stockfish Dev is very good considering it plays with a Net that is about 1/4 the size of Default Net.
Program Elo + - Games Score Av.Op. Draws
1 Stockfish 130721 x64 bmi2 : 2401 7 7 700 50.2 % 2399 93.0 %
2 Stockfish miniNNUE 170721 bmi2 : 2400 7 7 700 49.8 % 2401 93.0 %
Individual statistics:
1 Stockfish 130721 x64 bmi2 : 2401 700 (+ 26,=651,- 23), 50.2 %
Stockfish miniNNUE 170721 bmi2 : 700 (+ 26,=651,- 23), 50.2 %
2 Stockfish miniNNUE 170721 bmi2 : 2400 700 (+ 23,=651,- 26), 49.8 %
Stockfish 130721 x64 bmi2 : 700 (+ 23,=651,- 26), 49.8 %
Game Conditions: Cutechess Gui, 1 Core (Core-i7 9750h), 2 min + 0.5 sec TC, Balsa 5 Moves Opening Book, 128 Mb Hash, Ponder Off
https://www.mediafire.com/file/4zgtt4t2 ... 8.pgn/file
The performance of Stockfish miniNNUE against Stockfish Dev is very good considering it plays with a Net that is about 1/4 the size of Default Net.
-
- Posts: 671
- Joined: Sun Jan 26, 2020 10:38 pm
- Location: Turkey
- Full name: Mehmet Karaman
Re: Stockfish NNUE SV Tests
Stockfish miniNNUE vs Stockfish Dev (Fischer Random Chess):
Program Elo + - Games Score Av.Op. Draws
1 Stockfish miniNNUE 170721 bmi2 : 2409 12 12 700 52.6 % 2391 79.3 %
2 Stockfish 230721 x64 bmi2 : 2391 12 12 700 47.4 % 2409 79.3 %
Individual statistics:
1 Stockfish miniNNUE 170721 bmi2 : 2409 700 (+ 91,=555,- 54), 52.6 %
Stockfish 230721 x64 bmi2 : 700 (+ 91,=555,- 54), 52.6 %
2 Stockfish 230721 x64 bmi2 : 2391 700 (+ 54,=555,- 91), 47.4 %
Stockfish miniNNUE 170721 bmi2 : 700 (+ 54,=555,- 91), 47.4 %
Game Conditions: Cutechess Gui, 1 Core (Core-i7 9750h), 2 min + 0.5 sec TC, Chess960 3 Moves Opening Book, 128 Mb Hash, Ponder Off
https://www.mediafire.com/file/v0lzyjtw ... 9.pgn/file
The performance of Stockfish miniNNUE against Stockfish Dev is great (+18 elo) at FRC match.
Program Elo + - Games Score Av.Op. Draws
1 Stockfish miniNNUE 170721 bmi2 : 2409 12 12 700 52.6 % 2391 79.3 %
2 Stockfish 230721 x64 bmi2 : 2391 12 12 700 47.4 % 2409 79.3 %
Individual statistics:
1 Stockfish miniNNUE 170721 bmi2 : 2409 700 (+ 91,=555,- 54), 52.6 %
Stockfish 230721 x64 bmi2 : 700 (+ 91,=555,- 54), 52.6 %
2 Stockfish 230721 x64 bmi2 : 2391 700 (+ 54,=555,- 91), 47.4 %
Stockfish miniNNUE 170721 bmi2 : 700 (+ 54,=555,- 91), 47.4 %
Game Conditions: Cutechess Gui, 1 Core (Core-i7 9750h), 2 min + 0.5 sec TC, Chess960 3 Moves Opening Book, 128 Mb Hash, Ponder Off
https://www.mediafire.com/file/v0lzyjtw ... 9.pgn/file
The performance of Stockfish miniNNUE against Stockfish Dev is great (+18 elo) at FRC match.
-
- Posts: 3554
- Joined: Thu Jun 07, 2012 11:02 pm
-
- Posts: 671
- Joined: Sun Jan 26, 2020 10:38 pm
- Location: Turkey
- Full name: Mehmet Karaman
Re: Stockfish NNUE SV Tests
Stockfish miniNNUE vs Stockfish Dev:
Program Elo + - Games Score Av.Op. Draws
1 Stockfish 240721 x64 bmi2 : 2400 5 5 800 50.0 % 2400 96.5 %
2 Stockfish miniNNUE 170721 bmi2 : 2400 5 5 800 50.0 % 2400 96.5 %
Individual statistics:
1 Stockfish 240721 x64 bmi2 : 2400 800 (+ 14,=772,- 14), 50.0 %
Stockfish miniNNUE 170721 bmi2: 800 (+ 14,=772,- 14), 50.0 %
2 Stockfish miniNNUE 170721 bmi2: 2400 800 (+ 14,=772,- 14), 50.0 %
Stockfish 240721 x64 bmi2 : 800 (+ 14,=772,- 14), 50.0 %
Game Conditions: Arena Gui, 6 Cores (Core-i7 9750h), 1 min + 0.5 sec TC, Balsa 5 Moves Opening Book, 1024 Mb Hash, Ponder Off
https://www.mediafire.com/file/r8w74ybt ... 1.pgn/file
After 800 games 1 min+ 0.5 sec time control match at 6 cores Stockish Dev can't beat Stockfish miniNNUE 170721. After 770 games Stockfish miniNNUE 170721 was ahead of with a score of 386-384 (+ 14,=744,- 12)
We only know which NNUE nets are best for 1 min + 0.6 sec (1 core) and faster time control matches. We don't have enough information about which NNUE nets are strong in games longer than 1 min + 0.6 sec time control matches or in matches where Stockfish chess engines are played with multiprocessors
Program Elo + - Games Score Av.Op. Draws
1 Stockfish 240721 x64 bmi2 : 2400 5 5 800 50.0 % 2400 96.5 %
2 Stockfish miniNNUE 170721 bmi2 : 2400 5 5 800 50.0 % 2400 96.5 %
Individual statistics:
1 Stockfish 240721 x64 bmi2 : 2400 800 (+ 14,=772,- 14), 50.0 %
Stockfish miniNNUE 170721 bmi2: 800 (+ 14,=772,- 14), 50.0 %
2 Stockfish miniNNUE 170721 bmi2: 2400 800 (+ 14,=772,- 14), 50.0 %
Stockfish 240721 x64 bmi2 : 800 (+ 14,=772,- 14), 50.0 %
Game Conditions: Arena Gui, 6 Cores (Core-i7 9750h), 1 min + 0.5 sec TC, Balsa 5 Moves Opening Book, 1024 Mb Hash, Ponder Off
https://www.mediafire.com/file/r8w74ybt ... 1.pgn/file
After 800 games 1 min+ 0.5 sec time control match at 6 cores Stockish Dev can't beat Stockfish miniNNUE 170721. After 770 games Stockfish miniNNUE 170721 was ahead of with a score of 386-384 (+ 14,=744,- 12)
We only know which NNUE nets are best for 1 min + 0.6 sec (1 core) and faster time control matches. We don't have enough information about which NNUE nets are strong in games longer than 1 min + 0.6 sec time control matches or in matches where Stockfish chess engines are played with multiprocessors
-
- Posts: 671
- Joined: Sun Jan 26, 2020 10:38 pm
- Location: Turkey
- Full name: Mehmet Karaman
Re: Stockfish NNUE SV Tests
Fire vs Stockfish:
Program Elo + - Games Score Av.Op. Draws
1 Fire 8.NN MC x64 bmi2 : 2431 15 15 1100 58.8 % 2369 48.9 %
2 Stockfish 310720 x64 bmi2 : 2369 15 15 1100 41.2 % 2431 48.9 %
Individual statistics:
1 Fire 8.NN MC x64 bmi2 : 2431 1100 (+378,=538,-184), 58.8 %
Stockfish 310720 x64 bmi2 : 1100 (+378,=538,-184), 58.8 %
2 Stockfish 310720 x64 bmi2 : 2369 1100 (+184,=538,-378), 41.2 %
Fire 8.NN MC x64 bmi2 : 1100 (+184,=538,-378), 41.2 %
Game Conditions: Cutechess Gui, 1 Core (Core-i7 9750h), 10 sec + 0.2 sec TC, Balsa 5 Moves Opening Book, 64 Mb Hash, Ponder Off
https://www.mediafire.com/file/c2uj4isv ... 4.pgn/file
Stockfish 310720 is the last Stockfish Dev version without NNUE evaluation.
Program Elo + - Games Score Av.Op. Draws
1 Fire 8.NN MC x64 bmi2 : 2431 15 15 1100 58.8 % 2369 48.9 %
2 Stockfish 310720 x64 bmi2 : 2369 15 15 1100 41.2 % 2431 48.9 %
Individual statistics:
1 Fire 8.NN MC x64 bmi2 : 2431 1100 (+378,=538,-184), 58.8 %
Stockfish 310720 x64 bmi2 : 1100 (+378,=538,-184), 58.8 %
2 Stockfish 310720 x64 bmi2 : 2369 1100 (+184,=538,-378), 41.2 %
Fire 8.NN MC x64 bmi2 : 1100 (+184,=538,-378), 41.2 %
Game Conditions: Cutechess Gui, 1 Core (Core-i7 9750h), 10 sec + 0.2 sec TC, Balsa 5 Moves Opening Book, 64 Mb Hash, Ponder Off
https://www.mediafire.com/file/c2uj4isv ... 4.pgn/file
Stockfish 310720 is the last Stockfish Dev version without NNUE evaluation.