Stockfish NNUE SV Tests

[mehmet123]

Fat Titz vs Stockfish Dev (10 sec + 0.2 sec TC):

Program Elo + - Games Score Av.Op. Draws

1 Stockfish 220821 x64 bmi2 : 2422 17 16 360 56.4 % 2378 78.3 %
2 Fat Titz x64 bmi2 : 2378 16 17 360 43.6 % 2422 78.3 %

Individual statistics:

1 Stockfish 220821 x64 bmi2 : 2422 360 (+ 62,=282,- 16), 56.4 %

Fat Titz x64 bmi2 : 360 (+ 62,=282,- 16), 56.4 %

2 Fat Titz x64 bmi2 : 2378 360 (+ 16,=282,- 62), 43.6 %

Stockfish 220821 x64 bmi2 : 360 (+ 16,=282,- 62), 43.6 %


Game Conditions: Cutechess Gui, 1 Core (Core-i7 9750h), 10 sec + 0.2 sec TC, Balsa 5 Moves Opening Book, 64 Mb Hash, Ponder Off
https://www.mediafire.com/file/6d906r87 ... 9.pgn/file


Fat Titz vs Stockfish Dev (1 min + 0.5 sec TC):

Program Elo + - Games Score Av.Op. Draws

1 Stockfish 220821 x64 bmi2 : 2410 12 11 360 52.9 % 2390 89.7 %
2 Fat Titz x64 bmi2 : 2390 11 12 360 47.1 % 2410 89.7 %

Individual statistics:

1 Stockfish 220821 x64 bmi2 : 2410 360 (+ 29,=323,- 8), 52.9 %

Fat Titz x64 bmi2 : 360 (+ 29,=323,- 8), 52.9 %

2 Fat Titz x64 bmi2 : 2390 360 (+ 8,=323,- 29), 47.1 %

Stockfish 220821 x64 bmi2 : 360 (+ 8,=323,- 29), 47.1 %


Game Conditions: Cutechess Gui, 1 Core (Core-i7 9750h), 1 min + 0.5 sec TC, Balsa 5 Moves Opening Book, 64 Mb Hash, Ponder Off
https://www.mediafire.com/file/r9bhihis ... 0.pgn/file


Fat Titz vs Stockfish Dev (10 min TC):

Program Elo + - Games Score Av.Op. Draws

1 Fat Titz x64 bmi2 : 2401 5 4 360 50.1 % 2400 98.6 %
2 Stockfish 220821 x64 bmi2 : 2400 4 5 360 49.9 % 2400 98.6 %

Individual statistics:

1 Fat Titz x64 bmi2 : 2401 360 (+ 3,=355,- 2), 50.1 %

Stockfish 220821 x64 bmi2 : 360 (+ 3,=355,- 2), 50.1 %

2 Stockfish 220821 x64 bmi2 : 2400 360 (+ 2,=355,- 3), 49.9 %

Fat Titz x64 bmi2 : 360 (+ 2,=355,- 3), 49.9 %


Game Conditions: Cutechess Gui, 1 Core (Core-i7 9750h), 10 min TC, Balsa 5 Moves Opening Book, 128 Mb Hash, Ponder Off
https://www.mediafire.com/file/xseaysi2 ... 1.pgn/file


Fat Titz vs Stockfish Dev:

10 sec + 0.2 TC: - 44 elo
1 min + 0.5 TC: -20 elo
10 min + 0 TC: +1 elo


Fat Titz chess engine has a troll name, maybe that's why some people may not take this chess engine too seriously. The performance of Fat Titz is weaker against Stockfish Dev at fast time controls but it scales very well with increasing time. It signals that we will see more bigger nets in chess engines in the near future.

[mehmet123]

Fat Titz 1.1 vs Fat Titz 1.0:

Program Elo + - Games Score Av.Op. Draws

1 Fat Titz 1.1 x64 bmi2 : 2405 9 9 1000 51.4 % 2395 83.1 %
2 Fat Titz 1.0 x64 bmi2 : 2395 9 9 1000 48.5 % 2405 83.1 %

Individual statistics:

1 Fat Titz 1.1 x64 bmi2 : 2405 1000 (+ 99,=831,- 70), 51.5 %

Fat Titz 1.0 x64 bmi2 : 1000 (+ 99,=831,- 70), 51.4 %

2 Fat Titz 1.0 x64 bmi2 : 2395 1000 (+ 70,=831,- 99), 48.5 %

Fat Titz 1.1 x64 bmi2 : 1000 (+ 70,=831,- 99), 48.5 %


Game Conditions: Cutechess Gui, 1 Core (Core-i7 9750h), 10 sec + 0.2 sec, Balsa 5 Moves Opening Book, 128 Mb Hash, Ponder Off
https://www.mediafire.com/file/d09hqz2x ... 5.pgn/file


Fat Titz 1.1 vs Fat Titz 1.0:

Program Elo + - Games Score Av.Op. Draws

1 Fat Titz 1.1 x64 bmi2 : 2403 8 7 600 50.9 % 2397 92.8 %
2 Fat Titz 1.0 x64 bmi2 : 2397 7 8 600 49.1 % 2403 92.8 %

Individual statistics:

1 Fat Titz 1.1 x64 bmi2 : 2403 600 (+ 27,=557,- 16), 50.9 %

Fat Titz 1.0 x64 bmi2 : 600 (+ 27,=557,- 16), 50.9 %

2 Fat Titz 1.0 x64 bmi2 : 2397 600 (+ 16,=557,- 27), 49.1 %

Fat Titz 1.1 x64 bmi2 : 600 (+ 16,=557,- 27), 49.1 %


Game Conditions: Cutechess Gui, 1 Core (Core-i7 9750h), 1 min + 0.5 sec, Balsa 5 Moves Opening Book, 512 Mb Hash, Ponder Off
https://www.mediafire.com/file/a92vd38t ... 6.pgn/file

[mehmet123]

Sugar vs Stockfish:

Program Elo + - Games Score Av.Op. Draws

1 SugaR AI 2.40 bmi2 : 2401 6 6 770 50.1 % 2400 94.9 %
2 Stockfish 310821 x64 bmi2 : 2400 6 6 770 49.9 % 2400 94.9 %

Individual statistics:

1 SugaR AI 2.40 bmi2 : 2401 770 (+ 20,=731,- 19), 50.1 %

Stockfish 310821 x64 bmi2 : 770 (+ 20,=731,- 19), 50.1 %

2 Stockfish 310821 x64 bmi2 : 2400 770 (+ 19,=731,- 20), 49.9 %

SugaR AI 2.40 bmi2 : 770 (+ 19,=731,- 20), 49.9 %


Game Conditions: Cutechess Gui, 1 Core (Core-i7 9750h), 1 min + 0.5 sec TC, Balsa 5 Moves Opening Book, 128 Mb Hash, Ponder Off
https://www.mediafire.com/file/nu0du1pu ... 8.pgn/file

[mehmet123]

KomodoDragon vs Stockfish Dev:

Program Elo + - Games Score Av.Op. Draws

1 Stockfish 230921 x64 bmi2 : 2443 17 16 500 62.1 % 2357 69.0 %
2 KomodoDragon 2.5 x64 avx2 : 2357 16 17 500 37.9 % 2443 69.0 %

Individual statistics:

1 Stockfish 230921 x64 bmi2 : 2443 500 (+138,=345,- 17), 62.1 %

KomodoDragon 2.5 x64 avx2 : 500 (+138,=345,- 17), 62.1 %

2 KomodoDragon 2.5 x64 avx2 : 2357 500 (+ 17,=345,-138), 37.9 %

Stockfish 230921 x64 bmi2 : 500 (+ 17,=345,-138), 37.9 %


Game Conditions: Cutechess Gui, 1 Core (Core-i7 9750h), 10 sec + 0.2 sec TC, Balsa 5 Moves Opening Book, 64 Mb Hash, Ponder Off
https://www.mediafire.com/file/vnap2ff4 ... 8.pgn/file

I guess this power difference will be much less in longer games.



Dragon 2.5 vs Dragon 2:

Program Elo + - Games Score Av.Op. Draws

1 Dragon 2.5 x64 avx2 : 2444 17 16 500 62.4 % 2356 68.8 %
2 Dragon 2 x64 avx2 : 2356 16 17 500 37.6 % 2444 68.8 %

Individual statistics:

1 Dragon 2.5 x64 avx2 : 2444 500 (+140,=344,- 16), 62.4 %

Dragon 2 x64 avx2 : 500 (+140,=344,- 16), 62.4 %

2 Dragon 2 x64 avx2 : 2356 500 (+ 16,=344,-140), 37.6 %

Dragon 2.5 x64 avx2 : 500 (+ 16,=344,-140), 37.6 %


Game Conditions: Cutechess Gui, 1 Core (Core-i7 9750h), 10 sec + 0.2 sec TC, Balsa 5 Moves Opening Book, 64 Mb Hash, Ponder Off
https://www.mediafire.com/file/p01dyrby ... 6.pgn/file

Incredible performance (+122 elo) for Dragon 2.5 chess engine. But in longer games the difference in elo between two chess engines will be less.

[mehmet123]

KomodoDragon 2.5 vs Stockfish Dev:

Program Elo + - Games Score Av.Op. Draws

1 Stockfish 230921 x64 bmi2 : 2413 10 8 400 53.8 % 2387 92.5 %
2 KomodoDragon 2.5 x64 avx2 : 2387 8 10 400 46.2 % 2413 92.5 %

Individual statistics:

1 Stockfish 230921 x64 bmi2 : 2413 400 (+ 30,=370,- 0), 53.8 %

KomodoDragon 2.5 x64 avx2 : 400 (+ 30,=370,- 0), 53.8 %

2 KomodoDragon 2.5 x64 avx2 : 2387 400 (+ 0,=370,- 30), 46.2 %

Stockfish 230921 x64 bmi2 : 400 (+ 0,=370,- 30), 46.2 %


Game Conditions: Cutechess Gui, 1 Core (Core-i7 9750h), 10 min TC, Balsa 5 Moves Opening Book, 512 Mb Hash, Ponder Off
https://www.mediafire.com/file/zp1m1gx7 ... 0.pgn/file


Elo difference between Stockfish 230921 and Dragon 2.5:
10 sec + 0.2 sec TC: + 86 elo
10 min + 0 sec TC: +26 elo


Elo difference between Stockfish 020521 and Dragon 2:
10 sec + 0.2 sec TC: + 118 elo
1 min+ 0.5 sec TC: +72 elo
5 min+ 0 sec TC:+56 elo
forum3/viewtopic.php?f=6&t=74518&start=170

Dragon is constantly reducing the difference in elo with Stockfish. Stockfish is no longer unbeatable in alpabeta engines.

[Jouni]

+ 30,=370,- 0 means SF was unbeatable in test . Still of course new Dragon has great progress.

[mehmet123]

Redfish vs Stockfish :

Program Elo + - Games Score Av.Op. Draws

1 Stockfish 260921 x64 bmi2 : 2401 6 5 330 50.3 % 2399 98.2 %
2 Redfish 220921 x64 bmi2 : 2399 5 6 330 49.7 % 2401 98.2 %

Individual statistics:

1 Stockfish 260921 x64 bmi2 : 2401 330 (+ 4,=324,- 2), 50.3 %

Redfish 220921 x64 bmi2 : 330 (+ 4,=324,- 2), 50.3 %

2 Redfish 220921 x64 bmi2 : 2399 330 (+ 2,=324,- 4), 49.7 %

Stockfish 260921 x64 bmi2 : 330 (+ 2,=324,- 4), 49.7 %


Game Conditions: Arena Gui, 6 Cores (Core-i7 9750h), 2 min + 0.5 sec TC, Balsa 5 Moves Opening Book, 1024 Mb Hash, Ponder Off
https://www.mediafire.com/file/xspap620 ... 1.pgn/file


Redfish has a small net (~15 Mb ). The size of Redfish net is ~1/3 size of Stockfish Dev net.
The performance of Redfish is very good against Stockfish Dev. Stockfish Dev struggled with a much more recent version than Redfish. ( 4 new updates: 1.28 elo, 1.67 elo, 3.21 elo, 1.96 elo).
This result was not surprising for me. Stockfish miniNNUE also showed a similar performance against Stockfish Dev at two months ago.
forum3/viewtopic.php?f=6&t=74518&start=220
The performance of Redfish isn't so good against Stockfish dev at 1 core , 30 sec+ 0.5 sec and 1 core , 1 min + 0.5 sec time controls. But we can say that Redfish performs very well against Stockfish Dev in tests where more cores are used or more longer games than bullet/blitz games (1 core).

[mehmet123]

Stockfish PB vs Stockfish Dev:

Program Elo + - Games Score Av.Op. Draws

1 Stockfish PB 011021 x64 bmi2 : 2403 8 8 1300 50.8 % 2397 81.8 %
2 Stockfish 260921 x64 bmi2 : 2397 8 8 1300 49.2 % 2403 81.8 %

Individual statistics:

1 Stockfish PB 011021 x64 bmi2: 2403 1300 (+128,=1064,-108), 50.8 %

Stockfish 260921 x64 bmi2 : 1300 (+128,=1064,-108), 50.8 %

2 Stockfish 260921 x64 bmi2 : 2397 1300 (+108,=1064,-128), 49.2 %

Stockfish PB 011021 x64 bmi2 : 1300 (+108,=1064,-128), 49.2 %


Game Conditions: Cutechess Gui, 1 Core (Core-i7 9750h), 10 sec + 0.2 sec TC,3 Moves Opening Book, 64 Mb Hash, Ponder Off
Stockfish PB 011021 is a ChessMan compile chess engine and it supports two polyglot (.bin) books.
https://www.mediafire.com/file/bad96or1 ... 2.pgn/file

Great performance for Stockfish PB 011021 at this time condition (10 sec + 0.2 sec). I guess the difference will be a little less in longer time games.

[mehmet123]

Stockfish PB vs Stockfish Dev:

Program Elo + - Games Score Av.Op. Draws

1 Stockfish PB 011021 x64 bmi2 : 2401 6 6 1200 50.2 % 2399 90.7 %
2 Stockfish 260921 x64 bmi2 : 2399 6 6 1200 49.8 % 2401 90.7 %

Individual statistics:

1 Stockfish PB 011021 x64 bmi2: 2401 1200 (+ 59,=1088,- 53), 50.2 %

Stockfish 260921 x64 bmi2 : 1200 (+ 59,=1088,- 53), 50.2 %

2 Stockfish 260921 x64 bmi2 : 2399 1200 (+ 53,=1088,- 59), 49.8 %

Stockfish PB 011021 x64 bmi2 : 1200 (+ 53,=1088,- 59), 49.8 %


Game Conditions: Cutechess Gui, 1 Core (Core-i7 9750h), 1 min sec + 0.5 sec TC,3 Moves Opening Book, 1284 Mb Hash, Ponder Off
https://www.mediafire.com/file/gtwwwgwp ... 3.pgn/file

There is no significant power difference between two engines , but at least for my tests I can easily say that Stockfish PB is currently the most powerful Stockfish derivative chess engine.

[mehmet123]

Bluefish vs SugaR AI:

Program Elo + - Games Score Av.Op. Draws

1 SugaR AI 2.40 bmi2 : 2404 14 13 200 51.2 % 2396 92.5 %
2 Bluefish 190921 x64 bmi2 : 2396 13 14 200 48.8 % 2404 92.5 %


Individual statistics:

1 SugaR AI 2.40 bmi2 : 2404 200 (+ 10,=185,- 5), 51.2 %

Bluefish 190921 x64 bmi2 : 200 (+ 10,=185,- 5), 51.2 %

2 Bluefish 190921 x64 bmi2 : 2396 200 (+ 5,=185,- 10), 48.8 %

SugaR AI 2.40 bmi2 : 200 (+ 5,=185,- 10), 48.8 %


Game Conditions: Cutechess Gui, 1 Core (Core-i7 9750h), 5 min TC, 3 Moves Opening Book, 512 Mb Hash, Ponder Off
https://www.mediafire.com/file/p94uc1qc ... 4.pgn/file