CEGT - rating lists June 30th 2019

[Daniel Shawul]

Nice!! I never had INT8 in tournaments before -- good to know it actually doubles nps.
Maybe I will ask TCEC to use it too since it will double nps, especially given their RTX tensor cores.
It only hurt performance by -40 elo when it was restricted to use same number of nps as half precision.

Note that lc0 does not have INT8 support so it will not benefit from it even if you had tensor cores.
The GTX 1650 have not tensor cores but i think its FP16 performance is still twice the performance of FP32, and INT8
now seems to be twice FP16 atleast with scorpio.

I need to figure out the right number of threads mt=(32/64/128) and delay(0/1) combination that
will maximize performance. I am inclined towards delay=1 for very few cores but there must be a formula to automatically
pick up the right parameters.

regard,
Daniel

[Daniel Shawul]

Ok Werner, I have come to some kind of conclusion now. Assuming you are going to use 4 CPU cores, I suspect the best settings are one of
following in decreasing chance:

a) delay=0 mt=64
b) delay=1 mt=128
c) delay=0 mt=128
d) delay=1 mt=256 (very unlikely)

Note that if you use just 2-cpu cores mt>=128 could be a 100x slower.

If you can just do a single run for each of the settings on the command line using INT8 we can pick one and move on.
If what you were using before, does not perform significantly worse than the best setting we find here, you can just resume the tournament.

regards,
Daniel

[Werner]

delay 0 mt 128
# nodes = 391116 <0% qnodes> time = 9232ms nps = 42365 eps = 0 nneps = 4939
# Tree: nodes = 62300 depth = 20 pps = 4911 visits = 45192
# qsearch_calls = 0 search_calls = 0

delay 1 mt 128
# nodes = 247055 <0% qnodes> time = 10341ms nps = 23890 eps = 0 nneps = 2957
# Tree: nodes = 41461 depth = 19 pps = 2932 visits = 30141
# qsearch_calls = 0 search_calls = 0

delay 0 mt 64 (the engine does calibrating before running)
# nodes = 399689 <0% qnodes> time = 9060ms nps = 44115 eps = 0 nneps = 5183
# Tree: nodes = 64332 depth = 19 pps = 5172 visits = 46703
# qsearch_calls = 0 search_calls = 0

delay 1 mt 256 (the engine does calibrating new )
nodes = 333431 <0% qnodes> time = 9857ms nps = 33826 eps = 0 nneps = 3968
# Tree: nodes = 52894 depth = 20 pps = 3926 visits = 38397
# qsearch_calls = 0 search_calls = 0

delay 0 mt 16
# nodes = 293402 <0% qnodes> time = 9342ms nps = 31406 eps = 0 nneps = 3775
# Tree: nodes = 48529 depth = 20 pps = 3772 visits = 35244
# qsearch_calls = 0 search_calls = 0

and the Tournament is stopped

...and you did understand: FP16 is even 1.7 times faster on my Card if you use another math model

[Daniel Shawul]

Thanks Werner.
Ok lets go with the first one i.e. "delay=0 and mt=64".

Btw this is the first INT8 use for neural networks in chess, so I am curious to see how it performs.

I appreciate your help a lot.

regards,
Daniel

[Werner]

Ok Daniel,
I will repeat the match this night and tomorrow as I am not here and back in the afternoon.
best wishes
Werner

[Werner]

He Daniel,
here is the result
ScorpioNet I56600K 2019 40/4

1 StoofVleesII a10 3364 +173 +25/=23/-2 73.00% 36.5/50
2 Scorpio 2.9.0 MCTS+NN Maddex 3191 -173 +2/=23/-25 27.00% 13.5/50

And here one of the games Scorpio won:
[Event "ScorpioNet"]
[Site "I56600K"]
[Date "2019.07.02"]
[Round "40"]
[White "StoofVleesII a10"]
[Black "Scorpio 2.9.0 MCTS+NN Maddex"]
[Result "0-1"]
[ECO "D45"]
[PlyCount "76"]
[EventDate "2019.??.??"]
[TimeControl "40/240:40/240:40/240"]

1. d4 d5 2. c4 c6 3. Nf3 Nf6 4. Nc3 e6 5. e3 a6 6. b3 Bb4 7. Bd2 Bd6 8. Qc2
Nbd7 9. e4 {0.25/15 8} dxe4 {-0.29/73 14} 10. Nxe4 {0.40/14 5} Nxe4 {-0.47/69 5
} 11. Qxe4 {0.37/14 5} c5 {-0.23/68 4} 12. Bd3 {0.40/14 5} Nf6 {-0.54/70 5} 13.
Qh4 {0.39/14 5} cxd4 {-0.69/71 6} 14. O-O {0.45/14 5} b6 {-0.86/72 7} 15. Nxd4
{0.39/14 8} Be5 {1.06/69 5} 16. Be3 {0.04/14 5} g5 {0.72/68 4} 17. Qh6 {
-0.31/14 5} Bxd4 {1.05/68 4} 18. Rad1 {-0.49/14 5} Ng4 {0.42/68 3} 19. Qh5 {
0.01/14 5} Nxe3 {1.13/68 4} 20. fxe3 {0.07/14 5} Bxe3+ {1.18/68 4} 21. Kh1 {
-0.03/11 0} Ra7 {1.05/68 4} 22. Be4 {-0.67/15 8} Qe7 {2.27/68 5} 23. b4 {
-0.92/17 27} Rc7 {3.35/68 4} 24. Rf3 {-1.35/17 55} Bf4 {3.66/68 4} 25. Qh6 {
-1.50/14 5} Rxc4 {3.95/70 7} 26. Qg7 {-1.37/14 3} Rf8 {3.96/68 5} 27. Rfd3 {
-1.47/13 3} Rxe4 {4.46/68 5} 28. Rc3 {-2.23/15 6} Bb7 {6.05/69 6} 29. Rcd3 {
-2.83/16 18} Bd5 {6.53/68 6} 30. Rc3 {-3.31/16 25} Qd7 {5.97/71 9} 31. Kg1 {
-2.86/15 9} Re2 {6.20/71 9} 32. g4 {-2.04/12 1} Rg2+ {7.38/71 9} 33. Kf1 {
-1.29/9 0} Rxh2 {6.58/72 10} 34. Qf6 {-2.02/12 0} Rh1+ {6.82/70 8} 35. Kf2 {
-1.78/11 0} Rxd1 {6.85/69 7} 36. Rc8+ {-2.82/12 1} Qxc8 {7.90/67 6} 37. Qxf4 {
-3.13/9 0} gxf4 {#2/70 10} 38. g5 {-2.82/9 0} Qc2# {#1/70 11 normal} 0-1

[Daniel Shawul]

Hi Werner,

Thanks very much for the test !!

I am happy that INT8 actually performs really well. The long held belief on it was that the additional speedup gained
will not compensate for the reduction in accuracy of the net. I think in this test it more than compensated for that.
My test also put it at -40 elo with the same number of nodes as FP16, but i never tested on RTX machines that have int8 support like yours ...

With gpu cards without FP16 support, which most are btw, is the only option to double nps.
Lc0 should seriously look at implementing as it could easily double nps.

regards,
Daniel

[Werner]

Hi Daniel,
I think for the GTX 10... cards CUDA (int8) is the best Option, for RTX 20... Cards FP16 ( NHWC with fp16 ) and for GTX 16...Cards FP16 ( NCHW with fp16). But maybe here I am not correct informed.

On the other side I think the newest LC0 Nets maybe much stronger with Scorpio than these Maddex net you use. Perhaps I will repeat my test using such a net.

[Daniel Shawul]

I suspect even on the RTX cards INT8 may improve over / or equal the performance of FP16 ..
Scorpio only supports 30x nets of lczero -- there are some tensorrt issues i need to resolve to support the nets with SE.
However, the net 32930 maybe a +70 elo over the maddex net according to my tests. You can download the
UFF file from https://github.com/dshawul/Scorpio/rele ... lczero.zip
I think the rest of the elo diff can be accounted for scorpio's "somewhat dependence" on a strong CPU -- even though
no real work is doen on it managing 128 threads maybe needs >=8 cores i think ..

regards,
Daniel

[Werner]

Hi Daniel,
which net is used when I install the engine with the delivered maddex net and when the engine is running I Change the maddex.net with lc0 net 42668 (unzipped) and rename it with net-maddex.uff. The engine Plays - does not crash. Here the files I use:

nnprobe-windows-gpu\ <DIR> 02.07.2019 19:45 ----
net-maddex.uff.64_2.trt 40.449.848 06.07.2019 13:45 -a--
scorpio.ini 5.306 06.07.2019 13:38 -a--
calibrate.dat 1.434.288.128 06.07.2019 13:36 -a--
scorpio.exe 470.016 30.06.2019 23:16 -a--
calibrate.epd 2.478.901 29.06.2019 00:10 -a--
scorpio.bat 196 29.06.2019 00:10 -a--
net-maddex.uff 68.408.808 27.06.2019 13:55 -a--