Android Test Tournament - Lc0 24.1 (591226) 4CPU

[MMarco]

I've done a little tourney on my phone (4×2.2 GHz Cortex-A73) at 240s/40 + 1s, so that should be comparable to CCRL Blitz list testing conditions. Hash = 64, syzygy 5-men, adjudication at 3 moves in a row at +5 on both sides (Lc0 doesn't score high easily, so that looks safe). The offset was done to match best CCRL Blitz list again. All other engine but Lc0 used only 1 CPU. Apparently, Deep Fritz 15 for android is in fact Ginkgo 2.0: http://chesstroid.blogspot.com/2017/11/ ... ngine.html . I used a set of 25 opening positions that came with Fritz 17 (Nunn2).

Lc0 settings (different from defaults) were:
NNCacheSize = 800 000
MinibatchSize = 8
MaxPrefetch = 0
LogitQ = on

Code: Select all

Rank Name                         Elo    +    - games score oppo. draws
   1 Laser 1.7 for Android       3295   67   63    50   69%  3173   34%
   2 Lc0 24.1 (591226) 4CPU      3173   31   31   200   52%  3159   42%
   3 Deep Fritz 15 (Ginkgo 2.0)  3159   59   59    50   48%  3173   56%
   4 Pedone 2.0                  3142   61   62    50   45%  3173   42%
   5 Vajolet2 2.4                3042   63   67    50   30%  3173   36%

[MMarco]

Some more results:

Code: Select all

   # PLAYER                    : RATING    POINTS  PLAYED    (%)
   1 Komodo 13.1               : 3403.1      39.0      50   78.0%
   2 Laser 1.7                 : 3319.5      34.5      50   69.0%
   3 Gull 3  4CPU              : 3174.9      25.0      50   50.0%
   4 Lc0 24.1 (591226)  4CPU   : 3174.9     203.5     400   50.9%
   5 RubiChess 210320          : 3160.4      24.0      50   48.0%
   6 Deep Fritz 15             : 3160.4      24.0      50   48.0%
   7 Pedone 2.0                : 3138.5      22.5      50   45.0%
   8 Vajolet2 2.4              : 3021.8      15.0      50   30.0%
   9 Fruit reloaded 3.2.1      : 2976.6      12.5      50   25.0%

White advantage = 63.38
Draw rate (equal opponents) = 50.00 %

[Fuddur]

I've done a little tourney on my phone (4×2.2 GHz Cortex-A73) at 240s/40 + 1s, so that should be comparable to CCRL Blitz list testing conditions. Hash = 64, syzygy 5-men, adjudication at 3 moves in a row at +5 on both sides (Lc0 doesn't score high easily, so that looks safe). The offset was done to match best CCRL Blitz list again. All other engine but Lc0 used only 1 CPU. Apparently, Deep Fritz 15 for android is in fact Ginkgo 2.0: http://chesstroid.blogspot.com/2017/11/ ... ngine.html . I used a set of 25 opening positions that came with Fritz 17 (Nunn2).

Lc0 settings (different from defaults) were:
NNCacheSize = 800 000
MinibatchSize = 8
MaxPrefetch = 0
LogitQ = on

Code: Select all

Rank Name                         Elo    +    - games score oppo. draws
   1 Laser 1.7 for Android       3295   67   63    50   69%  3173   34%
   2 Lc0 24.1 (591226) 4CPU      3173   31   31   200   52%  3159   42%
   3 Deep Fritz 15 (Ginkgo 2.0)  3159   59   59    50   48%  3173   56%
   4 Pedone 2.0                  3142   61   62    50   45%  3173   42%
   5 Vajolet2 2.4                3042   63   67    50   30%  3173   36%

Nice!
Why lc0 is on 4 cores?

[MMarco]

Code: Select all

Rank Name                         Elo    +    - games score oppo. draws
   1 Laser 1.7 for Android       3295   67   63    50   69%  3173   34%
   2 Lc0 24.1 (591226) 4CPU      3173   31   31   200   52%  3159   42%
   3 Deep Fritz 15 (Ginkgo 2.0)  3159   59   59    50   48%  3173   56%
   4 Pedone 2.0                  3142   61   62    50   45%  3173   42%
   5 Vajolet2 2.4                3042   63   67    50   30%  3173   36%

Nice!
Why lc0 is on 4 cores?
[/quote]

I was curious to see the increase in strength going from 1 to 4 CPU. Given the low depths it reaches on my phone on 1 CPU and the time control (sometimes d=4 or d=5), it tends to blunder a lot. On 4 CPU, Lc0 seems much stronger (compared to what a regular AB engine would gain going from 1 to 4 CPU). I set the offset in my rating list to match CCRL blitz list on which Lc0 v24 591226 is rated 2988 on 1 CPU (see https://ccrl.chessdom.com/ccrl/404/cgi/ ... 226_64-bit). So there is a gain of 180 elo for LC0 4 CPU here. Well, to make sure I should redo my test on my phone on 1 CPU, but now I need my phone back!

[MMarco]

On my laptop (ryzen 7) the results were even better for Lc0 4CPU. I'm a bit surprised. I set ratings of other engines equal to their CCRL Blitz list rating. Time control was 40/2m repeating + 1s/move. I used 100 positions from Albert Silver (the 2010 set). Again, settings for Lc0 were:

NNCacheSize = 800 000
MinibatchSize = 8
MaxPrefetch = 0
LogitQ = on

Code: Select all

   # PLAYER                    : RATING    POINTS  PLAYED    (%)
   1 Xiphos 0.6                : 3337.0      59.5     100   59.5%
   2 Komodo 10                 : 3337.0      52.0     100   52.0%
   3 Lc0 24.1 (591226) 4CPU    : 3287.4     212.5     400   53.1%
   4 Fritz 17                  : 3274.0      42.5     100   42.5%
   5 Senpai 2.0                : 3086.0      33.5     100   33.5%

White advantage = 68.69
Draw rate (equal opponents) = 50.00 %

[Chessqueen]

On my laptop (ryzen 7) the results were even better for Lc0 4CPU. I'm a bit surprised. I set ratings of other engines equal to their CCRL Blitz list rating. Time control was 40/2m repeating + 1s/move. I used 100 positions from Albert Silver (the 2010 set). Again, settings for Lc0 were:

NNCacheSize = 800 000
MinibatchSize = 8
MaxPrefetch = 0
LogitQ = on

Code: Select all

   # PLAYER                    : RATING    POINTS  PLAYED    (%)
   1 Xiphos 0.6                : 3337.0      59.5     100   59.5%
   2 Komodo 10                 : 3337.0      52.0     100   52.0%
   3 Lc0 24.1 (591226) 4CPU    : 3287.4     212.5     400   53.1%
   4 Fritz 17                  : 3274.0      42.5     100   42.5%
   5 Senpai 2.0                : 3086.0      33.5     100   33.5%

White advantage = 68.69
Draw rate (equal opponents) = 50.00 %

I have a question for you, can any of these Android engines beat Shredder 8 on a regular P.C which is rated around 2800 ? The reason for asking is that I am NOT convinced the these Android are really rated over 2800

[MMarco]

On my laptop (ryzen 7) the results were even better for Lc0 4CPU. I'm a bit surprised. I set ratings of other engines equal to their CCRL Blitz list rating. Time control was 40/2m repeating + 1s/move. I used 100 positions from Albert Silver (the 2010 set). Again, settings for Lc0 were:

NNCacheSize = 800 000
MinibatchSize = 8
MaxPrefetch = 0
LogitQ = on

Code: Select all

   # PLAYER                    : RATING    POINTS  PLAYED    (%)
   1 Xiphos 0.6                : 3337.0      59.5     100   59.5%
   2 Komodo 10                 : 3337.0      52.0     100   52.0%
   3 Lc0 24.1 (591226) 4CPU    : 3287.4     212.5     400   53.1%
   4 Fritz 17                  : 3274.0      42.5     100   42.5%
   5 Senpai 2.0                : 3086.0      33.5     100   33.5%

White advantage = 68.69
Draw rate (equal opponents) = 50.00 %

I have a question for you, can any of these Android engines beat Shredder 8 on a regular P.C which is rated around 2800 ? The reason for asking is that I am NOT convinced the these Android are really rated over 2800

I'm almost 100% sure these android engines would clobber Shredder 8. They are the same engines as on PC, they just run slower (i.e. on my phone they run around 50% of the speed of my laptop on one thread). That isn't enough to give a chance to Shredder 8 against any of the engines listed above. You could pick another android engine to have a faire match up against Shredder 8 though,, maybe Cheese 1.9, Fruit 2.1 or Rhetoric for android.

If you have an android device and Shredder UCI on PC you could set up a match like this: http://aartbik.blogspot.com/2012/03/con ... emote.html

[MMarco]

On my laptop (ryzen 7) the results were even better for Lc0 4CPU. I'm a bit surprised. I set ratings of other engines equal to their CCRL Blitz list rating. Time control was 40/2m repeating + 1s/move. I used 100 positions from Albert Silver (the 2010 set). Again, settings for Lc0 were:

NNCacheSize = 800 000
MinibatchSize = 8
MaxPrefetch = 0
LogitQ = on

Code: Select all

   # PLAYER                    : RATING    POINTS  PLAYED    (%)
   1 Xiphos 0.6                : 3337.0      59.5     100   59.5%
   2 Komodo 10                 : 3337.0      52.0     100   52.0%
   3 Lc0 24.1 (591226) 4CPU    : 3287.4     212.5     400   53.1%
   4 Fritz 17                  : 3274.0      42.5     100   42.5%
   5 Senpai 2.0                : 3086.0      33.5     100   33.5%

White advantage = 68.69
Draw rate (equal opponents) = 50.00 %

I repeated the games with Senpai and Xiphos for Lc0 1CPU. The rating difference between 1 CPU and 4 CPU is close to 200 elo when I use Xiphos, Komodo Fritz and Senpai ratings set to their respective CCRL Blitz list rating (done with Ordo):

Code: Select all

   # PLAYER                    : RATING    POINTS  PLAYED    (%)
   1 Xiphos 0.6                : 3337.0     132.0     200   66.0%
   2 Komodo 10                 : 3337.0      52.0     100   52.0%
   3 Lc0 24.1 (591226) 4CPU    : 3293.6     212.5     400   53.1%
   4 Fritz 17                  : 3274.0      42.5     100   42.5%
   5 Lc0 24.1 (591226) 1CPU    : 3096.7      76.0     200   38.0%
   6 Senpai 2.0                : 3086.0      85.0     200   42.5%

White advantage = 68.59
Draw rate (equal opponents) = 50.00 %

[Fuddur]

On my laptop (ryzen 7) the results were even better for Lc0 4CPU. I'm a bit surprised. I set ratings of other engines equal to their CCRL Blitz list rating. Time control was 40/2m repeating + 1s/move. I used 100 positions from Albert Silver (the 2010 set). Again, settings for Lc0 were:

NNCacheSize = 800 000
MinibatchSize = 8
MaxPrefetch = 0
LogitQ = on

Code: Select all

   # PLAYER                    : RATING    POINTS  PLAYED    (%)
   1 Xiphos 0.6                : 3337.0      59.5     100   59.5%
   2 Komodo 10                 : 3337.0      52.0     100   52.0%
   3 Lc0 24.1 (591226) 4CPU    : 3287.4     212.5     400   53.1%
   4 Fritz 17                  : 3274.0      42.5     100   42.5%
   5 Senpai 2.0                : 3086.0      33.5     100   33.5%

White advantage = 68.69
Draw rate (equal opponents) = 50.00 %

I repeated the games with Senpai and Xiphos for Lc0 1CPU. The rating difference between 1 CPU and 4 CPU is close to 200 elo when I use Xiphos, Komodo Fritz and Senpai ratings set to their respective CCRL Blitz list rating (done with Ordo):

Code: Select all

   # PLAYER                    : RATING    POINTS  PLAYED    (%)
   1 Xiphos 0.6                : 3337.0     132.0     200   66.0%
   2 Komodo 10                 : 3337.0      52.0     100   52.0%
   3 Lc0 24.1 (591226) 4CPU    : 3293.6     212.5     400   53.1%
   4 Fritz 17                  : 3274.0      42.5     100   42.5%
   5 Lc0 24.1 (591226) 1CPU    : 3096.7      76.0     200   38.0%
   6 Senpai 2.0                : 3086.0      85.0     200   42.5%

White advantage = 68.59
Draw rate (equal opponents) = 50.00 %

Hi,
What's the cores/threads others engine using?
Or only leela is using 4 cores and others 1 core.
And where you test leela? in android device or PC.

[MMarco]

Hi,
What's the cores/threads others engine using?
Or only leela is using 4 cores and others 1 core.
And where you test leela? in android device or PC.

It changed along the way of this thread, but the last results I posted were done on my PC, ryzen 7. All other engines were on 1 thread. I used the DNNL BLAS build for Leela.