TCEC Season 12 with Stockfish Zero aka Leela Chess Zero

[Hai]

It will be strong enough to play Season 12.
But can it participate?

[CMCanavessi]

Yes, it can, even on CPUs. If it will be strong enough or not, that's a difficult answer. It depends on when Season 12 starts.

[Dann Corbit]

It will be strong enough to play Season 12.

Not a given.
Progress is clearly flattening out:
http://lczero.org/
And the graph is exponential along the y axis.
The 4500 Elo it has reached is about 2000 Elo CCRL or CEGT.

But can it participate?

Hard to know.
You could run it on a CPU, but it would lose a lot of steam.
It really needs a rack of those $3000 Nvidia cards with TPUs in them.
https://www.nvidia.com/en-us/titan/titan-v/
But now, how does that compare with the hardware for the even hardware TCEC challenge?
It doesn't. So it would also require a rule change.

[Vizvezdenec]

if it will run on CPUs that TCEC uses it will score 0 points in div 4 no matter what progress it has (honestly I think that it will score smth like that even with dedicated GPUs, but on CPUs it's 100% given).

[David Xu]

That's an interesting claim. I suppose we'll see.

[MonteCarlo]

With the current smallish network, you'd actually need at least a couple really expensive GPUs to match the performance the cpu-only version would get on 43 CPU cores.

So, if the requirement for fairness was something like cost of hardware, it would currently be better off playing on the 43 CPU cores than playing on GPUs.

Of course, unless a miracle happens, it won't be anywhere close to strong enough to compete in division 4 any time soon. That's the real issue

[Robert Pope]

Can you please stop referring to it as Stockfish Zero? That's deceptive, at best.

[Alexander Schmidt]

Progress is clearly flattening out:
http://lczero.org/
And the graph is exponential along the y axis.

I am not in familiar with LeelaCZ but with LeelaZ. If the progress stalls they will switch to a bigger Network. Before that they will squeeze everything out of the small network because testing is faster with it. There are parameters to adjust and most likely bugs to find.

For me it is just a question of time when they or another Zero project will catch up Stockfish.

[Werewolf]

Progress is clearly flattening out:

I'm not so sure.

[Dann Corbit]

Progress is clearly flattening out:

I'm not so sure.

The general shape of this curve:
http://lczero.org/

approximates the shape of the square root function pretty well.

Near the origin, the slope is nearly vertical.

Now the slope fluctuates close to zero, but still gaining a little bit.

at 1036299 games, the score is 4173.35
At 2213791 games, the score is 4694.18

rise/run = (4694.18 - 4173.35) / (2213791 - 1036299 ) = .00044

That's pretty close to a horizontal line.

Now, the y scale is an exponential scale (being Elo), so it is still learning like mad.
But it is not making progress like it was earlier.