TCEC Question

[Leo]

Why is SF doing so well against LCO in the latest TCEC?

[Dann Corbit]

In less than 1000 games practically any outcome is possible amongst approximate equals.
I guess that they are very close to equal, but SF had some fortunate outcomes.
And if SF is stronger, it is not by an enormous margain, as evidenced by the draw count.

[Leo]

OK.

[ernest]

And if SF is stronger, it is not by an enormous margain, as evidenced by the draw count.

Any idea if TCEC-hardware wise, the current Lc0-Stockfish contest is "fair" ?
Looking for some LeelaRatio clues, I noticed that going out of book, the Stockfish Knodes/sec were more than 2000 times those of Lc0. Isn't that a bit much (for fairness...) ?

[Leo]

And if SF is stronger, it is not by an enormous margain, as evidenced by the draw count.

Any idea if TCEC-hardware wise, the current Lc0-Stockfish contest is "fair" ?
Looking for some LeelaRatio clues, I noticed that going out of book, the Stockfish Knodes/sec were more than 2000 times those of Lc0. Isn't that a bit much (for fairness...) ?

I haven't heard anyone complaining about the fairness for a long time. I think they have it set up fair myself.

"The hardware for CPU engines was doubled to 88 cores, while the hardware for GPU engines was raised to 4 RTX 2080 Ti's."

[Cornfed]

In less than 1000 games practically any outcome is possible amongst approximate equals.
I guess that they are very close to equal, but SF had some fortunate outcomes.
And if SF is stronger, it is not by an enormous margain, as evidenced by the draw count.

I think the proverbial 'sample size' answer just kind of begs the question.

What does "fortunate" mean? Did LZ0 stay out late partying the night before?

After 71 games SF leads 37.5 vs 33.5 (and game 72 looks like it will end in the Fish's favor as well...). A 4 to 5 pt lead at this point is actually reasonably significant. That said, there are more games to be played and Game 72 has LCZero defending the Latvian Counter Gambit...which is bad. Has SF yet to defend it? I don't know. The Devil is in the details.

EDIT: It has, the game before and SF lost...just as LCZero looks to at the moment.

[ernest]

I haven't heard anyone complaining about the fairness for a long time.

Not complaining, just asking !

(looking at the Knodes/sec)

[Ovyron]

the Stockfish Knodes/sec were more than 2000 times those of Lc0

The "nodes" of Lc0 have absolutely nothing to do with the "nodes" of Stockfish because the architectures of both programs are completely different. Note the definition of NPS:

Nodes per second, including NNCache hits and terminal node hits, but excluding nodes carried over from tree reuse. In other words, total playouts generated per second.

Leela nodes = playouts (Stockfish doesn't do playouts at all)

[ernest]

the Stockfish Knodes/sec were more than 2000 times those of Lc0

The "nodes" of Lc0 have absolutely nothing to do with the "nodes" of Stockfish because the architectures of both programs are completely different. Note the definition of NPS:

Nodes per second, including NNCache hits and terminal node hits, but excluding nodes carried over from tree reuse. In other words, total playouts generated per second.

Leela nodes = playouts (Stockfish doesn't do playouts at all)

Yes, but LeelaRatio (one of the tools for "fairness") is precisely based on ratio of Knodes/sec...
Admittedly, not everybody likes LeelaRatio...

[Ovyron]

Yeah, we can't even have these discussions without putting quotes around the word "fairness."

It's like trying to make a Megahertz battle against a Megabyte "fair." People have to get into their heads these are different things.