Lc0 strange results

[Werewolf]

I tried two matches with Lc0 Big Net, 10,000 series, Net 10527, August 8th release, Nvidia 1060

The first match was against:
Houdini 6, 8 physical cores, Broadwell 6950X @ 4.2 GHz, HT OFF. No tablebases.

Noomen short lines, 20 game match @ G5 min + 30 sec increment/move (designed to negate issues with Leela's time management)

4 wins each, 12 draws.
10-10

OK, the sample size is small but this is an excellent result for Lc0, much better than I expected given Houdini's excellent hardware.

Then I tried a new match against the current release of asmFish (July 23rd 2018). 8 physical cores, Broadwell 6950X @ 4.2 GHz, HT OFF. Full Syzygy tablebases for asmFish, none for Lc0 of course.

Same T/C: Noomen short lines, 20 game match @ G5 min + 30 sec increment/move

Result: 9 wins for asmFish and 11 draws.
14.5-5.5

A terrible result! I know asmFish is, what? 100 elo stronger than Houdini 6? And how much do tablebases add?
What does the score say about the elo difference, is it +200 for asmFish? I'm surprised by this.

[mar]

20 game match

What do you expect to get from 20 games except noise?

[Werewolf]

More tests will follow.

The 9 wins to zero is not totally irrelevant though.

[ernest]

20 game match

What do you expect to get from 20 games except noise?

Indeed 14.5 - 5,5 is 72.5%

But 95% error-bar is around +- 11%...