Page 3 of 10

Re: 87% of the poll voted that SF will win Tcec Sufi 19.

Posted: Mon Oct 05, 2020 7:03 pm
by peter
CMCanavessi wrote:
Mon Oct 05, 2020 3:58 pm
Leela leading now after an amazing game 33. Check it out if you can, it was really cool. Reverse is playing right now.
After 50 moves it looks like one more 1:1 pair:

Ongoing:

Re: 87% of the poll voted that SF will win Tcec Sufi 19.

Posted: Mon Oct 05, 2020 9:21 pm
by peter
peter wrote:
Mon Oct 05, 2020 7:03 pm
Ongoing
Last moves I gave in manually and mistyped 51...Kh5, correct was ...Kh6.

Here's the rest of the game:

Re: 87% of the poll voted that SF will win Tcec Sufi 19.

Posted: Tue Oct 06, 2020 11:22 am
by MMarco
Laskos wrote:
Mon Oct 05, 2020 1:19 pm
So, you tackled too the issue with possible bad scaling of SV 3010 and later nets, right? Interesting, if J-92 nets scale well indeed as you found out, this Sufi might look different from expected. I might check too the scaling of J92-190 versus SV 3972mlh used in Season 18.
Yes I did a quick test to mickmick yours I found here: viewtopic.php?t=74239&start=10#p848375
I just dig it up from my computer. The results are a bit different from those I had in memory, but close enough I think!

I did the tests before J92-190 was selected for SuFi (J92-210 and J92-220 were the other nets considered).

On a rtx 2060 at 10s + 0.1s: (I can't remember the opening set. Maybe past TCEC).

Code: Select all

   # PLAYER      :  RATING  ERROR  PLAYED   (%)   CFS    W    D    L  D(%)
   1 J92-220     :    50.2   15.6     400  56.5    75  110  232   58  58.0
   2 J92-210     :    40.1   15.0     500  55.2   100  119  314   67  62.8
   3 T60-3010    :     0.0    9.3     900  44.2   ---  125  546  229  60.7

White advantage = 106.74 +/- 6.89
Draw rate (equal opponents) = 71.17 % +/- 1.82
Now at 40s + 0.4s (you had a longer time control, 60s + 0.6 and a 2070, so unfortunately this is only half of your LTC at best). I didn't do the test with J92-220 when it became clear it would be chosen for SuFi:

Code: Select all

   # PLAYER      :  RATING  ERROR  PLAYED   (%)   CFS    W    D    L  D(%)
   1 J92-210     :    32.9   10.7     240  54.6   100   39  184   17  76.7
   2 T60-3010    :     0.0   10.7     240  45.4   ---   17  184   39  76.7

White advantage = 50.42 +/- 11.32
Draw rate (equal opponents) = 82.04 % +/- 2.60
The collapse that happened at LTC in your test didn't occur here. J92-210 lost only 7 ratings points when the time control was multiplied by four. It would be surprinsing that J92-190 be scalling diffenrently than J92-210 or 220 according to JHortos.

Re: 87% of the poll voted that SF will win Tcec Sufi 19.

Posted: Wed Oct 07, 2020 3:56 am
by Nay Lin Tun
Still tied after 42 games. It is very likely that these two engines are within +/- 50 elo, most likely +/- 20 elo.

Thus being said, NNUE +100 elo hype was wrong against Leela.

In case, they are not equal in strength,
1. Is SF +50 elo better than Leela or
2. Is Leela + 50 elo better than SF? :mrgreen: :lol: :evil:

Re: 87% of the poll voted that SF will win Tcec Sufi 19.

Posted: Wed Oct 07, 2020 5:18 am
by Dann Corbit
Moving target.
They changed the Leela ratio for the this contest.

Nobody can say what the right ratio is anyway,

For that matter, both engines are moving targets.

Re: 87% of the poll voted that SF will win Tcec Sufi 19.

Posted: Thu Oct 08, 2020 1:01 am
by ernest
Dann Corbit wrote:
Wed Oct 07, 2020 5:18 am

They changed the Leela ratio for the this contest.
Who says ?
Isn't the hardware the same as for TCEC 18 ?

Re: 87% of the poll voted that SF will win Tcec Sufi 19.

Posted: Thu Oct 08, 2020 1:22 am
by Dann Corbit
I stand corrected.
I just checked and the hardware and time control is identical.
Not sure where I got the impression that something was changed.

Re: 87% of the poll voted that SF will win Tcec Sufi 19.

Posted: Thu Oct 08, 2020 1:41 am
by ernest
Still, I wonder (and of course I am not the only one...) why Lc0 is "resisting" so well against SF NNUE in Sufi 19.

As if the NNUE improvement almost did not help under the current playing conditions. Or...?

Re: 87% of the poll voted that SF will win Tcec Sufi 19.

Posted: Thu Oct 08, 2020 2:34 am
by mwyoung
ernest wrote:
Thu Oct 08, 2020 1:41 am
Still, I wonder (and of course I am not the only one...) why Lc0 is "resisting" so well against SF NNUE in Sufi 19.

As if the NNUE improvement almost did not help under the current playing conditions. Or...?

That is because testing has shown SF NNUE is not much stronger then Lc0. At long time controls. The confusion comes from only looking or testing at STC results. And assuming that result also applies to LTC games.

Re: 87% of the poll voted that SF will win Tcec Sufi 19.

Posted: Fri Oct 09, 2020 2:21 pm
by Laskos
ernest wrote:
Thu Oct 08, 2020 1:01 am
Dann Corbit wrote:
Wed Oct 07, 2020 5:18 am

They changed the Leela ratio for the this contest.
Who says ?
Isn't the hardware the same as for TCEC 18 ?
The Lc0 binaries are significantly faster now (+40%?). Also, the score now is back to almost regular: 3:1 =25 pairwise, with very high pairwise draw ratio. Maybe this pairwise draw rate is one of the reasons Elo difference seems small.