87% of the poll voted that SF will win Tcec Sufi 19.

Discussion of anything and everything relating to chess playing software and machines.

Moderators: Harvey Williamson, Dann Corbit, hgm

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
peter
Posts: 2171
Joined: Sat Feb 16, 2008 6:38 am
Full name: Peter Martan

Re: 87% of the poll voted that SF will win Tcec Sufi 19.

Post by peter » Mon Oct 05, 2020 7:03 pm

CMCanavessi wrote:
Mon Oct 05, 2020 3:58 pm
Leela leading now after an amazing game 33. Check it out if you can, it was really cool. Reverse is playing right now.
After 50 moves it looks like one more 1:1 pair:

Ongoing:
Peter.

peter
Posts: 2171
Joined: Sat Feb 16, 2008 6:38 am
Full name: Peter Martan

Re: 87% of the poll voted that SF will win Tcec Sufi 19.

Post by peter » Mon Oct 05, 2020 9:21 pm

peter wrote:
Mon Oct 05, 2020 7:03 pm
Ongoing
Last moves I gave in manually and mistyped 51...Kh5, correct was ...Kh6.

Here's the rest of the game:
Peter.

MMarco
Posts: 141
Joined: Sat Apr 11, 2020 11:09 pm
Full name: Marc-O Moisan-Plante

Re: 87% of the poll voted that SF will win Tcec Sufi 19.

Post by MMarco » Tue Oct 06, 2020 11:22 am

Laskos wrote:
Mon Oct 05, 2020 1:19 pm
So, you tackled too the issue with possible bad scaling of SV 3010 and later nets, right? Interesting, if J-92 nets scale well indeed as you found out, this Sufi might look different from expected. I might check too the scaling of J92-190 versus SV 3972mlh used in Season 18.
Yes I did a quick test to mickmick yours I found here: viewtopic.php?t=74239&start=10#p848375
I just dig it up from my computer. The results are a bit different from those I had in memory, but close enough I think!

I did the tests before J92-190 was selected for SuFi (J92-210 and J92-220 were the other nets considered).

On a rtx 2060 at 10s + 0.1s: (I can't remember the opening set. Maybe past TCEC).

Code: Select all

   # PLAYER      :  RATING  ERROR  PLAYED   (%)   CFS    W    D    L  D(%)
   1 J92-220     :    50.2   15.6     400  56.5    75  110  232   58  58.0
   2 J92-210     :    40.1   15.0     500  55.2   100  119  314   67  62.8
   3 T60-3010    :     0.0    9.3     900  44.2   ---  125  546  229  60.7

White advantage = 106.74 +/- 6.89
Draw rate (equal opponents) = 71.17 % +/- 1.82
Now at 40s + 0.4s (you had a longer time control, 60s + 0.6 and a 2070, so unfortunately this is only half of your LTC at best). I didn't do the test with J92-220 when it became clear it would be chosen for SuFi:

Code: Select all

   # PLAYER      :  RATING  ERROR  PLAYED   (%)   CFS    W    D    L  D(%)
   1 J92-210     :    32.9   10.7     240  54.6   100   39  184   17  76.7
   2 T60-3010    :     0.0   10.7     240  45.4   ---   17  184   39  76.7

White advantage = 50.42 +/- 11.32
Draw rate (equal opponents) = 82.04 % +/- 2.60
The collapse that happened at LTC in your test didn't occur here. J92-210 lost only 7 ratings points when the time control was multiplied by four. It would be surprinsing that J92-190 be scalling diffenrently than J92-210 or 220 according to JHortos.

Nay Lin Tun
Posts: 651
Joined: Mon Jan 16, 2012 5:34 am

Re: 87% of the poll voted that SF will win Tcec Sufi 19.

Post by Nay Lin Tun » Wed Oct 07, 2020 3:56 am

Still tied after 42 games. It is very likely that these two engines are within +/- 50 elo, most likely +/- 20 elo.

Thus being said, NNUE +100 elo hype was wrong against Leela.

In case, they are not equal in strength,
1. Is SF +50 elo better than Leela or
2. Is Leela + 50 elo better than SF? :mrgreen: :lol: :evil:

Dann Corbit
Posts: 11704
Joined: Wed Mar 08, 2006 7:57 pm
Location: Redmond, WA USA
Contact:

Re: 87% of the poll voted that SF will win Tcec Sufi 19.

Post by Dann Corbit » Wed Oct 07, 2020 5:18 am

Moving target.
They changed the Leela ratio for the this contest.

Nobody can say what the right ratio is anyway,

For that matter, both engines are moving targets.
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.

ernest
Posts: 1923
Joined: Wed Mar 08, 2006 7:30 pm

Re: 87% of the poll voted that SF will win Tcec Sufi 19.

Post by ernest » Thu Oct 08, 2020 1:01 am

Dann Corbit wrote:
Wed Oct 07, 2020 5:18 am

They changed the Leela ratio for the this contest.
Who says ?
Isn't the hardware the same as for TCEC 18 ?

Dann Corbit
Posts: 11704
Joined: Wed Mar 08, 2006 7:57 pm
Location: Redmond, WA USA
Contact:

Re: 87% of the poll voted that SF will win Tcec Sufi 19.

Post by Dann Corbit » Thu Oct 08, 2020 1:22 am

I stand corrected.
I just checked and the hardware and time control is identical.
Not sure where I got the impression that something was changed.
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.

ernest
Posts: 1923
Joined: Wed Mar 08, 2006 7:30 pm

Re: 87% of the poll voted that SF will win Tcec Sufi 19.

Post by ernest » Thu Oct 08, 2020 1:41 am

Still, I wonder (and of course I am not the only one...) why Lc0 is "resisting" so well against SF NNUE in Sufi 19.

As if the NNUE improvement almost did not help under the current playing conditions. Or...?

mwyoung
Posts: 2556
Joined: Wed May 12, 2010 8:00 pm

Re: 87% of the poll voted that SF will win Tcec Sufi 19.

Post by mwyoung » Thu Oct 08, 2020 2:34 am

ernest wrote:
Thu Oct 08, 2020 1:41 am
Still, I wonder (and of course I am not the only one...) why Lc0 is "resisting" so well against SF NNUE in Sufi 19.

As if the NNUE improvement almost did not help under the current playing conditions. Or...?

That is because testing has shown SF NNUE is not much stronger then Lc0. At long time controls. The confusion comes from only looking or testing at STC results. And assuming that result also applies to LTC games.
Professing themselves to be wise, they became fools,
take on me. Foes 0.

User avatar
Laskos
Posts: 10942
Joined: Wed Jul 26, 2006 8:21 pm
Full name: Kai Laskos

Re: 87% of the poll voted that SF will win Tcec Sufi 19.

Post by Laskos » Fri Oct 09, 2020 2:21 pm

ernest wrote:
Thu Oct 08, 2020 1:01 am
Dann Corbit wrote:
Wed Oct 07, 2020 5:18 am

They changed the Leela ratio for the this contest.
Who says ?
Isn't the hardware the same as for TCEC 18 ?
The Lc0 binaries are significantly faster now (+40%?). Also, the score now is back to almost regular: 3:1 =25 pairwise, with very high pairwise draw ratio. Maybe this pairwise draw rate is one of the reasons Elo difference seems small.

Post Reply