TCEC Division 3 results simulator

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Rebel, chrisw

User avatar
CMCanavessi
Posts: 1142
Joined: Thu Dec 28, 2017 4:06 pm
Location: Argentina

Re: TCEC Division 3 results simulator

Post by CMCanavessi »

Uri Blass wrote: Thu Aug 16, 2018 10:31 pm
CMCanavessi wrote: Thu Aug 16, 2018 10:16 pm So after the "non-optimal" fix they applied in the 2nd TCEC server (the 2xgpu one) leela is 5.5/6, where's all the people that were saying that the level of play that leela was showing at the beginning was real and the ones that knew that something was wrong were just "fanboys"? I understand variance of outcome, but this is too extreme. From being humiliated and not winning a single game for what, 20 games (?) to getting 5.5/6 is a long way...

There are rumors now that the previous NN games that happened before the ugly fix will be replayed (Leela and DeusX). That's gonna create another flamewar in itself :D
Fixing the hardware setting for lc0 in division 3 is not fair when tcec did not fix the fact that ivanhoe used only one core in division 4 because of rule that they fix nothing unless the engine crash(even if it is not the exe file but some setting).

Replaying previous NN games is not fair unless you go back to division 4 and replay all Ivanhoe games and if Ivanhoe or Wasp promote instead of Deus as a result also give Ivanhoe or Wasp a chance to compete in division 3 instead of Deus.
It's a completely different thing. The ivanhoe case was an ENGINE problem. The binary used had broken SMP. It was its own fault.
In this case, the problem is the SERVER, and it affects all engines running on it (leela and deusx), and it's a TCEC problem, not engine-related.

If you can't see the difference between those 2 situations, then it's pointless to discuss. Still, I agree that replaying all those games would look "weird" and I personally don't like it.
Follow my tournament and some Leela gauntlets live at http://twitch.tv/ccls
chrisw
Posts: 4319
Joined: Tue Apr 03, 2012 4:28 pm

Re: TCEC Division 3 results simulator

Post by chrisw »

CMCanavessi wrote: Thu Aug 16, 2018 10:28 pm And what if

Arasan 0-1 Ethereal
and
lc0 1-0 Arasan

That's the interesting case. Also considering Arasan 0.5-0.5 Ethereal.
I already wrote about that. The potential tie breaks between Arasan and LC0 fall in favour of Arasan.
LC0 falls on the first tie breaker (direct encounters) unless LC0-Arasan is win for LC0, in which case first tiebreak is drawn.
LC0 will almost certainly fail the second tie break condition (win count).
Maybe I should rephrase that: LC0 will probably fail the second tie break condition, more often than not.

Which means LC0 needs 0.5 more pts than Arasan to go through. Tough one.
Last edited by chrisw on Thu Aug 16, 2018 10:59 pm, edited 1 time in total.
Uri Blass
Posts: 10301
Joined: Thu Mar 09, 2006 12:37 am
Location: Tel-Aviv Israel

Re: TCEC Division 3 results simulator

Post by Uri Blass »

Note that I assume that ivanhoe did not use more than one core only because of wrong setting.
If the problem is in the exe and ivanhoe that they got did not run without crashes with more than one core then I do not think TCEC need to repeat the games of it.
Milos
Posts: 4190
Joined: Wed Nov 25, 2009 1:47 am

Re: TCEC Division 3 results simulator

Post by Milos »

CMCanavessi wrote: Thu Aug 16, 2018 10:35 pmIt's a completely different thing. The ivanhoe case was an ENGINE problem. The binary used had broken SMP. It was its own fault.
In this case, the problem is the SERVER, and it affects all engines running on it (leela and deusx), and it's a TCEC problem, not engine-related.

If you can't see the difference between those 2 situations, then it's pointless to discuss. Still, I agree that replaying all those games would look "weird" and I personally don't like it.
You are the one who can see because you are so blinded by your own fanboyism.
Whose fault is Ivanhoe?
Binary was provided by Kranium Fire's author (so totally unrelated person to Ivanhoe engine, who btw. had strong disagreement when Ivanhoe was previously allowed to competed in TCEC) who didn't care to look, or maybe even slipped broken compile intentionally since he openly said he doesn't care for Ivanhoe and want for NN engine(s) to go further.
Then their is incompetent money hungry TCEC owner Anton, who doesn't care about anything but the amount of viewers that bring him earnings and that basically made a charade out of TCEC. Who didn't bother even once to start the binary before tournament started or to actually use the binary that was used previously when Ivanhoe was participating in TCEC.
Mess with Ivanhoe is much more a fault of organizers and handicap was for an order of magnitude worse than with Lc0 and no one even bothered to suggest it was not fair from Lc0 team. So much about objectivity.
User avatar
CMCanavessi
Posts: 1142
Joined: Thu Dec 28, 2017 4:06 pm
Location: Argentina

Re: TCEC Division 3 results simulator

Post by CMCanavessi »

Uri Blass wrote: Thu Aug 16, 2018 10:42 pm Note that I assume that ivanhoe did not use more than one core only because of wrong setting.
That part is wrong, it wasn't a settings issue. Settings were correct for THAT version.

If the problem is in the exe and ivanhoe that they got did not run without crashes with more than one core then I do not think TCEC need to repeat the games of it.
Yes, the problem was with the submitted exe.
Follow my tournament and some Leela gauntlets live at http://twitch.tv/ccls
chrisw
Posts: 4319
Joined: Tue Apr 03, 2012 4:28 pm

Re: TCEC Division 3 results simulator

Post by chrisw »

After Ethereal win against Arasan

Ethereal 100% to promote
Arasan 62%
LC0 33%
Pedone 4%

Code: Select all

Engine,   Tournament Elo, Initial Elo, First, First two
Ethereal       3317         3300       1.000  1.000
Arasan         3151         3124       0.000  0.620
lc0            3135         3100       0.000  0.331
Pedone         3116         3092       0.000  0.045
Nemorino       3092         3099       0.000  0.000
Hannibal       3114         3158       0.000  0.000
Bobcat         2977         3029       0.000  0.000
DeusX          3130         3130       0.000  0.005
If LC0 vs Arasan 1-0 LC0 70%, Arasan 24% Pedone 5%
If LCO vs Arasan Draw Arasan 65% LC0 28% Pedone 6%
IF LC0 vs Arasan 0-1 Arasan 92% LC0 3% Pedone 4%
chrisw
Posts: 4319
Joined: Tue Apr 03, 2012 4:28 pm

Re: TCEC Division 3 results simulator

Post by chrisw »

106 games
Sim promotion chances ...
Ethereal 100%
Arasan 74%
LC0 25%

Code: Select all

Engine,   Tournament Elo, Initial Elo, First, First two
Ethereal       3318         3300       1.000  1.000
Arasan         3149         3124       0.000  0.744
lc0            3133         3100       0.000  0.251
Pedone         3113         3092       0.000  0.006
Hannibal       3119         3158       0.000  0.000
Nemorino       3092         3099       0.000  0.000
Bobcat         2975         3029       0.000  0.000
DeusX          3129         3130       0.000  0.000
frankp
Posts: 228
Joined: Sun Mar 12, 2006 3:11 pm

Re: TCEC Division 3 results simulator

Post by frankp »

There is a certain irony in leela's race for second place not being helped by a draw with its clone (leela-dx).

On the other hand, now the hardware problem is fixed, it seems to be back to being leela with a much better, and probably anticipated, performance in the latter stages of the competition.

Interested to see how far this experiment runs; and whether we are genuinely witnessing the beginning of the end of the pure AB engine dominance in chess. Surprised we have not seen other NN engines emerging (discounting the leela-dx clone), because I doubt that leela (NN structure, back end 'search' etc) is the optimised approach for chess. Great credit to the leela team and all involved in a terrific project.

Congratulations to John, assuming arasan gets promoted. And of course Ethereal, which seems to be a class above the rest of the field - overall anyway.
chrisw
Posts: 4319
Joined: Tue Apr 03, 2012 4:28 pm

Re: TCEC Division 3 results simulator

Post by chrisw »

Arasan - Bobcat, the decider, if Arasan wins it promotes, if not LC0 promotes. Sim suggests the chances are:

Arasan 59% LC0 40%

Code: Select all

Engine,   Tournament Elo, Initial Elo, First, First two
Ethereal       3314         3300       1.000  1.000
Arasan         3142         3124       0.000  0.593
lc0            3132         3100       0.000  0.407
DeusX          3129         3130       0.000  0.000
Hannibal       3120         3158       0.000  0.000
Nemorino       3093         3099       0.000  0.000
Bobcat         2981         3029       0.000  0.000
Pedone         3117         3092       0.000  0.000
chrisw
Posts: 4319
Joined: Tue Apr 03, 2012 4:28 pm

Re: TCEC Division 3 results simulator

Post by chrisw »

[d]1r1qr1k1/1p3pb1/pn1p2p1/2p2P1p/P1n1P3/1RN4Q/1PP1B1PP/2B2RK1 b - - 2 23

Arasan ought to win this, and go through to Division 2