Page 8 of 19
Re: TCEC season 13, 2 NN engines will be participating, Leela and Deus X
Posted: Thu Aug 02, 2018 3:30 am
by Nay Lin Tun
whereagles wrote: ↑Thu Aug 02, 2018 1:52 am
Aren't you guys being a bit too strict??? TCEC is for amusement only, and two NN, trained under very different circumstances, add a nice twist to this season. The shared base code situation is obviously borderline on the regulations, but the competition will certainly gain in interest.
Yes TCEC is a private computer tournament, so basically " My own tournament, my own rules".
In fact most people dont care much anymore on official organizations like ICGA but rather on TCEC because of their reputation on "strict regulation and fair competition"
That being said, TCEC didn't accept engines of controversial origin before, but accepting this controversial one now.
Re: TCEC season 13, 2 NN engines will be participating, Leela and Deus X
Posted: Thu Aug 02, 2018 3:59 am
by noobpwnftw
Funny that after the authors of both were to made a statement and made clear on their stands, people seem to on the one hand disregard or question the authority parties while on the other hand they just want one among them to do whatever they want.
For those who want to have DeusX disqualified, your easy pass is to get Leela team to say that and it'd probably work. Oh, wait, I can recall some "bad sportsmanship" accusations was made recently, who knows but in order not to let it backfire, best they can do is to keep whining.
Re: TCEC season 13, 2 NN engines will be participating, Leela and Deus X
Posted: Thu Aug 02, 2018 8:47 am
by Gian-Carlo Pascutto
Can we have Komodo 12(+) regular and Komodo 12(+) MCTS too then?
It would be at least as interesting.
Re: TCEC season 13, 2 NN engines will be participating, Leela and Deus X
Posted: Thu Aug 02, 2018 9:20 am
by noobpwnftw
This will actually be even more interesting: eventually Komodo team will pick what version(s) to participate, but who is going to make that choice for all Leelas?
Re: TCEC season 13, 2 NN engines will be participating, Leela and Deus X
Posted: Thu Aug 02, 2018 10:20 am
by frankp
Entirely a matter for TCEC to determine what can participate. Their decisions will affect only how people view their competition.
I am actually interested to see how Deusx plays, particularly as it is a clone of leela/lc0 but trained with human games - it is said.
On the other hand, however much effort was involved in producing the NN weighting factors, fundamentally I cannot see how this is different to changing the numerical evaluation factors (pst etc) in SF. It would play differently too - and be of interest.
Interesting decision and precedent for the future perhaps.
Re: TCEC season 13, 2 NN engines will be participating, Leela and Deus X
Posted: Thu Aug 02, 2018 10:27 am
by crem
Regarding Leela and DeusX, there are two separate questions.
First, in original announcement of DeusX and interview from author, there was no mention how it is related to LCZero, and it made impression that ASilver has a separate independent implementation of the engine and NN training scripts, which is wrong and unfair to LCZero team. That confusion was resolved with a special statement issued on chessdom website.
Second question is whether Lc0 + custom network is a separate enough engine to be allowed to participate in TCEC.
It's true that different NN on the same engine may have very different strengths and styles, so from the chess perspective they look different and it should be interesting to watch games of both.
It's also true that training a NN is not a trivial task. It requires lots of trial and error and parameter tuning.
So, when there is only a few (2-3) NNs running on the same engine, I think it's fine to allow them all.
But while training NN is not a trivial, it's still much easier than writing 3000+ Elo engine from scratch with no prior experience. NN can be trained within a month of spending 1 hour every evening, while writing engine (especially A/B engine) requires much more time, concentration and skills (for a person who didn't do that before).
With that it's possible that for the next TCEC season much more custom networks will appear (trained from tactical puzzles, Tal games, engine games, manually training against specific blunder types, endgame tablebase training, chess960 selfplay training, training draws as loss, whatever), and then having Lc0 + 10 different networks as different TCEC participants do look wrong (to me), even if they have very different style.
Or maybe that's fine too, who knows.
Re: TCEC season 13, 2 NN engines will be participating, Leela and Deus X
Posted: Thu Aug 02, 2018 10:42 am
by noobpwnftw
I'd be happier if they keep what it was used to be - given the same hardware of their choice.
Allowing NN engines to run on different hardware is enough compromise in my opinion.
Asking for not letting some other engines to participate is only possible if, for example crem and et.al. find it not appropriate to have DeusX running its NN with their engine, and file a complaint. Otherwise it is just more whining as much as I think no engine should receive special treatment of hardware, which is just my personal bias, like it or not.
IIRC, TCEC gets to pick their participants via invitations, since when they need to be care for what you and I think? There are alternatives like YCLET, and why not run your own tournaments like CCLS and a dozen of other streams?
My suggestion is rather simple: if NN people can't settle with their "controversy", disqualify them all and restore the traditional tournament setup.
I can summarize the whole story like this, Leela wanted to be in such a tournament for more publicity, then people start blackmailing TCEC over their "reputation" in order to get everything they want, GPUs, better GPUs, disqualifying other competitors, really decent.
Re: TCEC season 13, 2 NN engines will be participating, Leela and Deus X
Posted: Thu Aug 02, 2018 12:04 pm
by IQ
I am late to the party and never would have expected to fully agree with Milos.
The snowflakes from the lc0 team have been conned. Albert Silver pulled a fast-one on them and now tries to lull them into compliance with all his sweet talk. The video that was released is absolutely outrageous and shows Albert Silvers true colors. He did not write the tools, nor the search, and did not participate in the work and research that went on when lc0 was still leela zero, and even before that when it originated from leela go. And now he wants to be seen as some kind of AI-expert? Did coding an engine come down to changing a couple of numbers in a scrip written by somebody else, setting a couple of filters in megabase 2018 and letting the computer run for a couple of hours? And then having the audacity to secretly enter a competition, stealing the main teams efficient implementaion of search, NN query, and uci code? How does he think DeusX would do without search? The same logic would allow anybody to enter SF clones with their own individual clop tuned values. And the myth that different NNs play sooooooooooooooooo differently is just a myth. In fact the competition should be even more furious, but as TCEC gains from the increased attention they have no incentive to adhere to their own rules.
This is like Hamilton stealing Vettels Ferrari for the upcoming GP, calling Vettel from his cell-phone while driving of the Ferrari lot requesting some tuning tipps and telling him its all just a big misunderstanding. The lc0 team is much to nice to realize that this is a well planned publicity stunt. Hats off to Albert Silver for getting away with this! The added fineprint in the chessdom page hardly makes a dent and whenever Albert Silver tells this story at his villians anonymous group they all break into laughter.
The same evil laugh I hear at the end of the video when he says:
""As to whether it will be commercial, who can say... If I come out with something that's really spectacular and that outdoes Alphazero, very possibly. Right now at the moment, it's a private project."
Re: TCEC season 13, 2 NN engines will be participating, Leela and Deus X
Posted: Thu Aug 02, 2018 4:23 pm
by Error323
IQ wrote: ↑Thu Aug 02, 2018 12:04 pm
I am late to the party and never would have expected to fully agree with Milos.
The snowflakes from the lc0 team have been conned. Albert Silver pulled a fast-one on them and now tries to lull them into compliance with all his sweet talk. The video that was released is absolutely outrageous and shows Albert Silvers true colors. He did not write the tools, nor the search, and did not participate in the work and research that went on when lc0 was still leela zero, and even before that when it originated from leela go. And now he wants to be seen as some kind of AI-expert? Did coding an engine come down to changing a couple of numbers in a scrip written by somebody else, setting a couple of filters in megabase 2018 and letting the computer run for a couple of hours? And then having the audacity to secretly enter a competition, stealing the main teams efficient implementaion of search, NN query, and uci code? How does he think DeusX would do without search? The same logic would allow anybody to enter SF clones with their own individual clop tuned values. And the myth that different NNs play sooooooooooooooooo differently is just a myth. In fact the competition should be even more furious, but as TCEC gains from the increased attention they have no incentive to adhere to their own rules.
This is like Hamilton stealing Vettels Ferrari for the upcoming GP, calling Vettel from his cell-phone while driving of the Ferrari lot requesting some tuning tipps and telling him its all just a big misunderstanding. The lc0 team is much to nice to realize that this is a well planned publicity stunt. Hats off to Albert Silver for getting away with this! The added fineprint in the chessdom page hardly makes a dent and whenever Albert Silver tells this story at his villians anonymous group they all break into laughter.
The same evil laugh I hear at the end of the video when he says:
""As to whether it will be commercial, who can say... If I come out with something that's really spectacular and that outdoes Alphazero, very possibly. Right now at the moment, it's a private project."
This was my reasoning and initial reaction also, though not public. I won't lie, when reading this it kinda comes back. This could have been done with a lot more grace. I still think it's ridiculous to call it a new engine. I can train a good network on human data in less than a week. Whatever.
Re: TCEC season 13, 2 NN engines will be participating, Leela and Deus X
Posted: Thu Aug 02, 2018 4:40 pm
by Milos
Error323 wrote: ↑Thu Aug 02, 2018 4:23 pm
IQ wrote: ↑Thu Aug 02, 2018 12:04 pm
I am late to the party and never would have expected to fully agree with Milos.
The snowflakes from the lc0 team have been conned. Albert Silver pulled a fast-one on them and now tries to lull them into compliance with all his sweet talk. The video that was released is absolutely outrageous and shows Albert Silvers true colors. He did not write the tools, nor the search, and did not participate in the work and research that went on when lc0 was still leela zero, and even before that when it originated from leela go. And now he wants to be seen as some kind of AI-expert? Did coding an engine come down to changing a couple of numbers in a scrip written by somebody else, setting a couple of filters in megabase 2018 and letting the computer run for a couple of hours? And then having the audacity to secretly enter a competition, stealing the main teams efficient implementaion of search, NN query, and uci code? How does he think DeusX would do without search? The same logic would allow anybody to enter SF clones with their own individual clop tuned values. And the myth that different NNs play sooooooooooooooooo differently is just a myth. In fact the competition should be even more furious, but as TCEC gains from the increased attention they have no incentive to adhere to their own rules.
This is like Hamilton stealing Vettels Ferrari for the upcoming GP, calling Vettel from his cell-phone while driving of the Ferrari lot requesting some tuning tipps and telling him its all just a big misunderstanding. The lc0 team is much to nice to realize that this is a well planned publicity stunt. Hats off to Albert Silver for getting away with this! The added fineprint in the chessdom page hardly makes a dent and whenever Albert Silver tells this story at his villians anonymous group they all break into laughter.
The same evil laugh I hear at the end of the video when he says:
""As to whether it will be commercial, who can say... If I come out with something that's really spectacular and that outdoes Alphazero, very possibly. Right now at the moment, it's a private project."
This was my reasoning and initial reaction also, though not public. I won't lie, when reading this it kinda comes back. This could have been done with a lot more grace. I still think it's ridiculous to call it a new engine. I can train a good network on human data in less than a week. Whatever.
There is no difference in training NN and tuning eval coefficient for classical engine running CLOP or which ever way one prefers. It is not a science because there is no science behind, it is typical trial and error. Albert is trying to present it as some humongous effort and huge contribution and is piggybacking on ppl's lack of knowledge about NN training which is quite frankly laughable.
Just changing backend from TF to CUDNN requires for at least an order of magnitude more effort, not to mention amount of actual knowledge.