Page 4 of 19

Re: TCEC season 13, 2 NN engines will be participating, Leela and Deus X

Posted: Tue Jul 31, 2018 3:20 pm
by Branko Radovanovic
Milos wrote: Tue Jul 31, 2018 3:07 pm Because things like "compute some chess evaluation terms in the conventional way, and provide them as additional inputs to the net" make no sense at all.
Conventional chess evaluation terms are nothing but numbers in centi-pawns. NN takes board representation in some format and extracts features. These two are worlds apart. Combining them is pointless and very difficult if not impossible.
In that case I may have been unclear in using the word "term" - what I meant are raw counts (such as "number of doubled pawns"), i.e. what is usually called "variable", "predictor", or "feature".

Leela already uses "artificial", pre-computed inputs such as 50-move-rule counter, so I see no reason not to introduce more similar "shortcuts" and let the NN deal with them.

Re: TCEC season 13, 2 NN engines will be participating, Leela and Deus X

Posted: Tue Jul 31, 2018 3:21 pm
by Gian-Carlo Pascutto
Milos wrote: Tue Jul 31, 2018 3:07 pm Because things like "compute some chess evaluation terms in the conventional way, and provide them as additional inputs to the net" make no sense at all.
Output of computations of conventional chess evaluation terms is nothing but numbers in centi-pawns. NN takes board representation in some format and extracts features. These two are worlds apart. Combining them is pointless and very difficult if not impossible.
You can provide evaluation terms as bitmaps. I provide attack maps, passed pawns, open files, square color, etc.

You can also handle simple scalars, in at least 2 ways. You convert them to a larger amount of one-hot encoded bitmaps (wasteful for very fine grained ones, but perfectly possible for "opposite bishops", "drawish endgame", etc). If you are willing to skip having them available in the convolutional part of the network you can use a few fully-connected layers at the end and feed scalars to those directly. It's not very difficult and not impossible - I've already done so. Whether it helps, see the previous caveat regarding network size.

Whether it is pointless: it depends. Having some extra input knowledge certainly helps very small/fast networks. But I didn't manage to beat Stockfish 9 with the result.

Re: TCEC season 13, 2 NN engines will be participating, Leela and Deus X

Posted: Tue Jul 31, 2018 3:46 pm
by Branko Radovanovic
Gian-Carlo Pascutto wrote: Tue Jul 31, 2018 3:14 pm The people involved in the Leela Zero Chess project obviously like to keep the "Zero" aspect and not introduce bias by pre-selecting evaluation terms. That doesn't mean it's not a very obvious enhancement for anyone else to use that doesn't feel bound by such a restriction and wants to train some nets on their own.
I agree - when I said "obvious", I didn't mean to disparage what Leela developers are doing. I respect their decision to adhere to the "Zero" principle. It has definitely not been proven inferior either.

The question is whether eval is more like counting or more like pattern recognition. My guess is that it's both and - if that is true - hybrid approach might be promising.

Re: TCEC season 13, 2 NN engines will be participating, Leela and Deus X

Posted: Tue Jul 31, 2018 3:48 pm
by Milos
Gian-Carlo Pascutto wrote: Tue Jul 31, 2018 3:21 pm You can also handle simple scalars, in at least 2 ways. You convert them to a larger amount of one-hot encoded bitmaps (wasteful for very fine grained ones, but perfectly possible for "opposite bishops", "drawish endgame", etc). If you are willing to skip having them available in the convolutional part of the network you can use a few fully-connected layers at the end and feed scalars to those directly. It's not very difficult and not impossible - I've already done so. Whether it helps, see the previous caveat regarding network size.

Whether it is pointless: it depends. Having some extra input knowledge certainly helps very small/fast networks. But I didn't manage to beat Stockfish 9 with the result.
It is pointless because you are reinventing the wheel. Providing one-hot encoded bitmaps at input will probably have no effect at output. And I really see no point in skipping convolutional network and feeding scalars directly to fully-connected layers at the end when you can achieve much more controlled effect by using classical evaluation terms as additional bonuses on top of final score that NN provides.

Re: TCEC season 13, 2 NN engines will be participating, Leela and Deus X

Posted: Tue Jul 31, 2018 4:17 pm
by Gian-Carlo Pascutto
Milos wrote: Tue Jul 31, 2018 3:48 pm It is pointless because you are reinventing the wheel. Providing one-hot encoded bitmaps at input will probably have no effect at output.
Reinventing the wheel in what sense? For a large enough network, the network can compute those itself, I don't disagree there if you read what I wrote. That's why I doubt this'll ever make much difference in a LC0 style "huge network" engine.
And I really see no point in skipping convolutional network and feeding scalars directly to fully-connected layers at the end when you can achieve much more controlled effect by using classical evaluation terms as additional bonuses on top of final score that NN provides.
And treating the entire NN as an evaluation correction? Sure, that could works too. Incorporating them in the FC layer(s) at the end allows for some non-linearity, and in the LC0 design you already have this FC layer anyway, so why not. I've never tested the difference between these two, but I imagined that if I feed all my evaluation terms into the top of the FC stack then it might learn some non-linearity between them and what the NN computes.

Your case is the same as mine if the training learns to just put +1 and -1 -1 -1 coefficients to those terms, respectively.

Re: TCEC season 13, 2 NN engines will be participating, Leela and Deus X

Posted: Tue Jul 31, 2018 10:35 pm
by CMCanavessi
I think this move may backfire and will hurt TCEC more than it will do good. People are not stupid and want diversity, not 32 clones of SF. Deus X will use lc0 with a custom net. Shouldn't be allowed imho.

Re: TCEC season 13, 2 NN engines will be participating, Leela and Deus X

Posted: Tue Jul 31, 2018 10:59 pm
by jkiliani
CMCanavessi wrote: Tue Jul 31, 2018 10:35 pm I think this move may backfire and will hurt TCEC more than it will do good. People are not stupid and want diversity, not 32 clones of SF. Deus X will use lc0 with a custom net. Shouldn't be allowed imho.
I agree, it seems rather difficult to me to justify allowing Deus X while not letting developers of other strong engines enter clones with changed parameters under a new name. Let's see how TCEC spins this once the complaints to this move start coming in...

For Season 13, it seems the decision is made, but for the season after this they'll have to come up with a new set of rules and follow it consistently.

Re: TCEC season 13, 2 NN engines will be participating, Leela and Deus X

Posted: Tue Jul 31, 2018 11:12 pm
by AndrewGrant
jkiliani wrote: Tue Jul 31, 2018 10:59 pm
CMCanavessi wrote: Tue Jul 31, 2018 10:35 pm I think this move may backfire and will hurt TCEC more than it will do good. People are not stupid and want diversity, not 32 clones of SF. Deus X will use lc0 with a custom net. Shouldn't be allowed imho.
I agree, it seems rather difficult to me to justify allowing Deus X while not letting developers of other strong engines enter clones with changed parameters under a new name. Let's see how TCEC spins this once the complaints to this move start coming in...

For Season 13, it seems the decision is made, but for the season after this they'll have to come up with a new set of rules and follow it consistently.
I don't understand the motive here at all. It just seems so forced. We have the first season of TCEC where Leela is going to be on a real, mature net, powered by what I would say is a pretty good GPU setup. Leela's debut last season was just a little sneak peak. This time around Leela is going to thrash d4->d1 without issue. Why are we taking away the spotlight for a clone?

Albert gets this 25 minute interview with TCEC, the release of the engine is kept secret and rumored, people start to think some corporate entity is on its way in, others suggest people like Robert Houdart, ..., and when the news finally drops .... its just one guy who fed Leela's training code with different (already played) games.

I don't have anything against the guy personally, I only learned his name a few days ago. But his engine really has no business competing in TCEC. At least not while it is (so similar to) Leela.

TCEC has been forcing this NN GPU thing extremely hard, even delaying the season for (a month now?) just to get it to work. But this bit is too far for me.

Furthermore, were I the creater of Leela, or if someone submitted an Ethereal version trained on Stockfish games instead of the Laser games I use, I would be disgusted.

Re: TCEC season 13, 2 NN engines will be participating, Leela and Deus X

Posted: Tue Jul 31, 2018 11:15 pm
by CMCanavessi
AndrewGrant wrote: Tue Jul 31, 2018 11:12 pm
jkiliani wrote: Tue Jul 31, 2018 10:59 pm
CMCanavessi wrote: Tue Jul 31, 2018 10:35 pm I think this move may backfire and will hurt TCEC more than it will do good. People are not stupid and want diversity, not 32 clones of SF. Deus X will use lc0 with a custom net. Shouldn't be allowed imho.
I agree, it seems rather difficult to me to justify allowing Deus X while not letting developers of other strong engines enter clones with changed parameters under a new name. Let's see how TCEC spins this once the complaints to this move start coming in...

For Season 13, it seems the decision is made, but for the season after this they'll have to come up with a new set of rules and follow it consistently.
I don't understand the motive here at all. It just seems so forced. We have the first season of TCEC where Leela is going to be on a real, mature net, powered by what I would say is a pretty good GPU setup. Leela's debut last season was just a little sneak peak. This time around Leela is going to thrash d4->d1 without issue. Why are we taking away the spotlight for a clone?

Albert gets this 25 minute interview with TCEC, the release of the engine is kept secret and rumored, people start to think some corporate entity is on its way in, others suggest people like Robert Houdart, ..., and when the news finally drops .... its just one guy who fed Leela's training code with different (already played) games.

I don't have anything against the guy personally, I only learned his name a few days ago. But his engine really has no business competing in TCEC. At least not while it is (so similar to) Leela.

TCEC has been forcing this NN GPU thing extremely hard, even delaying the season for (a month now?) just to get it to work. But this bit is too far for me.

Furthermore, were I the creater of Leela, or if someone submitted an Ethereal version trained on Stockfish games instead of the Laser games I use, I would be disgusted.
Beware, TCEC Season 14 may have Leela Chess Ether ==> lc0 + network trained from Ethereal games :roll:

At least not while it is (so similar to) Leela.
Not only it's similar, it's the exact same binary (v0.16, as per ASilver's own words) with a custom net.

Re: TCEC season 13, 2 NN engines will be participating, Leela and Deus X

Posted: Tue Jul 31, 2018 11:26 pm
by AndrewGrant
CMCanavessi wrote: Tue Jul 31, 2018 11:15 pm Beware, TCEC Season 14 may have Leela Chess Ether ==> lc0 + network trained from Ethereal games :roll:
Guess I better start saving PGNs....
CMCanavessi wrote: Tue Jul 31, 2018 11:15 pm Not only it's similar, it's the exact same binary (v0.16, as per ASilver's own words) with a custom net.
Really? Not modified AT ALL? I assume he will at least change some UCI settings ... ?