A Crossroad in Computer Chess; Or Desperate Flailing for Relevance

[Rebel]

...
I could be all wrong. I could be out of touch. But if I'm not, then the future of computer chess, the future of unique and diverse engines, depends upon all of us, as individuals, to encourage and promote new ideas while discouraging those who take from Stockfish without trying their hand at the problem. I'm already concerned when I see engines with Stockfish nets being placed onto rating lists.

In every research work there is the possibility of the partial or the total setback, I think.
Without any doubt the method of Alpha Zero Team is nearly automatized the development of chess engines.
This is the natural course of every technical innovations, earlier or later it comes a new trick what makes being superfluous the old ones.

His (rightly) concern is that engine authors download the latest SF NNUE net and gain a free xx elo. Example, suppose I add NNUE (I won't, but suppose) to ProDeo and get and 150 elo gain on the rating lists because of that. 6-7 months later -- having changed not one bit on my engine -- I download the latest SF net, it gives me a free 30-50 elo, I release it and rub my hands seeing ProDeo climb and climb in the rating lists. I would say that's a very unhealthy situation and surely this (kind of things) is what's going to happen in the near future.

So while NNUE is a fantastic new development regarding increasing strength it has an unwished negative side effect and I can understand that some engine authors already now consider to stop.

Looks to me there is a similar development in computer chess as in computer Shogi, where one combines for instace YaneuraOu (GPL) search (16 entries in 2019), adopted from Stockfish's search, with various evaluations, in 2019 many NNUE networks, trained by different programs, plus some sophisticated super-trained opening book. Take a look at the wcsc29 site 参加チーム (Participating teams) page the 使用ライブラリ (Library used) column ...

Any info how they are dealing with the problem?

[Terje]

...
I could be all wrong. I could be out of touch. But if I'm not, then the future of computer chess, the future of unique and diverse engines, depends upon all of us, as individuals, to encourage and promote new ideas while discouraging those who take from Stockfish without trying their hand at the problem. I'm already concerned when I see engines with Stockfish nets being placed onto rating lists.

In every research work there is the possibility of the partial or the total setback, I think.
Without any doubt the method of Alpha Zero Team is nearly automatized the development of chess engines.
This is the natural course of every technical innovations, earlier or later it comes a new trick what makes being superfluous the old ones.

His (rightly) concern is that engine authors download the latest SF NNUE net and gain a free xx elo. Example, suppose I add NNUE (I won't, but suppose) to ProDeo and get and 150 elo gain on the rating lists because of that. 6-7 months later -- having changed not one bit on my engine -- I download the latest SF net, it gives me a free 30-50 elo, I release it and rub my hands seeing ProDeo climb and climb in the rating lists. I would say that's a very unhealthy situation and surely this (kind of things) is what's going to happen in the near future.

So while NNUE is a fantastic new development regarding increasing strength it has an unwished negative side effect and I can understand that some engine authors already now consider to stop.

Looks to me there is a similar development in computer chess as in computer Shogi, where one combines for instace YaneuraOu (GPL) search (16 entries in 2019), adopted from Stockfish's search, with various evaluations, in 2019 many NNUE networks, trained by different programs, plus some sophisticated super-trained opening book. Take a look at the wcsc29 site 参加チーム (Participating teams) page the 使用ライブラリ (Library used) column ...

Any info how they are dealing with the problem?

They don't seem to see it as a problem.

[Gerd Isenberg]

...
I could be all wrong. I could be out of touch. But if I'm not, then the future of computer chess, the future of unique and diverse engines, depends upon all of us, as individuals, to encourage and promote new ideas while discouraging those who take from Stockfish without trying their hand at the problem. I'm already concerned when I see engines with Stockfish nets being placed onto rating lists.

In every research work there is the possibility of the partial or the total setback, I think.
Without any doubt the method of Alpha Zero Team is nearly automatized the development of chess engines.
This is the natural course of every technical innovations, earlier or later it comes a new trick what makes being superfluous the old ones.

His (rightly) concern is that engine authors download the latest SF NNUE net and gain a free xx elo. Example, suppose I add NNUE (I won't, but suppose) to ProDeo and get and 150 elo gain on the rating lists because of that. 6-7 months later -- having changed not one bit on my engine -- I download the latest SF net, it gives me a free 30-50 elo, I release it and rub my hands seeing ProDeo climb and climb in the rating lists. I would say that's a very unhealthy situation and surely this (kind of things) is what's going to happen in the near future.

So while NNUE is a fantastic new development regarding increasing strength it has an unwished negative side effect and I can understand that some engine authors already now consider to stop.

Looks to me there is a similar development in computer chess as in computer Shogi, where one combines for instace YaneuraOu (GPL) search (16 entries in 2019), adopted from Stockfish's search, with various evaluations, in 2019 many NNUE networks, trained by different programs, plus some sophisticated super-trained opening book. Take a look at the wcsc29 site 参加チーム (Participating teams) page the 使用ライブラリ (Library used) column ...

Any info how they are dealing with the problem?

May be some former participants of CSA events could explain. CSA seem to define library entries, search and eval (?), and teams need to apply an appeal document in order to participiate, explaining what they intend to use, how to train etc.!?

https://www.semanticscholar.org/paper/C ... per-header

[Albert Silver]

...
I could be all wrong. I could be out of touch. But if I'm not, then the future of computer chess, the future of unique and diverse engines, depends upon all of us, as individuals, to encourage and promote new ideas while discouraging those who take from Stockfish without trying their hand at the problem. I'm already concerned when I see engines with Stockfish nets being placed onto rating lists.

In every research work there is the possibility of the partial or the total setback, I think.
Without any doubt the method of Alpha Zero Team is nearly automatized the development of chess engines.
This is the natural course of every technical innovations, earlier or later it comes a new trick what makes being superfluous the old ones.

His (rightly) concern is that engine authors download the latest SF NNUE net and gain a free xx elo. Example, suppose I add NNUE (I won't, but suppose) to ProDeo and get and 150 elo gain on the rating lists because of that. 6-7 months later -- having changed not one bit on my engine -- I download the latest SF net, it gives me a free 30-50 elo, I release it and rub my hands seeing ProDeo climb and climb in the rating lists. I would say that's a very unhealthy situation and surely this (kind of things) is what's going to happen in the near future.

So while NNUE is a fantastic new development regarding increasing strength it has an unwished negative side effect and I can understand that some engine authors already now consider to stop.

I honestly don't see the problem you describe. Let's suppose you do add NNUE support to ProDeo, and as a result, it can support not only SF's net, but also a number of others, including one of your own making/training. yes, the net is modular in such a case, much as a GUI can run a variety of UCI engines, a Playstation can run a variety of games, and so on. The claim that you are 'gaining' 30, 50, or 150 Elo should you or anyone else run an outside net seems silly. Are you really gaining this Elo? Or are you just saying: "if you use the SF net inside ProDeo it will play at X Elo". Naturally, if you obfuscate this by claiming authorship of another net you are in a different situation, but such shenanigans aside, how a user chooses to use your engine and what net seems to me to be their concern, no?

When I played in Leiden in December 2018 (and won) I was very forthcoming about using the GPLed Lc0 binary to run my homebrewed NN "DeusX". I explained what had gone into it: human games, engine games, selfplay games, tablebases, etc. to set it apart from Leela's net, and it was up to the organizers to decide whether this was an acceptable entry or not. Other tournaments also had no issues, and authors such as Mark Lefler have also voiced support. Should someone decide do run the Fat Fritz NN (assuming they are legal owners) in Scorpio, Allie, or some other, I cannot imagine either the authors of those engines or myself objecting.

[AndrewGrant]

When I played in Leiden in December 2018 (and won) I was very forthcoming about using the GPLed Lc0 binary to run my homebrewed NN "DeusX". I explained what had gone into it: human games, engine games, selfplay games, tablebases, etc. to set it apart from Leela's net

That is a nice change of pace for you
https://blog.lczero.org/blog/2018/08/up ... s-team-on/

[Albert Silver]

When I played in Leiden in December 2018 (and won) I was very forthcoming about using the GPLed Lc0 binary to run my homebrewed NN "DeusX". I explained what had gone into it: human games, engine games, selfplay games, tablebases, etc. to set it apart from Leela's net

That is a nice change of pace for you
https://blog.lczero.org/blog/2018/08/up ... s-team-on/

I owe that post, and all the BS in it, full credit for the very existence of Fat Fritz. Initially, DeusX was nothing more than a science project, a proof of concept. It was that post and the subsequent persecutions that drove me to where I am. Without that, it would certainly have died off.

[dkappe]

When I played in Leiden in December 2018 (and won) I was very forthcoming about using the GPLed Lc0 binary to run my homebrewed NN "DeusX". I explained what had gone into it: human games, engine games, selfplay games, tablebases, etc. to set it apart from Leela's net

That is a nice change of pace for you
https://blog.lczero.org/blog/2018/08/up ... s-team-on/

As the editor of the blog at that time, I was aware of the behind the scenes discussions around that topic. Here is my note to the post:

Editor's note: if you're curious as to why I stopped blogging for LCZ, it's this statement by the "core" dev team. I considered it ill advised and intemperate. I published it under protest and then washed my hands of the whole thing.

It’s so unlike you to publish a post in ignorance, Andy.

[AndrewGrant]

When I played in Leiden in December 2018 (and won) I was very forthcoming about using the GPLed Lc0 binary to run my homebrewed NN "DeusX". I explained what had gone into it: human games, engine games, selfplay games, tablebases, etc. to set it apart from Leela's net

That is a nice change of pace for you
https://blog.lczero.org/blog/2018/08/up ... s-team-on/

As the editor of the blog at that time, I was aware of the behind the scenes discussions around that topic. Here is my note to the post:

Editor's note: if you're curious as to why I stopped blogging for LCZ, it's this statement by the "core" dev team. I considered it ill advised and intemperate. I published it under protest and then washed my hands of the whole thing.

It’s so unlike you to publish a post in ignorance, Andy.

Well, the DeusX thing is one of the weirder things in TCEC history.
Its a major reason for the concern over people lying about origins of their NNUEs.
Since we've already shown that one can attempt to lie, get called out, but still compete, only to be removed after the fact.
No one was impressed by DeusX at the time, except for those who were deceived by Albert's sweet song.

The Leela project gets far too little credit. They did what AlphaZero did, but a billion times better. They beat Stockfish with publically available and reasonably obtained hardware. Then they went a step further, produced a CPU version which could compete with all but Stockfish. A massive feat. Its a shame to see any project use and abuse the work that went into Leela.

[dkappe]

Well, the DeusX thing is one of the weirder things in TCEC history.
Its a major reason for the concern over people lying about origins of their NNUEs.
Since we've already shown that one can attempt to lie, get called out, but still compete, only to be removed after the fact.
No one was impressed by DeusX at the time, except for those who were deceived by Albert's sweet song.

The Leela project gets far too little credit. They did what AlphaZero did, but a billion times better. They beat Stockfish with publically available and reasonably obtained hardware. Then they went a step further, produced a CPU version which could compete with all but Stockfish. A massive feat. Its a shame to see any project use and abuse the work that went into Leela.

As someone in the know about what went on and as someone involved with the leela project close to the start, I can say that you don’t know what you’re talking about.

[AndrewGrant]

Well, the DeusX thing is one of the weirder things in TCEC history.
Its a major reason for the concern over people lying about origins of their NNUEs.
Since we've already shown that one can attempt to lie, get called out, but still compete, only to be removed after the fact.
No one was impressed by DeusX at the time, except for those who were deceived by Albert's sweet song.

The Leela project gets far too little credit. They did what AlphaZero did, but a billion times better. They beat Stockfish with publically available and reasonably obtained hardware. Then they went a step further, produced a CPU version which could compete with all but Stockfish. A massive feat. Its a shame to see any project use and abuse the work that went into Leela.

As someone in the know about what went on and as someone involved with the leela project close to the start, I can say that you don’t know what you’re talking about.

Could you please explain it to me then? If I'm missing crucial information, I'm perfectly happy to change my view when presented with additional information which brings my current view into question.