One for Andrew Grant et al. - NNUE?

[connor_mcmonigle]

...

I see, this seems to be about competition, tournaments and rating lists, what
is allowed and what not. I am not in the 3000+ Elo ballpark and currently not
active as chess programmer, so I guess it is up to the top programmers and
the organizers and audience to define what kind of competition is desired and
fair. But I wish to point out, that by restricting competition in this way, you
also restrict development process, imagine one comes up with an non-standard,
non-SF search algorithm, which offers new features, but uses the SF-NNUE
implementation and networks. Such an engine would offer something new,
innovative in combination of non-original work, and would be not allowed to
participate in tournaments and rating lists. Take as current example Ceres, the
alternative implementation of Lc0, it uses the Lc0 backends and networks but
has an alternative implementation of search, according to your statements, such
an engine would be not allowed to participate in tournaments and rating lists,
is this desired?

--
Srdja

That's an interesting concept.

In my humble opinion, permitting a large number engines all using an evaluation functionally identical to that found in Stockfish to flood rating lists would lead to an even greater stagnation in innovation. Were the copying of Stockfish's evaluation function to become widely accepted by tournaments and rating lists, any new engine developer would be foolish to use anything other than Stockfish's evaluation function for position evaluation in their engine. There's less motivation to explore your own solutions/novel ideas when you can just copy the best in the world. Diversity is generally good for evolutionary progress to an extent.

This isn't necessarily counter to your point. If the goal of rating lists and tournaments in their specification of uniqueness constraints for participants is understood to be to maximize progress in computer chess, there obviously exists some optima between the extremes of overly restrictive and too permissive (only one participant can use alpha-beta search vs. every participant is a verbatim copy of Stockfish).

My estimate is that allowing the copying of Stockfish's evaluation function pushes us way too far in the direction of too permissive if progress in computer chess is the end goal.

[smatovic]

...

I see, this seems to be about competition, tournaments and rating lists, what
is allowed and what not. I am not in the 3000+ Elo ballpark and currently not
active as chess programmer, so I guess it is up to the top programmers and
the organizers and audience to define what kind of competition is desired and
fair. But I wish to point out, that by restricting competition in this way, you
also restrict development process, imagine one comes up with an non-standard,
non-SF search algorithm, which offers new features, but uses the SF-NNUE
implementation and networks. Such an engine would offer something new,
innovative in combination of non-original work, and would be not allowed to
participate in tournaments and rating lists. Take as current example Ceres, the
alternative implementation of Lc0, it uses the Lc0 backends and networks but
has an alternative implementation of search, according to your statements, such
an engine would be not allowed to participate in tournaments and rating lists,
is this desired?

--
Srdja

That's an interesting concept.

In my humble opinion, permitting a large number engines all using an evaluation functionally identical to that found in Stockfish to flood rating lists would lead to an even greater stagnation in innovation. Were the copying of Stockfish's evaluation function to become widely accepted by tournaments and rating lists, any new engine developer would be foolish to use anything other than Stockfish's evaluation function for position evaluation in their engine. There's less motivation to explore your own solutions/novel ideas when you can just copy the best in the world. Diversity is generally good for evolutionary progress to an extent.

This isn't necessarily counter to your point. If the goal of rating lists and tournaments in their specification of uniqueness constraints for participants is understood to be to maximize progress in computer chess, there obviously exists some optima between the extremes of overly restrictive and too permissive (only one participant can use alpha-beta search vs. every participant is a verbatim copy of Stockfish).

My estimate is that allowing the copying of Stockfish's evaluation function pushes us way too far in the direction of too permissive if progress in computer chess is the end goal.

Thanks, I get your point.

--
Srdja

[jdart]

I have never been in computer chess to win competitions or top rating lists. I accept that other people have that motivation but it is not what has kept me going all these years. People are still worried that someone will top them because they have some secret sauce, or some borrowed sauce. If you are not worried about your rating, then you don't care.

But as for this:
>Copying SF NNUE code is OK/NOK (even if your program is under GPL)

Violating GPL is not ok, although I believe in most cases there's no effective legal recourse. However, assuming the user is under a compatible license - Isn't GPL exactly for this? Clone and modify. Github is full of examples of that.

[smatovic]

I do get Connor's point, if you set the bar higher for entering competition by
forcing own implementation of NNUE you set a ground for the potential diversity
amongst implementations, assuming that programmers hereby understand the
technique and are therefore able to improve, add, modify, etc. it.

If this works out in the long run, if we loose diversity in other parts of the
engines by this, dunno.

--
Srdja

[AndrewGrant]

I do get Connor's point, if you set the bar higher for entering competition by
forcing own implementation of NNUE you set a ground for the potential diversity
amongst implementations, assuming that programmers hereby understand the
technique and are therefore able to improve, add, modify, etc. it.

If this works out in the long run, if we loose diversity in other parts of the
engines by this, dunno.

--
Srdja

Well you don't even have to speculate.
We've already seen a dozen engines copy paste Stockfish and their nets, without adding anything back to it.
Who has done something new?

[smatovic]

I do get Connor's point, if you set the bar higher for entering competition by
forcing own implementation of NNUE you set a ground for the potential diversity
amongst implementations, assuming that programmers hereby understand the
technique and are therefore able to improve, add, modify, etc. it.

If this works out in the long run, if we loose diversity in other parts of the
engines by this, dunno.

--
Srdja

Well you don't even have to speculate.
We've already seen a dozen engines copy paste Stockfish and their nets, without adding anything back to it.
Who has done something new?

Hmm, if you take a look at the programmers section this NN on CPU stuff is
currently taking off, in the bigger picture NNUE as used by SF is just one of
zillion possible implementations. Programmers already explained their intend,
they wish to experiment with NNUE, what is the gain, the quickest way is to
copy n paste other implementations and use others nets for this, some may stop
here, others may move on. I am not the guy who will tell them that this is
wrong, I am also not the guy who will define who may enter a tournament or not,
this is up to you guys who are active programmers.

--
Srdja

[xr_a_y]

Who has done something new?

At least some tried to do something and participate at least by sharing their results and thoughts.

[JohnWoe]

I have never been in computer chess to win competitions or top rating lists. I accept that other people have that motivation but it is not what has kept me going all these years. People are still worried that someone will top them because they have some secret sauce, or some borrowed sauce. If you are not worried about your rating, then you don't care.

But as for this:
>Copying SF NNUE code is OK/NOK (even if your program is under GPL)

Violating GPL is not ok, although I believe in most cases there's no effective legal recourse. However, assuming the user is under a compatible license - Isn't GPL exactly for this? Clone and modify. Github is full of examples of that.

This.

If the license isn't violated then why not use existing code? NIH syndrome at its best.
The problem is that if you don't use SF NNUE. You leave tons of Elo on the table.
If one really want to compete then obviously one need to close the source. Otherwise one is playing with open cards.

[xr_a_y]

The problem is that if you don't use SF NNUE. You leave tons of Elo on the table.

Well ...
One might say :

The problem is that if you don't use SF. You leave tons of Elo on the table.

Where is the limit ?