Before things become more messy than they already are.

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Rebel, chrisw

User avatar
Rebel
Posts: 7006
Joined: Thu Aug 18, 2011 12:04 pm
Full name: Ed Schröder

Before things become more messy than they already are.

Post by Rebel »

With the current SF14 subject evolving I would like to do another programmer poll as I have done in the past, see for instance:

http://www.top-5000.nl/appeal.htm

This time about the use of networks which maybe can lead to a new programmer code, see:

http://rebel13.nl/download/programmer-c ... honor.html

Before creating the poll page I like to propose the following poll questions open for discussion and improvement.

1. My preference is that I am allowed to use any existing network for my engine.
2. My preference is that I am allowed to use any game collection to create my own network.
3. My preference is that I am in favor that any programmer creates his own network either from human game collections or from his own engine.

Shoot !
90% of coding is debugging, the other 10% is writing bugs.
Madeleine Birchfield
Posts: 512
Joined: Tue Sep 29, 2020 4:29 pm
Location: Dublin, Ireland
Full name: Madeleine Birchfield

Re: Before things become more messy than they already are.

Post by Madeleine Birchfield »

I would vote for the second option
2. My preference is that I am allowed to use any game collection to create my own network.
as this is already the norm with tuning handcrafted evaluation functions such as Pesto.

The firat choice is akin to taking an evaluation function directly from another engine, which most would agree would not count as unique, while the third choice is akin to forcing engines to tune using only their own engine, whereas many engines have used CCRL data or the Zurichess game collections in tuning in the past.

For this reason engines like Stockfish 14 and Nemorino 6.00 are fine, while engines like Mayham, Fire, and BBC 1.4 are too derivative. People should learn to train or tune their own networks, just as they would learn to tune their own handcrafted evaluations.
User avatar
hgm
Posts: 27829
Joined: Fri Mar 10, 2006 10:06 am
Location: Amsterdam
Full name: H G Muller

Re: Before things become more messy than they already are.

Post by hgm »

Whatever anyone's preference, it should be clear that programmers are allowed to do whatever they want, as long as they don't violate copyright licences. And that as long as they honestly specify what they have been doing, there is never anything unethical about that.
User avatar
Rebel
Posts: 7006
Joined: Thu Aug 18, 2011 12:04 pm
Full name: Ed Schröder

Re: Before things become more messy than they already are.

Post by Rebel »

hgm wrote: Fri Jul 02, 2021 9:24 pm Whatever anyone's preference, it should be clear that programmers are allowed to do whatever they want, as long as they don't violate copyright licences. And that as long as they honestly specify what they have been doing, there is never anything unethical about that.
What you say is self understood but the context of this thread is about fair competition.

Personally I agree whit what Madeleine said, [2] is not perfect but [3] limits the freedom of programmers too much, I will go for [2].

If there are no further comments on the 3 questions I will create a programmer and user poll tomorrow.
90% of coding is debugging, the other 10% is writing bugs.
connor_mcmonigle
Posts: 533
Joined: Sun Sep 06, 2020 4:40 am
Full name: Connor McMonigle

Re: Before things become more messy than they already are.

Post by connor_mcmonigle »

Personally, it would be neat if option 3 was the norm, but obviously everyone is free to do what they enjoy in computer chess provided they don't do anything illegal or are otherwise dishonest about the origins of their engine.

I believe there's somewhat of a dearth of technical information in a digestible format about top engines in general which limits the capacity of rating lists and tournaments to make informed decisions in selecting engines deemed "sufficiently original/different". This leads to silly situations such as testers refusing to test Stockfish 14 as it was trained on Lc0 self play data. I think it would be far more productive to have some sort of space (with some moderation to enforce honesty) for authors to self report technical details about their engines.

It's obviously impossible to develop an exact set of criterion for what constitutes an original work, but I do think it's possible to at least maintain some standards (provided sufficient information is available). It's always going to be subjective and I don't think the above 3 options comes close to capturing the complexity of the situation. For example, if the author an engine, such as RubiChess which copies Stockfish inference code, training code and network topology, decided to train on Stockfish data, their engine would be far less "original" than an engine implementing a meaningfully different network topology, training code, inference code, etc. You're never going to please everyone, but throwing out standards and testing anything and everything is untenable.
Last edited by connor_mcmonigle on Fri Jul 02, 2021 10:50 pm, edited 1 time in total.
Madeleine Birchfield
Posts: 512
Joined: Tue Sep 29, 2020 4:29 pm
Location: Dublin, Ireland
Full name: Madeleine Birchfield

Re: Before things become more messy than they already are.

Post by Madeleine Birchfield »

connor_mcmonigle wrote: Fri Jul 02, 2021 10:37 pm such as RubiChess which copies on Stockfish inference code
Not anymore: the Stockfish team rewrote its own nnue inference code, so it is now different from all the other engines who still use the old nodchip inference code.
connor_mcmonigle
Posts: 533
Joined: Sun Sep 06, 2020 4:40 am
Full name: Connor McMonigle

Re: Before things become more messy than they already are.

Post by connor_mcmonigle »

Madeleine Birchfield wrote: Fri Jul 02, 2021 10:43 pm
connor_mcmonigle wrote: Fri Jul 02, 2021 10:37 pm such as RubiChess which copies on Stockfish inference code
Not anymore: the Stockfish team rewrote its own nnue inference code, so it is now different from all the other engines who still use the old nodchip inference code.
I'm aware. Maybe I could have written "copies Stockfish's old inference code", but I don't think it's relevant to the discussion
User avatar
hgm
Posts: 27829
Joined: Fri Mar 10, 2006 10:06 am
Location: Amsterdam
Full name: H G Muller

Re: Before things become more messy than they already are.

Post by hgm »

Rebel wrote: Fri Jul 02, 2021 9:53 pmWhat you say is self understood but the context of this thread is about fair competition.
Well, deciding what is 'fair competition' is pretty much the prerogative of those organizing that competition. And diversity is usually a good thing. So my preference is that everything would be 'allowed', and that everyone interested in our fantastic hobby can have fun with it, without being pestered by others because they are not doing exactly the same thing as they do. Be it people that are only interested in training networks, writing an engine with HCE from scratch, or marginally improving some existing engine.

There seems to be room for all.
User avatar
Rebel
Posts: 7006
Joined: Thu Aug 18, 2011 12:04 pm
Full name: Ed Schröder

Re: Before things become more messy than they already are.

Post by Rebel »

3 programmers, 3 different votes, poll could be fun.
90% of coding is debugging, the other 10% is writing bugs.
Edsel Apostol
Posts: 803
Joined: Mon Jul 17, 2006 5:53 am
Full name: Edsel Apostol

Re: Before things become more messy than they already are.

Post by Edsel Apostol »

I prefer 2. For those of us who have limited hardware it will take us forever to generate games/positions for our engines to learn.