With the current SF14 subject evolving I would like to do another programmer poll as I have done in the past, see for instance:
http://www.top-5000.nl/appeal.htm
This time about the use of networks which maybe can lead to a new programmer code, see:
http://rebel13.nl/download/programmer-c ... honor.html
Before creating the poll page I like to propose the following poll questions open for discussion and improvement.
1. My preference is that I am allowed to use any existing network for my engine.
2. My preference is that I am allowed to use any game collection to create my own network.
3. My preference is that I am in favor that any programmer creates his own network either from human game collections or from his own engine.
Shoot !
Before things become more messy than they already are.
Moderators: hgm, Rebel, chrisw
-
- Posts: 7006
- Joined: Thu Aug 18, 2011 12:04 pm
- Full name: Ed Schröder
Before things become more messy than they already are.
90% of coding is debugging, the other 10% is writing bugs.
-
- Posts: 512
- Joined: Tue Sep 29, 2020 4:29 pm
- Location: Dublin, Ireland
- Full name: Madeleine Birchfield
Re: Before things become more messy than they already are.
I would vote for the second option
The firat choice is akin to taking an evaluation function directly from another engine, which most would agree would not count as unique, while the third choice is akin to forcing engines to tune using only their own engine, whereas many engines have used CCRL data or the Zurichess game collections in tuning in the past.
For this reason engines like Stockfish 14 and Nemorino 6.00 are fine, while engines like Mayham, Fire, and BBC 1.4 are too derivative. People should learn to train or tune their own networks, just as they would learn to tune their own handcrafted evaluations.
as this is already the norm with tuning handcrafted evaluation functions such as Pesto.2. My preference is that I am allowed to use any game collection to create my own network.
The firat choice is akin to taking an evaluation function directly from another engine, which most would agree would not count as unique, while the third choice is akin to forcing engines to tune using only their own engine, whereas many engines have used CCRL data or the Zurichess game collections in tuning in the past.
For this reason engines like Stockfish 14 and Nemorino 6.00 are fine, while engines like Mayham, Fire, and BBC 1.4 are too derivative. People should learn to train or tune their own networks, just as they would learn to tune their own handcrafted evaluations.
-
- Posts: 27829
- Joined: Fri Mar 10, 2006 10:06 am
- Location: Amsterdam
- Full name: H G Muller
Re: Before things become more messy than they already are.
Whatever anyone's preference, it should be clear that programmers are allowed to do whatever they want, as long as they don't violate copyright licences. And that as long as they honestly specify what they have been doing, there is never anything unethical about that.
-
- Posts: 7006
- Joined: Thu Aug 18, 2011 12:04 pm
- Full name: Ed Schröder
Re: Before things become more messy than they already are.
What you say is self understood but the context of this thread is about fair competition.hgm wrote: ↑Fri Jul 02, 2021 9:24 pm Whatever anyone's preference, it should be clear that programmers are allowed to do whatever they want, as long as they don't violate copyright licences. And that as long as they honestly specify what they have been doing, there is never anything unethical about that.
Personally I agree whit what Madeleine said, [2] is not perfect but [3] limits the freedom of programmers too much, I will go for [2].
If there are no further comments on the 3 questions I will create a programmer and user poll tomorrow.
90% of coding is debugging, the other 10% is writing bugs.
-
- Posts: 533
- Joined: Sun Sep 06, 2020 4:40 am
- Full name: Connor McMonigle
Re: Before things become more messy than they already are.
Personally, it would be neat if option 3 was the norm, but obviously everyone is free to do what they enjoy in computer chess provided they don't do anything illegal or are otherwise dishonest about the origins of their engine.
I believe there's somewhat of a dearth of technical information in a digestible format about top engines in general which limits the capacity of rating lists and tournaments to make informed decisions in selecting engines deemed "sufficiently original/different". This leads to silly situations such as testers refusing to test Stockfish 14 as it was trained on Lc0 self play data. I think it would be far more productive to have some sort of space (with some moderation to enforce honesty) for authors to self report technical details about their engines.
It's obviously impossible to develop an exact set of criterion for what constitutes an original work, but I do think it's possible to at least maintain some standards (provided sufficient information is available). It's always going to be subjective and I don't think the above 3 options comes close to capturing the complexity of the situation. For example, if the author an engine, such as RubiChess which copies Stockfish inference code, training code and network topology, decided to train on Stockfish data, their engine would be far less "original" than an engine implementing a meaningfully different network topology, training code, inference code, etc. You're never going to please everyone, but throwing out standards and testing anything and everything is untenable.
I believe there's somewhat of a dearth of technical information in a digestible format about top engines in general which limits the capacity of rating lists and tournaments to make informed decisions in selecting engines deemed "sufficiently original/different". This leads to silly situations such as testers refusing to test Stockfish 14 as it was trained on Lc0 self play data. I think it would be far more productive to have some sort of space (with some moderation to enforce honesty) for authors to self report technical details about their engines.
It's obviously impossible to develop an exact set of criterion for what constitutes an original work, but I do think it's possible to at least maintain some standards (provided sufficient information is available). It's always going to be subjective and I don't think the above 3 options comes close to capturing the complexity of the situation. For example, if the author an engine, such as RubiChess which copies Stockfish inference code, training code and network topology, decided to train on Stockfish data, their engine would be far less "original" than an engine implementing a meaningfully different network topology, training code, inference code, etc. You're never going to please everyone, but throwing out standards and testing anything and everything is untenable.
Last edited by connor_mcmonigle on Fri Jul 02, 2021 10:50 pm, edited 1 time in total.
-
- Posts: 512
- Joined: Tue Sep 29, 2020 4:29 pm
- Location: Dublin, Ireland
- Full name: Madeleine Birchfield
Re: Before things become more messy than they already are.
Not anymore: the Stockfish team rewrote its own nnue inference code, so it is now different from all the other engines who still use the old nodchip inference code.connor_mcmonigle wrote: ↑Fri Jul 02, 2021 10:37 pm such as RubiChess which copies on Stockfish inference code
-
- Posts: 533
- Joined: Sun Sep 06, 2020 4:40 am
- Full name: Connor McMonigle
Re: Before things become more messy than they already are.
I'm aware. Maybe I could have written "copies Stockfish's old inference code", but I don't think it's relevant to the discussionMadeleine Birchfield wrote: ↑Fri Jul 02, 2021 10:43 pmNot anymore: the Stockfish team rewrote its own nnue inference code, so it is now different from all the other engines who still use the old nodchip inference code.connor_mcmonigle wrote: ↑Fri Jul 02, 2021 10:37 pm such as RubiChess which copies on Stockfish inference code
-
- Posts: 27829
- Joined: Fri Mar 10, 2006 10:06 am
- Location: Amsterdam
- Full name: H G Muller
Re: Before things become more messy than they already are.
Well, deciding what is 'fair competition' is pretty much the prerogative of those organizing that competition. And diversity is usually a good thing. So my preference is that everything would be 'allowed', and that everyone interested in our fantastic hobby can have fun with it, without being pestered by others because they are not doing exactly the same thing as they do. Be it people that are only interested in training networks, writing an engine with HCE from scratch, or marginally improving some existing engine.
There seems to be room for all.
-
- Posts: 7006
- Joined: Thu Aug 18, 2011 12:04 pm
- Full name: Ed Schröder
Re: Before things become more messy than they already are.
3 programmers, 3 different votes, poll could be fun.
90% of coding is debugging, the other 10% is writing bugs.
-
- Posts: 803
- Joined: Mon Jul 17, 2006 5:53 am
- Full name: Edsel Apostol
Re: Before things become more messy than they already are.
I prefer 2. For those of us who have limited hardware it will take us forever to generate games/positions for our engines to learn.
Edsel Apostol
https://github.com/ed-apostol/InvictusChess
https://github.com/ed-apostol/InvictusChess