Re: Copyright and Machine Learning IP
Posted: Wed Aug 15, 2018 9:38 pm
You've described a process where a network has learnt "game rules" by observation of those rules in action. But I don't get the bit where the "rule creation creativity" was passed across to the NN. I would get it if you said the "rules" were passed across. Because, now the NN encodes the rules, but not the quality required to create them.syzygy wrote: ↑Wed Aug 15, 2018 7:57 pmYes, I see now. I skipped a paragraph I shouldn't have skipped.chrisw wrote: ↑Wed Aug 15, 2018 7:19 pmYes, I agreed with you earlier and switched to:syzygy wrote: So my point is that the set of weights will be just as (un)copyrighted as the equivalent source or object code. The extra mechanical transformation (the "direct map") makes no difference.
as the creative and potentially copyright giving step.Chrisw wrote: Agreed. For this particular ingenious machine, the key step is the process "transformation of input game database into a probe-able knowledge engine", it wasn't possible to probe the database for this "knowledge", but it is possible to probe its transformation, or whatever you want to call it.
OK, then I'd say the transformation is still mechanical and irreversible. Since it is mechanical, no copyrightable elements are added. Since it is irreversible, some copyrightable elements might be lost. (In view of section 178, UK copyright law seems to accept that (non-trivial) mechanical transformations can in theory add copyrightable elements, though.)
And I think what normally, if not always, will happen is that everything copyrightable about the initial set of games will be lost.
But I might be able to come up with an exception. The game of chess itself, as a set of game rules, is probably copyrightable (were it not that the rules of chess are in the public domain). Suppose you start with an uninitialised NN for playing a turn-based two-person board game, where the NN does not have any hard-coded game rules because of how the inputs are connected. Now you (creatively) come up with a set of game rules, generate a large amount of games adhering to those game rules, and train the network on those games.
If you play the resulting NN, you will probably be able to recognise some of the game rules (even though the NN might not have acquired the rules with 100% accuracy). So some of the creativity that went into coming up with the game rules will have been retained in the NN weights.
Going back to Lc0, something similar might be possible. Select training games having some silly pattern not connected to the rules of chess and hope you can find back that pattern in the games played by the resulting NN (showing that the NN has some observable creative characteristic created by the person selecting the training games). This does not seem to be easy to achieve, but may not be completely impossible.
Whether your above example of learn rules, or our field interest, the example of learn chess play knowledge, we are considering the process:
apply position inputs to NN(i) and feedforward to output
backproject the output error game result delta through the connection weights to give NN(i+1)
repeat with another position etc.
NN(0) is random and contains no useful chess knowledge (presumably).
NN(i) contains probe-able unique chess knowledge. Unique because size of net and floating point weights.
NN(i+1) contains increased (on balance), but still unique chess knowledge.
The tiny increase in knowledge of the network was extracted/stored by mechanical/functional process.
We have absolutely no idea at all what the nature of this increased knowledge is. Presumably it is a tiny intangible piece of a subset of everything all at once. It's also unique knowledge. It will exist once and then never exist by itself again.
Maybe it is of consequence that the "thing(s)" learnt/knowledge extracted are not constant/fixed for any one position. What is learnt/knowledge extracted also depends on the state of NN(i).
So, we did a series of irreversible unique knowledge transfers, where the knowledge transferred is some function of a) the chess position and b) the NN state. The transfer is also transformative. We re-present the knowledge as if in a lookup table. Whilst in the games, the knowledge is smeared everywhere and accessible only to interpretation by a powerful human brain.
Extraction and transformation process, while mechanical, creates each step, unique and unfathomable knowledge, then made fathomable by storage in NN lookup table.
Please award this ingenious machine and its creators some sort of copyright protection in order to encourage their work and work of others. According to earlier reference, some jurisdictions already do.