Copyright and Machine Learning IP

[chrisw]

syzygy wrote: So my point is that the set of weights will be just as (un)copyrighted as the equivalent source or object code. The extra mechanical transformation (the "direct map") makes no difference.

Yes, I agreed with you earlier and switched to:

Chrisw wrote: Agreed. For this particular ingenious machine, the key step is the process "transformation of input game database into a probe-able knowledge engine", it wasn't possible to probe the database for this "knowledge", but it is possible to probe its transformation, or whatever you want to call it.

as the creative and potentially copyright giving step.

Yes, I see now. I skipped a paragraph I shouldn't have skipped.

OK, then I'd say the transformation is still mechanical and irreversible. Since it is mechanical, no copyrightable elements are added. Since it is irreversible, some copyrightable elements might be lost. (In view of section 178, UK copyright law seems to accept that (non-trivial) mechanical transformations can in theory add copyrightable elements, though.)

And I think what normally, if not always, will happen is that everything copyrightable about the initial set of games will be lost.

But I might be able to come up with an exception. The game of chess itself, as a set of game rules, is probably copyrightable (were it not that the rules of chess are in the public domain). Suppose you start with an uninitialised NN for playing a turn-based two-person board game, where the NN does not have any hard-coded game rules because of how the inputs are connected. Now you (creatively) come up with a set of game rules, generate a large amount of games adhering to those game rules, and train the network on those games.

If you play the resulting NN, you will probably be able to recognise some of the game rules (even though the NN might not have acquired the rules with 100% accuracy). So some of the creativity that went into coming up with the game rules will have been retained in the NN weights.

Going back to Lc0, something similar might be possible. Select training games having some silly pattern not connected to the rules of chess and hope you can find back that pattern in the games played by the resulting NN (showing that the NN has some observable creative characteristic created by the person selecting the training games). This does not seem to be easy to achieve, but may not be completely impossible.

You've described a process where a network has learnt "game rules" by observation of those rules in action. But I don't get the bit where the "rule creation creativity" was passed across to the NN. I would get it if you said the "rules" were passed across. Because, now the NN encodes the rules, but not the quality required to create them.

Whether your above example of learn rules, or our field interest, the example of learn chess play knowledge, we are considering the process:

apply position inputs to NN(i) and feedforward to output
backproject the output error game result delta through the connection weights to give NN(i+1)
repeat with another position etc.

NN(0) is random and contains no useful chess knowledge (presumably).
NN(i) contains probe-able unique chess knowledge. Unique because size of net and floating point weights.
NN(i+1) contains increased (on balance), but still unique chess knowledge.
The tiny increase in knowledge of the network was extracted/stored by mechanical/functional process.
We have absolutely no idea at all what the nature of this increased knowledge is. Presumably it is a tiny intangible piece of a subset of everything all at once. It's also unique knowledge. It will exist once and then never exist by itself again.
Maybe it is of consequence that the "thing(s)" learnt/knowledge extracted are not constant/fixed for any one position. What is learnt/knowledge extracted also depends on the state of NN(i).

So, we did a series of irreversible unique knowledge transfers, where the knowledge transferred is some function of a) the chess position and b) the NN state. The transfer is also transformative. We re-present the knowledge as if in a lookup table. Whilst in the games, the knowledge is smeared everywhere and accessible only to interpretation by a powerful human brain.

Extraction and transformation process, while mechanical, creates each step, unique and unfathomable knowledge, then made fathomable by storage in NN lookup table.

Please award this ingenious machine and its creators some sort of copyright protection in order to encourage their work and work of others. According to earlier reference, some jurisdictions already do.

[syzygy]

You've described a process where a network has learnt "game rules" by observation of those rules in action. But I don't get the bit where the "rule creation creativity" was passed across to the NN. I would get it if you said the "rules" were passed across. Because, now the NN encodes the rules, but not the quality required to create them.

If the rules are creative, and the rules are passed (to some extent) to the NN, then the creativity conceived by the creator of the rules (so the copyright on the rules) is passed to the NN.

Or if you consider this in the other direction, if you find something creative about the NN, the question is who conceived this creative aspect. If a human intellectually conceived it, that human has a copyright on it. If it is attributable only to the AI, then EU law seems to say nobody owns a copyright on that aspect, and UK law seems to say the programmer responsible for the AI owns the copyright. (So under UK law the question arises who is the programmer: the programmer of the NN engine, the programmer of the NN training program, the guy feeding the training program with training data, perhaps even the chess players creating the training games, or all of them? This seems very complicated.)

Whether your above example of learn rules, or our field interest, the example of learn chess play knowledge, we are considering the process:

apply position inputs to NN(i) and feedforward to output
backproject the output error game result delta through the connection weights to give NN(i+1)
repeat with another position etc.

NN(0) is random and contains no useful chess knowledge (presumably).
NN(i) contains probe-able unique chess knowledge. Unique because size of net and floating point weights.
NN(i+1) contains increased (on balance), but still unique chess knowledge.
The tiny increase in knowledge of the network was extracted/stored by mechanical/functional process.
We have absolutely no idea at all what the nature of this increased knowledge is. Presumably it is a tiny intangible piece of a subset of everything all at once. It's also unique knowledge. It will exist once and then never exist by itself again.
Maybe it is of consequence that the "thing(s)" learnt/knowledge extracted are not constant/fixed for any one position. What is learnt/knowledge extracted also depends on the state of NN(i).

I think I can turn your words into this: no human is intellectually responsible for the particular knowledge learned in each iteration.

That would mean that no human can claim to have intellectually conceived the NN or anything that the NN produces. So there is no copyright (under EU law).

(To be clear: there may be examples like the game-rules example. Unlikely in the Lc0 context, but in the Lc0 context I see anyway very little opportunity for copyright for reasons already discussed, whether EU law or UK law.)

Please award this ingenious machine and its creators some sort of copyright protection in order to encourage their work and work of others. According to earlier reference, some jurisdictions already do.

I don't know if there is sufficient justification for awarding the "creators" of the AI (whoever they are in case of an NN-based AI) a copyright on the AI/NN or the material produced by it. But that's something for politicians to decide.

[chrisw]

You've described a process where a network has learnt "game rules" by observation of those rules in action. But I don't get the bit where the "rule creation creativity" was passed across to the NN. I would get it if you said the "rules" were passed across. Because, now the NN encodes the rules, but not the quality required to create them.

If the rules are creative, and the rules are passed (to some extent) to the NN, then the creativity conceived by the creator of the rules (so the copyright on the rules) is passed to the NN.

Or if you consider this in the other direction, if you find something creative about the NN, the question is who conceived this creative aspect. If a human intellectually conceived it, that human has a copyright on it. If it is attributable only to the AI, then EU law seems to say nobody owns a copyright on that aspect, and UK law seems to say the programmer responsible for the AI owns the copyright. (So under UK law the question arises who is the programmer: the programmer of the NN engine, the programmer of the NN training program, the guy feeding the training program with training data, perhaps even the chess players creating the training games, or all of them? This seems very complicated.)

Whether your above example of learn rules, or our field interest, the example of learn chess play knowledge, we are considering the process:

apply position inputs to NN(i) and feedforward to output
backproject the output error game result delta through the connection weights to give NN(i+1)
repeat with another position etc.

NN(0) is random and contains no useful chess knowledge (presumably).
NN(i) contains probe-able unique chess knowledge. Unique because size of net and floating point weights.
NN(i+1) contains increased (on balance), but still unique chess knowledge.
The tiny increase in knowledge of the network was extracted/stored by mechanical/functional process.
We have absolutely no idea at all what the nature of this increased knowledge is. Presumably it is a tiny intangible piece of a subset of everything all at once. It's also unique knowledge. It will exist once and then never exist by itself again.
Maybe it is of consequence that the "thing(s)" learnt/knowledge extracted are not constant/fixed for any one position. What is learnt/knowledge extracted also depends on the state of NN(i).

I think I can turn your words into this: no human is intellectually responsible for the particular knowledge learned in each iteration.

That would mean that no human can claim to have intellectually conceived the NN or anything that the NN produces. So there is no copyright (under EU law).

(To be clear: there may be examples like the game-rules example. Unlikely in the Lc0 context, but in the Lc0 context I see anyway very little opportunity for copyright for reasons already discussed, whether EU law or UK law.)

Please award this ingenious machine and its creators some sort of copyright protection in order to encourage their work and work of others. According to earlier reference, some jurisdictions already do.

I don't know if there is sufficient justification for awarding the "creators" of the AI (whoever they are in case of an NN-based AI) a copyright on the AI/NN or the material produced by it. But that's something for politicians to decide.

Good. No copyright. My politics would be much happier if robot created "work" was public and copyright free. Less power for the robot-owning classes. In the case of LC0 NNs though, it is worrying that an enduser can create new weights, claim he trained on some kind of creatively selected set and he would then claim to have weights copyright even though a) is pretty much unproveable and b) the enduser "work" was small beans in comparison.