Stockfish: Our lawsuit against ChessBase

[LittleNasia]

Stockfish criteria to test progresses are often criticized. Eg: in the first phase, 1 second to verify a potential improvement is ridiculous. Strategic understanding of some positions is still very weak for all top programs. Playing against the previous version gives only a partial view. A GM could judge much better an engine, playing more games @40/2h. The contribute of the largest panel of engines is very important, also if they are only a simple fine tuning of the same source code.

But I really don't like to interact with you... although I still love Ethereal and I really cannot ignore it

Kind regards, Alex

I don't really think Stockfish way of testing is in any way bad. The very fact that it works, and that other methods do not work as well is enough proof, but you can also look at it that way:

- Stockfish patches are really tiny, changing some tuning constants and testing that constant alone, some new network that is stronger by 3 Elo (or weaker by 0.5 elo), changing some tiny detail in extensions etc. These changes will produce no meaningful changes in most of the positions. It might flip two best moves in another position (assuming same number of nodes searched), however both moves will still be very good. In some cases, the new best move will actually be worse. However, since there's no all-knowing entity in chess, we simply don't know if the new best move is better or worse. A GM sitting for hours on Stockfish trying to spot the 0.9 elo improvement will just waste their hours, just to find out that the engine just gives the same best moves, or that all moves are still good. Adding the fact that using different hash size can change the moves as well, that threads introduce some randomness that changes the moves as well, there's just no way for anyone to spot a difference.

- for eval patches, any positive change will make it play better positionally in general. It's just giving the score to a position at the end of PV, no matter how long it is. For search patches you would obviously want the engine to search as much as possible. However, it takes milliseconds to get to depth 9/10. At that point, you will still have cut way more nodes than not, and so there is already a "large sample size" of nodes pruned/not pruned and extensions done/not done to test the search changes. In some positions it might not make a difference. In some positions you might find that the patch is worse at depth 40. But more often than not, testing many many different positions just turns out to be better than doing depth 40 searches. If we had a way to determine whether a position might be "problematic", a way to detect potential positions that might become worse at depth 40, while leaving the rest out at the low depths from fast play, then we could search those very few positions to depth 40, while leaving the rest as is to make the tests give better results. However, I can't even imagine something like this being possible, and I would argue that it's impossible.

[AlexChess]

Stockfish criteria to test progresses are often criticized. Eg: in the first phase, 1 second to verify a potential improvement is ridiculous. Strategic understanding of some positions is still very weak for all top programs. Playing against the previous version gives only a partial view. A GM could judge much better an engine, playing more games @40/2h. The contribute of the largest panel of engines is very important, also if they are only a simple fine tuning of the same source code.

But I really don't like to interact with you... although I still love Ethereal and I really cannot ignore it

Kind regards, Alex

I don't really think Stockfish way of testing is in any way bad. The very fact that it works, and that other methods do not work as well is enough proof, but you can also look at it that way:

- Stockfish patches are really tiny, changing some tuning constants and testing that constant alone, some new network that is stronger by 3 Elo (or weaker by 0.5 elo), changing some tiny detail in extensions etc. These changes will produce no meaningful changes in most of the positions. It might flip two best moves in another position (assuming same number of nodes searched), however both moves will still be very good. In some cases, the new best move will actually be worse. However, since there's no all-knowing entity in chess, we simply don't know if the new best move is better or worse. A GM sitting for hours on Stockfish trying to spot the 0.9 elo improvement will just waste their hours, just to find out that the engine just gives the same best moves, or that all moves are still good. Adding the fact that using different hash size can change the moves as well, that threads introduce some randomness that changes the moves as well, there's just no way for anyone to spot a difference.

- for eval patches, any positive change will make it play better positionally in general. It's just giving the score to a position at the end of PV, no matter how long it is. For search patches you would obviously want the engine to search as much as possible. However, it takes milliseconds to get to depth 9/10. At that point, you will still have cut way more nodes than not, and so there is already a "large sample size" of nodes pruned/not pruned and extensions done/not done to test the search changes. In some positions it might not make a difference. In some positions you might find that the patch is worse at depth 40. But more often than not, testing many many different positions just turns out to be better than doing depth 40 searches. If we had a way to determine whether a position might be "problematic", a way to detect potential positions that might become worse at depth 40, while leaving the rest out at the low depths from fast play, then we could search those very few positions to depth 40, while leaving the rest as is to make the tests give better results. However, I can't even imagine something like this being possible, and I would argue that it's impossible.

Thank you very much for the detailed and clear explanation

[syzygy]

Next you'll argue Houdini source code is available and therefore not a GPL violation. "Just ask for source on some Russian forum"

It's how the Houdini source was obtained that bothers me.

Evidence gathered by guessing passwords isn't illegal in Germany.
Let's say you did something bad and the evidence is in your car then there are 2 possibilities:
a) a person broke in -> this is illegal and the evidence won't be used in court (unless the police gets a permission from a judge or you save a life by breaking into the car)
b) you left your roof and doors open and it was clearly visible -> the evidence will be used in court

Getting in by guessing passwords is clearly the electronic version of breaking in.

[syzygy]

As far as I am aware almost all countries in the EU. Making laws based on Religion can be dangerous. The reason why guessing passwords is ok is that even though somebody technically did something unethical one is still partly responsible for ones privacy and safety. A recycled password is basically an open door which was again demonstrated.

A bad password is still a password and therefore not an open door.

No insurance will pay in case of a theft if you left your door open. Nothing was broken and nobody was harmed and something incriminating was found by accident so logically there is no reason to not permit it as evidence.

Do you realise that you yourself are calling it theft?
Where in the world is theft legal? Where in the world is theft not harmful? Where in the world is theft "by accident"?

That the insurance of the car owner may or may not pay is completely irrelevant and does not permit you to commit theft. You are not allowed to steal, whether or not the item was behind a lock and whether or not the lock was weak or strong.

What you are writing here is devoid of consistency and coherence.

[syzygy]

Would be nice if you stopped putting things in my mouth I never said. All I said was that I think it's likely that the leaked source code will be permitted as evidence. I never said cucumber didn't commit a crime.

You said there was no hacking, and it seems to me the only possible crime committed by cucumber is hacking. So it is difficult to see how you did not say that cucumber didn't commit a crime.

[syzygy]

It's just saddens me when lawsuits or other types of divisive action occur.

Just curious, if CB loses, will you remove FF from the CCRL list?

ChessBase already conceded selling Houdini, and Houdini is still there. So the answer is no I suspect.

At the very least Houdini 5/6 should have a big fat separate tag that signifies it's an illegal clone (explicitly) and that if it's still on the list, it's only for legacy reasons.

Just curious, who defines that is an illegal clone? Which authoroty? In what kind of process? Did you take Houdart to court? Did Team SF come up with a clone-planel?

--
Srdja

Very lazy way to avoid any real argument. Devoid of intellectual merit. Who defines anything? Who says what you are experiencing is not only in your own mind? Who says the air you breath is real?

You did not answer my question. The ICGA had at least a panel and a process.

--
Srdja

Clone is the wrong term. They should have said GPLv3 violation, which in this case boils down to the question whether H6 infringes SF's copyright. And this would then be a matter for the court to decide on the basis of the evidence. (Of course -- assuming H6 infringed SF's copyright -- there is still another question, namely whether ChessBase's GPLv3 violation really "counts" as one given that the company most likely was not aware that H6 infringed SF's copyright. For example, if you bought H6 and then sold your copy to a friend, you unknowingly committed a GPLv3 violation yourself. Can the SF copyright holders now prevent you from downloading a copy of SF by revoking your rights under the license? Probably not. Same applies to CB, I'd say.)

[syzygy]

I do not care if FF is actually legal or not, all i know is what they did is not allowed by common chess community standards and they need to face the consquences for that.

That's what gets me about all these discussions of legality in public forums. Even if everything was explicitly and unambiguously legal the displayed behavior still deserves "judgement". Every developer should be insulted by this kind of rebranding and representation independent of their involvement. The tunnel vision on legality frightens me since it hints at the seemingly prevalent idea that legal=acceptable and exploitation of anything or anyone is ok as long as no one has thought to formally make it illegal.

If you sue someone in court (title of this thread: "Stockfish: Our lawsuit against ChessBase"), you better stick to the "tunnel vision on legality".

[syzygy]

- FF2 runs on a slightly modified version of SF. This can be distributed in binary form according to the GPLv3 as long as you provide the source. See github,
- Stockfish is credited prominently in the marketing copy of FF2. (I suspect they did that because Stockfish is a selling point, but that’s just a guess.) It’s on the box.
- FF2 was trained on data generated using FF1. Not hard to believe. I generated a ton of data using Bad Gyal, Toga, Ice, etc., though not nearly as much as Albert.

So there you go. Common chess community standards(tm) adhered to. You are now free to enjoy FF2 without guilt. You’re welcome.

I suppose there still are the issues of:
1) Whether a NN is a computer program, and the weights are the corresponding machine code.
2) Whether this program counts as linked with a conventional executable when the latter uses the output of the NN it interprets for further calculation
3) What would count as the 'source code' of the NN in the GPL sense.

It will be interesting what the court decides on this.

I can imagine an outcome where (1) and (2) are decided as 'yes, it does', and (3) would be the set of training positions.

In my view the (dynamically loaded) NN is simply a parameter file with an uncopyrightable structure, and the particular choice of values is even less copyrightable.

CB did not violate SF's copyright by not releasing the FF2 net under the GPLv3.
At the same time, there is no copyright on FF2, so anyone can do with it what they want.

[syzygy]

I already posted this earlier in the thread, and thats why I said "goal posts moved" when you dismissed it. I suppose you just did not read it:

Dem Titel Fat Fritz 2.0 (DVD) liegen leider nicht die vollständigen GPL3-Lizenzbedingungen bei. Aus diesem Grund sind wir gezwungen, den Titel zurückzurufen und den Verkauf der DVD-Version zu stoppen. Bitte beenden Sie den Verkauf der DVD-Version am Mittwoch, den 31. März 2021. Fat Fritz 2.0 (DVD) darf ab dem 31. März 2021 nicht weiter verkauft werden.

So what they admitted is that they forgot to include a copy of GPLv3's Copying.txt on the DVD.

[syzygy]

In case you didn’t notice: https://stockfishchess.org/blog/2021/ou ... chessbase/

Even though we had our first successes, leading to a recall of the Fat Fritz 2 DVD and the termination of the sales of Houdini 6, we were unable to finalize our dispute out of court.

Can the SF team, or the "team of maintainers and developers of Stockfish", elaborate on what this means?

The Stockfish maintainers have (to my knowledge) never posted informations about the lawsuit in talkchess and will probably be wise and serious enough to never do that in the future.

Whether or not you have the law on your side does not depend on whether or not you lay open your cards on a public forum.