rkiss and other dependencies in syzygy

[Don]

I'm looking at implementing syzygy in Komodo but I see all sorts of include files that do not correspond to any known libraries.

Some examples:

#include "position.h"
#include "movegen.h"
#include "rkiss.h"
#include "bitboard.h"
#include "search.h"

This is not in the source code the readme file says comes from Stockfish, this is actually part of Ronald's probing code.

But these files are not included from the git site. Where do I get them? What else is missing that will make this not work?

[Michel]

These are header files from stockfish.

It is an interesting fact that they fall under the GPL:-)

[Don]

These are header files from stockfish.

It is an interesting fact that they fall under the GPL:-)

From Stockfish????

I'm not including Stockfish code in Komodo no matter what. This is just crazy and not what I was told about being able to include syzygy in a commercial product. Does this mean Houdart is violating GPL or is his implementation only private?

[syzygy]

These are header files from stockfish.

It is an interesting fact that they fall under the GPL:-)

From Stockfish????

I'm not including Stockfish code in Komodo no matter what. This is just crazy and not what I was told about being able to include syzygy in a commercial product. Does this mean Houdart is violating GPL or is his implementation only private?

I explained this to you just a few days ago:

There should not be any licensing problems for commercial programs.

The generator is GPL, but the probing code is free of any restrictions, so do with it what you want.
See https://github.com/syzygy1/tb under "Terms of use":
"The files tbcore.c, tbcore.h, tbprobe.cpp and tbprobe.h in interface/ may be freely modified and redistributed in source and/or binary format."
And there are similar notices at the top of these files.

Some lines in tbprobe.cpp invoke functions of Stockfish, but that does not mean that tbprobe.cpp as a text file is covered by Stockfish's GPL. And you'll have to adapt those lines to using Komodo functions anyway.

The files in interface/ are somewhat outdated compared to the current Stockfish sources, but that should not matter as you're going to remove that dependency anyway.

And from https://github.com/syzygy1/tb:

The directory interface/ contains probing code. It does not come in the form of a shared library, and requires some work to integrate into an engine. The main reason for this is efficiency. There are four files: tbcore.c, tbcore.h, tbprobe.cpp, tbprobe.h.

The files tbcore.c and tbcore.h should not require much changes, although engine authors might want to replace some printf()s with suitable logging statements. The files tbprobe.cpp and tbprobe.h do require some changes but these should be fairly straightforward when following the comments. The only reason for tbprobe.cpp having the .cpp extension is that I have used Stockfish as example. The probing code expects WDL files in $RTBWDIR and DTZ files in $RTBZDIR.

The files main.cpp, search.cpp and types.h are from Stockfish with calls to the probing code added (see // TB comments). The change in types.h is necessary in order to make room for "tablebase win in n" values distinct from "mate in n" values. Please note that the integration of probing code into Stockfish is merely intended as a proof of concept. It is far from perfect and might have some bugs.

The point is: my probing code needs a move generator and it does not make sense to include a separate move generator when any engine already has one.

I could have provided my original probing code that calls my own engine's move generator, but it seems easier for everybody else if the example probing code uses a move generator that publicly available.

Again: You are not going to use anything from Stockfish. There is nothing from Stockfish in tbcore.c, tbcore.h, tbprobe.cpp, tbprobe.h.

I am a lawyer. I know how copyright works.

[Don]

These are header files from stockfish.

It is an interesting fact that they fall under the GPL:-)

From Stockfish????

I'm not including Stockfish code in Komodo no matter what. This is just crazy and not what I was told about being able to include syzygy in a commercial product. Does this mean Houdart is violating GPL or is his implementation only private?

I explained this to you just a few days ago:

There should not be any licensing problems for commercial programs.

The generator is GPL, but the probing code is free of any restrictions, so do with it what you want.
See https://github.com/syzygy1/tb under "Terms of use":
"The files tbcore.c, tbcore.h, tbprobe.cpp and tbprobe.h in interface/ may be freely modified and redistributed in source and/or binary format."
And there are similar notices at the top of these files.

Some lines in tbprobe.cpp invoke functions of Stockfish, but that does not mean that tbprobe.cpp as a text file is covered by Stockfish's GPL. And you'll have to adapt those lines to using Komodo functions anyway.

The files in interface/ are somewhat outdated compared to the current Stockfish sources, but that should not matter as you're going to remove that dependency anyway.

I was just surprised to see that stockfish code seemed to be required to make this work. You have your own licence on top of code that call GPL code that is incompatible (I cannot put SF code in a commercial engine.)

You said the probing code is free of any restrictions, but you didn't say "as long as it's completely modified to not include the parts that are heavily restricted, namely the stockfish code."

Anyway, I think I get the point now, The probing code is raw and I have a lot of code to rewrite.

Don

[syzygy]

You said the probing code is free of any restrictions, but you didn't say "as long as it's completely modified to not include the parts that are heavily restricted, namely the stockfish code."

The code is just a piece of text. The textual content DOES NOT CONTAIN ANYTHING FROM STOCKFISH. You are free to print it, both completely unmodified and completely modified, on a t-shirt and sell it for profit. I cannot make this any clearer.

What you should not do is copy the move generator from Stockfish into your program.

Anyway, I think I get the point now, The probing code is raw and I have a lot of code to rewrite.

To get it working just follow the comments in tbprobe.cpp.
Adapting my original probing code to use the move generator of Stockfish did not take me much time, and I was not familiar with Stockfish and still had to more cleanly separate my probing code from the rest of my engine.

[syzygy]

Originally I planned to provide just pseudo code for generating moves etc. but surely having real code calling a real code generator gives a much clearer example to work from.

[Don]

You said the probing code is free of any restrictions, but you didn't say "as long as it's completely modified to not include the parts that are heavily restricted, namely the stockfish code."

The code is just a piece of text. The textual content DOES NOT CONTAIN ANYTHING FROM STOCKFISH. You are free to print it on a t-shirt and sell it for profit. I cannot make this any clearer.

What you should not do is copy the move generator from Stockfish into your program.

Anyway, I think I get the point now, The probing code is raw and I have a lot of code to rewrite.

To get it working just follow the comments in tbprobe.cpp.
Adapting my original probing code to use the move generator of Stockfish did not take me much time, and I was not familiar with Stockfish and still had to more cleanly separate my probing code from the rest of my engine.

I guess I'm just annoyed that I have to figure out what rkiss and all the other stuff is that the code references and I have to go to Stockfish. I don't have anything like rkiss in Komodo - at least I don't think I do. Then I look and see this:

static RKISS rk;

And not a single reference to it in your code. So I basically have to take the whole thing apart and learn stockfish just to implement your databases. It's odd that you have to do stockfish initializations from your code so it tells me that it's going to be a kind of dependency hell and a lot of work.

Anyway, forgive me for being so annoyed here, I'm sure it's worth the aggravation - I just didn't expect this.

Don

[Don]

Originally I planned to provide just pseudo code for generating moves etc. but surely having real code calling a real code generator gives a much clearer example to work from.

If I get any ideas I might suggest them or even contribute a patch that you can accept or reject. Sometimes another implementer can see things you might not necessarily see. It seems that you could, for example, provide some prototypes for functions that the implementer fills in using the most efficient method for his program.

Don

[Don]

Originally I planned to provide just pseudo code for generating moves etc. but surely having real code calling a real code generator gives a much clearer example to work from.

Just so that I don't spin my wheels, which version of Stockfish do I have to download to understand this code? Is the latest official release good enough or do I have to get one of the new development versions?