One caveat though, some programmers say that the code from Eugene can not really be licensed under the GPL because it still remains privately owned code. That is a reason why for instance Tord did not want to implement EGTBs at least not Nalimov's code in Glaurung. Maybe you will have to consider another form of licensing? Crafty uses Nalimov code but a different license. I would not really know the pros and cons. I don't want to do difficult or anything but it's probably better to look at that now than later? I do hope you can go through with Protector!
I now read some messages in the thread again and realize Ryan Benitez already posted this above so you will already know about this.
Regards,
Eelco
Debugging is twice as hard as writing the code in the first
place. Therefore, if you write the code as cleverly as possible, you
are, by definition, not smart enough to debug it.
-- Brian W. Kernighan
A lot has been said about Robbolito, however not much has been said about the Robbo tablebases. Robbolito comes with a program called RobboBuild, an endgame tablebase generator that produces 3, 4, and 5 piece tablebases taking up just 3.5gb, it also generates 6 piece tables, and the licence is unrestricted. from the readme:-
For the building of your 2s and 3s:
$ mkdir RobboTotalBase
$ mkdir RobboTotalBase/3
$ ./RobboBuild --mem --recurse P
And then 4s and 5s
$ mkdir RobboTotalBase/4
$ ./RobboBuild --recurse --mem PP Pp
$ mkdir RobboTotalBase/5
$ ./RobboBuild --recurse --mem PPP PPp
This last can take about 10-20 hours, and fit in 1GB of memory in the build.
The ending disk usage will be around 3.5GB.
So too the 6s, but not for 4-tuples and their inductions can be built.
If the --mem option is not present, the build goes by disk. The dated
tests have not assumed pawns with 6s, so be warned!
These will take many hours. If you have enough, the --mem option will help.
Adding compression to disk builds was too hairy with meeting and beating
the Revolution deadline, but should see the future.
* Compression
RobboTotalBase uses by design 64K blocks of RLE+BWT+MTF+Huffman-compressed DTC
data, with wins just having one bit. This compares to other schemes in the end.
Full Size RobboSize Index Total PerCent
3s: 410382 37176+ 272 37448 9.1%
4s: 157440864 15885237+ 10780 15935813 10.1%
5s: 37429189056 3659510583+2289088 3661799671 9.8% (3.41GB)
The Nalimov takes 7.2GB for 5s, but they are DTM. For DTC, they are still
4.5GB, largely for the RobboEdge in its one-sided nature. The FEG are 5.6GB
with DTM, and should compare the RobboScheme if outfitted with DTC.
For 6s, a superstructural indexing scheme can be added for the future, and with
some attempt for a compatability to the current to exist.
* RobboTripleBases
These are other compressed bases, that yield only result information, but
can be accessed faster. Similar are ShredderBases and ScorpioBases. You
might propose a comparison, but I cannot avail.
The TripleBases are loaded at startup, just once. Their space is 3.5MB for 4s,
and 645MB for 5s. This last can be reduced to 500MB by a switch that computes
the TripleBase differently, but has slower access. There is no capacity to
use a TripleBase for 6s, until a later release.
On disk storage, compressing the TripleBases finds its form, but more brains
are needed to extend the method.
The building of a TripleBase depends on the TotalBase, and can also recurse.
But a warning, the piece order is important, and the superior choices I found
are in with included scripts. Build time wanders around 2-3 hours for your 5s.
$ mkdir RobboTripleBase
$ mkdir RobboTripleBase/2
$ mkdir RobboTripleBase/3
$ mkdir RobboTripleBase/4
$ mkdir RobboTripleBase/5
$ sh triple3.sh
$ sh triple4.sh
$ sh triple5.sh
* In Use
RobboLito on startup will load all found TripleBases, and access indices
for the TotalBases. Then the TripleBase gets used in search, until a root
position for TotalBases is attained, when a switch occurs. Enjoy!
Eelco de Groot wrote:Glad you got the permission Raimund!
One caveat though, some programmers say that the code from Eugene can not really be licensed under the GPL that is a reason why for instance Tord did not want to implement EGTBs at least not Nalimov's code in Glaurung. Maybe you will have to consider another form of licensing? Crafty uses Nalimov code but a different license. I would not really know the pros and cons. I don't want to do difficult or anything but it's probably better to look at that now than later? I do hope you can go through with Protector!
Regards,
Eelco
Thanks, Eelco.
Meanwhile I'm of course aware of these pros and cons. The topic has become rather complex and I will try to discuss things with Andrew Kadatch and Eugene Nalimov. I will have to consider:
- Not GPL could mean I cannot host the sources on Sourceforge anymore. Not funny.
- Playing without tablebases seems to hurt Protector's rating severly, although I've added some eval terms for very basic endings (KNB vs K seems to work at 5 sec per move). But I still don't have sufficient terms for e.g. KP vs. KP without passers. So adding the tablebase code is still the most favourable option.
And last but not least the permission from Eugene Nalimov itself is still missing. But after all there's some progress in the case now...
RaimundHeid wrote:adding the tablebase code is still the most favourable option.
And last but not least the permission from Eugene Nalimov itself is still missing. But after all there's some progress in the case now...
Good to hear you have that permission, I am sure that Nalimov will give his as well.
BTW, I can't see why each and every author has to apply for a licence. Will they refuse to give their permission to some of them? Thereby they would put those at handicap, is that fair? And what if the tablebase authors become permanently unreachable?
There must be some reason why they haven't issued a general licence which does not require individual permission, but I can't guess what.
I suppose by demanding a permission they want to keep things under control. Don't know if this really works. Might work well for them but for the developers it's not very comfortable.
I've sent a mail to Andrew Kadatch to discuss the topic with him. I assume they don't want to publish their code under GPL since that would mean anyone can modify it and publish the modifications (and in the worst case Kadatch/Nalimov will get requests to fix the bugs). I suppose an easy solution would be an address where the official version of the tablebase code is located. But currently this is only speculation. Will keep you updated.
SzG wrote:BTW, I can't see why each and every author has to apply for a licence. Will they refuse to give their permission to some of them? Thereby they would put those at handicap, is that fair?.
I believe Diep has been refused permission. I am therefor of the opinion that Nalimov tablebases should not be allowed in competitive events.
I've given up hope now. Eugene Nalimov doesn't reply in any way. I'm pretty sure my mails have reached him since Andrew Kadatch has forwarded them to him. But I never got any reply at all from him, neither positive nor negative.
So my new goal will be to add some endgame knowlegde. The first results are promising (e.g. KBNK is much easier than I thought and takes only a few lines of code) and I assume that the rating has not dropped more than 10-15 points after removing the egtb access code.
Nonetheless I think it's a pity because I loved these "Mate in 95" announcements in KNNKP and KQPKQ endings too much...
Someone please close the thread, don't wait for updates anymore.
SzG wrote: And what if the tablebase authors become permanently unreachable?
... a fate awaiting all of us
.
I hope this fate finds it's way fast to a certain member here so that we can get rid of his spamming posts
Dr.D
_No one can hit as hard as life.But it ain’t about how hard you can hit.It’s about how hard you can get hit and keep moving forward.How much you can take and keep moving forward….
RaimundHeid wrote:I read that several other freeware authors got the permission to at least *use* (maybe not 'publish') the sources
... Besides, I have never seen such a written permission ...
The permission looks like this:
Andrew Kadatch wrote:
He Raimund,
Please accept my sincere apologies for the delay -- I do not check that account often.
Yes, you can. If the program is commercial I would like to ask you to send me 2 copies.
Eugene is currently not available, he'll be back after Nov. 2nd -- I'll forward your message to him.
Good luck!
-- Andrew
I received this message today. Hope is rising again. Will keep you updated ...
Congratulations Raimund,I'm so happy the situation has cleared up for you
One of the finest chess engines I've ever tested....Protector 1.2.8 x64 using 4CPU is laready in my rating list....I will add this new version as soon as possible....
Dr.D
_No one can hit as hard as life.But it ain’t about how hard you can hit.It’s about how hard you can get hit and keep moving forward.How much you can take and keep moving forward….
because it still remains privately owned code. That is a reason why for instance Tord did not want to implement EGTBs at least not Nalimov's code in Glaurung. Maybe you will have to consider another form of licensing? Crafty uses Nalimov code but a different license. I would not really know the pros and cons. I don't want to do difficult or anything but it's probably better to look at that now than later? I do hope you can go through with Protector!
