syzygy question (for Ronald)

[Daniel Shawul]

I have added a BSD license and removed some clauses that I thought were too restrictive and updated the git repo.
I don't know how restrictive this license is but I would be glad to modify it as required because I don't want to drive anyone away.
----------------
Copyright (c) 2005 - 2013 Daniel Shawul Abdi
All rights reserved.

Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE,
EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

-----------------------

[lucasart]

I'm really considering implementing syzygy TB in DiscoCheck. It has all the nice properties that I like and no other TB has:
* 5-men WDL holds in 378 MB only!
* probing code is: open source, concise, portable, non intrusive, and fast!

Meh...scorpio bitbases 'has all the nice properties that you like'.
* 5-men WDL holds in 200 MB only!
* probing code is: open source, concise, portable, non intrusive, and fast!
Something else I don't get is, this 'philosophy of not using bitbases' which is utter BS if you ask me. It is more like I don't want to be associated with someone elses code if it don't get much elo for me (though chess community wants your engine to have it!). I don't envy if someone's TBs become popular but you don't have to go and unfairly criticize other TBs that has been serving good in the past.
Peace!

Sorry, I didn't realize that Scorpio's EGBB were open source. I thought the probing code came in binary form only, as a Windows DLL. 200 MB is very compact indeed: is that bitbase information comparable to a WDL table (result of the game, accounting for 50-move rule) ?

[syzygy]

Sorry, I didn't realize that Scorpio's EGBB were open source. I thought the probing code came in binary form only, as a Windows DLL. 200 MB is very compact indeed: is that bitbase information comparable to a WDL table (result of the game, accounting for 50-move rule) ?

As far as I know, those 200 MB represent the "smaller half" of each table, which is about a third of both sides. The disadvantage is that probing a position for the wrong side to move translates into 1 ply of probes for the right side.

My DTZ tables also store the "smaller half" of each table, which is not a problem because the DTZ tables are only probed at the root.

[lucasart]

Is there a non trivial gain (elo-wise) in the root_probe() stuff ?

No one to my knowledge has ever shown a non-trivial gain from any tablebases with an engine with decent endgame evaluation. We need a good implementation of tablebases to do proper testing, but the multitude of different Stockfish versions is inhibiting this. It can be done, of course, but it takes some will power.

I used to think that too. But recently, with the implementation of syzygy tb in stockfish, it has been shown that there is a measurable gain. Of course, it all depends on what you consider to be a non trivial gain. Jean Francois Romang measured +8 elo in 60"+0.05" (single threaded).

Yet Stockfish has an enormous amount of endgame specific knowledge. But all the endgame knowledge of Stockfish is in the eval. It mostly attempts to recognize draws or drawish positions, and sometimes helps to win some tricky ones (eg. KBNK).

I think that most of the gain lies in TB probing, which allows a lot of 100% accurate pruning in the search, when there are few pieces left on the board. With 6-men TB, and assuming the cost of probing is negligible, it makes sense that there is a measurable ELO improvement.

If you look at Ronald's implementation in Stockfish, you'll see that the cost of probing is not big at all. At the root node, he sets a variable

Code: Select all

use_tb = nb of pieces if <= 6, 0 otherwise

and TB probing looks like

Code: Select all

if &#40;nb of pieces <= use_tb&#41; &#123;...&#125;

That way the cost in the general case is negligible enough for the tradeoff to be positive.

Also, using a TB makes late endgames look cleaner. Often it doesn't change the outcome of the game, but it improves the manner in which this outcome is achieved. That's why users like it, even though the elo gain is tiny.

[Michel]

I have added a BSD license and removed some clauses that I thought were too restrictive and updated the git repo.
I don't know how restrictive this license is but I would be glad to modify it as required because I don't want to drive anyone away.

Ok great. Hopefully they will now be reconsidered for Debian. Gaviota TB's
have recently been imported into Debian.

[jshriver]

What is the address for the scorpio githuh? Ive been hosting a set for years but people say its and older version. Would like to gen the 5 man set and replace my copy.

-Josh

[Daniel Shawul]

Sorry, I didn't realize that Scorpio's EGBB were open source. I thought the probing code came in binary form only, as a Windows DLL. 200 MB is very compact indeed: is that bitbase information comparable to a WDL table (result of the game, accounting for 50-move rule) ?

Yes it is very compact many have used it on mobile devices even on older ones. I am not all that optimistic about 6-men bitbases because it looses all the nice properties you mentioned. It becomes bulky and slow so you might as well have TBs there. I don't like TBs so it will be WDL for 6 men too even if it may need effort to win most games with WDL alone. Diep have used them before so they must be Ok to use alone. The source code for probing, generation, and compression is there if you need it but you don't need to see all that junk. It is very non-intrusive because all you need to do is add a 100 lines interface to egbbdlls. If it was up to me every engine should have had them just because of this neat approach, compared to bulky Nalimov TB probing code that is statically linked. I can understand why people would need convincing for those but 100 lines of code that is much faster than TBs, come on? So people who rejected Scorpio bitbases on the grounds of philosophy, better have some good reason for making a switch now, instead of repeating stuff that is not new as new like you just did. You seem to be brand new to this TB stuff from the way you present the probing when tb < 6 as a novel approach, but I can guarantee if there is any improvement over scorpio bitbases it will be very minimal, if at all.

The probing code has many other features as well. Using scorpio bitbases alone can can win many difficult endgames by modifying the egbb scores, which I think is the first to do so. There is caching if you need it besides the system cache. There is also multi-thread support if you need it. My bitbases will give same elo gain as any bitbases out there for 5-men ,or your money back? I am not interested in the buisness of TBs anymore but the uninformed stuff that keeps written here is unbelivooble

[Daniel Shawul]

The format have changed since I modified the code for 6 men and above, which I never completed, so please don't regenerate. But since you are hosting the code, could you please take the license file and add it to the package you host? I am happy both ways but it seems to have prevented some from using it as they wish in the absence a liscense.
Thanks for hosting
P.s: Address of github is in my signature