Performance of Syzygy and Scorpio

[Houdini]

Syzygy WDL doesn't help SF at all.

I'd expect the same for Houdini.
Thank you for running these useful tests that demonstrate the limitations of using only WDL information.
I'll refer to this thread whenever someone asks me whether the DTZ files are useful for Houdini + Syzygy.

[Daniel Shawul]

Syzygy WDL doesn't help SF at all.

Hilarious. I thought that when the author himself asks for it to be tested, he might have implemented it correctly this time... I guess we still leave in darkness there when it comes to using bitbases alone for progress. I have told him before, and this confirms that the DTZ tables were necessary for him because of lack of knowldege of using bitbases alone. He won't admit it though. Why else would one spend 110 GB of extra space for DTZ alone?? Doesn't make sense.

Now try the Scorpio bitbases with Stockfish and see how it fares. Source is here.
https://github.com/dshawul/Stockfish
You can test the effect of egbbs by first setting a wrong path(disable) and enabling it.

[ouachita]

Interesting albeit unsurprising results:
- The Houdini Syzygy implementation produces a perfect 500/1000 score.
- Toga's "exemplary" egbb implementation only scores 470/1000.

Robert, You are referring to both 3-4-5 men and 6-men Syzygy table bases?

[Houdini]

Interesting albeit unsurprising results:
- The Houdini Syzygy implementation produces a perfect 500/1000 score.
- Toga's "exemplary" egbb implementation only scores 470/1000.

Robert, You are referring to both 3-4-5 men and 6-men Syzygy table bases?

Yes. Kai's test was for 3/4/5 men positions, but the comment is valid for any number of pieces.
With DTZ (distance-to-zero) or DTM (distance-to-mate) bases like Syzygy, Nalimov or Gaviota you'll convert all winning positions, whereas with WDL (win-draw-loss) bases like Scorpio you'll have some misses.
It's up to you to decide whether this is worth the extra disk space (70 GB for the 6-men Syzygy DTZ) - for me it's a no-brainer.

[syzygy]

I cannot imagine any EGTB without DTZ would be of help. The engine simply doesn't have the capacity to understand many wins as the wins require moves which if "fixed" in the algorithmic sense ... they would make the engine play poorly in the majority of other positions.

Most TB wins, once reached on the board, aren't very difficult to convert. TBs help to reach those positions on the board starting from positions with more pieces. For this WDL is sufficient. This is not an aspect that Kai is testing, since he starts with 5-piece positions.

[Daniel Shawul]

Admit it DTZ/DTM is a wasted space. Nalimov is perfect but we know it brought exactly 0 elos! Kai tested it also so Nalimov + Shredder bitbases performed equally with other bitbases. So anything you use at the root brings nothing practically otherwise everyone would be using Nalimov alone.

Also as Kai showed Syzygy bitbases alone brought 0 elo to stockfish. What a disgrace to the bitbase community.

Code: Select all

3 SF 04.02 NO TB                 : 422.5/1000 
4 SF 04.02 Syzygy WDL            : 419.0/1000 

We will see about that when the Stockfish uses the proper bitbases This on the request of the Syzygy author himself. I can't stop laughing.

Toga is ofcourse excpected to score lower than Stockfish because it is a weeker engine but I doubt you are smart enough to see that ...

[syzygy]

Syzygy WDL doesn't help SF at all.

That's a bit surprising as SF in such positions only probes WDL after a capture or pawn move. The probes after a capture into a 4-piece position probably won't help compared to default SF (since default SF most likely evaluates those 4-piece positions perfectly anyway), but those after pawn moves should be of benefit.

The WDL tables also prevent SF from looking at suboptimal moves and that should help as well.

What kind of positions are you testing on? With pawns or without?

[Daniel Shawul]

Syzygy WDL doesn't help SF at all.

That's a bit surprising as SF in such positions only probes WDL after a capture or pawn move. The probes after a capture into a 4-piece position probably won't help compared to default SF (since default SF most likely evaluates those 4-piece positions perfectly anyway), but those after pawn moves should be of benefit.

The WDL tables also prevent SF from looking at suboptimal moves and that should help as well.

What kind of positions are you testing on? With pawns or without?

It is not surprising for me at all. I warned you many times before there is an art to making progress with bitbases alone but you wouldn't listen. The search will find ways to delay capturing even if you probe after captures. It will prefer a winning non-capture move and make the caputre much later unless you have a heuristic eval added to it to prefer 4 men over 5 men. That interacts with the distance from the root, hashtable etc.. I looked into Stockfish syzygy stockfish code and there is no way that would work with bitbases alone. Atleast now you know for sure.

[syzygy]

The search will find ways to delay capturing even if you probe after captures. It will prefer a winning non-capture move and make the caputre much later unless you have a heuristic eval added to it to prefer 4 men over 5 men.

Apparently I have to explain it better.

SF+WDL starting from a TB win as root position probes after captures and pawn moves and essentially treats captures and pawn moves into won positions as "mate". Progress towards such a winning capture and pawn moves is ensured in the same way as any engine ensures progress towards a mate. SF+WDL therefore gives preference to a path into a winning capture or pawn move over any other path. It will certainly not prefer a winning non-capture over a winning capture.

The only positions for which this does not work are positions where winning captures and pawn moves are outside the search horizon. Such positions exist, but are not common.

[Laskos]

Syzygy WDL doesn't help SF at all.

That's a bit surprising as SF in such positions only probes WDL after a capture or pawn move. The probes after a capture into a 4-piece position probably won't help compared to default SF (since default SF most likely evaluates those 4-piece positions perfectly anyway), but those after pawn moves should be of benefit.

The WDL tables also prevent SF from looking at suboptimal moves and that should help as well.

What kind of positions are you testing on? With pawns or without?

Many do have pawns, generally, from KBNK and KBNKN to KRPKR and such. I could upload the EPD file of hard wins if you need it.