Performance of Syzygy and Scorpio

[Mike S.]

I have to ponder about this thought... Considering engines of almost equal strength, going into a close late endgame sounds plausible.

But you have a point.

[Adam Hair]

I extracted 700 positions containing 6 to 10 men from a database of chess engine matches I have. These 700 positions are judged by Stockfish (with no endgame bases) at depth 20 to be between +75 cp and -75 cp. I went ahead and did some testing using the compiles from my previous test and the compiles of the latest sources from GitHub for Syzygy and Scorpio. Every version of Stockfish with Scorpio bases and Stockfish with Syzygy bases played 1400 games (reverse colors) against Stockfish 090214 without endgame bases. Here are the results (time control was 20"+0.2"):

Impressive testing effort!

It is interesting that SF 130214 does worse than SF 040214 without TBs but does better with TBs.

I am not sure why it did worse. I compiled both SF 040214 and SF 130214 in the same manner, so that should not introduce a source of error. I think that I will rerun the test to see if the results are repeated.

[Adam Hair]

Thanks for this test! It confirms that the Syzygys are a breakthrough in endgame tablebases technology. It is not too long ago that +30 Elo were the result after a year of hard programming work.

P.S. I assume 5-men tables have been used, only?

I would take the +30 Elo with a grain of salt. Under tournament conditions I would expect the gain to be much less. But I do think the test shows that Scorpio and Syzygy WDL bases do help a strong engine like Stockfish.

Yes, I am using 3-4-5 men bases. I believe that my computer/accessories are too antiquated to make use of 6 men bases. But I do have enough RAM to make good use of 5 men bases.