What happens using egbb

[Laskos]

I tested Houdini 3 with Scorpio egbb with the default Hard and Soft Probe Depth. It quickly became apparent that H3 without egbb beats H3 Scorpio egbb at 60''+0.15'' TC using standard set of opening positions.

Code: Select all

 
    Program                            Score     %    Av.Op.  Elo    +   -    Draws

  1 H3                             : 124.0/220  56.4    -22     22   28  28   61.8 %
  2 H3 Scorpio                     :  96.0/220  43.6     22    -22   28  28   61.8 %

44 +/- 28 Elo points 2SD, LOS 99.9% as a stopping rule.

Then at the same TC I fed endgame 3-4-5 men starting positions, where the bitbases hit at the root. The result is even worse with egbb

Code: Select all

   Program                            Score     %    Av.Op.  Elo    +   -    Draws

  1 H3                             :  71.5/102  70.1    -74     74   54  53   40.2 %
  2 H3 Scorpio                     :  30.5/102  29.9     74    -74   53  54   40.2 %

I am using the renamed egbbdll64.dll from Daniel site, and egbb archives from Houdini site. The UCI info is the following:

setoption name ScorpioPath value C:\LittleBlitzer\Scorpio
info string EgbbProbe 3.3.1 by Daniel Shawul
info string Loading egbbs...
info string 180 Scorpio Egbb files loaded
info string Scorpio 5 men EGTB available - 32 MB cache

Is w/d/l/ information actually hard to convert, and does more harm than good in games?

[Laskos]

Is w/d/l/ information actually hard to convert, and does more harm than good in games?

I am in the process of abandoning egbb's in my tournaments because I've seen a lot of games where the engine using them could make no progress.

I believe your worse results using egbb's are due to the lower number of won positions actually converted into wins.

I too suspected that, as the 3-4-5 men endgame positions of the second test were TB wins, and the result using egbb were abysmal. But I wouldn't have imagined that it's so much worse than the engine evaluating positions without any kind of bases.

[Daniel Shawul]

That is because he probably didn't implement egbb probing correctly. Scorpio does it so well that DTM/DTC are not required at all (say one in a 10000). You should try with scorpio vs scorpio-egbb, probably Houdart implemented the same way as for Nalimov's, which brought some bad name for egbbs. My six men egbbs will also use the same technique because I hardly see scorpio not be able to convert any endings in 5-man but I will check later once 6 man are finished. Diep said they were mostly ok there too so I bet DTC/DTM is just a ton of burden for an insignificant gain.

Btw all pawn-less 6 man are generated now and they are about 17GB for me. I predict the final to be 50-60GB but I could have done better than that if I thinkered a bit with it compression scheme. But the size is too big for me, that I don't feel like downloading the data from the cluster to laptop And I also don't feel like optimizing the generator.

[syzygy]

Is w/d/l/ information actually hard to convert, and does more harm than good in games?

I can't speak of egbb, but the obvious way of dealing with w/d/l information is assigning d a draw score and treating w/l as mate scores in the same way that the engine treats other mate scores. To make sure that a real mate is preferred over a tablebase win, set the w to MATE - 100 but make sure the search and hash table still treat those scores in the same way as real mate scores.

In this way having the w/d/l information should never hurt, unless probing overhead reduces nps too much.

[Daniel Shawul]

Show me position where scorpio had problems. I bet they are far and a few. I am not responsible for programmers not be able to use information, other than straight forward DTM that requires just a probe(). For egbbs, there is even a very simple implementation that has no hassle by just put the probe() inside eval. Other WDL databases do not work like that because they just return either Win,Loss or Draw but scorpio-egbbs work like majic there.

[syzygy]

Is w/d/l/ information actually hard to convert, and does more harm than good in games?

I am in the process of abandoning egbb's in my tournaments because I've seen a lot of games where the engine using them could make no progress.

Is the lack of progress typically occurring after a 5-men position has been reached, or before?

If the lack of progress occurs before a 5-men position has been reached, then the problem is probably that the tablebase results are not treated by the search and hash table in the same way as real mate scores.

Normally the search awards a mate that is 20 ply from the root a score of MATE - 20. If it does not subtract the 20, it will not see a difference between a mate in 18 ply and a mate in 20 ply and might fail to make progress. (To make this work fully correctly some adjustments are needed when storing/retrieving the score in/from the hash table.)

If the lack of progress occurs after a 5-men position has been reached, then the problem might be that the search continues to probe the tables for 5-men positions. A much better strategy is to use the tables only to filter out the moves at the root that lose a half point or a whole point and search either without, or only probe 4-men positions (or positions after a pawn move).

[Houdini]

I tested Houdini 3 with Scorpio egbb with the default Hard and Soft Probe Depth. It quickly became apparent that H3 without egbb beats H3 Scorpio egbb at 60''+0.15'' TC using standard set of opening positions.

Code: Select all

 
    Program                            Score     %    Av.Op.  Elo    +   -    Draws

  1 H3                             : 124.0/220  56.4    -22     22   28  28   61.8 %
  2 H3 Scorpio                     :  96.0/220  43.6     22    -22   28  28   61.8 %

As you've noticed, Houdini 3 with Scorpio egbb has some difficulties with converting won positions. I recommend using Houdini 3+egbb only as analysis tool, not for playing matches.

Houdini 4 will no longer support Scorpio egbb.
Instead Houdini 4 will support Syzygy bases, I'm currently running extensive tests with 5 and 6-men tablebases to make sure that conversion of won positions is flawless so that they can effectively be used in matches.

Cheers,
Robert

[Daniel Shawul]

Good riddance...

[Houdini]

Good riddance...

Agreed, Houdini 4 users will discover that the Syzygy bases are a vastly superior solution.

All the best,
Robert

[Daniel Shawul]

You must think audience is gullible dont ya? Scorpio egbbs have worked flawless with many engines for 10 freaking years!. That is my CV (insert curse word)! But I am glad you are finally gone because my egbb's had to pay for your incompetence.