Gaviota TBs [0.1.6.1], bitbase-like interface

[michiguel]

First, I included tbcache_flush(), becasue sometimes you want to erase it. For instance, for epd test, etc. and have reproducibility. I cleaned up and remove a lot of commented code that was not needed.

But mainly:

I included a function that ask Win-Draw-Loss info from the TBs, without the distance to mate. In this way, this function can be used when the distance is not needed. For now, this function internally calls the regular one. However, the idea is that it will call something that will give a better performance. For instance, it could be an actual bitbase. I am implementing a second cache that will store the WDL info, saving a LOT of space or multiplying the efficiency of the cache for the same amount of memory. You could look at is as multiplying the cache memory 8x or look at it a having bitbases that are built on the fly. In any case, the performance should be better than plain TBs.

For now, I provide you with the interface, to see if you have any comments, and to start thinking about the implementation in your own program. I implemented the interface in Gaviota already .
Of course, it comes in two flavors, soft and hard, like the regular functions (soft probes only the cache, never the HD).

If you use it now, it may not be different from the regular TBs, but you will notice a boost in performance next time, just by upgrading the files.

I believe that having a common interface for TBs or bitbases should be the way to go. The engine should not be interested in knowing what happen behind the curtains. The engine knows that sometimes the actual distance to mate is not needed. The library should know how to optimize the flow of information based on that and keep those things separate. TBs or bitbases? none of the engine's business!

You can download the update here, version [0.1.6.1]
http://github.com/michiguel/Gaviota-Tablebases

Code: Select all

This is the regular function

extern int /*bool*/ tb_probe_hard
(unsigned stm,
unsigned epsq,
unsigned castles,
const unsigned *inp_wSQ,
const unsigned *inp_bSQ,
const unsigned char *inp_wPC,
const unsigned char *inp_bPC,
/*@out@*/ unsigned *tbinfo,
/*@out@*/ unsigned *plies);

This is the WDL function, without returning "plies". 

extern int /*bool*/ tb_probe_WDL_hard
(unsigned stm,
unsigned epsq,
unsigned castles,
const unsigned *inp_wSQ,
const unsigned *inp_bSQ,
const unsigned char *inp_wPC,
const unsigned char *inp_bPC,
/*@out@*/ unsigned *tbinfo); 

[Aaron Becker]

I think a unified interface for bitbases and tablebases is a very nice idea, but one thing to keep in mind is that often for bitbases you need to ensure some progress constraints in the search to make sure you're actually making progress and not just shuffling between won positions. For example, here's my unified probing function in daydreamer:

Code: Select all

static bool check_eg_database(position_t* pos,
        float depth,
        int ply,
        int alpha,
        int beta,
        int* score)
{
    if (options.use_gtb) {
        // For DTM tablebases, just look.
        if (probe_gtb_soft(pos, score)) {
            ++root_data.stats.egbb_hits;
            return true;
        }
    } else if (options.use_egbb) {
        // For bitbases, ensure some progress constraints first.
        if (pos->fifty_move_counter != 0 &&
                &#40;ply <= 2*&#40;depth_to_index&#40;depth&#41; + ply&#41;/3&#41;) return false;
        if &#40;probe_egbb&#40;pos, score, ply&#41;) &#123;
            ++root_data.stats.egbb_hits;
            return true;
        &#125;
    &#125;
    return false;
&#125;

[michiguel]

I think a unified interface for bitbases and tablebases is a very nice idea, but one thing to keep in mind is that often for bitbases you need to ensure some progress constraints in the search to make sure you're actually making progress and not just shuffling between won positions. For example, here's my unified probing function in daydreamer:

Code: Select all

static bool check_eg_database(position_t* pos,
        float depth,
        int ply,
        int alpha,
        int beta,
        int* score)
{
    if (options.use_gtb) {
        // For DTM tablebases, just look.
        if (probe_gtb_soft(pos, score)) {
            ++root_data.stats.egbb_hits;
            return true;
        }
    } else if (options.use_egbb) {
        // For bitbases, ensure some progress constraints first.
        if (pos->fifty_move_counter != 0 &&
                &#40;ply <= 2*&#40;depth_to_index&#40;depth&#41; + ply&#41;/3&#41;) return false;
        if &#40;probe_egbb&#40;pos, score, ply&#41;) &#123;
            ++root_data.stats.egbb_hits;
            return true;
        &#125;
    &#125;
    return false;
&#125;

The engine developer could keep track of this information. The gtb API, I think, should be at a lower level.

Anyway, having TB info readily available make bitbase tricks unnecessary. In gaviota I have a function that probes "tables" in general taking alpha, beta, depth, etc. as input. The function decides the internal probing mechanism based on alpha and beta.

Internally if alpha and beta are not mate scores, it will probe WDL functions, otherwise it will probe the regular one. So, WDL functions are used for cutoffs. When a won position is approached, the bounds reach mate values and make the true TBs to be probed.

Unfortunately, I cannot make this "wrapper" part of the APIs because it is engine dependent. Different engines have different ranges of alpha, beta, infinite etc.

Miguel

[Aaron Becker]

Yes, I agree that this cannot be part of the gtb library. I was responding to this:

I believe that having a common interface for TBs or bitbases should be the way to go. The engine should not be interested in knowing what happen behind the curtains. The engine knows that sometimes the actual distance to mate is not needed. The library should know how to optimize the flow of information based on that and keep those things separate. TBs or bitbases? none of the engine's business!

If the engine doesn't know what's going on behind the curtains, how do we ensure progress in the case of bitbases? Maybe the library can be made smart enough to always use dtm scores when progress is an issue, I'm not sure.

edit: now I see that I misinterpreted your intent here. If by a common interface you mean that gtb should understand both kinds of queries, we don't have any disagreement. For some reason I thought you were proposing a single function that would use either DTM or WDL based on some internal conditions, which is problematic. I fail at reading comprehension today.

[michiguel]

Yes, I agree that this cannot be part of the gtb library. I was responding to this:

I believe that having a common interface for TBs or bitbases should be the way to go. The engine should not be interested in knowing what happen behind the curtains. The engine knows that sometimes the actual distance to mate is not needed. The library should know how to optimize the flow of information based on that and keep those things separate. TBs or bitbases? none of the engine's business!

If the engine doesn't know what's going on behind the curtains, how do we ensure progress in the case of bitbases? Maybe the library can be made smart enough to always use dtm scores when progress is an issue, I'm not sure.

edit: now I see that I misinterpreted your intent here. If by a common interface you mean that gtb should understand both kinds of queries, we don't have any disagreement. For some reason I thought you were proposing a single function that would use either DTM or WDL based on some internal conditions, which is problematic. I fail at reading comprehension today.

We are in no disagreement. I guess I was confusing in a couple of points. The engine does not need to know what happen behind the WDL function (is it a real bitbase? just a TB that does not return distance? a special cache built from the TB? a function that calculates WDL for certain positions?). The engine trusts that the WDL function is possibly faster than the regular, at least the same, but never slower.

At a higher level, the engine does not need to know what happen behind the wrapper. Behind the wrapper, this function decides when to go for the WDL functions or the regular ones, knowing alpha, beta, depth etc.

Miguel