EGTB probe in search

[zenpawn]

I'm adding Gaviota 5-piece tablebases to RookieMonster. At the root, this is easy, but in search it can slow things down a bit. After having it in both alphabeta and quiesce, I've left it only in qsearch. Is this best? Are there other tricks? Perhaps adding probe results to the transposition table with an infinite depth?

Thanks,
-Erin

[syzygy]

After having it in both alphabeta and quiesce, I've left it only in qsearch. Is this best?

The other way around is much better: probe in alphabeta and do not probe in qsearch.

In particular, once a branch of the search tree has entered the TBs (in your case a position with 5 pieces or less), there is never a good reason to delay probing until deeper in the tree. Either probe right away upon entering the TB position (and use the result to cut the whole subtree) or don't probe in that subtree at all (because, again, if it would ever be good to probe a node somewhere in that subtree, it would always have been better to do the probe already at the parent node of the subtree where the search entered the TBs).

To control the level of probing, limit it to nodes with remaining depth greater than or equal to some configurable threshold. (So not probing in the qsearch is equivalent to probing only nodes with non-negative remaining depth.)

Perhaps adding probe results to the transposition table with an infinite depth?

That is a possibility.

[zenpawn]

Thank you. That makes more sense, to catch it as early as possible. Do you then not probe in qsearch at all?

Regarding the depth remaining setting, wouldn't it make more sense to do that via a ply setting instead? My engine is lucky to get to 9-ply in a rich middlegame position at 5/1 time controls, so any depth remaining setting would have to be pretty low.

[pedrox]

I'm adding Gaviota 5-piece tablebases to RookieMonster. At the root, this is easy, but in search it can slow things down a bit. After having it in both alphabeta and quiesce, I've left it only in qsearch. Is this best? Are there other tricks? Perhaps adding probe results to the transposition table with an infinite depth?

Thanks,
-Erin

The following seems to work well for me.

In search I has:

Code: Select all

		/* Probamos tablebases Gaviota */
		total_piezas = peonesblancos + caballosblancos + alfilesblancos + torresblancas
						+ damasblancas + peonesnegros + caballosnegros + alfilesnegros + torresnegras + damasnegras + 2;
		if &#40;egtb_is_loaded && total_piezas <= egtb_men&#41; &#123;
			int score;
			if (&#40;ply == 1&#41; && probe_tablebases&#40;&score, DTM_hard&#41; == 1&#41; &#123;
				if &#40;score > 0&#41;	score = score - ply;
				else if &#40;score < 0&#41;	score = score + ply;
				return score;
			&#125;
			if (&#40;depth >= 4*PLY || &#40;nodo_pv && depth >= 1*PLY&#41;) && probe_tablebases&#40;&score, &#40;alpha<=-MATE+200 || beta>=MATE-200&#41;? DTM_hard&#58;WDL_hard&#41; == 1&#41; &#123;
				if &#40;score > 0&#41;	score = score - ply;
				else if &#40;score < 0&#41;	score = score + ply;
				return score;
			&#125;
			if (&#40;nodo_pv || depth >= 1*PLY&#41; && probe_tablebases&#40;&score, &#40;alpha<=-MATE+200 || beta>=MATE-200&#41;? DTM_soft&#58;WDL_soft&#41; == 1&#41; &#123;
				if &#40;score > 0&#41;	score = score - ply;
				else if &#40;score < 0&#41;	score = score + ply;
				return score;
			&#125;
		&#125;

In qsearch I has:

Code: Select all

	/* comprobamos tablebases de Gaviota */
	if (egtb_is_loaded && nodo_pv) {
		total_piezas = peonesblancos + caballosblancos + alfilesblancos + torresblancas
						+ damasblancas + peonesnegros + caballosnegros + alfilesnegros + torresnegras + damasnegras + 2;
		if &#40;total_piezas <= egtb_men&#41; &#123;
			int score;
			if &#40;probe_tablebases&#40;&score, &#40;alpha <= -MATE+ply || beta >= MATE-ply&#41; ? DTM_soft &#58; WDL_soft ) == 1&#41; &#123;
				if &#40;score > 0&#41;	score = score - ply;
				else if &#40;score < 0&#41;	score = score + ply;
				return score;
			&#125;
		&#125;
	&#125;

[syzygy]

Thank you. That makes more sense, to catch it as early as possible. Do you then not probe in qsearch at all?

Probing in qsearch is probably too expensive. In return for a probe, the search can skip only a single qsearch. If you probe closer to the root, the search can skip a far larger subtree.

Regarding the depth remaining setting, wouldn't it make more sense to do that via a ply setting instead? My engine is lucky to get to 9-ply in a rich middlegame position at 5/1 time controls, so any depth remaining setting would have to be pretty low.

A depth remaining setting will result in TBs being probed farther from the root as ply depth increases. That's exactly what one would want, it seems to me.

So if the depth remaining probe limit setting is 5, then the first 5 search iteration only the root position will be probed. In the 6th iteration, also the positions 1 ply from the root will be probed. In the 10th iteration, the positions 1-5 ply from the root will be probed. (Of course will be probed only if they actually enter the TBs.)

[zenpawn]

Regarding the depth remaining setting, wouldn't it make more sense to do that via a ply setting instead? My engine is lucky to get to 9-ply in a rich middlegame position at 5/1 time controls, so any depth remaining setting would have to be pretty low.

A depth remaining setting will result in TBs being probed farther from the root as ply depth increases. That's exactly what one would want, it seems to me.

So if the depth remaining probe limit setting is 5, then the first 5 search iteration only the root position will be probed. In the 6th iteration, also the positions 1 ply from the root will be probed. In the 10th iteration, the positions 1-5 ply from the root will be probed. (Of course will be probed only if they actually enter the TBs.)

I guess there's a logic to that, but why not figure out you're about to enter a TB position in ply 1 as soon as possible, i.e, in the first iteration rather than waiting for the 6th?

[syzygy]

I guess there's a logic to that, but why not figure out you're about to enter a TB position in ply 1 as soon as possible, i.e, in the first iteration rather than waiting for the 6th?

I don't know what you mean by "ply 1" here.

As I said earlier, if you probe at all, then you must probe immediately upon entering the TB (or at the root if the root happens to be in the TB, but this is a special case that has to be treated separately).

So if the root position has 7 pieces and your TBs cover 5-piece positions, then you probe immediately after the 2nd capture of a branch.

At the root, you can't yet tell which branches will have a 2nd capture. At 1 ply from the root, you cannot tell. You will just have to walk the tree and check at every node where you allow probes whether the number of pieces has come down to 5. If yes, then probe (and return immediately... no need to search the child nodes, which is the main advantage of using TBs). If no, then search on.

[zenpawn]

I guess there's a logic to that, but why not figure out you're about to enter a TB position in ply 1 as soon as possible, i.e, in the first iteration rather than waiting for the 6th?

I don't know what you mean by "ply 1" here.

As I said earlier, if you probe at all, then you must probe immediately upon entering the TB (or at the root if the root happens to be in the TB, but this is a special case that has to be treated separately).

So if the root position has 7 pieces and your TBs cover 5-piece positions, then you probe immediately after the 2nd capture of a branch.

At the root, you can't yet tell which branches will have a 2nd capture. At 1 ply from the root, you cannot tell. You will just have to walk the tree and check at every node where you allow probes whether the number of pieces has come down to 5. If yes, then probe (and return immediately... no need to search the child nodes, which is the main advantage of using TBs). If no, then search on.

Understood. By ply 1, I was referring to the quoted portion from you, namely, "In the 6th iteration, also the positions 1 ply from the root will be probed." I was wondering why wait until the 6th iteration to probe at that first ply.

[syzygy]

Understood. By ply 1, I was referring to the quoted portion from you, namely, "In the 6th iteration, also the positions 1 ply from the root will be probed." I was wondering why wait until the 6th iteration to probe at that first ply.

Well, you have to take a decision on how to limit the probes. And once you have taken that decision, you have to stick to it or you could as well not have taken the decision.

So the decision is to use minimum remaining depth as a criterion. If that is set to 5, and at 1 ply from the root the remaining depth is less than 5 (which will approximately be the case in iterations 1-4, but extensions and reductions can change that), then you don't probe. Because that's what you decided.

If you make an exception for positons 1 ply from the root, why not make an exception for positions 2 ply from the root? And if you make an exception for positions 2 ply from the root, why not for positions 3 ply from the root?

The whole idea behind the criterion is that you should only probe if the cost of the probe is sufficiently offset by the advantages of probing. The advantages of probing are (1) better accuracy and (2) tree cutoffs (you don't search the subtree at remaining_depth). So the higher remaining depth, the higher the advantage of (2) and so the more likely it is that the probe will help rather than hurt.

Of course you are free to make an exception for positions 1 ply from the root and no exception for position 2 ply and more from the root, but it seems a bit arbitrary to me.

[zenpawn]

Right, I guess I was proposing using, say, ply < 3 or something, but I can see that remaining depth does have the advantage of eventually consulting the TB further into the variations.

Sadly, I've now tried quite a few of the options described in this thread, including remaining depth > 5, and all of them are giving worse results in matches vs Isa. Perhaps at the level of our engines, the extra knowledge causes it to exclude branches that would otherwise have given good practical chances.