Worrying about what end-users want has never seemed a priority of the Stockfish team. Their goal has been to develop a world-class chess engine with source code that is open and clean. This seems like a perfectly fine goal, and I'm puzzled why this one behavior appears to be getting special attention.syzygy wrote:
If the aim is to remove "unnatural behaviour", then I guess the underlying aim is to improve SF for end users. What do end users want? Do they want to be informed that the position they are analysing is a TB win? Or do they prefer to be kept in the dark about whether the engine has already solved the position with the help of TBs?
Do users prefer to be kept in the dark? Marco seems to think so.
Natural TB
Moderators: hgm, Rebel, chrisw
-
- Posts: 6442
- Joined: Tue Jan 09, 2007 12:31 am
- Location: PA USA
- Full name: Louis Zulli
Re: Natural TB (take 2)
-
- Posts: 5566
- Joined: Tue Feb 28, 2012 11:56 pm
Re: Natural TB (take 2)
Yes, that was an inside joke.zullil wrote:Worrying about what end-users want has never seemed a priority of the Stockfish team.syzygy wrote:If the aim is to remove "unnatural behaviour", then I guess the underlying aim is to improve SF for end users. What do end users want? Do they want to be informed that the position they are analysing is a TB win? Or do they prefer to be kept in the dark about whether the engine has already solved the position with the help of TBs?
Do users prefer to be kept in the dark? Marco seems to think so.
If it had been up to Marco, the TB code would have been where the large page code is today. It accidentally was integrated during a short period in which Marco had relinquished control over SF. Since he took back control he has not stopped complaining about the "completely broken design" etc.
(Of course some parts ot the initial patch were hacky, since I simply added TB support with as few changes to SF as possible. The root probing coding admittedly is somewhat "broken" in the sense that it does not play well with multipv and searchmoves. I submitted a patch to clean this up, but there was no willingness to consider it.)
-
- Posts: 4367
- Joined: Fri Mar 10, 2006 5:23 am
- Location: http://www.arasanchess.org
Re: Natural TB
I think the mate problem could be solved by also allowing the use of a DTM tablebase such as Nalimov or Gaviota. If this reports a mate of a distance that is safely within the 50-move counter, then play that mating move.
--Jon
--Jon
-
- Posts: 27817
- Joined: Fri Mar 10, 2006 10:06 am
- Location: Amsterdam
- Full name: H G Muller
Re: Natural TB
More generally, you could take the move with the smallest DTM that according to the DTZ50 is won.
-
- Posts: 2684
- Joined: Sat Jun 14, 2008 9:17 pm
Re: Natural TB (take 2)
I don't think I understand the above, for instance in the following positionsyzygy wrote: I'm not sure it is helpful to treat TB wins and TB losses differently in the middle of the tree (unless you are talking about win/loss relative to the root position). It should not matter whether a TB position is entered by a capture by the winning side (in which case the TB probe returns a TB loss) or by a (re)capture by the losing side (in which case the TB probe returns a TB win).
[D]5k2/8/1p6/r7/R7/K7/8/8 w - - 0 1
TB report a draw for Rxa5 and in the follow up
[D]5k2/8/1p6/R7/8/K7/8/8 b - - 0 1
It is still a draw for bXa5.
Can you please better explain your point?
-
- Posts: 5566
- Joined: Tue Feb 28, 2012 11:56 pm
Re: Natural TB (take 2)
I was clearly not talking about TB draws, so I suggest you carefully re-read what I wrote.mcostalba wrote:Can you please better explain your point?
-
- Posts: 2684
- Joined: Sat Jun 14, 2008 9:17 pm
Re: Natural TB (take 2)
Ok, in this position after the winning side capture Rxa5 black is losing and TB return loss.syzygy wrote:I was clearly not talking about TB draws, so I suggest you carefully re-read what I wrote.mcostalba wrote:Can you please better explain your point?
[D]4k3/8/8/p7/R7/8/8/4K3 w - - 0 1
Instead in this position
[D] 4k3/8/1p6/R7/R7/8/8/4K3 b - - 0 1
after losing side capture bxa5, TB return a win score for white.
Is this what you mean? Your point is that in one case search returns TB loss, in teh other case it returns a normal score (probably very high) and you think this asymmetry could be an issue? Is it correct?
-
- Posts: 5566
- Joined: Tue Feb 28, 2012 11:56 pm
Re: Natural TB (take 2)
I said I see no reason to treat TB wins inside the tree different from TB losses inside the tree.
You treat them differently: TB loss returns immediately, TB win is ignored.
To explain why there is no reason to treat them differently, I point out that if say, white can reach a TB win from some (non-TB) position in the tree, then the point in the subtree at which the winning path enters the TBs could be a TB win (for white) or a TB loss (for black). In fact, some winning paths may enter the TBs in a TB win position and other paths may enter the TBs in a TB loss position. This is sort of random, and it therefore seems very illogical to treat the two cases differently.
Example: a winning KRPPKRP position may have one winning path entering 6-piece TBs in a losing KRPPKR position (white captured into the TBs, so black is to move and loses) and another winning path entering 6-piece TBs in a winning KRPKRP position (black captured into the TBs, so white is to move and wins). You would return the TB loss from the lost KRPPKR position, but you would ignore the TB win in the KRPKRP position. That does not seem to make a lot of sense.
So my proposal:
- a TB loss/win returns immediately with losing/winning TB score if that score can logically be used for an alpha or beta cutoff (so when you don't need to know the exact distance-to-mate to know that you can take the cutoff: it would be silly to search on).
- if alpha, beta are such that you want to know the distance-to-mate (either winning or losing), then continue the search, but make sure that if no mate score is found, then the value returned still reflects that the position is a known (TB) win or loss.
With this approach, SF's TB probing behaviour is unchanged as long as the aspiration window does not approach mate values. So any extra search effort compared to the current implementation is made only when the search has already determined the game to be won or lost, which means it should not cost any Elo. (There is of course no way to gain Elo from resolving TB wins or losses to mates, since DTZ probing at the root is infallible as it is implemented now).
You treat them differently: TB loss returns immediately, TB win is ignored.
To explain why there is no reason to treat them differently, I point out that if say, white can reach a TB win from some (non-TB) position in the tree, then the point in the subtree at which the winning path enters the TBs could be a TB win (for white) or a TB loss (for black). In fact, some winning paths may enter the TBs in a TB win position and other paths may enter the TBs in a TB loss position. This is sort of random, and it therefore seems very illogical to treat the two cases differently.
Example: a winning KRPPKRP position may have one winning path entering 6-piece TBs in a losing KRPPKR position (white captured into the TBs, so black is to move and loses) and another winning path entering 6-piece TBs in a winning KRPKRP position (black captured into the TBs, so white is to move and wins). You would return the TB loss from the lost KRPPKR position, but you would ignore the TB win in the KRPKRP position. That does not seem to make a lot of sense.
So my proposal:
- a TB loss/win returns immediately with losing/winning TB score if that score can logically be used for an alpha or beta cutoff (so when you don't need to know the exact distance-to-mate to know that you can take the cutoff: it would be silly to search on).
- if alpha, beta are such that you want to know the distance-to-mate (either winning or losing), then continue the search, but make sure that if no mate score is found, then the value returned still reflects that the position is a known (TB) win or loss.
With this approach, SF's TB probing behaviour is unchanged as long as the aspiration window does not approach mate values. So any extra search effort compared to the current implementation is made only when the search has already determined the game to be won or lost, which means it should not cost any Elo. (There is of course no way to gain Elo from resolving TB wins or losses to mates, since DTZ probing at the root is infallible as it is implemented now).
-
- Posts: 2684
- Joined: Sat Jun 14, 2008 9:17 pm
Re: Natural TB (take 2)
Yes, to use TB scores in analogy with upper/lower bounds of TT case makes definitely sense, and I have done it:syzygy wrote: But Michel is right. The "clean" solution to the problem has been pointed out many times in this thread. The simple idea of the solution is that you don't take the TB cutoff if the TB win or loss value is inside the (alpha, beta) window.
Code: Select all
// Step 4a. Tablebase probe
if (!PvNode && !rootNode && ss->tbCardinality)
{
int piecesCount = pos.count<ALL_PIECES>();
if ( piecesCount <= ss->tbCardinality
&& (piecesCount < ss->tbCardinality || depth >= TB::ProbeDepth)
&& pos.rule50_count() == 0
&& !pos.can_castle(ANY_CASTLING))
{
TB::ProbeState err;
TB::WDLScore wdl = Tablebases::probe_wdl(pos, &err);
if (err != TB::ProbeState::FAIL)
{
thisThread->tbHits.fetch_add(1, std::memory_order_relaxed);
int drawScore = TB::UseRule50 ? 1 : 0;
value = wdl < -drawScore ? -VALUE_MATE + MAX_PLY + ss->ply
: wdl > drawScore ? VALUE_MATE - MAX_PLY - ss->ply
: VALUE_DRAW + 2 * wdl * drawScore;
// Use TB scores win/loss as bounds (like TT upperbound and
// lowerbound scores), a draw score is always considered.
if (value >= beta ? wdl >= TB::WDLDraw : wdl <= TB::WDLDraw)
{
// In case of a draw or a loss, save TB score in TT and return
// In case of a winning position keep searching the sub-tree
// with a much reduced depth and do not probe TB anymore.
if (value <= VALUE_DRAW)
{
tte->save(posKey, value_to_tt(value, ss->ply), BOUND_EXACT,
std::min(DEPTH_MAX - ONE_PLY, depth + 6 * ONE_PLY),
MOVE_NONE, VALUE_NONE, TT.generation());
return value;
}
depth = std::max(depth / 2, ONE_PLY);
ss->tbCardinality = 0;
}
}
}
}
-
- Posts: 2684
- Joined: Sat Jun 14, 2008 9:17 pm
Re: Natural TB (take 2)
Yes, I treat them differently, and not only when entering in TB, but alwayssyzygy wrote: You treat them differently: TB loss returns immediately, TB win is ignored.
<cut>
That does not seem to make a lot of sense.
The reason for this is to avoid odd sacrifices and TB scores to be exposed to user. If I don't overwrite search value in case of a winning position I assume the above two "features" of TB behavior will not be exposed.
On the other hand it is not like I don't' use TB anymore because:
- I still don't search in draw and losing sub-trees so keeping original TB efficiency
- I spend much less time than usual in winning sub-trees (it is a compromise, I don't' return immediately like in pure TB case, so some efficiency is lost, but because search is greatly reduced, I look for this to be harmless).
You may say that in win position, returning a score form a reduced depth search, instead of a TB win could be very bad. I don't' think so. I don't' throw away TB info, but I implicitly use the very critical knowledge that searching those sub-tree will never fail low even in very deep searches. That's why I search swallow: I already know a priori it will not fail low. This is a very important information. Reason why engines search very deep the PV is not to find a better PV, but to prove the current PV does not fail low. We already know it after a TB probing, so we implicitly use this to do a swallow search and save resources.
Code: Select all
So my proposal:
- a TB loss/win returns immediately with losing/winning TB score if that score can logically be used for an alpha or beta cutoff (so when you don't need to know the exact distance-to-mate to know that you can take the cutoff: it would be silly to search on).
- if alpha, beta are such that you want to know the distance-to-mate (either winning or losing), then continue the search, but make sure that if no mate score is found, then the value returned still reflects that the position is a known (TB) win or loss.
My aim is to keep the TB ELO advantage but do not show to the user we are using TB. User looking at the engine should not realize if we are using TB or not. I don't claim my aim is what everybody expects / likes, I just say that this patch is wrote for this reason, not for others.