hgm wrote: Well, if the root is a win then every searched line will be reduced, and reducing everything doesn't alter the shape of the tree, but just causes the engine to lie more about the depth it reaches.
Yes, that's correct.
Please note that this is what happens already today with current implementation: if root position is in TB, then probing is disabled. Regarding to lie about searched depth, I am not sure because PV nodes are not TB probed and depth there is real, side nodes already lie a lot about actual search depth due to aggressive LMR. If LMR is lying then, yes, I lie...as all the other engines that use LMR btw.
hgm wrote:
If the root is a win not yet in any EGT, all lines going in the EGT will be reduced, but the lines that avoid conversion will not. If the conversion itself is no obvious progress (i.e. a trade, rather than gaining material), this will make it less likely that it will go for a conversion, as when you are winning, the deeper you search, the better the score usually gets.
This is a speculative statement. May be it is maybe not, but without actual testing it is just a speculation about what could happen. Indeed what actually happens in the engine search is very complex and difficult to deduce just out of our minds.
That is what I have been using in Matefinder for quite some time now.
- }
This is also very interesting. I see you don't use pos.rule50_count() == 0 condition to probe TB.
I always got it as a kind of optimization heuristic, instead of probing everywhere just probe when there is a capture or a pawn move.
I did some quick test and with Natural TB this condition seems largely irrelevant because without it probed nodes are increased by just less than 10% so if the aim is the speed optimization I would say this gives at best a limited advantage.
Well, so if the root is in the EGT, all it amounts to is more aggressive LMR. It is not clear why you would expect the optimum reduction for LMR to be different in this situation than in others. Large LMR tends to make the engine more speculative; it tends to stick with what it liked best at low depth, and will have grave difficulty finding out that an alternative is better. It can more easily detect it when its first instincts were not so good as it initially thought. But as the PV line is already guaranteed to be a win by the EGT, it is not likely to run into any trouble. Effectively it decides on the move at lower depth, and then just spends more time on confirming the obvious.
And indeed, everything must be tested. It is just as speculative to think the proposed strategy has any benefits at all.
syzygy wrote:Obviously disabling TB probing introduces search instability and it prevents SF from finding the winning line to a TB win.
It is too much to ask for an example position? The only obvious thing is that for me and you the word "obvious" has a different meaning.
No example is needed to understand that a search that does not probe TBs in the PV will not be able to detect that the PV reaches a winning TB ending. So, if the havoc your approach wreaks outside the PV does not result in all TB wins going undetected, it will be the lack of TB probing in the PV that finishes that job.
Here it is common for people (not you in particular) to invoke "search instability" as a kind of amulet when they have no more arguments, it is a kind of last resort answer, is like the shooting ink of the octopus
Most people participating in this thread have a good basic understanding of TB probing and the ability to reason. You do not accept reasons and arguments, which makes communication futile.
All of the stuff being discussed here is very well understood by those who have been working with and thinking about TBs, but it seems to be very new to you as most of your posts here show. Yet you do not take the trouble of first understanding the basics before trying to see if you can improve on it.
So this is not a case where you are like Einstein introducing revolutionary ideas that shock the experts. Einstein first mastered the basics, so he knew what he was doing.
Have you talked about your ideas with Joona? He does seem to have good insight in this area (judging from some posts he made quite some time ago). You might be more willing to accept his words.
hgm wrote: And indeed, everything must be tested. It is just as speculative to think the proposed strategy has any benefits at all.
Yes, currently I have verified on some test positions that with this code there are no sacrifice, TB scores, etc. But more test positions are needed possibly by other interested people. This kind of test is relatively easy.
Then the big test will be to verify Natural TB does not regress compared with current implementation. This test is definitely much more difficult and it will be long.
hgm wrote: Well, if the root is a win then every searched line will be reduced, and reducing everything doesn't alter the shape of the tree, but just causes the engine to lie more about the depth it reaches.
Yes, that's correct.
Please note that this is what happens already today with current implementation: if root position is in TB, then probing is disabled.
Which only works because the non-winning moves are tossed out first...
That is what I have been using in Matefinder for quite some time now.
- }
This is also very interesting. I see you don't use pos.rule50_count() == 0 condition to probe TB.
I always got it as a kind of optimization heuristic, instead of probing everywhere just probe when there is a capture or a pawn move.
It is not a heuristic.
In the normal case, the condition is superfluous: the first position in a branch to reach TBs will immediately follow a capture. The rest of the subtree is TB-pruned. So TB probes are done only if rule50_count == 0 anyway.
The reason why it is there is to handle the case where DTZ is not available and the root position is in the TBs and winning. The search then probes WDL after captures and pawn moves to try to make progress towards a winning 50-move zeroing move.
syzygy wrote: Have you talked about your ideas with Joona? He does seem to have good insight in this area (judging from some posts he made quite some time ago). You might be more willing to accept his words.
I am willing to accept the words of everybody, especially your words considered that you are the original author.
But of course I question the arguments if I found them not convincing for me. As you all are questioning my proposal and this is ok and it helps me to improve it.
Still nobody came out with a definitive sound argument why this approach should not work. The only fact is that treatment of winning positions is not so efficient as original implementation where we have an hard fail high. But it is very difficult to speculate, without testing, that this could have an impact in real games.
Regarding not probing at PV nodes, this could be another source of reduced efficiency, but it is difficult to say for sure without testing, for instance at first glance you may say the same for TT where instead we have proved not probing TT at PV nodes does not yield any ELO regression.
