Stockfish depth vs. others; challenge

[lkaufman]

when you talk about selective depths what do you means?
I have always used the deepest node searched by qsearch function, but I have just noticed Stockfish uses the deepest node searched by the main search function.

Well, I never even look at "selective depth" as it can have different meanings for different engines, and anyway it's not an important number. Maybe if everyone used the Stockfish definition it would be somewehat useful. I'm always referring to the normal depth, i.e. the iteration counter.

[Steve Maughan]

Hi Marco,

That explains the difference I noted. I think most engines show the deepest nodes searched in the qsearch - Maverick does this. So if Stockfish has a different definition then this is the explanation.

Thanks,

Steve

[PK]

My guess is that Stockfish benefits primarily from synergy, i.e. it can do big reductions because it already reaches amazing depths (i.e. because it already reduces a lot).

I remember that I failed miserably trying to implement some of Stockfish reductions/prunings in Rodent a year or two ago. similiar attempts made this year were much more successful, with one caveat: Rodent is now mature enough to absorb the ideas from Stockfish code, but each and every time I have to use more conservative numbers, since risk/reward ratio is different at lower depths.

[mcostalba]

Yes, selective depth is measured only on main search. On qsearch there can be positions that greatly recurs in qsearch, but is not really searching it is just resolving the captures. So measuring on main search seems a bit more consistent to me.

The main extension of SF is not check extension, but singular search....and it is also the most important. It is this one that can grow a much bigger selective depth out of the normal iteration depth.

[sandro Necchi]

Marco,
The depth search of Stockfish is impressive and I see improvements on the search also on very recent versions, but still it looks to me that the setting is not optimized. With a setting I made "called" Ratassada, I am getting better results regardless the engine version.
Basically I see Stockfish strongera in middlegame and endgames even without TB.
I admit my books probably help a lot the engine by giving suitable openings, but I get at least +85 Elo against Houdini 3 Pro with Perfect 2012 CTG, also at fast blitz games.
Basically the changes on the setting were to improve first part of the game and midlle game without modifying/effecting the end game.
I admit the number of games is not significant to make conclusions.
I am willing to share this setting for intensive testings...
Ciao
Sandro

[mcostalba]

I am willing to share this setting for intensive testings...

Thanks for your willing to share: please post them on SF dev forum here:

https://groups.google.com/forum/?fromgr ... ishcooking


And we will properly test them.

[Daniel Shawul]

If we compare to Ivanhoe, the extension rules are mor stingy in Stockfish, but not way more. Maybe this accounts for a quarter ply or so. The futility pruning seems fairly similar to Ivanhoe in terms of net effect, I doubt that they differ by more than a quarter play anyway. I'll accept your estimate that SF LMR adds 2 ply. Regarding evaluation, stockfish is certainly faster than Komodo, but probably not faster (certainly not much faster) than Ivanhoe. More aggressive null move (which you didn't mention) might add another ply. So maybe we have accounted for 3.5 plies difference. But we typically see differences around 10 plies, and much more than that in endgames! So there is still a lot to explain.

This seems to me like all guesses. Why not turn off all prunings and then add them one by one, or better yet start from glaurung and add the improvments. That must tell where the depth is coming from though I suspect it is a collective effect from small contributions. Heavy reductions + almost no extensions + light q-search already gets you big depths as Ryan pointed out.

[Henk]

In my chess program the fastest way to reach huge search depths is to increment R in both LMR and Null move. But that makes my program only play worse.

Same for Komodo. Presumably if Stockfish decreased R to Ippo-like levels in both LMR and Null move it would get weaker, else they would do that. So why can Stockfish get away with reductions that everyone else finds to be quite bad?

Maybe the first step is to make a chess program scalable. If depth is increased and the program plays worse, than that means it is not scalable. So reductions/extensions/pruning like for instance futility pruning and razoring should only use boundaries that are not constants but formulas computed out of current maximum search depth. Same holds for R should be something like for instance c1 + c2 * maxDepth. [Perhaps now c1 gives another minor scalability problem]

[lkaufman]

If we compare to Ivanhoe, the extension rules are mor stingy in Stockfish, but not way more. Maybe this accounts for a quarter ply or so. The futility pruning seems fairly similar to Ivanhoe in terms of net effect, I doubt that they differ by more than a quarter play anyway. I'll accept your estimate that SF LMR adds 2 ply. Regarding evaluation, stockfish is certainly faster than Komodo, but probably not faster (certainly not much faster) than Ivanhoe. More aggressive null move (which you didn't mention) might add another ply. So maybe we have accounted for 3.5 plies difference. But we typically see differences around 10 plies, and much more than that in endgames! So there is still a lot to explain.

This seems to me like all guesses. Why not turn off all prunings and then add them one by one, or better yet start from glaurung and add the improvments. That must tell where the depth is coming from though I suspect it is a collective effect from small contributions. Heavy reductions + almost no extensions + light q-search already gets you big depths as Ryan pointed out.

I agree, it must be the sum of lots of things. But I note that Stockfish DD actually does FULL ply check extensions (when not losing) as well as singular extensions, so I don't think lack of extensions can explain very much. Really the puzzle for me is more how Stockfish can get away with so much reducing/pruning when no one else seems to be able to do so. If I made Komodo reduce and prune as much as SF it would not be a top engine.

[lkaufman]

In my chess program the fastest way to reach huge search depths is to increment R in both LMR and Null move. But that makes my program only play worse.

Same for Komodo. Presumably if Stockfish decreased R to Ippo-like levels in both LMR and Null move it would get weaker, else they would do that. So why can Stockfish get away with reductions that everyone else finds to be quite bad?

Maybe the first step is to make a chess program scalable. If depth is increased and the program plays worse, than that means it is not scalable. So reductions/extensions/pruning like for instance futility pruning and razoring should only use boundaries that are not constants but formulas computed out of current maximum search depth. Same holds for R should be something like for instance c1 + c2 * maxDepth. [Perhaps now c1 gives another minor scalability problem]

We have found what you say to be true for null move but not for much else. But it works for Stockfish somehow.