syzygy wrote: ↑Sat May 12, 2018 11:56 am
Ozymandias wrote: ↑Sat May 12, 2018 11:44 am
syzygy wrote: ↑Sat May 12, 2018 11:31 am
Testing TBs at fixed depth will not give a complete picture. On the one hand, they take time to probe. On the other hand, they allow the search to go deeper. By fixing the depth you eliminate these two important components from the equation.
What do you mean by "time to probe"? CPU throttle? There isn't.
Time to probe is simply time to probe. It takes many cpu cycles to probe a position, and data may have to be read from SSD.
Not shooting for the obvious, but everything takes time, aren't those CPU cycles equivalent to the ones needed to reach the same depth without TBs?
syzygy wrote: ↑Sat May 12, 2018 11:56 amOzymandias wrote: ↑Sat May 12, 2018 11:44 amAs the test shows, more depth equals less of a gap, if you increase said depth you'll hardly register any differences at all.
No, your test artificially limits the depth reached by the engine playing with TBs. At the same time, it artificially removes the speed penalty of probing.
Limiting depth is less of an intrusion than limiting time (more on that later), and both the engines playing with TBs, as well as the one without, were depth limited. I think that's fair testing.
What's "the speed penalty of probing"? When the HDD doesn't get in the way, I've noticed none.
syzygy wrote: ↑Sat May 12, 2018 11:56 amOzymandias wrote: ↑Sat May 12, 2018 11:44 amJust because it's the more chosen time control, it doesn't mean you should jump onboard blindly.
Playing with fixed depth is simply not a time control at all.
Only in the sense that the TC isn't fixed, total time per game is variable because the engine requires more or less time, depending on how difficult the positions involved turn out to be. For engine testing, it makes sense to externally influence the engine, imposing a deadline, but looking at TBs performance is different. Let me explain with an example, I played two quick matches between asmFish 9 at depth 12, and asmFish 9 with a TC of 25 moves per second (equivalent total time on that PC); in one of them, asmFish 9 25/1 used 6-men and in the other, no TBs. The difference in
engine's strength was between the margin of error, does that mean that TBs only add a few Elo points? No, it means that when the engine has to manage time allocation, it saves more than enough for the "ending" (past move 53) and that
doubling time or probing TBs will have little effect.