That's the first step to solving any of these kinds of problems, proper instrumentation. If it is expensive to compute, make it something that can be compiled out for real games, compiled in for testing and improving.zullil wrote:Perhaps it would be interesting to have a SF that reports the idleness of the search threads, say as a percentage of the total search time. Or some similar core measurement of the parallel implementation.Joerg Oster wrote:There is one big mistake in this argumentation, though.lucasart wrote:Thanks. I'm glad at least one person understands the obvious.jhellis3 wrote:No it isn't. The reason it isn't can be evidenced by what he was replying to. Lucas's statement was not incorrect because Elo > all. So using any reasoning whatsoever (no matter how independently sound) to claim that one should worship at a different altar than Elo is wrong.
You can measure the elo gain and be happy with it, but only with this, you have no clue whether your smp implementation is performing at a top, medium or low level.
To know this, you have to do some other measurements. For instance, time to depth, nodes per second, idle time of the threads, etc.
Did you know before TCEC that SF's nps is (maybe) somewhat low compared to Komodo's in the endgame on 16 cores? No? Elo doesn't tell you that? What a pity ...
Of course, gaining playing strength is the main goal. Nobody denies this.
I added the "cpu %" number in Crafty so I could see exactly that and determine whether the performance issues might be the actual parallel search waiting time (the % value) or the more problematic cache/memory issues that can't easily be measured.