TalkChess.com

Komodo is partially crippled if speed is 22% less than usual.

syzygy wrote: Obviously there is still a lot of room for discussion on whether the exact number is 22% or 23% or maybe 22.78%.

Or even 8% as Mark calculated on his own 22-core machine. General observations aren't enough. The only way is after the tournament to run the exe on some identical positions on infinite analysis and compare, and perhaps run the internal benchmark as well. Until that is done the 23% number is pure speculation.

Modern Times wrote:

syzygy wrote: Obviously there is still a lot of room for discussion on whether the exact number is 22% or 23% or maybe 22.78%.

Or even 8% as Mark calculated on his own 22-core machine. General observations aren't enough. The only way is after the tournament to run the exe on some identical positions on infinite analysis and compare, and perhaps run the internal benchmark as well. Until that is done the 23% number is pure speculation.

If you care much about 22.78% versus 23%, then yes, it is pure speculation as you say.

I think most people will happily accept that there is a substantial slowdown of around 23% by simply inspecting the nps numbers reported by Komodo over a series of positions in a number of games.

But feel free to attack Mark by arguing that he cannot know that it is 23% and not 22.78%

No attacks on Mark, to the contrary his 8% number is the only verifiable one at the moment.

From games 25-67, the scores have been level.
Those five unanswered losses in the first 24 games really hurt Komodo.

Modern Times wrote:No attacks on Mark, to the contrary his 8% number is the only verifiable one at the moment.

Do you realise the 23% number comes from him?

And do you understand how he got to that number? (Hint: not by "extrapolating" the 8% number.)

Why are people comparing the nps of Komodo in stage 2 (different engine), with Komodo of another stage?
SF dev versions have very different nps from a version to the next. After applying several many patches you cannot expect the nps to roughly stay the same. I am not saying it is meaningless to do so, but it isn't meaningul either. It cannot give much information. What count is elo/strength, not raw nps.
If the bug is only a slowdown (not a functional change), then the correct way to estimate the slowdown is to let the 2 engines, normal and bugged, on many different positions and report the average nps or something like that. Just like the bench command of stockfish and other engines. Comparing the nps between 2 differently functional engines doesn't give as much information as I think people are attributing.

Isaac wrote:Why are people comparing the nps of Komodo in stage 2 (different engine), with Komodo of another stage?

If an internal change in Komodo had explained the nps difference, we trust that Mark would have said so.

Here we have a case where there is probably no slowdown on 1 core, an 8% slowdown with 24 cores and a 23% slowdown with 43 cores. That cannot be explained by a regular change in the eval or search code. Of course it is possible that Mark overhauled the threading code, but it is not very likely.

SF dev versions have very different nps from a version to the next. After applying several many patches you cannot expect the nps to roughly stay the same.

SF's nps roughly does stay the same over many patches.

I am not saying it is meaningless to do so, but it isn't meaningul either.

If Mark thinks it is meaningful, who are we to judge differently?

syzygy wrote:SF's nps roughly does stay the same over many patches.

Ok thanks, I didn't know this. I thought there was a great variability between each dev versions, even though the average nps on the bench positions stays around a mean that doesn't change much with time anymore.

Isaac wrote:

syzygy wrote:SF's nps roughly does stay the same over many patches.

Ok thanks, I didn't know this. I thought there was a great variability between each dev versions, even though the average nps on the bench positions stays around a mean that doesn't change much with time anymore.

The bench number itself (total nodes) varies a lot from patch to patch in a completely unpredictable manner.