Houdini with a six point lead near the halfway point of TCEC
Moderators: hgm, Rebel, chrisw
-
- Posts: 2821
- Joined: Fri Sep 25, 2015 9:38 pm
- Location: Sortland, Norway
Re: Houdini with a six point lead near the halfway point of
Komodo is partially crippled if speed is 22% less than usual.
-
- Posts: 3546
- Joined: Thu Jun 07, 2012 11:02 pm
Re: Houdini with a six point lead near the halfway point of
Or even 8% as Mark calculated on his own 22-core machine. General observations aren't enough. The only way is after the tournament to run the exe on some identical positions on infinite analysis and compare, and perhaps run the internal benchmark as well. Until that is done the 23% number is pure speculation.syzygy wrote: Obviously there is still a lot of room for discussion on whether the exact number is 22% or 23% or maybe 22.78%.
-
- Posts: 5563
- Joined: Tue Feb 28, 2012 11:56 pm
Re: Houdini with a six point lead near the halfway point of
If you care much about 22.78% versus 23%, then yes, it is pure speculation as you say.Modern Times wrote:Or even 8% as Mark calculated on his own 22-core machine. General observations aren't enough. The only way is after the tournament to run the exe on some identical positions on infinite analysis and compare, and perhaps run the internal benchmark as well. Until that is done the 23% number is pure speculation.syzygy wrote: Obviously there is still a lot of room for discussion on whether the exact number is 22% or 23% or maybe 22.78%.
I think most people will happily accept that there is a substantial slowdown of around 23% by simply inspecting the nps numbers reported by Komodo over a series of positions in a number of games.
But feel free to attack Mark by arguing that he cannot know that it is 23% and not 22.78%
-
- Posts: 3546
- Joined: Thu Jun 07, 2012 11:02 pm
Re: Houdini with a six point lead near the halfway point of
No attacks on Mark, to the contrary his 8% number is the only verifiable one at the moment.
-
- Posts: 41428
- Joined: Sun Feb 26, 2006 10:52 am
- Location: Auckland, NZ
Re: Houdini with a six point lead near the halfway point of
From games 25-67, the scores have been level.
Those five unanswered losses in the first 24 games really hurt Komodo.
Those five unanswered losses in the first 24 games really hurt Komodo.
gbanksnz at gmail.com
-
- Posts: 5563
- Joined: Tue Feb 28, 2012 11:56 pm
Re: Houdini with a six point lead near the halfway point of
Do you realise the 23% number comes from him?Modern Times wrote:No attacks on Mark, to the contrary his 8% number is the only verifiable one at the moment.
And do you understand how he got to that number? (Hint: not by "extrapolating" the 8% number.)
-
- Posts: 265
- Joined: Sat Feb 22, 2014 8:37 pm
Re: Houdini with a six point lead near the halfway point of
Why are people comparing the nps of Komodo in stage 2 (different engine), with Komodo of another stage?
SF dev versions have very different nps from a version to the next. After applying several many patches you cannot expect the nps to roughly stay the same. I am not saying it is meaningless to do so, but it isn't meaningul either. It cannot give much information. What count is elo/strength, not raw nps.
If the bug is only a slowdown (not a functional change), then the correct way to estimate the slowdown is to let the 2 engines, normal and bugged, on many different positions and report the average nps or something like that. Just like the bench command of stockfish and other engines. Comparing the nps between 2 differently functional engines doesn't give as much information as I think people are attributing.
SF dev versions have very different nps from a version to the next. After applying several many patches you cannot expect the nps to roughly stay the same. I am not saying it is meaningless to do so, but it isn't meaningul either. It cannot give much information. What count is elo/strength, not raw nps.
If the bug is only a slowdown (not a functional change), then the correct way to estimate the slowdown is to let the 2 engines, normal and bugged, on many different positions and report the average nps or something like that. Just like the bench command of stockfish and other engines. Comparing the nps between 2 differently functional engines doesn't give as much information as I think people are attributing.
-
- Posts: 5563
- Joined: Tue Feb 28, 2012 11:56 pm
Re: Houdini with a six point lead near the halfway point of
If an internal change in Komodo had explained the nps difference, we trust that Mark would have said so.Isaac wrote:Why are people comparing the nps of Komodo in stage 2 (different engine), with Komodo of another stage?
Here we have a case where there is probably no slowdown on 1 core, an 8% slowdown with 24 cores and a 23% slowdown with 43 cores. That cannot be explained by a regular change in the eval or search code. Of course it is possible that Mark overhauled the threading code, but it is not very likely.
SF's nps roughly does stay the same over many patches.SF dev versions have very different nps from a version to the next. After applying several many patches you cannot expect the nps to roughly stay the same.
If Mark thinks it is meaningful, who are we to judge differently?I am not saying it is meaningless to do so, but it isn't meaningul either.
-
- Posts: 265
- Joined: Sat Feb 22, 2014 8:37 pm
Re: Houdini with a six point lead near the halfway point of
Ok thanks, I didn't know this. I thought there was a great variability between each dev versions, even though the average nps on the bench positions stays around a mean that doesn't change much with time anymore.syzygy wrote:SF's nps roughly does stay the same over many patches.
-
- Posts: 5563
- Joined: Tue Feb 28, 2012 11:56 pm
Re: Houdini with a six point lead near the halfway point of
The bench number itself (total nodes) varies a lot from patch to patch in a completely unpredictable manner.Isaac wrote:Ok thanks, I didn't know this. I thought there was a great variability between each dev versions, even though the average nps on the bench positions stays around a mean that doesn't change much with time anymore.syzygy wrote:SF's nps roughly does stay the same over many patches.