Page 1 of 6

Houdini with a six point lead near the halfway point of TCEC

Posted: Tue Nov 28, 2017 3:10 pm
by AdminX
Chessdom Write Up:

Statement by team Komodo

"During Game 1 of the Superfinal Anton Mihailov, TCEC’s tournament director, emailed us noting significantly lower nodes per second from the version of Komodo we had used in Stage 2, 1959.00. Anton checked TCEC’s server for correct settings and found nothing amiss there.
We began our investigation by making Windows versions of many of the promotions between version 1959.00 and 1970.00 (the Superfinal version), using 23 threads on a 24 core machine. In this testing we found a roughly 8% drop in speed when running 1960.00 versus 1959.00, and similar speed drops in all the versions tested up through 1970.00. We examined the code change between 1959.00 and 1960.00 which was a minor addition to some backward pawn score (two or three lines of code).
Finding nothing in the code that could explain a significant slowdown, we recompiled the source code turning off PGO (profile-guided optimization). This did not help. Then we recompiled turning off LTO (Link Time Optimization). This resulted in restoration of the lost speed. The compiler we used throughout was MinGW 4.7.3.
It is not completely clear what exactly happened between Stage 2 version 1959.00 and 1960.00. There was no evidence of this speed reduction in 1959.00 or Komodo 11.2.2, our latest public release. We speculate that the speed reduction happens once some size of compiled code is exceeded.
It is important to point out that the approximately 8% speed reduction we noted on our best hardware (24 cores) is apparently as high as 23% on TCEC’s 44-core machine based on Komodo’s relative nodes per second vs. Houdini in Stage 2. A difference of this magnitude is what Anton noted in his original email signalling that there might be a problem.
In summary, there is indeed a slowdown in the version now running in TCEC, which appears to be due to a compiler bug. We believe that the compiler bug is more significant than any program changes since 11.2.2.
We accept full responsibility for not discovering the problem before the start of the final. Although the bug has probably cost us some points it probably does not fully explain the current five point score deficit."

http://www.chessdom.com/houdini-with-a- ... t-of-tcec/

Re: Houdini with a six point lead near the halfway point of

Posted: Tue Nov 28, 2017 3:27 pm
by velmarin
The Komodo team looked for one more thing, and lost strength.
Blaming the compiler is simply absurd.
Houdart is right to decline a change.

Re: Houdini with a six point lead near the halfway point of

Posted: Tue Nov 28, 2017 4:41 pm
by Dan Cooper
velmarin wrote:The Komodo team looked for one more thing, and lost strength.
Blaming the compiler is simply absurd.
Houdart is right to decline a change.
People always find the weirdest spin for these.

Re: Houdini with a six point lead near the halfway point of

Posted: Tue Nov 28, 2017 4:42 pm
by mehmet karaman
I think Komodo has time management problem. It has been using less time for critical positions.

Re: Houdini with a six point lead near the halfway point of

Posted: Tue Nov 28, 2017 5:51 pm
by Dirt
velmarin wrote:The Komodo team looked for one more thing, and lost strength.
Blaming the compiler is simply absurd.
No, the compiler shares the blame.
velmarin wrote:Houdart is right to decline a change.
Yes, I agree with that.

Re: Houdini with a six point lead near the halfway point of

Posted: Tue Nov 28, 2017 6:24 pm
by Ras
velmarin wrote:Houdart is right to decline a change.
Especially because he is using a 2.5 months old version. Houdart has decided that ruling out regressions is more important than gaining additional strength. The Komodo team has decided the other way round and went for more bleeding edge.

It would be nonsense to allow one team where the respective decision fired back to have both the cake and eat it. That would be like taking back moves in a tournament game.

That's also why it doesn't matter whether it's the compiler or whatever.

Re: Houdini with a six point lead near the halfway point of

Posted: Tue Nov 28, 2017 6:44 pm
by Leo
Thanks for the post Ted. I didn't know that.

Re: Houdini with a six point lead near the halfway point of

Posted: Tue Nov 28, 2017 6:59 pm
by royb
AdminX wrote:Chessdom Write Up:

Statement by team Komodo

"During Game 1 of the Superfinal Anton Mihailov, TCEC’s tournament director, emailed us noting significantly lower nodes per second from the version of Komodo we had used in Stage 2, 1959.00. Anton checked TCEC’s server for correct settings and found nothing amiss there.
We began our investigation by making Windows versions of many of the promotions between version 1959.00 and 1970.00 (the Superfinal version), using 23 threads on a 24 core machine. In this testing we found a roughly 8% drop in speed when running 1960.00 versus 1959.00, and similar speed drops in all the versions tested up through 1970.00. We examined the code change between 1959.00 and 1960.00 which was a minor addition to some backward pawn score (two or three lines of code).
Finding nothing in the code that could explain a significant slowdown, we recompiled the source code turning off PGO (profile-guided optimization). This did not help. Then we recompiled turning off LTO (Link Time Optimization). This resulted in restoration of the lost speed. The compiler we used throughout was MinGW 4.7.3.
It is not completely clear what exactly happened between Stage 2 version 1959.00 and 1960.00. There was no evidence of this speed reduction in 1959.00 or Komodo 11.2.2, our latest public release. We speculate that the speed reduction happens once some size of compiled code is exceeded.
It is important to point out that the approximately 8% speed reduction we noted on our best hardware (24 cores) is apparently as high as 23% on TCEC’s 44-core machine based on Komodo’s relative nodes per second vs. Houdini in Stage 2. A difference of this magnitude is what Anton noted in his original email signalling that there might be a problem.
In summary, there is indeed a slowdown in the version now running in TCEC, which appears to be due to a compiler bug. We believe that the compiler bug is more significant than any program changes since 11.2.2.
We accept full responsibility for not discovering the problem before the start of the final. Although the bug has probably cost us some points it probably does not fully explain the current five point score deficit."

http://www.chessdom.com/houdini-with-a- ... t-of-tcec/
So, a 23% speed reduction would reduce the playing strength of Komodo 1970 by how much? 15 Elo?

Re: Houdini with a six point lead near the halfway point of

Posted: Tue Nov 28, 2017 7:05 pm
by M ANSARI
I agree you have to accept a mistake and deal with it till next TCEC. It is a shame to not have the best version though. 23% is quite a big difference if both engines are very close in strength. It did seem that Houdini was seeing a little deeper in positions. But have to give credit to Houdini team as the engine is really playing well. Unfortunate that this was not a 3 way race with SF in the hunt.

Re: Houdini with a six point lead near the halfway point of

Posted: Tue Nov 28, 2017 7:15 pm
by AdminX
M ANSARI wrote:I agree you have to accept a mistake and deal with it till next TCEC. It is a shame to not have the best version though. 23% is quite a big difference if both engines are very close in strength. It did seem that Houdini was seeing a little deeper in positions. But have to give credit to Houdini team as the engine is really playing well. Unfortunate that this was not a 3 way race with SF in the hunt.
Also, maybe I am wrong but I don't think it would have helped in game 12.