Houdini with a six point lead near the halfway point of TCEC

Discussion of anything and everything relating to chess playing software and machines.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
User avatar
AdminX
Posts: 5193
Joined: Mon Mar 13, 2006 1:34 pm
Location: Acworth, GA
Contact:

Houdini with a six point lead near the halfway point of TCEC

Post by AdminX » Tue Nov 28, 2017 3:10 pm

Chessdom Write Up:

Statement by team Komodo

"During Game 1 of the Superfinal Anton Mihailov, TCEC’s tournament director, emailed us noting significantly lower nodes per second from the version of Komodo we had used in Stage 2, 1959.00. Anton checked TCEC’s server for correct settings and found nothing amiss there.
We began our investigation by making Windows versions of many of the promotions between version 1959.00 and 1970.00 (the Superfinal version), using 23 threads on a 24 core machine. In this testing we found a roughly 8% drop in speed when running 1960.00 versus 1959.00, and similar speed drops in all the versions tested up through 1970.00. We examined the code change between 1959.00 and 1960.00 which was a minor addition to some backward pawn score (two or three lines of code).
Finding nothing in the code that could explain a significant slowdown, we recompiled the source code turning off PGO (profile-guided optimization). This did not help. Then we recompiled turning off LTO (Link Time Optimization). This resulted in restoration of the lost speed. The compiler we used throughout was MinGW 4.7.3.
It is not completely clear what exactly happened between Stage 2 version 1959.00 and 1960.00. There was no evidence of this speed reduction in 1959.00 or Komodo 11.2.2, our latest public release. We speculate that the speed reduction happens once some size of compiled code is exceeded.
It is important to point out that the approximately 8% speed reduction we noted on our best hardware (24 cores) is apparently as high as 23% on TCEC’s 44-core machine based on Komodo’s relative nodes per second vs. Houdini in Stage 2. A difference of this magnitude is what Anton noted in his original email signalling that there might be a problem.
In summary, there is indeed a slowdown in the version now running in TCEC, which appears to be due to a compiler bug. We believe that the compiler bug is more significant than any program changes since 11.2.2.
We accept full responsibility for not discovering the problem before the start of the final. Although the bug has probably cost us some points it probably does not fully explain the current five point score deficit."

http://www.chessdom.com/houdini-with-a- ... t-of-tcec/
"Good decisions come from experience, and experience comes from bad decisions."
__________________________________________________________________
Ted Summers

User avatar
velmarin
Posts: 1600
Joined: Mon Feb 21, 2011 8:48 am

Re: Houdini with a six point lead near the halfway point of

Post by velmarin » Tue Nov 28, 2017 3:27 pm

The Komodo team looked for one more thing, and lost strength.
Blaming the compiler is simply absurd.
Houdart is right to decline a change.

Dan Cooper
Posts: 184
Joined: Sun Nov 01, 2015 2:15 am

Re: Houdini with a six point lead near the halfway point of

Post by Dan Cooper » Tue Nov 28, 2017 4:41 pm

velmarin wrote:The Komodo team looked for one more thing, and lost strength.
Blaming the compiler is simply absurd.
Houdart is right to decline a change.
People always find the weirdest spin for these.

mehmet karaman
Posts: 142
Joined: Tue Jan 28, 2014 7:37 am
Location: TURKEY

Re: Houdini with a six point lead near the halfway point of

Post by mehmet karaman » Tue Nov 28, 2017 4:42 pm

I think Komodo has time management problem. It has been using less time for critical positions.

Dirt
Posts: 2851
Joined: Wed Mar 08, 2006 9:01 pm
Location: Irvine, CA, USA

Re: Houdini with a six point lead near the halfway point of

Post by Dirt » Tue Nov 28, 2017 5:51 pm

velmarin wrote:The Komodo team looked for one more thing, and lost strength.
Blaming the compiler is simply absurd.
No, the compiler shares the blame.
velmarin wrote:Houdart is right to decline a change.
Yes, I agree with that.
Deasil is the right way to go.

Ras
Posts: 1161
Joined: Tue Aug 30, 2016 6:19 pm
Contact:

Re: Houdini with a six point lead near the halfway point of

Post by Ras » Tue Nov 28, 2017 6:24 pm

velmarin wrote:Houdart is right to decline a change.
Especially because he is using a 2.5 months old version. Houdart has decided that ruling out regressions is more important than gaining additional strength. The Komodo team has decided the other way round and went for more bleeding edge.

It would be nonsense to allow one team where the respective decision fired back to have both the cake and eat it. That would be like taking back moves in a tournament game.

That's also why it doesn't matter whether it's the compiler or whatever.

Leo
Posts: 844
Joined: Fri Sep 16, 2016 4:55 pm
Location: USA/Minnesota
Full name: Leo

Re: Houdini with a six point lead near the halfway point of

Post by Leo » Tue Nov 28, 2017 6:44 pm

Thanks for the post Ted. I didn't know that.
Advanced Micro Devices fan.

royb
Posts: 519
Joined: Wed Mar 08, 2006 11:53 pm

Re: Houdini with a six point lead near the halfway point of

Post by royb » Tue Nov 28, 2017 6:59 pm

AdminX wrote:Chessdom Write Up:

Statement by team Komodo

"During Game 1 of the Superfinal Anton Mihailov, TCEC’s tournament director, emailed us noting significantly lower nodes per second from the version of Komodo we had used in Stage 2, 1959.00. Anton checked TCEC’s server for correct settings and found nothing amiss there.
We began our investigation by making Windows versions of many of the promotions between version 1959.00 and 1970.00 (the Superfinal version), using 23 threads on a 24 core machine. In this testing we found a roughly 8% drop in speed when running 1960.00 versus 1959.00, and similar speed drops in all the versions tested up through 1970.00. We examined the code change between 1959.00 and 1960.00 which was a minor addition to some backward pawn score (two or three lines of code).
Finding nothing in the code that could explain a significant slowdown, we recompiled the source code turning off PGO (profile-guided optimization). This did not help. Then we recompiled turning off LTO (Link Time Optimization). This resulted in restoration of the lost speed. The compiler we used throughout was MinGW 4.7.3.
It is not completely clear what exactly happened between Stage 2 version 1959.00 and 1960.00. There was no evidence of this speed reduction in 1959.00 or Komodo 11.2.2, our latest public release. We speculate that the speed reduction happens once some size of compiled code is exceeded.
It is important to point out that the approximately 8% speed reduction we noted on our best hardware (24 cores) is apparently as high as 23% on TCEC’s 44-core machine based on Komodo’s relative nodes per second vs. Houdini in Stage 2. A difference of this magnitude is what Anton noted in his original email signalling that there might be a problem.
In summary, there is indeed a slowdown in the version now running in TCEC, which appears to be due to a compiler bug. We believe that the compiler bug is more significant than any program changes since 11.2.2.
We accept full responsibility for not discovering the problem before the start of the final. Although the bug has probably cost us some points it probably does not fully explain the current five point score deficit."

http://www.chessdom.com/houdini-with-a- ... t-of-tcec/
So, a 23% speed reduction would reduce the playing strength of Komodo 1970 by how much? 15 Elo?

User avatar
M ANSARI
Posts: 3411
Joined: Thu Mar 16, 2006 6:10 pm

Re: Houdini with a six point lead near the halfway point of

Post by M ANSARI » Tue Nov 28, 2017 7:05 pm

I agree you have to accept a mistake and deal with it till next TCEC. It is a shame to not have the best version though. 23% is quite a big difference if both engines are very close in strength. It did seem that Houdini was seeing a little deeper in positions. But have to give credit to Houdini team as the engine is really playing well. Unfortunate that this was not a 3 way race with SF in the hunt.

User avatar
AdminX
Posts: 5193
Joined: Mon Mar 13, 2006 1:34 pm
Location: Acworth, GA
Contact:

Re: Houdini with a six point lead near the halfway point of

Post by AdminX » Tue Nov 28, 2017 7:15 pm

M ANSARI wrote:I agree you have to accept a mistake and deal with it till next TCEC. It is a shame to not have the best version though. 23% is quite a big difference if both engines are very close in strength. It did seem that Houdini was seeing a little deeper in positions. But have to give credit to Houdini team as the engine is really playing well. Unfortunate that this was not a 3 way race with SF in the hunt.
Also, maybe I am wrong but I don't think it would have helped in game 12.
"Good decisions come from experience, and experience comes from bad decisions."
__________________________________________________________________
Ted Summers

Post Reply