Houdini with a six point lead near the halfway point of TCEC

[IanO]

Amusing that Komodo made back two points since this post, including the first black victory in the match! As we should all be aware, don't count your chickens until they've hatched.

[egiovannotti]

For me, the progress of Houdini goes hand in hand with the Stockfish and if the latter does not have significant progress there is very little to improve in Houdini and rather, with so little time, Houdart runs the risk of introducing a bug.

[kk]

Not quite sure how Houdini only progresses when Stockfish progresses when Houdini is getting superior results to Stockfish? Looks to me like Stockfish has hit a wall anD development improvement has slowed somewhat.

[velmarin]

For me, the progress of Houdini goes hand in hand with the Stockfish and if the latter does not have significant progress there is very little to improve in Houdini and rather, with so little time, Houdart runs the risk of introducing a bug.

Your comment is OK.
No one can doubt Houdart's talent,
Perhaps he has managed to introduce things of Houdini_4 into the current Houdini, together with a good testing frame.

[TommyTC]

Houdart is right to decline a change.

Yes, and in my view the request to replace Komodo should have been rejected without consulting Houdart.

I agree also, but I guess there was a slight chance that he would have agreed to it so they put the question.

And exactly that is why, in my view, they should not have put the question to him.

I think all agree that 23% lower nps is not a play-limiting bug "like crashing".

It is not fair to shift the blame/responsibility for not replacing Komodo to Houdart. (But luckily it seems that most people understand his refusal.)

Unfortunately some uninformed people see 23% lower nps and think it is a huge disadvantage and that Houdart is "the bad guy" for not allowing a Komodo update.

Each season there are likely to be situations where the rules don't seem to give a 100% clear direction. In those circumstances I think it is a good idea for the tournament organizer (Anton) to consult with the participants, and anyone else, to obtain opinions on how to continue.

After those discussion, he should then take 100% responsibility for the decision and should state something like, "After consulting with the participants in this tournament I have made the decision to...."

This then leaves open the possibility that all participants might agree on a solution that is disadvantageous to their own participation in the event, yet absolves them of ever having to take blame for any other decision.

[Modern Times]

The 23% is also just a guess from my reading of this.

That was a rough observation by the tournament director. NPS as we know varies hugely during the phase of the game. Of course he would have been smart enough for example to look at it out of the opening for example, but even then there is variation. Mark said 8% on his 22 core machine and then assumed that it was a reasonable extrapolation to go from 8% to 23% when going from 22 to 44 cores. I don't accept that the 23% is real necessarily, only way is to run a proper benchmark outside gameplay. It may be 15% for example or even closer to the 8% Mark measured. And even if it is 23% during the opening for example it may be entirely different during the mid-game and end-game.

[shrapnel]

For me, the progress of Houdini goes hand in hand with the Stockfish and if the latter does not have significant progress there is very little to improve in Houdini and rather, with so little time, Houdart runs the risk of introducing a bug.

Spoken well and truly with Tongue-in-Cheek
Many people didn't get what you were hinting at though...
For the record, I fully agree with you.

[Jouni]

Just for curiosity I played one top engine with 23% time handicap against itself. First 100 games (default engine first):

Code: Select all

+20/=70/-10 55.00%   55.0/100
+10/=70/-20 45.00%   45.0/100

and next 100

Code: Select all

+23/=62/-15 54.00%   54.0/100
+15/=62/-23 46.00%   46.0/100

In these fast games 23% can easily result with 10 wins less .

[Milos]

Just for curiosity I played one top engine with 23% time handicap against itself. First 100 games (default engine first):

Code: Select all

+20/=70/-10 55.00%   55.0/100
+10/=70/-20 45.00%   45.0/100

and next 100

Code: Select all

+23/=62/-15 54.00%   54.0/100
+15/=62/-23 46.00%   46.0/100

In these fast games 23% can easily result with 10 wins less .

First there is not even a remotely reliable figure regarding what is actual slow down of K 1960-1970 vs. 1959 on TCEC hardware. Talking about 23% when on 22 cores it is 8% is just ridiculous.
It is quite easy to compare Komodo in early opening phase of the games in stage 2 and now in stage 3. In stage 2 it was typically around 47Mnps and now is 40Mnps. So that is reduction of 15% at best.
Second, on TCEC conditions doubling single core nodes gives around 30-40Elo, doubling it in terms of additional cores hardly 20Elo. Question is how much of the so-called K bug is related to single core performance. The indication is quite strong that it is exclusively SMP speed loss. In that case 15% speed loss would translate into log(0.85)/log(0.5)*20 = 4.7 Elo.
That is like 0.5 points more for K in 100 games match against H.
And Robert Houdard with his 9Elo estimation was more than generous.

[syzygy]

The 23% is also just a guess from my reading of this.

Not really:

It is important to point out that the approximately 8% speed reduction we noted on our best hardware (24 cores) is apparently as high as 23% on TCEC’s 44-core machine based on Komodo’s relative nodes per second vs. Houdini in Stage 2. A difference of this magnitude is what Anton noted in his original email signalling that there might be a problem.

Anton only noted a difference "of this magnitude". The 23% number is based on the nps numbers reported by Komodo in the Superfinal and in Stage 2. It is not difficult to verify that the difference in nps is substantial.

That Komodo 1970 seemed to be about 20% slower had already been noticed by others:
http://talkchess.com/forum/viewtopic.php?t=65795

Obviously there is still a lot of room for discussion on whether the exact number is 22% or 23% or maybe 22.78%.