Something Hikaru Said

[syzygy]

In twelve official (45' + 15") knight odds games Komodo performed a bit below 2000 FIDE. So I would expect a 2500 FIDE GM to score about 95% taking knight odds from Komodo today, as that is what he should score against an opponent slightly below 2000 in normal chess.

95% is more than I would have dared to guess.

So Komodo easily outplays a 2500 Elo GM at normal chess, but only scores about 5% when giving knight odds.

Scoring 5% means that on average once every 10 games the GM makes a half-point error that is taken advantage of by Komodo. How many errors would the GM make that are not punished by Komodo? If that also happens once every 10 games (and I doubt that Komodo lets that happen so often), then perfect play by the engine would still score at most 10%.

The only thing that could help the engine is a special mode that looks for complications or traps that could make things difficult for a human GM of about 2500 Elo. I'm guessing such a special mode would help more than a million-fold increase in nps. But I don't think it will be enough to get the score anywhere near 50%.

IE perhaps find a line where at ply 40 there is only one correct response to win, or other such ideas that are not being done today.

With knight odds there will be billions of ways to win. Being able to access TBs does not change that. You can look for something, but if it does not exist you won't find it.

I'm sure something can be gained with tricks, but it will only help in a fraction of the games and therefore not suffice to reach 50%.

[syzygy]

Is 1 % the blunder ratio nowadays for SF / Komodo in games with adequate time on the clock?

Sorry, that question is too ambiguous. You would have to define "blunder".

Let's assume chess is a theoretical draw. How many wins is a human GM expected to score against Komodo in 100 games?

[JJJ]

Without handicap ? 1 on 5000 maybe.

[Uri Blass]

In twelve official (45' + 15") knight odds games Komodo performed a bit below 2000 FIDE. So I would expect a 2500 FIDE GM to score about 95% taking knight odds from Komodo today, as that is what he should score against an opponent slightly below 2000 in normal chess.

95% is more than I would have dared to guess.

So Komodo easily outplays a 2500 Elo GM at normal chess, but only scores about 5% when giving knight odds.

Scoring 5% means that on average once every 10 games the GM makes a half-point error that is taken advantage of by Komodo. How many errors would the GM make that are not punished by Komodo? If that also happens once every 10 games (and I doubt that Komodo lets that happen so often), then perfect play by the engine would still score at most 10%.

The only thing that could help the engine is a special mode that looks for complications or traps that could make things difficult for a human GM of about 2500 Elo. I'm guessing such a special mode would help more than a million-fold increase in nps. But I don't think it will be enough to get the score anywhere near 50%.

IE perhaps find a line where at ply 40 there is only one correct response to win, or other such ideas that are not being done today.

With knight odds there will be billions of ways to win. Being able to access TBs does not change that. You can look for something, but if it does not exist you won't find it.

I'm sure something can be gained with tricks, but it will only help in a fraction of the games and therefore not suffice to reach 50%.

95% is less than what I guess and I believe that the GM can score more than 95% against Komodo without a knight.

Uri

[bob]

In twelve official (45' + 15") knight odds games Komodo performed a bit below 2000 FIDE. So I would expect a 2500 FIDE GM to score about 95% taking knight odds from Komodo today, as that is what he should score against an opponent slightly below 2000 in normal chess.

95% is more than I would have dared to guess.

So Komodo easily outplays a 2500 Elo GM at normal chess, but only scores about 5% when giving knight odds.

Scoring 5% means that on average once every 10 games the GM makes a half-point error that is taken advantage of by Komodo. How many errors would the GM make that are not punished by Komodo? If that also happens once every 10 games (and I doubt that Komodo lets that happen so often), then perfect play by the engine would still score at most 10%.

The only thing that could help the engine is a special mode that looks for complications or traps that could make things difficult for a human GM of about 2500 Elo. I'm guessing such a special mode would help more than a million-fold increase in nps. But I don't think it will be enough to get the score anywhere near 50%.

IE perhaps find a line where at ply 40 there is only one correct response to win, or other such ideas that are not being done today.

With knight odds there will be billions of ways to win. Being able to access TBs does not change that. You can look for something, but if it does not exist you won't find it.

I'm sure something can be gained with tricks, but it will only help in a fraction of the games and therefore not suffice to reach 50%.

Don't make the mistake of assuming that Komodo of today is playing near-perfect chess. Also don't assume that komodo 100 or 1000 years in the future will only gain 200 Elo or so...

That's all unknown waters...

[Nordlandia]

Is 1 % the blunder ratio nowadays for SF / Komodo in games with adequate time on the clock?

Sorry, that question is too ambiguous. You would have to define "blunder".

I.e dubvious move/weak moves. Of course top engine don't blunders nowadays, although every now and then some weak moves are played.

This also depends on positional complexity. Closed positions do indeed increase the likelihood of weaker moves than open positions, - as seen from human perspective.

[syzygy]

Without handicap ? 1 on 5000 maybe.

Yes, without handicap.

So 1 in 5000 could be an estimate for the number of Komodo half-point errors.

Now consider a knight odds game where the human makes a half-point error but still beats Komodo. That means Komodo made a half-point error as well. So that should happen only 1 in 5000 games where the human made an error. That means improving Komodo further should only make a difference in those 1 in 5000 games (of the already few games where the human made an error). In other words, there is hardly anything to gain at knight odds by further improving the engine.

This is of course not a mathematical proof, but it is an argument that would seem to have some force.

Don't make the mistake of assuming that Komodo of today is playing near-perfect chess. Also don't assume that komodo 100 or 1000 years in the future will only gain 200 Elo or so...

I have extensively discussed why further improvements will hardly gain anything at knight odds.

[Laskos]

Without handicap ? 1 on 5000 maybe.

Yes, without handicap.

So 1 in 5000 could be an estimate for the number of Komodo half-point errors.

Not necessarily. Maybe Komodo is making 10 half- or full- point errors every 60 moves, so almost every game it's making such errors. Only that human is making 20 such errors every 60 moves. Almost every game too.

Now consider a knight odds game where the human makes a half-point error but still beats Komodo. That means Komodo made a half-point error as well. So that should happen only 1 in 5000 games where the human made an error. That means improving Komodo further should only make a difference in those 1 in 5000 games (of the already few games where the human made an error). In other words, there is hardly anything to gain at knight odds by further improving the engine.

This is of course not a mathematical proof, but it is an argument that would seem to have some force.

Don't make the mistake of assuming that Komodo of today is playing near-perfect chess. Also don't assume that komodo 100 or 1000 years in the future will only gain 200 Elo or so...

I have extensively discussed why further improvements will hardly gain anything at knight odds.

[syzygy]

Without handicap ? 1 on 5000 maybe.

Yes, without handicap.

So 1 in 5000 could be an estimate for the number of Komodo half-point errors.

Not necessarily. Maybe Komodo is making 10 half- or full- point errors every 60 moves, so almost every game it's making such errors. Only that human is making 20 such errors every 60 moves. Almost every game too.

Well, you're right that it is not correct to equate the Komodo half-point error rate to the win rate of a human GM against Komodo without handicap. Clearly the human will often not be able to convert such an error into a win.

Still the reasoning seems to be valid.

Instead of referring to 1 in 5000 as the Komodo half-point error rate, I should have called it the rate at which Komodo loses theoretically drawn positions against a 2500 Elo GM. (This is still a tricky generalisation, of course. There are huge differences between "theoretically drawn positions" once imperfect play of engines and GMs is involved.)

Let's take X for the fraction of knight odds games that an 2500 Elo GM plays flawlessly (i.e. keeping a winning position from start to end).

The fraction of knight odds games where Komodo reaches a theoretically drawn (or winning) position is then 1-X. Of those games, the human might still win 1 in 5000.

So the fraction of games won by the human is X + (1-X) * (1/5000).

Taking Larry's numbers, we get X + (1-X) * (1/5000) = 0.90. (The games that are not won should be mostly draws, so the score is indeed 95%.)

This gives X = 0.89998.

Now replace Komodo by an engine that plays perfect chess. The fraction of games won by the 2500 Elo GM is now "only" X. So the 2500 Elo GM will "only" score 94.998% or so.

This does not take into account the possibility that a stronger engine might make it harder for the human to play a perfect knight odds game. But I fail to see how e.g. speeding up Komodo by a factor of 1 mlllion would make it more difficult. Special anti-human tricks on the other hand should be able to make a (limited) difference.

[bob]

Without handicap ? 1 on 5000 maybe.

Yes, without handicap.

So 1 in 5000 could be an estimate for the number of Komodo half-point errors.

Now consider a knight odds game where the human makes a half-point error but still beats Komodo. That means Komodo made a half-point error as well. So that should happen only 1 in 5000 games where the human made an error. That means improving Komodo further should only make a difference in those 1 in 5000 games (of the already few games where the human made an error). In other words, there is hardly anything to gain at knight odds by further improving the engine.

This is of course not a mathematical proof, but it is an argument that would seem to have some force.

Don't make the mistake of assuming that Komodo of today is playing near-perfect chess. Also don't assume that komodo 100 or 1000 years in the future will only gain 200 Elo or so...

I have extensively discussed why further improvements will hardly gain anything at knight odds.

ALL based on current Komodo vs GM results....