future of top engines:how much more elo?

[jp]

The maximum possible elo on the human scale has been estimated by an IM computer scientist to be 3500-3600, so if we assume that's reliable the maximum possible improvement of engines depends on how the engine elo scales are related to the human scale.

[Guenther]

The maximum possible elo on the human scale has been estimated by an IM computer scientist to be 3500-3600, so if we assume that's reliable the maximum possible improvement of engines depends on how the engine elo scales are related to the human scale.

Any sources for that claim? I say it is not even unreliable, but plain fantasy, except with a quantum jump in Human brain functions.
(weren't seen in the last couple of hundreds years, and please don't start about a new kind of hominidae in may be 30.000 years from now.)

Some 'so called' historical rating lists give already players > 100 years ago, 2800+ elo and players 150 years ago, over 2700 elo.
I also doubt those numbers compared to current elo ratings, but still we have already 50 years of pro chess and 100 years of modern chess
and there is no indication for any skyrocking for another +650/700 elo in any reasonable future.

Also, there are plenty of 'IM computer scientists', chance is big there are some whackos among them - already know a couple of them...

[jp]

I say it is not even unreliable, but plain fantasy, except with a quantum jump in Human brain functions.
(weren't seen in the last couple of hundreds years, and please don't start about a new kind of hominidae in may be 30.000 years from now.)

I think you are misunderstanding. The claim is: that is the upper bound on the elo possible given the game of chess and the elo system. It's not claiming a human or computer can reach that upper bound. It's claiming that perfect play will not get you an elo greater than 3500-3600 on the elo scale FIDE uses for humans (unless you try to game the system somehow).


Geddit now?

[Ovyron]

That's interesting, because my own estimation is that 3600 elo will be reached within 3 years, so what will happen then? Will elo break or will improvement stop?

[Guenther]

I say it is not even unreliable, but plain fantasy, except with a quantum jump in Human brain functions.
(weren't seen in the last couple of hundreds years, and please don't start about a new kind of hominidae in may be 30.000 years from now.)

I think you are misunderstanding. The claim is: that is the upper bound on the elo possible given the game of chess and the elo system. It's not claiming a human or computer can reach that upper bound. It's claiming that perfect play will not get you an elo greater than 3500-3600 on the elo scale FIDE uses for humans (unless you try to game the system somehow).


Geddit now?

There is no need to add 'Human scale', if you talk about Elo.
(Anyhow, we calculate ratings practically the same way, no matter, if Humans or programs.)

If you wanted to talk about an estimate for an upper bound of chess ratings, we (most participants in this thread) had already agreed on around 4000-4500 Elo. (we already have around 3300-3500 now, so the estimate of 3500-3600 is obsolete - even Shredder 7-9 already reached 2650-2800 perf on ordinary hardware in 2003-2005, check the diff to current programs 15 years later and add improved hardware too)

[jp]

There is no need to add 'Human scale', if you talk about Elo.
(Anyhow, we calculate ratings practically the same way, no matter, if Humans or programs.)

Clearly, they are not calibrated the same (in the sense of 1500=1500; 3000 FIDE, etc. =3000 ccrl, etc.), and there has never been a definitive determination of what the relative offset or spread is.

That's the whole point. If you do calibrate the computer ratings to FIDE human elo, 4500 is highly likely going to be way too high an estimate for the ceiling. It's very different for games where there's no draw.

[Guenther]

There is no need to add 'Human scale', if you talk about Elo.
(Anyhow, we calculate ratings practically the same way, no matter, if Humans or programs.)

Clearly, they are not calibrated the same (in the sense of 1500=1500, 3000 FIDE, etc. =3000 ccrl, etc.), and there has never been agreement or certainty on this bulletin board or outside it about what the relative offset or spread is.

That's why I said 'calculate', ofc it just needs to be calibrated differently, but I left that out intentionally, because it was too simple for mentioning it here...
Whatever, this leads to nothing as expected.

[jp]

Whatever, this leads to nothing as expected.

If you're happy to have scales that don't tell us anything meaningful about the relation of different sets to each other, then you can keep being happy. But most people here are not happy e.g. (an extreme example) with the Leela self-elo graphs that just shoot off the scale towards infinity. They're not happy, even though those self-elo numbers obviously mean something.

[Ovyron]

Isn't this a problem with the elo system itself, though? Suppose Magnus Carlsen is rated 2800 and there's a chess engine that beats him 100% of the time, what rating do you give it?

I guess a better system would take a look at the actual chess moves played in a game, and tell us how those moves would have performed against a different opponent. Then engine A can beat MC 100% of the time, and engine B does too, but the new system says the former is 3600 rated and the latter is 4500 rated (or something), by checking the moves.

Since producing moves is much faster than producing chess games, a system like this could tell us more accurate rating than elo with less effort.

[dragontamer5788]

Isn't this a problem with the elo system itself, though? Suppose Magnus Carlsen is rated 2800 and there's a chess engine that beats him 100% of the time, what rating do you give it?

Under the original Bradley Terry system that Elo is based on, you assume that the "losing" player has one half-win out of the whole bunch.
So if you played 50 games and the computer won 50 times out of 50, you calculate a score based off of 50/50.5. That is equivalent to 100-to-1, or roughly +800 Elo or so. If it were instead 5000 games and the computer won 5000 games out of 5000, then the score is off of 5000 / 5000.5, or 10,000-to-1 (roughly 1600 Elo).

Under a system I explored (unpublished), where you create acyclic graphs between players and assign the hypothetical computer "infinite" more score than Magnus Carlsen. You can only create a Bradley-Terry (or Elo) score between players who have a loss or draw against each other. You compare players with a Topological sort. All cycles are resolved with Bradley Terry (or Elo), while acyclic / trees are sorted by a Topological sort of some kind.

The fundamental assumption of Bradley Terry (or Elo) is that the win/loss graph is a statistic, and therefore has a chance to be wrong. If the system never observes a loss, then you have a "division by zero" situation. The actual division by zero can be avoided by topological sorts + graph theory. But the original papers talking about the system has the +0.5 methodology to the loser, which seems to work well in practice.