future of top engines:how much more elo?

[Sean Evans]

I think the future is likely to hold some real surprises. Remember, it was only 50 years since all this started. I think that in the next 20 years, we may see ratings crack the 4,000 barrier. No, I don't believe chess will ever be "solved" but we may see computers playing fantastic and even artful chess that all players will be completely awed over.

I am still very optimistic about the future of chess engines and computer chess.

Thank you David, well said. One thought QUANTUM COMPUTER, this could change everything!

Cordially,

Sean

[Uri Blass]

after the latest stf6 komodo 8 etc, i wonder how much elo more can be achieve via programming,on the same hardware of course.
100,200,300 elo more? it could be a poll but anyway just a "food of thought"
my personal feeling not more than 100 elo. more? it would be a miracle
dont forget pls on the same hardware! lets say a medium pc 2 core etc..

Question is not clear.

Elo is dependent on time control so I guess it is possible to earn less at longer time control when the programs today play more often the best moves.

If we think about ccrl or cegt conditions then
my personal feeling is that more than 200 elo is possible(and the best software of 2025 is going to be more than 200 elo better than stockfish6 on the same hardware that ccrl or cegt use and the same time controls that they use).

[michiguel]

after the latest stf6 komodo 8 etc, i wonder how much elo more can be achieve via programming,on the same hardware of course.
100,200,300 elo more? it could be a poll but anyway just a "food of thought"
my personal feeling not more than 100 elo. more? it would be a miracle
dont forget pls on the same hardware! lets say a medium pc 2 core etc..

Assuming, after many extrapolations, that perfect engine has ~4,500 Elo points, till chess is solved, 600 points will be gained by software, 600 by hardware, so my bet would be 600.

At that point, Elo won't be an accurate measure of strength anymore because of the draw rates. Maybe Wilos will have to be used extending the scale way much longer.

Miguel

[Uri Blass]

after the latest stf6 komodo 8 etc, i wonder how much elo more can be achieve via programming,on the same hardware of course.
100,200,300 elo more? it could be a poll but anyway just a "food of thought"
my personal feeling not more than 100 elo. more? it would be a miracle
dont forget pls on the same hardware! lets say a medium pc 2 core etc..

Assuming, after many extrapolations, that perfect engine has ~4,500 Elo points, till chess is solved, 600 points will be gained by software, 600 by hardware, so my bet would be 600.

At that point, Elo won't be an accurate measure of strength anymore because of the draw rates. Maybe Wilos will have to be used extending the scale way much longer.

Miguel

I think that elo can never be accurate and even today it is dependent on the pool of players.

It may be interesting if somebody try the following experiment:

1)Start with the random player define the elo of it as 0.

2)Do a match of 100,000 games between the random player and the player who play random moves only in 99% of the cases and in 1% engine X's move at some small depth like depth 3.
calculate rating for the new engine rating based on the match.

3)You have something like

random100%=0
random99%=100(random99% may be 99% random moves and 1% gaviota's move at depth 3)

Continue in the same way
and calculate rating for random98% based on a match of 100,000 games against random99% and continue in this way until you get to random0%
that is the rating of your engine at depth 3.

Do the same experiement to calculate rating of your engine at depth 10.

My intuition is that the difference between depth 3 and depth 10 will be clearly smaller than the real rating difference between them with a normal pool of players.

[Sean Evans]

There's room for improvement everywhere, and we will continue to see it so long as a few keep carrying the ball forward.

Unlike Hyatt and Crafty who dropped the ball years ago

[Laskos]

after the latest stf6 komodo 8 etc, i wonder how much elo more can be achieve via programming,on the same hardware of course.
100,200,300 elo more? it could be a poll but anyway just a "food of thought"
my personal feeling not more than 100 elo. more? it would be a miracle
dont forget pls on the same hardware! lets say a medium pc 2 core etc..

Assuming, after many extrapolations, that perfect engine has ~4,500 Elo points, till chess is solved, 600 points will be gained by software, 600 by hardware, so my bet would be 600.

At that point, Elo won't be an accurate measure of strength anymore because of the draw rates. Maybe Wilos will have to be used extending the scale way much longer.

Miguel

I believe a close to perfect engine can score 97% (+600 Elo points) against top engines of today, on equal hardware. Many draws today are due to "wrong" draws, games which the perfect engine converts to wins. Many moves in draws today are probably not perfect moves, and it's enough for perfect engine to spot one such wrong move to convert. The problem of too many perfect draws will occur between close to perfect engines (like in checkers), not with regard to today's top engines, which probably blunder every odd move.

Also, the perfect engines should "know" they play a non-perfect engine, so the perfect engine doesn't go to a draw even if chess is draw.

[Laskos]

I think that elo can never be accurate and even today it is dependent on the pool of players.

It may be interesting if somebody try the following experiment:

1)Start with the random player define the elo of it as 0.

2)Do a match of 100,000 games between the random player and the player who play random moves only in 99% of the cases and in 1% engine X's move at some small depth like depth 3.
calculate rating for the new engine rating based on the match.

3)You have something like

random100%=0
random99%=100(random99% may be 99% random moves and 1% gaviota's move at depth 3)

Continue in the same way
and calculate rating for random98% based on a match of 100,000 games against random99% and continue in this way until you get to random0%
that is the rating of your engine at depth 3.

Do the same experiement to calculate rating of your engine at depth 10.

My intuition is that the difference between depth 3 and depth 10 will be clearly smaller than the real rating difference between them with a normal pool of players.

I did some Monte Carlo simulations assuming reasonable beta distributions of random mover inaccuracies, depth=3 inaccuracies and depth=10 inaccuracies. The beta distribution is useful to model the random variables limited to intervals of finite length, and works well in this case. My simulations are in steps of 5%, so "85%" means that 85% of moves are random, 15% are either depth=3 or depth=10. The results are here:

Code: Select all

Each 10000 simulations, depth=3

95% vs 100%   +114 Elo points
90% vs  95%   +116 Elo points
85% vs  90%   +118 Elo points
80% vs  85%   +117 Elo points
75% vs  80%   +121 Elo points 
70% vs  75%   +120 Elo points
65% vs  70%   +123 Elo points
60% vs  65%   +115 Elo points
55% vs  60%   +121 Elo points
50% vs  55%   +126 Elo points
45% vs  50%   +126 Elo points
40% vs  45%   +139 Elo points
35% vs  40%   +145 Elo points
30% vs  35%   +146 Elo points
25% vs  30%   +148 Elo points
20% vs  25%   +164 Elo points
15% vs  20%   +161 Elo points
10% vs  15%   +180 Elo points
 5% vs  10%   +179 Elo points
 0% vs   5%   +189 Elo points
_____________________________

Total     +2768 Elo points Gaviota depth=3 versus random engine 



Each 10000 simulations, depth=10

95% vs 100%   +124 Elo points
90% vs  95%   +131 Elo points
85% vs  90%   +126 Elo points
80% vs  85%   +128 Elo points
75% vs  80%   +136 Elo points 
70% vs  75%   +132 Elo points
65% vs  70%   +125 Elo points
60% vs  65%   +140 Elo points
55% vs  60%   +129 Elo points
50% vs  55%   +136 Elo points
45% vs  50%   +137 Elo points
40% vs  45%   +151 Elo points
35% vs  40%   +147 Elo points
30% vs  35%   +164 Elo points
25% vs  30%   +168 Elo points
20% vs  25%   +181 Elo points
15% vs  20%   +192 Elo points
10% vs  15%   +195 Elo points
 5% vs  10%   +196 Elo points
 0% vs   5%   +192 Elo points
_____________________________

Total     +3030 Elo points Gaviota depth=10 versus random engine

The difference with this pool of depth=10 compared to depth=3 is 262 Elo points. The real difference is at least 900 Elo points, confirming Uri's hypothesis of dependence on the pool of players.

[Zenmastur]

Regards,

Zenmastur

[Guenther]

Regards,

Zenmastur

Ok, so SF already made around 200 rating points in 4 years on software only. (Current SF Dev should be surely +20 better than SF10)
Uri predicted 10 years for this and Stavros did not even believe this would be possible at all.
Carldaman was not far away with 100-200 in 3-5 years. (still the guess was quite vague)

The question if the software mark to reach from that point in 2015 is around 600 still remains.
(not even starting to talk about NN+MCTS programs, as hardware/software improvement cannot be compared differentiated in a useful way
for that historical point in 2015 anyway)

Code: Select all

CCRL 40/4 Rating List — Single-CPU engines (Quote)

1	Stockfish 10 64-bit	3493	+13	−13	74.8%	−175.7	41.8%	2190
 	Stockfish 6 64-bit	3317	+12	−11	70.6%	−147.2	41.9%	2833

[Uri Blass]

Regards,

Zenmastur

Ok, so SF already made around 200 rating points in 4 years on software only. (Current SF Dev should be surely +20 better than SF10)
Uri predicted 10 years for this and Stavros did not even believe this would be possible at all.
Carldaman was not far away with 100-200 in 3-5 years. (still the guess was quite vague)

The question if the software mark to reach from that point in 2015 is around 600 still remains.
(not even starting to talk about NN+MCTS programs, as hardware/software improvement cannot be compared differentiated in a useful way
for that historical point in 2015 anyway)

Code: Select all

CCRL 40/4 Rating List — Single-CPU engines (Quote)

1	Stockfish 10 64-bit	3493	+13	−13	74.8%	−175.7	41.8%	2190
 	Stockfish 6 64-bit	3317	+12	−11	70.6%	−147.2	41.9%	2833

Here are the exact words from my post in this thread.

"the best software of 2025 is going to be more than 200 elo better than stockfish6 on the same hardware that ccrl or cegt use and the same time controls that they use"

I did not predict 10 years for 200 elo and I gave no prediction for exactly 200 elo.
200 elo was only a correct lower bound for 2025.

You can say correctly that the guess was quite vague but it was still better than the guess that 200 elo is impossible.