Can current top engines still be beaten?

[Alayan]

Well, there is an easy way to verify this. If you're patient.

Today, stash your software. Take your top engine, take your top book and tune it to play the more drawish line possible as black against anything white might try.

Wait a few years.

Take the new top engine of that time, give it a book (or settings) that is meant to strive for complex positions that are hard to simplify, and match it up against your top engine. Set up hardware to your satisfaction (from your rules, nothing forbids giving a hardware advantage to the new one, or a longer TC. Run the match. Get your results.

I think you'll be surprised.

Otherwise, if one is confident in finding a way to force away the defending engine from current mainlines into complex positions, this could already be demonstrated today.

[Uri Blass]

The question is: will there ever be an engine that wins 1 game vs that setup?

Oh yes, Stockfish is nowhere near perfection, I could say that a top correspondence chess player using 6 days per move could find a string of moves that'd beat your conditions, so if some engine of the future is able to find those moves then Stockfish is going to be defeated eventually. But that'd be overkill, the rule of ELO per doubling doesn't break, so all you need to do is match Stockfish dev on that machine with your conditions against Stockfish dev on that machine with your conditions but 6 hours + 2 minute increment time control. Then you will not only see your conditions losing a single game, it'll lose many, enough to follow the ELO per doubling curve.

These kinds of discussions aren't new, back in 2010 people would ask similar questions about Rybka 3 on a 256-Cluster, because the strongest hardware of the day was much faster than anything else seen before, and the engine was way ahead of anything else, producing the strongest chess ever seen. But even by 2015 R3 on that Cluster would look like a joke, let's not talk about the destruction that would happen today.

And it will happen again, and eventually Stockfish dev on your conditions will look like a joke. This is due to the exponential nature of chess, and the horizon effect, the trees get so dense that the relevant variation that wins for the opponent is outside of the horizon, as engines become better at exploiting the horizon of today's engines, you will see your conditions resemble more and more how blitz on old hardware looks like today, a level that used to require the strongest hardware and slow time controls of years ago.

In 2010 ICCF had less draws relative to the situation today.

Stockfish is not perfect but I believe that today it is strong enough not to lose the opening position at correspondence time control.

There are positions that stockfish play a losing move at correspondence time control and not a drawing move but I believe that you practically cannot get them in games.

I am even not sure if a book is needed for stockfish to draw every game.
Maybe stockfish11 with at least 100,000,000,000 nodes per move(stockfish search every move from clear hash) and no book is enough for that purpose.

[carldaman]

I doubt if chess engines can be beaten today at correspondence chess.
Correspondence chess is already dying based on percentage of draws and I often see tournament with more than 90% draws and I suspect that there are wins only because of clerical errors or games when both players play for a win and unclear positions but every player could get a draw if that was the target.

In the last ICCF world championship there are 127 draws out of 136 games

https://www.iccf.com/event?id=66745

The following top correspondence tournament may be the first correspondence tournament to be finished with 100% draws(one game is still not finished).

https://www.iccf.com/event?id=68635

In the following tournament we had 98 draws out of 105 games and the winners did not have more than 1 game that they won.

https://www.iccf.com/event?id=70460


In the following strong tournament the winners had only one win amd we had 129 draws out of 136 games.


https://www.iccf.com/event?id=72244

A lot could still depend on the time controls, even in correspondence games.

I'm tempted to believe that unaided top engines, running on some baseline hardware with reasonably limited time controls, can still be defeated at corr chess. A strong, skilled human can guide the play [while analyzing] and gain a huge depth advantage in critical lines. Not to mention the additional resources available to the human - databases, books, etc.

[Ovyron]

I'm tempted to believe that unaided top engines, running on some baseline hardware with reasonably limited time controls, can still be defeated at corr chess. A strong, skilled human can guide the play [while analyzing] and gain a huge depth advantage in critical lines. Not to mention the additional resources available to the human - databases, books, etc.

Yes! Specially if you know who you are facing! Sometimes the moves that would be the best against an unaided engine aren't so against a skilled human, and very different than against a unskilled human (against this last tier what you want is those adventurous openings that exist for a reason, you don't want the unskilled to just copy and paste 40 moves from a book to draw you.)

What is missing is a big incentive for people to try. Say, first, the ICCF offers $100000 to an skilled bookmaker that can create the most drawish book imaginable, that would play against others in a private tournament, where whoever builds a book that avoids losing with both white and black gets the bounty.

Then, offer a $1000000 prize to whoever can defeat Stockfish using this private book and top-of-the-line hardware at 5 days per move time control against the challenger at 50 days for every 10 moves repeating. "The challenger" can be a team of top players that would use this time to try to find a line that'd defeat this Stockfish.

With this big incentive I'd expect "the challenger" to be able to win their first game and claim and split the prize with the team, by just driving the game into a position where Stockfish just thinks it's a draw, but it's wrong (with the aid of Leela?) If those moves exist then I don't see why a solo skilled human player can't find them.

[OneTrickPony]

Assuming chess is already softly solved for practical purposes (you can draw with black assuming you have enough resources and time for a move within reasonable realms for both) it seems to me the most interesting project is to find ways which are objectively the best tries to win.
There is a concept of Kolmogorov complexity (https://en.wikipedia.org/wiki/Kolmogorov_complexity). Applying this to chess means defining complexity of a position by the minimum size of a chess engine which is able to defend it. Simplifying a bit we could define it as minimum nodes/move of current Stockfish (or Leela or whatever) that is able not to lose the position vs any entity.

With that in mind we could try finding opening strategy that steers the game into the most complex positions (according to the definition above). One very interesting question would be: what is an opening that guarantees the most complex position with black vs 1.e4.
I think measuring (or approximating) complexity of positions is something that will be of great interest for practical players in the future.
Even if we are able to say chess is "mathematically" a draw we can still define (mathematically!) which path is the most difficult one.

[Alayan]

I think measuring (or approximating) complexity of positions is something that will be of great interest for practical players in the future.
Even if we are able to say chess is "mathematically" a draw we can still define (mathematically!) which path is the most difficult one.

I completely agree.

In its way, Stockfish's contempt is a crude way of doing exactly this. A more sophisticated system would also favor imbalance, and any other position characteristics that make finding the best moves harder.

[OneTrickPony]

In its way, Stockfish's contempt is a crude way of doing exactly this. A more sophisticated system would also favor imbalance, and any other position characteristics that make finding the best moves harder.

If you have enough position - complexity data then you can also train NN to estimate complexity of a new position

[Ozymandias]

I think measuring (or approximating) complexity of positions is something that will be of great interest for practical players in the future.

The problem is that even the most difficult path can be drawn by any seriously prepared player. I said it three years ago, in (Freestyle) tournaments, the most important player isn't the one who ends up first, it's the one who "gives away" key points.

[Uri Blass]

I'm tempted to believe that unaided top engines, running on some baseline hardware with reasonably limited time controls, can still be defeated at corr chess. A strong, skilled human can guide the play [while analyzing] and gain a huge depth advantage in critical lines. Not to mention the additional resources available to the human - databases, books, etc.

Yes! Specially if you know who you are facing! Sometimes the moves that would be the best against an unaided engine aren't so against a skilled human, and very different than against a unskilled human (against this last tier what you want is those adventurous openings that exist for a reason, you don't want the unskilled to just copy and paste 40 moves from a book to draw you.)

What is missing is a big incentive for people to try. Say, first, the ICCF offers $100000 to an skilled bookmaker that can create the most drawish book imaginable, that would play against others in a private tournament, where whoever builds a book that avoids losing with both white and black gets the bounty.

Then, offer a $1000000 prize to whoever can defeat Stockfish using this private book and top-of-the-line hardware at 5 days per move time control against the challenger at 50 days for every 10 moves repeating. "The challenger" can be a team of top players that would use this time to try to find a line that'd defeat this Stockfish.

With this big incentive I'd expect "the challenger" to be able to win their first game and claim and split the prize with the team, by just driving the game into a position where Stockfish just thinks it's a draw, but it's wrong (with the aid of Leela?) If those moves exist then I don't see why a solo skilled human player can't find them.

I do not expect the challanger to be able to win their first game when the book is private.

I even do not except the challanger to be able to win their first game against stockfish with the empty book when stockfish use 5 days per move and good hardware.
Note that stockfish with many cores is not determinsitic so you have no way to find a single winning line that you can use to win against it.

I think that a more interesting challange can be to beat stockfish with a single thread at X nodes per move for maximal X that is possible.

Give people one year to prepare a win.
Stockfish is deterministic because it use only one thread and play every move from clear hash so it is possible to reproduce the games.

Stockfish get constant number of nodes per move.

If stockfish has some bug and crash in the middle of the search then the move of it is considered to be the last move that it suggested before the crash.

The winner is the person who produce a win against stockfish with the maximal X nodes per move and 1024 mbytes hash get a lot of money for it.

What is going to be X?

[EroSennin]

In the following tournament we had 98 draws out of 105 games and the winners did not have more than 1 game that they won.

https://www.iccf.com/event?id=70460

What about this game from the tournament? https://www.iccf.com/game?id=974017
Black played solidly but lost. I haven't analysed the game with an engine, but I would assume it would useless to do it anyway without a few days of work.