Can current top engines still be beaten?

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Rebel, chrisw

OneTrickPony
Posts: 157
Joined: Tue Apr 30, 2013 1:29 am

Can current top engines still be beaten?

Post by OneTrickPony »

I don't have experience playing correspondence or very long time controls engine assisted matches. I do a lot of opening analysis and while I believe there is edge for white everywhere in human terms it all end up in endgames which should be easy to hold for engines.
I would love to get some perspective on it from more experience engine users.
The conditions are:

-SF dev, 7man tablebases
-top of the line hardware available today (let's say 128 EPYC cores)
-very long time controls (something like 3 hours + 1 minute increment)
-all games start from starting position
-top of the line opening book geared toward most solid and drawish openings (Berlin, Ragozin, whatever is considered the most solid vs Catalan)
-100 games, SF plays black in every of them

The question is: will there ever be an engine that wins 1 game vs that setup? what about 5 games?
My question is motivated by my recent interest in opening theory of Ragozin. Unless there are some very unexpected resources (that is moves engines of today consider to be completely harmless) white can choose between the following endgames:
1)2rr2k1/pp1n1pp1/2p4p/3p1b2/P2P4/2P1P3/3NBPPP/R3K2R w KQ - 1 16
2)r4rk1/pp1b1pp1/5n1p/2Pp4/3N4/2P1PP2/P5PP/3RKB1R b K - 0 15
3)r1bq2k1/pp3ppp/8/8/2B2P2/4r3/PPQ3PP/R4RK1 w - - 0 18

All of those are considered the state of the theory right now (although black maybe have a better option if white goes for 3rd one). They are very playable for humans and was won by elite GMs in classical or rapid games vs other elite GMs. Still, for an engine those are easy positions which not much material left. I just don't believe state of the art engine of of today on top of the line hardware losses any of those to a godlike engine of tomorrow. If I am right it means somewhere before there need to exists moves which while considered harmless today are somehow very good. This I think is very unlikely because top human players (like Anand for example) devote a lot of resources to comb through the opening theory with very serious hardware and many people to help and they are still unable to come up with anything better for white not only in Ragozin but in other major openings as well. It seems correspondence chess at top level is dead as well and the only fun left is if both players collaborate to play a position outside of the very mainlines (which also happens in engine matches/tournaments).
User avatar
Ovyron
Posts: 4556
Joined: Tue Jul 03, 2007 4:30 am

Re: Can current top engines still be beaten?

Post by Ovyron »

OneTrickPony wrote: Sat Jan 18, 2020 10:40 am The question is: will there ever be an engine that wins 1 game vs that setup?
Oh yes, Stockfish is nowhere near perfection, I could say that a top correspondence chess player using 6 days per move could find a string of moves that'd beat your conditions, so if some engine of the future is able to find those moves then Stockfish is going to be defeated eventually. But that'd be overkill, the rule of ELO per doubling doesn't break, so all you need to do is match Stockfish dev on that machine with your conditions against Stockfish dev on that machine with your conditions but 6 hours + 2 minute increment time control. Then you will not only see your conditions losing a single game, it'll lose many, enough to follow the ELO per doubling curve.

These kinds of discussions aren't new, back in 2010 people would ask similar questions about Rybka 3 on a 256-Cluster, because the strongest hardware of the day was much faster than anything else seen before, and the engine was way ahead of anything else, producing the strongest chess ever seen. But even by 2015 R3 on that Cluster would look like a joke, let's not talk about the destruction that would happen today.

And it will happen again, and eventually Stockfish dev on your conditions will look like a joke. This is due to the exponential nature of chess, and the horizon effect, the trees get so dense that the relevant variation that wins for the opponent is outside of the horizon, as engines become better at exploiting the horizon of today's engines, you will see your conditions resemble more and more how blitz on old hardware looks like today, a level that used to require the strongest hardware and slow time controls of years ago.
OneTrickPony
Posts: 157
Joined: Tue Apr 30, 2013 1:29 am

Re: Can current top engines still be beaten?

Post by OneTrickPony »

the rule of ELO per doubling doesn't break
Not sure about that. All the engine matches are played using opening books to ensure variety and interesting positions. I am talking about "build the wall" approach with black and the wall building is quite well analyzed by now with literally thousands of core years invested into analyzing the main openings for black.
For example if you run self-play matches with SF on big hardware vs much worse hardware starting from some mainline positions in Petroff it will result in 100% draw ratio. I am not saying Petroff is the best way to build the wall vs 1.e4 but at least some branches of it are completely dead if it's the engine playing. Same goes for many other lines.
OneTrickPony
Posts: 157
Joined: Tue Apr 30, 2013 1:29 am

Re: Can current top engines still be beaten?

Post by OneTrickPony »

Can't edit anymore but recent example is AlphaZero vs Stockfish match. They had competent chess programmers with a lot of experience in engine testing and programming on the team. They knew opening book is supposed to be provided for SF for a match to be fair and the only reason official SF doesn't ship with a vast book is that accepted practice is that books are provided separately. If the SF's team goal was to ship the strongest chess playing entity possible they would include opening book. I think they either knew straight away or run the tests and wouldn't win any games if SF played with a good book. As it was all published wins come from blunders which would be covered by the book (or the lines wouldn't be entered at all).
This was ancient (by today's standard) version of SF running on not so impressive hardware as well.
User avatar
Ovyron
Posts: 4556
Joined: Tue Jul 03, 2007 4:30 am

Re: Can current top engines still be beaten?

Post by Ovyron »

The wall doesn't exist, if it did every correspondence chess player could make use of it when they're black and avoid losing.

What is happening is that Stockfish dev always lags behind the level of correspondence games, you may be amazed by the level that it has that you're proposing, but top correspondence players were displaying this level years ago, and there's still games being decided at this level. Others will point out to seemingly "draw death" and tournaments where you can't find a single loss or blunder for the players, but that's because those players are happy with a draw and they match their long known lines against each other and with nothing new to test them draw is inevitable.

But I hold that if someone with access to software and hardware from 2025 that had studied the weaknesses of today came and played with them, they could be defeated. It's just that the top always has access to what is state of the art today, so they can't surprise each other. But since those players that can be defeated are still years ahead of Stockfish dev (in a few years Stockfish at 3 hours/game will play like they do at 50 days per 10 moves repeating time control), then infallibility is still far away.

Note that I used to think like you, that I could be able to build "a wall" and avoid losing a game if I played for a draw, with the Petroff, even. I challenged this guy and played my best efforts for a wall, but he defeated me. That white line that he played still stands today as great, and makes it so the Petroff isn't even the best attempt for a wall after all.

You could see that game and laugh at the software and hardware used at the time, but people from the future are going to see what we use today and laugh as well.

----------------

As for your last message, you overestimate opening books. There are people that can get their hands on any book, see what lines it plays, defeat the weak ones, avoid the strong ones, and after a few weeks come up with a "counterbook" that easily defeats that one. If people had found a wall then this wouldn't happen, and someone could come up with a "drawbook" that specializes in not taking risks and trying to ensure that a draw happens. But nobody has been able to, and 2019 was the year of the busted variations (try downloading a book released from before July and try to contend, and you'll play into holes where the engine shows some -1.50 disadvantage for you!)

I know what you're saying, and I believe that it's possible, but we're not there yet. Before we're close we'll see engine development plateau, like, get alarmed if after 1 year the best engine could only improve 10 elo, that means it's near infallibility and it's hard to find lines where it loses. But there's still plenty of them, and 3 hour/game of today will be a 10 minute game from the future.
OneTrickPony
Posts: 157
Joined: Tue Apr 30, 2013 1:29 am

Re: Can current top engines still be beaten?

Post by OneTrickPony »

The wall doesn't exist, if it did every correspondence chess player could make use of it when they're black and avoid losing.
That's exactly what the are doing and that's why all the games among good correspondence players end in a draw unless someone feels adventurous.
The reason not every corr player does that is that they are not aware of the wall and/or don't have access to serious hardware.
and there's still games being decided at this level
Last time I checked correspondence World Championship is all draws among the top guys with maybe some outsiders being beaten when venturing into King's Indian or other speculative (read: crappy) opening. Can you point me to a top correspondence game in main line Berlin or Ragozin (or any other super solid variation) which was decisive?
Maybe I am not aware of the scene enough and the real top players don't care about World Championship and they play games in places I am not aware of. I was following Rybka forum for a while and while many interesting games were played there they were only interesting because people were adventurous in the openings.
with the Petroff, even. I challenged this guy
I am sorry to be blunt but that opening at the very least qualifies as being adventurous with black.
You got very complicated position with unbalanced pawn structure. In mainline Petroff you get this:

rn1q1rk1/pp3ppp/2pb4/5b2/2BP4/2P2N2/P4PPP/R1BQ1RK1 w - - 1 12

or similar if you know what you are doing.
(not talking about Nc3 variation, I've never looked at it in detail as Petroff doesn't seem to be the best wall out of available walls vs 1.e4)

Things weren't that different in Petroff theory in 2014. You just played for complications with black and got punished.
I know what you're saying, and I believe that it's possible, but we're not there yet. Before we're close we'll see engine development plateau
I don't think engines are close to perfect in general. I think they are close to (or even perfect) when it comes to defending initial position playing for a draw having access to theory of today and top notch hardware.
giovanni
Posts: 142
Joined: Wed Jul 08, 2015 12:30 pm

Re: Can current top engines still be beaten?

Post by giovanni »

OneTrickPony wrote: Sat Jan 18, 2020 10:40 am I don't have experience playing correspondence or very long time controls engine assisted matches. I do a lot of opening analysis and while I believe there is edge for white everywhere in human terms it all end up in endgames which should be easy to hold for engines.
I would love to get some perspective on it from more experience engine users.
The conditions are:

-SF dev, 7man tablebases
-top of the line hardware available today (let's say 128 EPYC cores)
-very long time controls (something like 3 hours + 1 minute increment)
-all games start from starting position
-top of the line opening book geared toward most solid and drawish openings (Berlin, Ragozin, whatever is considered the most solid vs Catalan)
-100 games, SF plays black in every of them

The question is: will there ever be an engine that wins 1 game vs that setup? what about 5 games?
My question is motivated by my recent interest in opening theory of Ragozin. Unless there are some very unexpected resources (that is moves engines of today consider to be completely harmless) white can choose between the following endgames:
1)2rr2k1/pp1n1pp1/2p4p/3p1b2/P2P4/2P1P3/3NBPPP/R3K2R w KQ - 1 16
2)r4rk1/pp1b1pp1/5n1p/2Pp4/3N4/2P1PP2/P5PP/3RKB1R b K - 0 15
3)r1bq2k1/pp3ppp/8/8/2B2P2/4r3/PPQ3PP/R4RK1 w - - 0 18

All of those are considered the state of the theory right now (although black maybe have a better option if white goes for 3rd one). They are very playable for humans and was won by elite GMs in classical or rapid games vs other elite GMs. Still, for an engine those are easy positions which not much material left. I just don't believe state of the art engine of of today on top of the line hardware losses any of those to a godlike engine of tomorrow. If I am right it means somewhere before there need to exists moves which while considered harmless today are somehow very good. This I think is very unlikely because top human players (like Anand for example) devote a lot of resources to comb through the opening theory with very serious hardware and many people to help and they are still unable to come up with anything better for white not only in Ragozin but in other major openings as well. It seems correspondence chess at top level is dead as well and the only fun left is if both players collaborate to play a position outside of the very mainlines (which also happens in engine matches/tournaments).
but don't we already have Father? :)
OneTrickPony
Posts: 157
Joined: Tue Apr 30, 2013 1:29 am

Re: Can current top engines still be beaten?

Post by OneTrickPony »

I've taken a look at the recent World Championship:
https://www.iccf.com/event?id=66745

All decisive games resulted from wild gambles by black in the opening, The winner got a huge gift when someone played Mar Del Plata King's Indian vs him (very likely losing opening imo and certainly one you are doomed to long defense with no chances for anything vs competent people). He also won a game with black in which white gambled recklessly in the opening and then blundered in the middle game with a move that can be rejected with around 5 seconds of analysis on middle range hardware from 6 years ago. Apparently even in such friendly environment it's pretty much completely dead anyway.
I mean maybe wild gambling is optimal strategy for your chances to win round robin correspondence tournament but it has nothing to do with trying not to lose at chess with black.
todd
Posts: 56
Joined: Thu Apr 19, 2018 9:09 pm

Re: Can current top engines still be beaten?

Post by todd »

OneTrickPony,

I came to the same conclusions about all the same lines and for pretty much the same reasons.

A player with sufficient familiarity with opening theory as black can avoid getting caught in a slow, maneuvering position where either player might win a long game. There seems to be a solid and forcing line against everything that leads to an endgame with insufficient white advantage to win against current engine-level defense.

The nature of these endgames is usually such that black may continue with forcing play, and the positions that appear are not the sort that tend to have deep zugzwang wins that engines often miss.
Uri Blass
Posts: 10267
Joined: Thu Mar 09, 2006 12:37 am
Location: Tel-Aviv Israel

Re: Can current top engines still be beaten?

Post by Uri Blass »

I doubt if chess engines can be beaten today at correspondence chess.
Correspondence chess is already dying based on percentage of draws and I often see tournament with more than 90% draws and I suspect that there are wins only because of clerical errors or games when both players play for a win and unclear positions but every player could get a draw if that was the target.

In the last ICCF world championship there are 127 draws out of 136 games

https://www.iccf.com/event?id=66745

The following top correspondence tournament may be the first correspondence tournament to be finished with 100% draws(one game is still not finished).

https://www.iccf.com/event?id=68635

In the following tournament we had 98 draws out of 105 games and the winners did not have more than 1 game that they won.

https://www.iccf.com/event?id=70460


In the following strong tournament the winners had only one win amd we had 129 draws out of 136 games.


https://www.iccf.com/event?id=72244