For how long will Stockfish be competitive versus the best NN ?

[drewdrew]

I think this position is a better test and I would certainly like to see your analysis of it. I originally thought this would be a “hard” test, but now I'm not so sure how hard it is. I think “most” (meaning 50% or more) ICCF 2200 players can/would solve this if it occurred in one of their games.

[d]4r3/5pk1/1q1r1p1p/1p1Pn2Q/1Pp4P/6P1/5PB1/R3R1K1 b - -

Jdart posted this position and Dann Corbit analyzed it to depth 74 and concluded it's probably a draw. So, much for big hardware and unassisted analysis. I analyzed it and concluded black wins. If a position like this occurred in a CC and you were playing black could you win it?

I'm very interested in seeing your analysis of this position.

Regards,

Zenmastur

Can you give some idea of what "a moderately large number" might be? An unassisted Stockfish-dev search yielded 2.91 in favor of Black.

Greater than +4 for black.

Unassisted Stockfish-dev (old-ish version):
info depth 72 seldepth 111 multipv 1 score cp 433 nodes 8708473100400 nps 374666434 hashfull 953 tbhits 21101722591 time 23243270

Using 7-man endgame tables?

No, only 6-men syzygy.

I'm surprised! I tried this and it takes way too long better to use 7-man for a position like this.

I'm also surprised, but maybe drewdrew got some benefit from having 384 threads and 128 GB hash.

My unassisted search (using 6-man tables) reached depth 89 but the eval was only -2.91. Following that with a bit of forward/backward searching, I got the root evaluation to -3.38. And then I gave up.

Possibly a lucky run. SF's eval was stuck at -2.00 from depth 40-ish to depth 70 where it suddenly started failing high.

[Ovyron]

But, I found a line specifically to avoid this position and ones like it. So, I'm pretty sure this position ( the original position posted by JDart) is won for black.

The question is, would you have found *the winning moves* if you had slower hardware?

What would be very interesting to see from you, if you can't find some winning move at low depth, is that position where you find the winning move thanks to your big hardware, and you don't say what's the move, to get and see if big hardware would have made a difference.

Because the difference in score is clear, I may not be able to get it past -2.50 (without a game being played; in a game every time my opponent plays a move the positions I have to check shrink exponentially), but if I don't miss a winning move, the score doesn't matter at all.

[Zenmastur]

But, I found a line specifically to avoid this position and ones like it. So, I'm pretty sure this position ( the original position posted by JDart) is won for black.

The question is, would you have found *the winning moves* if you had slower hardware?

The whole point to having fast hardware, besides the fact that it saves time, is that I don't have to care what would happen if I had slow hardware. It becomes a complete non-issue.

What would be very interesting to see from you, if you can't find some winning move at low depth, is that position where you find the winning move thanks to your big hardware, and you don't say what's the move, to get and see if big hardware would have made a difference.

The winning move isn't hidden. f5 is pretty easy to find. Knowing what should happen after f5 is played is key to winning as many of these moves aren't obvious even after extensive analysis. e.g. I ran into one position in which white had a drawing line. There was a non-obvious ( i.e. I had searched with SF to a depth of 72/109 and it didn't find the winning move) black move 20 plies prior that, after considerable maneuvering, lead to a 7-man position, that was a win in 72 moves. In the meantime, you had to convince SF to stay away from lines that lead to positions like this.

[d]8/1K6/8/2P5/2R5/7k/7q/8 b - -

Instead you need to find positions that eventually (~150 plies later) convert to ones like this:

[d]8/8/7p/5k2/3r3P/6KB/8/8 b - -

I can't imagine that you can do that with a machine that produces 2.5Mnps in a reasonable period of time.

Feel free to prove me wrong!

Regards,

Zenmastur

[Ovyron]

The winning move isn't hidden. f5 is pretty easy to find. Knowing what should happen after f5 is played is key to winning as many of these moves aren't obvious even after extensive analysis.

My point here is that you don't need to do that from the current position. You can do that as the game progresses and your opponent's moves reduce exponentially what you need to check.

It's like in our King's Gambit game, there's no way in hell my hardware could have allowed me to predict the game continuation that we played. There's no way in hell my hardware could have allowed me to build a book that made the moves I was going to play from 2.f4.

Did that matter? No, because by the time the moves happened, by your selections the tree became so small I was able to predict your next 3 moves in advance, and only stopped in case committing to my moves in advance was stupid (my prediction can only be right if I force myself to play those moves.)

The same could happen here, I can't find the winning line from the position before f5, but I don't need to, because by the time we've reached the key position where I need to find the winning move, the tree will have shrunk exponentially and what remains can be managed without big hardware.

And big hardware allowing you to not have to worry about it is a fallacy. Imagine that x10 hardware faster than yours existed and your opponent had it. Would you be scared now? But those moves that they'd play to beat you with bigger hardware already exist, so they could be found by anyone. Even I could find them. Except I can't because you won't play anymore, of course...

[jp]

The winning move isn't hidden. f5 is pretty easy to find. Knowing what should happen after f5 is played is key to winning as many of these moves aren't obvious even after extensive analysis.

My point here is that you don't need to do that from the current position. You can do that as the game progresses and your opponent's moves reduce exponentially what you need to check.

You won't always be able to play the move going step by step blindly without knowing the end analysis.

[Ovyron]

You won't always be able to play the move going step by step blindly without knowing the end analysis.

But Zenmastur also doesn't know the end analysis. He thinks he does, and certainly he could beat anyone with black (and perhaps I could do it too), but "big hardware" just provides the illusion that you know something.

I've spent the last decade rubbing shoulders with the people of "big hardware" of each era, because, you know, at some point in time, even my hardware was "big". Zenmastur sounds like all of them, just because they have the hardware and the software that is the current zenith they think they're closer to the truth.

At some point the Rybka Cluster was the biggest thing around, so if Zenmastur had it back then he could have analyzed f5 in this position and gained as much confidence as he has now, and claimed that something as big as the cluster was necessary to find the positions that he saw. As years went on better software appeared and it was demonstrated that Houdini 1.5 in a computer like mine was at the level of the cluster. How does that compare to Stockfish 11?

The illusion is that with so much effort spent reaching some depth in some positions with some strong hardware and software you can get to know something. But either you can't because of the complexity of chess (imagine 1.g4 was a draw with perfect play but Zenmastur has no way to see it with what he has) or you can, but you could have it done with a different analysis method on slower hardware (like... moves where Stockfish 5 with Aggressiveness 200 finds the move instantly, and it doesn't take much to try it.)

Even for the position that I failed that Zullil posted 16 minutes would have been enough, because it wasn't about seeing that a move was losing, having Ke2 with a bigger eval than Bg5 is enough to avoid losing (say, having Bg5 at -0.20 and Ke2 at -0.10 is enough to avoid losing even if the evals are nonsense), because after you do Bg5 never overtakes Ke2 again. A position where slow hardware can't find the best move with some method has yet to be posted. Or someone can try to defend that position, though I think I could win it more easily than how I beat mmt's 1.g4.

[Leto]

I think this position is a better test and I would certainly like to see your analysis of it. I originally thought this would be a “hard” test, but now I'm not so sure how hard it is. I think “most” (meaning 50% or more) ICCF 2200 players can/would solve this if it occurred in one of their games.

[d]4r3/5pk1/1q1r1p1p/1p1Pn2Q/1Pp4P/6P1/5PB1/R3R1K1 b - -

Jdart posted this position and Dann Corbit analyzed it to depth 74 and concluded it's probably a draw. So, much for big hardware and unassisted analysis. I analyzed it and concluded black wins. If a position like this occurred in a CC and you were playing black could you win it?

I'm very interested in seeing your analysis of this position.

Regards,

Zenmastur

Hi I am currently 2307 on ICCF (https://www.iccf.com/player?id=516044&tab=3) and I think I'm now playing at the 2500 ICCF level but I guess only time will tell if I'm right. I analysed your position for 90 minutes, here are the notes I took:

In first minute I had 1...f5 as top move.

After 30 minutes I had this as my top line: 1... f5 2. Qxf5 Rf6 3. Qc2
Qd4 4. Rad1 (4. Rab1 Nf3+ 5. Bxf3 Rxe1+ 6. Rxe1 {R}) 4... Nf3+ 5. Bxf3 Rxe1+
6. Rxe1 Rxf3 7. Kf1 c3 8. Qe2 Rf6 9. h5 Qxb4 10. Rd1 Qc5 {-1.01} *

After 45 minutes I realized that 7...c3 was a mistake and that 7...Qxd5 was best. My new top line was now:

1... f5 2. Qe2 (2. Qd1 Rdd8 3. Re3 Qf6 4. Raa3 {-1.90})(2. Rf1 {-1.95}
) 2... Rdd8 3. Red1 Qf6 4. Ra3 Nd3 5. Qc2 Qb2 (5... Re1+ 6. Rxe1 Nxe1 7. Qe2
Nxg2 8. Kxg2 Rxd5 9. Qe8 {-1.90}) 6. Qxb2+ Nxb2 7. Rd2 Re1+ 8. Kh2 Na4 (8...
Nd3 9. Ra5 {-2.00} (9. Ra7 {-1.94})) 9. Rf3 {-1.99}

After one hour this was my top line:

1... f5 2. Rad1 Rf6 3. Qe2 f4 4. gxf4 Rxf4 5. Qc2 Nf3+ 6. Bxf3 Rxe1+ {-1.97} *

After one hour 15 minutes my top line was:

1... f5 2. Qe2 Rdd8 3. Red1 Qf6 4. Ra3 Nd3 5. Qc2 Qb2 6. Qxb2+ Nxb2 7. Rd2
Re1+ 8. Kh2 Na4 9. Rf3 c3 10. Rc2 Rxd5 11. Re3 Rxe3 12. Bxd5 Rd3 13. Bb3 Kf6
14. Kg2 Ke5 *

After one hour 30 minutes my top line was:

1... f5 2. Qe2 Rdd8 3. Red1 Qf6 4. Ra3 Nd3 5. Qd2 f4 6. Rxd3 cxd3 7. Qxd3 Re7
8. Kh2 Rc7 9. Bf1 Rc4 10. Qd2 fxg3+ 11. fxg3 Re4 12. Bg2 Rg4 13. Bh3 Rg6 14.
Bg2 Qe5 15. Qd4 Qxd4 16. Rxd4 *

And here's a possible ending, showing that the above line is likely a win for black:

1... f5 2. Qe2 Rdd8 3. Red1 Qf6 4. Ra3 Nd3 5. Qd2 f4 6. Rxd3 cxd3 7. Qxd3 Re7
8. Kh2 Rc7 9. Bf1 Rc4 10. Qd2 fxg3+ 11. fxg3 Re4 12. Bg2 Rg4 13. Bh3 Rg6 14.
Bg2 Qe5 15. Qd4 Qxd4 16. Rxd4 f5 17. Bh3 Rgd6 18. Bxf5 Rxd5 19. Rg4+ Kf6 20.
Bh7 R8d6 21. Rg8 Rd4 22. Kh3 Rxb4 23. h5 Rb3 24. Rb8 Rc6 25. Bg8 Rb2 26. Bh7
Ke5 27. Bg6 Rc3 28. Re8+ Kd4 29. Rh8 Rbb3 30. Rxh6 Rxg3+ 31. Kh4 Rh3+ 32. Kg5
Rbg3+ 33. Kf6 b4 34. Rh8 b3 35. Rd8+ Kc3 36. Rc8+ Kb2 37. Rb8 Rh4 38. Rc8 Rh1
39. Re8 Rc3 40. Kg7 Rc5 41. Kh6 Rh3 42. Ra8 Kc3 43. Ra1 Kb4 44. Rb1 Rh2 45. Rf1
b2 46. Rf4+ Rc4 47. Rf1 Rc5 48. Rf4+ Ka5 49. Rf1 Rc1 50. Rf5+ Kb6 51. Rf8 Rc6
52. Kg7 Rg2 53. Rb8+ Kc7 54. Rb4 Rb6 55. Rxb6 Kxb6 56. Kf7 Rxg6 57. hxg6 b1=Q
58. g7 Qb3+ 59. Kf8 Qa3+ 60. Ke8 Qa8+ 61. Kf7 Qd5+ 62. Kf8 Qd8+ 63. Kf7 Qd5+
64. Kf8 Qd8+ 65. Kf7 Qd5+ 66. Kf8 {-1/0} *

[zullil]

There's an ongoing game at http://www.talkchess.com/forum3/viewtop ... =6&t=73843.

But it's not clear that your analysis is applicable or beneficial to either side, so I guess no harm to leave it here.

[yurikvelo]

500 Games
Book = 8moves_v3.pgn
Time = 10' + 3"
Ponder = On (game duration 20-35 minutes)
Sygyzy = 6-man SSD
Stockfish = 2020/06/06/16 @ 4 CPU (5200 MN/sec @ startpos)
Lc0 = weights_run1_63702.pb.gz (320x24) @ GTX-1650 (3095 nps @ startpos)

+30 -40 =430

Leela: 23 white wins, 7 black wins
SF: 27 white wins, 13 black wins

PGN of non-draw games (70 games)


Full PGN (500 games)