For how long will Stockfish be competitive versus the best NN ?

[Zenmastur]

I am sure that there are positions that you will be slower in finding the best move by your methods
otherwise you can get a huge software improvement by simply writing an engine that find the best moves faster.

Show me one.

Go and find a position where you need to give the engine a really long time and really high depth to find the best move, so that big hardware is "necessary" to find it.

Then I'll show you an analysis method that could have found the move much faster. Even in the faster mate challenge I found the winning moves within seconds, and the ones I missed weren't better, they only mated faster.

NOTE - Last time someone tried this Fruit 2.3.1 found the best move in 10 minutes

I used to think this was programmable, but I no longer think so. The analysis methods needed to find the best moves in some Spanish line can be completely different from those that find those moves in some Sicilian. Sometimes you need to extend the lines as deep as you can, sometimes you need to examine as many alternative moves as widely as you can. Sometimes you need to use an engine with original ideas that finds unorthodox lines that others were pruning, sometimes you need a tactical monster that makes sure you're not missing anything on a very sharp mainline.

Some people in the world might have found a "one size fits all" analysis method that helps them avoid losing all their games with certainty, but they may not be winning as many games against weaker opposition as they could.

It was probably possible many years ago when the search algorithm of engines was stupid but not today.

Better search algorithms support my point, those algorithms have gotten so good that they find the best moves in most positions without needing big hardware, that's why 8 years ago I'd lose most of my games in playchess but with software of today I draw most of my games.

I see you failed Zullil's test. That's surprising since it takes less than 60 seconds to determine that Bg5 has a problem and less than 2 minutes to determine Ke2 is the better move.

Code: Select all

 1 [-0.13]  51.Bg5  (0.00)
 2 [+0.35]  51.Bg5 Ke5  (0.00)
 3 [-0.19]  51.Bg5 Ke5 52.Bf4+ Kf5  (0.00)
 4 [+0.08]  51.Bd8 Ke5 52.Bc7+ Ke6  (0.00)
 5 [-0.13]  51.Bd8 Bc3 52.Kf4 Bb2 53.Bb6  (0.00)
 7 [+0.00]  51.Bg5 Ke5 52.Bf4+ Kf5 53.Bg3 Bd2 54.Bf2 Bc3  (0.00)
 8 [-0.07]  51.Bg5 Ke5 52.Be7 Bd2 53.Kf2 Bc1 54.Ke2 Be3  (0.00)
 9 [+0.00]  51.Bg5 Ke5 52.Be7 Bd2 53.Bf8 Kf5 54.Bc5 Be3 55.Ba7 Bc1 56.Bc5  (0.00)
10 [-0.63]  51.Bd8 Kf5 52.Be7 Ba5 53.Bc5 Ke5 54.Bf8 Bd8 55.Bg7+ Kf5 56.Bxd4 Bxh4 57.c5 Bg5 58.Kg3 Bc1  (0.01)
.
.
.
… snip snip...
.
.
.
70 [-1.09]  51.Bg5 Ba5 52.Ke4 Bb6 53.Bc1 Bc5 54.Kf3 Be7 55.Kg3 Kf5 56.Bb2 Bf6 57.Bc1 Be5+ 58.Kf3 Bg7 59.Kg3 Bf6 60.Bh6 Be5+ 61.Kf3 Bd6 62.Bc1 Be7 63.Kg3 Ke6 64.Bb2 Bd6+ 65.Kg2 Kf5 66.Kf3 Bf8 67.Kg3 Ke5 68.Kf3 Bc5 69.Bc1 Ke6 70.Bb2 Kf6 71.Kg3 Bd6+ 72.Kf3 Be7 73.Kg3 Bc5 74.Bc1 Kf7 75.Bb2 Bd6+ 76.Kf3 Bf8 77.Bc1 Be7 78.Kg3 Ke8 79.Bb2 Bd6+ 80.Kg2 Be5 81.Kf3 Bf6 82.Kg3 Kf7 83.Bc1 Bg7 84.Bb2 Ke6 85.Kf3 Bf6 86.Kg3 Be5+ 87.Kf3 Bg7 88.Bc1 Bf6 89.Kg3 Kd7 90.Bb2 Be5+ 91.Kf3 Ke6  (44.48)
71 [-1.19]  51.Bg5 Ba5  (48.35)
71 [-1.29]  51.Bg5 Bc3  (58.66)
71 [-1.43]  51.Bg5 Ba5  (72.03)
71 [-1.31]  51.Ke2  (105.05)
71 [-1.10]  51.Ke2 Bg3 52.Kf3 Bc7 53.Bc5 Bd8 54.Kg3 Bf6 55.Bb4 b6 56.Bf8 Bd8 57.Bb4 Kf5 58.Bd6 Bf6 59.Kh3 Ke6 60.Bb4 Be5 61.Kg2 Kd7 62.Bf8 Bf6 63.Kh3 Kc6 64.Bb4 Be5 65.Kg2 b5 66.cxb5+ Kxb5 67.Kf2 Bf6 68.Kg3 Kc6 69.Bf8 Be5+ 70.Kf3 Kd5 71.Be7 Bc7 72.Bf8 Bd8 73.Kg3 Ke6 74.Bc5 Bf6 75.Bb4 Be5+ 76.Kg2 Kf5 77.Kf3 Bc7 78.Be7 Bb6 79.Kg3 Ke6 80.Bb4 Bc7+ 81.Kg2 Bd8 82.Kh3 Kd7 83.Kg3 Kc6 84.Bf8 Kd5 85.Bb4 Bc7+ 86.Kg2 Kc6 87.Kf2 Bd8 88.Kg3  (117.49)
72 [-1.10]  51.Ke2 Bg3 52.Kf3 Bc7 53.Bb4 Kf5 54.Be7 Be5 55.Bc5 Bf6 56.Kg3 Bg7 57.Bd6 Bh6 58.Kf3 Bc1 59.Bc5 Ke5 60.Bb4 Bh6 61.Be7 Bd2 62.Bd8 Bc3 63.Be7 Ba5 64.Bf8 Be1 65.Be7 Kf5 66.Bg5 Bc3 67.Bc1 Ke5 68.Bh6 Be1 69.Bg5 Kf5 70.Be7 Bd2 71.Bc5 Ke5 72.Bf8 Be3 73.Be7 Bh6 74.Bc5 Bg7 75.Be7 Kf5 76.Kg3 Ke6 77.Bc5 Kf7 78.Kf2 Bf6 79.Kg3 Ke8 80.Bb4 Kd7 81.Bc5 Kc6 82.Bb4 Bd8 83.Bf8 Kd7 84.Bb4 b6 85.Bf8 Kc6 86.Bb4 b5 87.cxb5+ Kxb5 88.Bd6 Kb6  (123.73)
73 [-1.10]  51.Ke2 Bg3 52.Kf3 Bc7 53.Bb4 Bd8 54.Kg3 Bf6 55.Bc5 Be5+ 56.Kf2 Kf5 57.Kf3 Bf6 58.Kg3 Ke6 59.Bb4 b6 60.Kh3 Be5 61.Kg2 Bf4 62.Kf3 Kf5 63.Bf8 Be5 64.Be7 Bc7 65.Kg2 Ke6 66.Bf8 Bd8 67.Kg3 Ke5 68.Bg7+ Kd6 69.Bf8+ Kc6 70.Bb4 Bf6 71.Bf8 Kd7 72.Bb4 Kc7 73.Bf8 b5 74.cxb5 Kb6 75.Bd6 Kxb5 76.Kh3 Kc6 77.Bf8 Be5 78.Be7 Kd5 79.Kg2 Ke6 80.Bc5 Bc7 81.Bb4 Bd8 82.Kg3 Kf5 83.Bc5 Ba5 84.Kf3 Bc7 85.Be7 Ke5 86.Bg5 Bb6 87.Be7 Kd5 88.Bb4 Bc7 89.Kf2 Bd8 90.Kg3 Kc6 91.Bf8 Bc7+ 92.Kf3  (134.14)
.
.
.
… snip snip...
.
.
.

89 [-1.10]  51.Ke2 Ba5 52.Bc5 Ke5 53.Kf2 Bc7 54.Kf3 Bd8 55.Kg3 Bf6 56.Bf8 Ke6 57.Bc5 Kf5 58.Bd6 Bd8 59.Bf8 Bc7+ 60.Kf3 b6 61.Be7 Ke6 62.Bb4 Ke5 63.Bf8 Bd8 64.Kg3 Kf6 65.Bd6 Kf7 66.Bb4 Ke8 67.Kh3 Kd7 68.Kg3 Kc6 69.Bf8 Bf6 70.Bb4 b5 71.cxb5+ Kxb5 72.Bf8 Kc6 73.Bb4 Be5+ 74.Kg2 Kb5 75.Kf3 Bf6 76.Kg3 Kc6 77.Bf8 Bd8 78.Bb4 Bc7+ 79.Kg2 Kd5 80.Kh3 Ke6 81.Kg2 Bd8 82.Kg3 Kf5 83.Bc5 Bc7+ 84.Kf3 Ke5 85.Bf8 Kd5 86.Be7 Ke6 87.Bc5 Be5 88.Bb4 Bf6 89.Kg3 Ke5 90.Bc5 Bg7 91.Bb4 Kf5 92.Kf3 Bh6 93.Bc5 Bc1 94.Bb4 Ke5 95.Bf8 Kd5 96.Kf2 Be3+ 97.Kf3 Bd2 98.Be7 Bc1 99.Ke2  (601.05)
90 [-1.10]  51.Ke2 Ba5 52.Bc5 Ke5 53.Kf2 Bc7 54.Kf3 Bd8 55.Kg3 Bf6 56.Bf8 Bh8 57.Bh6 Ke6 58.Bf8 Be5+ 59.Kg2 Bf6 60.Kh3 b6 61.Kg3 Bd8 62.Bb4 Bc7+ 63.Kg2 Be5 64.Bf8 Kf6 65.Bb4 Kf5 66.Be7 Bc7 67.Kf3 Ke6 68.Bb4 Ke5 69.Bf8 Bd8 70.Kg3 Kf5 71.Bd6 Bf6 72.Bb4 Bg7 73.Be7 Ke6 74.Bb4 Be5+ 75.Kg2 Bh8 76.Kf3 Bf6 77.Kg3 Kf5 78.Bd6 Bd8 79.Kh3 Ke6 80.Bb4 Kd7 81.Kg3 Kc6 82.Bf8 Bf6 83.Bb4 Kc7 84.Bf8 Kc8 85.Kh3 Kb7 86.Bb4 Bd8 87.Kg3 b5 88.cxb5 Kb6 89.Bf8 Kxb5 90.Bb4 Kc6 91.Kh3 Bf6 92.Kg3 Be5+ 93.Kf3 Kd5 94.Kg2 Ke6 95.Kf3 Kf5 96.Bc5 Bf6 97.Kg3 Ke5 98.Bf8 Ke6 99.Bc5 Kf5 100.Bf8 Ke5  (659.13)
91 [-1.10]  51.Ke2 Ba5 52.Bc5 Ke5 53.Kf2 Bc7 54.Kf3 Bd8 55.Kg3 Bf6 56.Bf8 Bh8 57.Bh6 Ke6 58.Bf8 Be5+ 59.Kg2 Bf6 60.Kh3 b6 61.Kg3 Bd8 62.Bb4 Bc7+ 63.Kg2 Be5 64.Bf8 Kf6 65.Bb4 Kf5 66.Be7 Bc7 67.Kf3 Ke6 68.Bb4 Ke5 69.Bf8 Bd8 70.Kg3 Kf5 71.Bd6 Bf6 72.Bb4 Bg7 73.Be7 Ke6 74.Bb4 Be5+ 75.Kg2 Bh8 76.Kf3 Bf6 77.Kg3 Kf5 78.Bd6 Bd8 79.Kh3 Ke6 80.Bb4 Kd7 81.Kg3 Kc6 82.Bf8 Bf6 83.Bb4 Kc7 84.Bf8 Kc8 85.Kh3 Kb7 86.Bb4 Bd8 87.Kg3 b5 88.cxb5 Kb6 89.Bf8 Kxb5 90.Bb4 Kc6 91.Kh3 Bf6 92.Kg3 Be5+ 93.Kf3 Kd5 94.Kg2 Ke6 95.Kf3 Kf5 96.Bc5 Bf6 97.Kg3 Ke5 98.Bf8 Ke6 99.Bc5 Kf5 100.Bf8 Ke5  (665.32)

The numbers at the end of each iteration is the time in seconds. From iteration 71 to the end of the analysis Ke2 is chosen. The total time to find Ke2 with an empty transposition table using unassisted analysis is 117.49 seconds on my machine. So, this problem is clearly too easy.

It's comparatively easy to draw a correspondence time control game. Just look at all the drawn games on ICCF to see the proof. It's an order of magnitude harder to win such a game. (So, perhaps drawing someone in a CC game should be worth a tenth of a point.) In any case, this disparity makes winning a won position EXTREMELY important because such positions occur only rarely in top level ICCF play and must be taken advantage of when they occur.

I think this position is a better test and I would certainly like to see your analysis of it. I originally thought this would be a “hard” test, but now I'm not so sure how hard it is. I think “most” (meaning 50% or more) ICCF 2200 players can/would solve this if it occurred in one of their games.

[d]4r3/5pk1/1q1r1p1p/1p1Pn2Q/1Pp4P/6P1/5PB1/R3R1K1 b - -

Jdart posted this position and Dann Corbit analyzed it to depth 74 and concluded it's probably a draw. So, much for big hardware and unassisted analysis. I analyzed it and concluded black wins. If a position like this occurred in a CC and you were playing black could you win it?

I'm very interested in seeing your analysis of this position.

Regards,

Zenmastur

[zullil]

I think this position is a better test and I would certainly like to see your analysis of it. I originally thought this would be a “hard” test, but now I'm not so sure how hard it is. I think “most” (meaning 50% or more) ICCF 2200 players can/would solve this if it occurred in one of their games.

[d]4r3/5pk1/1q1r1p1p/1p1Pn2Q/1Pp4P/6P1/5PB1/R3R1K1 b - -

Jdart posted this position and Dann Corbit analyzed it to depth 74 and concluded it's probably a draw. So, much for big hardware and unassisted analysis. I analyzed it and concluded black wins. If a position like this occurred in a CC and you were playing black could you win it?

I'm very interested in seeing your analysis of this position.

Regards,

Zenmastur

Can you give some idea of what "a moderately large number" might be? An unassisted Stockfish-dev search yielded 2.91 in favor of Black.

[Zenmastur]

I think this position is a better test and I would certainly like to see your analysis of it. I originally thought this would be a “hard” test, but now I'm not so sure how hard it is. I think “most” (meaning 50% or more) ICCF 2200 players can/would solve this if it occurred in one of their games.

[d]4r3/5pk1/1q1r1p1p/1p1Pn2Q/1Pp4P/6P1/5PB1/R3R1K1 b - -

Jdart posted this position and Dann Corbit analyzed it to depth 74 and concluded it's probably a draw. So, much for big hardware and unassisted analysis. I analyzed it and concluded black wins. If a position like this occurred in a CC and you were playing black could you win it?

I'm very interested in seeing your analysis of this position.

Regards,

Zenmastur

Can you give some idea of what "a moderately large number" might be? An unassisted Stockfish-dev search yielded 2.91 in favor of Black.

Greater than +4 for black.

[Ovyron]

[d]4r3/5pk1/1q1r1p1p/1p1Pn2Q/1Pp4P/6P1/5PB1/R3R1K1 b - -

If a position like this occurred in a CC and you were playing black could you win it?

I'm very interested in seeing your analysis of this position.

That's difficult to answer without an actual game happening. Here I'd play 1...f5 and would not prepare another move until knowing what white plays. If 1...f5 is a known draw and black has another move that needs to be played I'd probably wouldn't have found it.

[drewdrew]

I think this position is a better test and I would certainly like to see your analysis of it. I originally thought this would be a “hard” test, but now I'm not so sure how hard it is. I think “most” (meaning 50% or more) ICCF 2200 players can/would solve this if it occurred in one of their games.

[d]4r3/5pk1/1q1r1p1p/1p1Pn2Q/1Pp4P/6P1/5PB1/R3R1K1 b - -

Jdart posted this position and Dann Corbit analyzed it to depth 74 and concluded it's probably a draw. So, much for big hardware and unassisted analysis. I analyzed it and concluded black wins. If a position like this occurred in a CC and you were playing black could you win it?

I'm very interested in seeing your analysis of this position.

Regards,

Zenmastur

Can you give some idea of what "a moderately large number" might be? An unassisted Stockfish-dev search yielded 2.91 in favor of Black.

Greater than +4 for black.

Unassisted Stockfish-dev (old-ish version):
info depth 72 seldepth 111 multipv 1 score cp 433 nodes 8708473100400 nps 374666434 hashfull 953 tbhits 21101722591 time 23243270

[zullil]

I think this position is a better test and I would certainly like to see your analysis of it. I originally thought this would be a “hard” test, but now I'm not so sure how hard it is. I think “most” (meaning 50% or more) ICCF 2200 players can/would solve this if it occurred in one of their games.

[d]4r3/5pk1/1q1r1p1p/1p1Pn2Q/1Pp4P/6P1/5PB1/R3R1K1 b - -

Jdart posted this position and Dann Corbit analyzed it to depth 74 and concluded it's probably a draw. So, much for big hardware and unassisted analysis. I analyzed it and concluded black wins. If a position like this occurred in a CC and you were playing black could you win it?

I'm very interested in seeing your analysis of this position.

Regards,

Zenmastur

Can you give some idea of what "a moderately large number" might be? An unassisted Stockfish-dev search yielded 2.91 in favor of Black.

Greater than +4 for black.

Unassisted Stockfish-dev (old-ish version):
info depth 72 seldepth 111 multipv 1 score cp 433 nodes 8708473100400 nps 374666434 hashfull 953 tbhits 21101722591 time 23243270

Using 7-man endgame tables?

[drewdrew]

I think this position is a better test and I would certainly like to see your analysis of it. I originally thought this would be a “hard” test, but now I'm not so sure how hard it is. I think “most” (meaning 50% or more) ICCF 2200 players can/would solve this if it occurred in one of their games.

[d]4r3/5pk1/1q1r1p1p/1p1Pn2Q/1Pp4P/6P1/5PB1/R3R1K1 b - -

Jdart posted this position and Dann Corbit analyzed it to depth 74 and concluded it's probably a draw. So, much for big hardware and unassisted analysis. I analyzed it and concluded black wins. If a position like this occurred in a CC and you were playing black could you win it?

I'm very interested in seeing your analysis of this position.

Regards,

Zenmastur

Can you give some idea of what "a moderately large number" might be? An unassisted Stockfish-dev search yielded 2.91 in favor of Black.

Greater than +4 for black.

Unassisted Stockfish-dev (old-ish version):
info depth 72 seldepth 111 multipv 1 score cp 433 nodes 8708473100400 nps 374666434 hashfull 953 tbhits 21101722591 time 23243270

Using 7-man endgame tables?

No, only 6-men syzygy.

[Zenmastur]

I did a multiPV 6 after f5 to get whites best defense. Only the top 4 looked playable. I took the top white reply and am reverse analyzing it now. I'm not quite done yet but it looks like it will be about -10 or so.

I'm pretty sure he is using 7-man. I tried it with 6-man and it was taking way too long and the scores weren't great. I actually generated a 9-man to help resolve some of the problems I was halving. E.G.:

[d]8/3P1pk1/7p/8/3r3P/6P1/6B1/6K1 w - - 0 1

As it turns out this is a draw. But, I found a line specifically to avoid this position and ones like it. So, I'm pretty sure this position ( the original position posted by JDart) is won for black.

Regards,

Zenmastur

[Zenmastur]

I think this position is a better test and I would certainly like to see your analysis of it. I originally thought this would be a “hard” test, but now I'm not so sure how hard it is. I think “most” (meaning 50% or more) ICCF 2200 players can/would solve this if it occurred in one of their games.

[d]4r3/5pk1/1q1r1p1p/1p1Pn2Q/1Pp4P/6P1/5PB1/R3R1K1 b - -

Jdart posted this position and Dann Corbit analyzed it to depth 74 and concluded it's probably a draw. So, much for big hardware and unassisted analysis. I analyzed it and concluded black wins. If a position like this occurred in a CC and you were playing black could you win it?

I'm very interested in seeing your analysis of this position.

Regards,

Zenmastur

Can you give some idea of what "a moderately large number" might be? An unassisted Stockfish-dev search yielded 2.91 in favor of Black.

Greater than +4 for black.

Unassisted Stockfish-dev (old-ish version):
info depth 72 seldepth 111 multipv 1 score cp 433 nodes 8708473100400 nps 374666434 hashfull 953 tbhits 21101722591 time 23243270

Using 7-man endgame tables?

No, only 6-men syzygy.

I'm surprised! I tried this and it takes way too long better to use 7-man for a position like this.

[zullil]

I think this position is a better test and I would certainly like to see your analysis of it. I originally thought this would be a “hard” test, but now I'm not so sure how hard it is. I think “most” (meaning 50% or more) ICCF 2200 players can/would solve this if it occurred in one of their games.

[d]4r3/5pk1/1q1r1p1p/1p1Pn2Q/1Pp4P/6P1/5PB1/R3R1K1 b - -

Jdart posted this position and Dann Corbit analyzed it to depth 74 and concluded it's probably a draw. So, much for big hardware and unassisted analysis. I analyzed it and concluded black wins. If a position like this occurred in a CC and you were playing black could you win it?

I'm very interested in seeing your analysis of this position.

Regards,

Zenmastur

Can you give some idea of what "a moderately large number" might be? An unassisted Stockfish-dev search yielded 2.91 in favor of Black.

Greater than +4 for black.

Unassisted Stockfish-dev (old-ish version):
info depth 72 seldepth 111 multipv 1 score cp 433 nodes 8708473100400 nps 374666434 hashfull 953 tbhits 21101722591 time 23243270

Using 7-man endgame tables?

No, only 6-men syzygy.

I'm surprised! I tried this and it takes way too long better to use 7-man for a position like this.

I'm also surprised, but maybe drewdrew got some benefit from having 384 threads and 128 GB hash.

My unassisted search (using 6-man tables) reached depth 89 but the eval was only -2.91. Following that with a bit of forward/backward searching, I got the root evaluation to -3.38. And then I gave up.