For how long will Stockfish be competitive versus the best NN ?

Discussion of computer chess matches and engine tournaments.

Moderators: hgm, Rebel, chrisw

Milos
Posts: 4190
Joined: Wed Nov 25, 2009 1:47 am

Re: For how long will Stockfish be competitive versus the best NN ?

Post by Milos »

Dann Corbit wrote: Wed Apr 15, 2020 4:05 pm Thw 1650 super is apparently about half as fast as a RTX 2060 so that is about 10K NPS:
http://rybkaforum.net/cgi-bin/rybkaforu ... ?tid=33328
That is ofc not true. 1650 is on par with 1070Ti, i.e. in best case around 7k nps from the start position. 2060 is over 21k nps from the start position. So difference is 3x at least.
And there is an additional caveat. 1650 achieves this performance in full FP16, meaning FP16 accumulation in addition to FP16 multiplication.
RTX architectures perform FP16 multiplication too, but FP32 accumulation, so even with the same Lc0 running on RTX should show a bit better performance (due to more accurate inference).
User avatar
Ovyron
Posts: 4556
Joined: Tue Jul 03, 2007 4:30 am

Re: For how long will Stockfish be competitive versus the best NN ?

Post by Ovyron »

jp wrote: Wed Apr 15, 2020 7:45 am Well, that's because (we believe) chess is a draw. You don't know that you haven't missed wins when your opponents made mistakes that ended as draws.
I know, I'm not blind. What I can say about that is that the wins I've missed would have been missed with better hardware, and would have been found with my hardware with better analysis methods. I failed to win because I became lazy and thought the games were already decided, so I played considerably below my level. My usual methods would have sufficed to win them if I used them...
User avatar
Ovyron
Posts: 4556
Joined: Tue Jul 03, 2007 4:30 am

Re: For how long will Stockfish be competitive versus the best NN ?

Post by Ovyron »

zullil wrote: Wed Apr 15, 2020 3:10 pmOK, here's an easy one for you:

[d]8/1p2B3/4k3/7p/p1Pp3P/P2P1K2/8/4b3 w - - 6 51

What move(s) are "best"?
I find suspect that you post a position where I have no problem reaching depth 60 very quick with my hardware, in discussions where a position where someone could outsearch me is called for. But now I'm really curious to see how you tell me Bg5 (found in 15 minutes) isn't the "best" move.
corres
Posts: 3657
Joined: Wed Nov 18, 2015 11:41 am
Location: hungary

Re: For how long will Stockfish be competitive versus the best NN ?

Post by corres »

Ovyron wrote: Thu Apr 16, 2020 8:16 am
zullil wrote: Wed Apr 15, 2020 3:10 pmOK, here's an easy one for you:

[d]8/1p2B3/4k3/7p/p1Pp3P/P2P1K2/8/4b3 w - - 6 51

What move(s) are "best"?
I find suspect that you post a position where I have no problem reaching depth 60 very quick with my hardware, in discussions where a position where someone could outsearch me is called for. But now I'm really curious to see how you tell me Bg5 (found in 15 minutes) isn't the "best" move.
It mainly depends on the developing of NVIDIA GPUs.
zullil
Posts: 6442
Joined: Tue Jan 09, 2007 12:31 am
Location: PA USA
Full name: Louis Zulli

Re: For how long will Stockfish be competitive versus the best NN ?

Post by zullil »

Ovyron wrote: Thu Apr 16, 2020 8:16 am
zullil wrote: Wed Apr 15, 2020 3:10 pmOK, here's an easy one for you:

[d]8/1p2B3/4k3/7p/p1Pp3P/P2P1K2/8/4b3 w - - 6 51

What move(s) are "best"?
I find suspect that you post a position where I have no problem reaching depth 60 very quick with my hardware, in discussions where a position where someone could outsearch me is called for. But now I'm really curious to see how you tell me Bg5 (found in 15 minutes) isn't the "best" move.
OK, I think you just lost the game, because your hardware/method failed you.

Perhaps someone with better hardware than mine can confirm that Ke2 is the only move here that holds the draw?
Alayan
Posts: 550
Joined: Tue Nov 19, 2019 8:48 pm
Full name: Alayan Feh

Re: For how long will Stockfish be competitive versus the best NN ?

Post by Alayan »

This is Stockfish-Komodo with black and white reversed, isn't it ?

Then Bg5 is guaranteed to lose, and Ke2 likely holds (or at least, we don't know how to beat it).
zullil
Posts: 6442
Joined: Tue Jan 09, 2007 12:31 am
Location: PA USA
Full name: Louis Zulli

Re: For how long will Stockfish be competitive versus the best NN ?

Post by zullil »

Alayan wrote: Thu Apr 16, 2020 1:58 pm This is Stockfish-Komodo with black and white reversed, isn't it ?

Then Bg5 is guaranteed to lose, and Ke2 likely holds (or at least, we don't know how to beat it).
Yes, it is. The original position was discussed at this forum, so I intentionally obfuscated the position so that searching the FEN would not be immediately useful. Here's what I currently see after Bg5:

-25.30 51... Ba5 52. Ke4 Bb6 53. Bc1 Bc5 54. Kf4 Be7 55. Bb2 Bf6 56. Ba1 Kd6 57. Kg3 Ke5 58. Bb2 Be7 59. Bc1 Ke6 60. Bb2 Bd6+ 61. Kf3 Kf5 62. Bc1 Be7 63. Kg3 Bc5 64. Kh3 Bd6 65. Kg2 Kg4 66. Bb2 Bc5 67. Kf2 Be7 68. Kf1 Bxh4 69. Ke2 Kf5 70. Ba1 Ke5 71. Kd1 Bg5 72. Ke2 Bc1 73. Ke1 Bxa3 74. Kf2 Bb4 75. Bb2 a3 76. Ba1 Bc3 77. Bxc3 dxc3 78. Ke3 c2 79. d4+ Ke6 80. d5+ Kd6 (depth 51, 1:01:15)

That's after a deep Stockfish search of the original position with MultiPV = 2, followed by forward/backward analysis of the Bg5 line.

20 threads, 64 GB hash table.
Milos
Posts: 4190
Joined: Wed Nov 25, 2009 1:47 am

Re: For how long will Stockfish be competitive versus the best NN ?

Post by Milos »

zullil wrote: Thu Apr 16, 2020 2:10 pm
Alayan wrote: Thu Apr 16, 2020 1:58 pm This is Stockfish-Komodo with black and white reversed, isn't it ?

Then Bg5 is guaranteed to lose, and Ke2 likely holds (or at least, we don't know how to beat it).
Yes, it is. The original position was discussed at this forum, so I intentionally obfuscated the position so that searching the FEN would not be immediately useful. Here's what I currently see after Bg5:

-25.30 51... Ba5 52. Ke4 Bb6 53. Bc1 Bc5 54. Kf4 Be7 55. Bb2 Bf6 56. Ba1 Kd6 57. Kg3 Ke5 58. Bb2 Be7 59. Bc1 Ke6 60. Bb2 Bd6+ 61. Kf3 Kf5 62. Bc1 Be7 63. Kg3 Bc5 64. Kh3 Bd6 65. Kg2 Kg4 66. Bb2 Bc5 67. Kf2 Be7 68. Kf1 Bxh4 69. Ke2 Kf5 70. Ba1 Ke5 71. Kd1 Bg5 72. Ke2 Bc1 73. Ke1 Bxa3 74. Kf2 Bb4 75. Bb2 a3 76. Ba1 Bc3 77. Bxc3 dxc3 78. Ke3 c2 79. d4+ Ke6 80. d5+ Kd6 (depth 51, 1:01:15)

That's after a deep Stockfish search of the original position with MultiPV = 2, followed by forward/backward analysis of the Bg5 line.

20 threads, 64 GB hash table.
I really don't know what are you taking about. SFdev, 6-men, MultiPV=2, depth 80, Bg5 and Kd2 are a transposition so lines are identical, score -1.07.
Line is 51. Bg5 Ba5 52. Ke2 Bc7 or 51.Ke2 Ba5 52. Bg5 Bc7
Alayan
Posts: 550
Joined: Tue Nov 19, 2019 8:48 pm
Full name: Alayan Feh

Re: For how long will Stockfish be competitive versus the best NN ?

Post by Alayan »

See http://talkchess.com/forum3/viewtopic.php?f=2&t=73557

This is proven to win by force.
zullil
Posts: 6442
Joined: Tue Jan 09, 2007 12:31 am
Location: PA USA
Full name: Louis Zulli

Re: For how long will Stockfish be competitive versus the best NN ?

Post by zullil »

Milos wrote: Thu Apr 16, 2020 2:56 pm
zullil wrote: Thu Apr 16, 2020 2:10 pm
Alayan wrote: Thu Apr 16, 2020 1:58 pm This is Stockfish-Komodo with black and white reversed, isn't it ?

Then Bg5 is guaranteed to lose, and Ke2 likely holds (or at least, we don't know how to beat it).
Yes, it is. The original position was discussed at this forum, so I intentionally obfuscated the position so that searching the FEN would not be immediately useful. Here's what I currently see after Bg5:

-25.30 51... Ba5 52. Ke4 Bb6 53. Bc1 Bc5 54. Kf4 Be7 55. Bb2 Bf6 56. Ba1 Kd6 57. Kg3 Ke5 58. Bb2 Be7 59. Bc1 Ke6 60. Bb2 Bd6+ 61. Kf3 Kf5 62. Bc1 Be7 63. Kg3 Bc5 64. Kh3 Bd6 65. Kg2 Kg4 66. Bb2 Bc5 67. Kf2 Be7 68. Kf1 Bxh4 69. Ke2 Kf5 70. Ba1 Ke5 71. Kd1 Bg5 72. Ke2 Bc1 73. Ke1 Bxa3 74. Kf2 Bb4 75. Bb2 a3 76. Ba1 Bc3 77. Bxc3 dxc3 78. Ke3 c2 79. d4+ Ke6 80. d5+ Kd6 (depth 51, 1:01:15)

That's after a deep Stockfish search of the original position with MultiPV = 2, followed by forward/backward analysis of the Bg5 line.

20 threads, 64 GB hash table.
I really don't know what are you taking about. SFdev, 6-men, MultiPV=2, depth 80, Bg5 and Kd2 are a transposition so lines are identical, score -1.07.
Line is 51. Bg5 Ba5 52. Ke2 Bc7 or 51.Ke2 Ba5 52. Bg5 Bc7
Please let us know if you discover that your initial analysis is incorrect.