For how long will Stockfish be competitive versus the best NN ?

Discussion of computer chess matches and engine tournaments.

Moderator: Ras

jp
Posts: 1483
Joined: Mon Apr 23, 2018 7:54 am

Re: For how long will Stockfish be competitive versus the best NN ?

Post by jp »

Uri Blass wrote: Tue Apr 14, 2020 3:44 pm
Ovyron wrote: Tue Apr 14, 2020 2:54 am
I totally disagree.
...
2)I am sure that there are positions that you will be slower in finding the best move by your methods
otherwise you can get a huge software improvement by simply writing an engine that find the best moves faster.

It was probably possible many years ago when the search algorithm of engines was stupid but not today.
Coding up his methods would be an interesting thing to try. Don't worry about huge improvement; any improvement would be an interesting result. (And of course the opposite...)
Milos
Posts: 4190
Joined: Wed Nov 25, 2009 1:47 am

Re: For how long will Stockfish be competitive versus the best NN ?

Post by Milos »

Uri Blass wrote: Tue Apr 14, 2020 3:44 pm 2)I am sure that there are positions that you will be slower in finding the best move by your methods
otherwise you can get a huge software improvement by simply writing an engine that find the best moves faster.

It was probably possible many years ago when the search algorithm of engines was stupid but not today.
Maybe because no one yet invented the way to effectively code in human intuition.
User avatar
Ovyron
Posts: 4562
Joined: Tue Jul 03, 2007 4:30 am

Re: For how long will Stockfish be competitive versus the best NN ?

Post by Ovyron »

Uri Blass wrote: Tue Apr 14, 2020 3:44 pm I am sure that there are positions that you will be slower in finding the best move by your methods
otherwise you can get a huge software improvement by simply writing an engine that find the best moves faster.
Show me one.

Go and find a position where you need to give the engine a really long time and really high depth to find the best move, so that big hardware is "necessary" to find it.

Then I'll show you an analysis method that could have found the move much faster. Even in the faster mate challenge I found the winning moves within seconds, and the ones I missed weren't better, they only mated faster.

NOTE - Last time someone tried this Fruit 2.3.1 found the best move in 10 minutes :P

I used to think this was programmable, but I no longer think so. The analysis methods needed to find the best moves in some Spanish line can be completely different from those that find those moves in some Sicilian. Sometimes you need to extend the lines as deep as you can, sometimes you need to examine as many alternative moves as widely as you can. Sometimes you need to use an engine with original ideas that finds unorthodox lines that others were pruning, sometimes you need a tactical monster that makes sure you're not missing anything on a very sharp mainline.

Some people in the world might have found a "one size fits all" analysis method that helps them avoid losing all their games with certainty, but they may not be winning as many games against weaker opposition as they could.
Uri Blass wrote: Tue Apr 14, 2020 3:44 pmIt was probably possible many years ago when the search algorithm of engines was stupid but not today.
Better search algorithms support my point, those algorithms have gotten so good that they find the best moves in most positions without needing big hardware, that's why 8 years ago I'd lose most of my games in playchess but with software of today I draw most of my games.
jp
Posts: 1483
Joined: Mon Apr 23, 2018 7:54 am

Re: For how long will Stockfish be competitive versus the best NN ?

Post by jp »

Ovyron wrote: Wed Apr 15, 2020 4:57 am
Uri Blass wrote: Tue Apr 14, 2020 3:44 pmIt was probably possible many years ago when the search algorithm of engines was stupid but not today.
Better search algorithms support my point, those algorithms have gotten so good that they find the best moves in most positions without needing big hardware, that's why 8 years ago I'd lose most of my games in playchess but with software of today I draw most of my games.
Well, that's because (we believe) chess is a draw. You don't know that you haven't missed wins when your opponents made mistakes that ended as draws.
zullil
Posts: 6442
Joined: Tue Jan 09, 2007 12:31 am
Location: PA USA
Full name: Louis Zulli

Re: For how long will Stockfish be competitive versus the best NN ?

Post by zullil »

Ovyron wrote: Wed Apr 15, 2020 4:57 am
Uri Blass wrote: Tue Apr 14, 2020 3:44 pm I am sure that there are positions that you will be slower in finding the best move by your methods
otherwise you can get a huge software improvement by simply writing an engine that find the best moves faster.
Show me one.

Go and find a position where you need to give the engine a really long time and really high depth to find the best move, so that big hardware is "necessary" to find it.

Then I'll show you an analysis method that could have found the move much faster.
OK, here's an easy one for you:

[d]8/1p2B3/4k3/7p/p1Pp3P/P2P1K2/8/4b3 w - - 6 51

What move(s) are "best"?
Morten Lohne
Posts: 11
Joined: Thu Jul 20, 2017 12:18 am

Re: For how long will Stockfish be competitive versus the best NN ?

Post by Morten Lohne »

Dann Corbit wrote: Tue Apr 14, 2020 11:25 pm
Morten Lohne wrote: Tue Apr 14, 2020 4:15 pm
Dann Corbit wrote: Mon Apr 13, 2020 11:51 am However, you can get a much cheaper card that uses less power and still provides very good analysis.
https://www.videocardbenchmark.net/common_gpus.html
Look at the price performance graph and you can see for $200 you can get a pretty good card for LC0 that will also use a lot less power than a 2080Ti.
While this is true for gaming, it is simple not true for running Leela. GPUs with special NN inference hardware will crush any GPU without without it. For now, your only choices for that are Nvidia RTX cards (2060, 2070, 2080 and their cousins). For instance, an RTX 2060 gets more than 3x more nps than a GTX 1070 ti, even though they are roughly equivalent for gaming.
The 1650 super is about half as fast as a 2080 super, and it is less than $200:
https://www.videocardbenchmark.net/comp ... 4123vs4167
It is very new, so it has a modern AMD instruction set.
Yes, it will not be as fast as a 2080 Super, but if you wait two or three times as long, your answer will be just as good. I have two 2080 Supers in the machine right next to me, but not everyone wants to spend a lot of money in order to push little wooden horsies around on some cardboard squares.
You're missing the point, the 1650 super is *not* half as fast when running Leela, because the 1650 doesn't have RTX cores. You cannot look at gaming benchmarks to compare, because they don't take that into account.

Take a look at this google spreadsheet of Leela benchmarks: https://docs.google.com/spreadsheets/d/ ... 1508569046 The RTX cards mop the floor with all the non-RTX cards. For instance, the 2080 gets 4x more nps than the 1070ti.
Dann Corbit
Posts: 12790
Joined: Wed Mar 08, 2006 8:57 pm
Location: Redmond, WA USA

Re: For how long will Stockfish be competitive versus the best NN ?

Post by Dann Corbit »

Morten Lohne wrote: Wed Apr 15, 2020 3:13 pm
Dann Corbit wrote: Tue Apr 14, 2020 11:25 pm
Morten Lohne wrote: Tue Apr 14, 2020 4:15 pm
Dann Corbit wrote: Mon Apr 13, 2020 11:51 am However, you can get a much cheaper card that uses less power and still provides very good analysis.
https://www.videocardbenchmark.net/common_gpus.html
Look at the price performance graph and you can see for $200 you can get a pretty good card for LC0 that will also use a lot less power than a 2080Ti.
While this is true for gaming, it is simple not true for running Leela. GPUs with special NN inference hardware will crush any GPU without without it. For now, your only choices for that are Nvidia RTX cards (2060, 2070, 2080 and their cousins). For instance, an RTX 2060 gets more than 3x more nps than a GTX 1070 ti, even though they are roughly equivalent for gaming.
The 1650 super is about half as fast as a 2080 super, and it is less than $200:
https://www.videocardbenchmark.net/comp ... 4123vs4167
It is very new, so it has a modern AMD instruction set.
Yes, it will not be as fast as a 2080 Super, but if you wait two or three times as long, your answer will be just as good. I have two 2080 Supers in the machine right next to me, but not everyone wants to spend a lot of money in order to push little wooden horsies around on some cardboard squares.
You're missing the point, the 1650 super is *not* half as fast when running Leela, because the 1650 doesn't have RTX cores. You cannot look at gaming benchmarks to compare, because they don't take that into account.

Take a look at this google spreadsheet of Leela benchmarks: https://docs.google.com/spreadsheets/d/ ... 1508569046 The RTX cards mop the floor with all the non-RTX cards. For instance, the 2080 gets 4x more nps than the 1070ti.
Thw 1650 super is apparently about half as fast as a RTX 2060 so that is about 10K NPS:
http://rybkaforum.net/cgi-bin/rybkaforu ... ?tid=33328

If you wait nine times as long, you will get the same analysis as the top entry in the spreadsheet.
So time is the antidote, and if you do not want to spend money, you will simply have to wait longer for your analysis.
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.
mmt
Posts: 343
Joined: Sun Aug 25, 2019 8:33 am
Full name: .

Re: For how long will Stockfish be competitive versus the best NN ?

Post by mmt »

SF will be competitive until the LC0 team focuses on the tactics and the endgame. If SF can reach EGTBs because it looks deeper and LC0 cannot, it's a lost cause in the cases when the endgame is not straightforward. Learning on games with a larger variety of openings will also help.
Jouni
Posts: 3641
Joined: Wed Mar 08, 2006 8:15 pm
Full name: Jouni Uski

Re: For how long will Stockfish be competitive versus the best NN ?

Post by Jouni »

In TCEC 176 cores SF is competitive with 4 x 4352 = 17408 CUDA cores. Neither is consumer product :).
Jouni
Dann Corbit
Posts: 12790
Joined: Wed Mar 08, 2006 8:57 pm
Location: Redmond, WA USA

Re: For how long will Stockfish be competitive versus the best NN ?

Post by Dann Corbit »

Jouni wrote: Wed Apr 15, 2020 7:47 pm In TCEC 176 cores SF is competitive with 4 x 4352 = 17408 CUDA cores. Neither is consumer product :).
Maybe not, but it sure makes for great chess. TCEC is my kind of contest.
I wish there was a simpler access to the kibitz data. They have a json listing, but it is not tied to the game directly.
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.