Threads test incl. Stockfish 7

Discussion of anything and everything relating to chess playing software and machines.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
fastgm
Posts: 409
Joined: Mon Aug 19, 2013 4:57 pm
Contact:

Threads test incl. Stockfish 7

Post by fastgm » Mon Jan 11, 2016 7:12 pm

I have included the new Stockfish 7 with Lazy SMP implemantation in my threads test.

Here the data of the test and the graphical presentation, see also: http://www.fastgm.de/threads5.html including the test conditions.

Andreas

Image

Image

Jouni
Posts: 2005
Joined: Wed Mar 08, 2006 7:15 pm

Re: Threads test incl. Stockfish 7

Post by Jouni » Mon Jan 11, 2016 7:59 pm

I was waiting for this already :) thanks. Nice progress, if not at Komodo level? SF7 gets +228 from 16 cores, but Komodo 8 in older test 265. Seems to be same conditions.
Jouni

User avatar
Nordlandia
Posts: 2447
Joined: Fri Sep 25, 2015 7:38 pm
Location: Sortland, Norway

Re: Threads test incl. Stockfish 7

Post by Nordlandia » Mon Jan 11, 2016 8:39 pm

Nice representation and graphics :)

bob
Posts: 20555
Joined: Mon Feb 27, 2006 6:30 pm
Location: Birmingham, AL

Re: Threads test incl. Stockfish 7

Post by bob » Mon Jan 11, 2016 11:28 pm

As a note, and you will probably hear this from others, very fast searches have always been a weakness of normal parallel searches. They begin to "bear down" as time stretches out. It would be much more interesting to stretch the time to 60 seconds per move rather than about 1 second since that is a more useful / common time control limit. Also it would be nice to plot X vs 2x where 2x is the single-cpu version gives 2x as much time per move, to get a "optimal" potential parallel speedup. Ditto for 4x, 8x and 16x. Be nice to see the actual upper bound in addition to the smp numbers...

Jouni
Posts: 2005
Joined: Wed Mar 08, 2006 7:15 pm

Re: Threads test incl. Stockfish 7

Post by Jouni » Tue Jan 12, 2016 3:18 pm

And how consistent this is with framework test? There 15cpu SF7 got +131 vs SF6. Here it is 62 + 44 = 106. In any case homepage is way too modest with "This release is around 60 ELO stronger than Stockfish 6 in self play." AT LEAST 100 ELO STRONGER!
Jouni

bob
Posts: 20555
Joined: Mon Feb 27, 2006 6:30 pm
Location: Birmingham, AL

Re: Threads test incl. Stockfish 7

Post by bob » Wed Jan 13, 2016 10:01 pm

Jouni wrote:And how consistent this is with framework test? There 15cpu SF7 got +131 vs SF6. Here it is 62 + 44 = 106. In any case homepage is way too modest with "This release is around 60 ELO stronger than Stockfish 6 in self play." AT LEAST 100 ELO STRONGER!
You do realize SF uses self-play for testing? And you should remember that a rating computed from playing yourself is usually exaggerated significantly compared to playing a gauntlet of other programs. As much as double in fact. So I think their estimate is pretty reasonable based on scaling it back to reflect games against others rather than against an older version of their own program...

Jouni
Posts: 2005
Joined: Wed Mar 08, 2006 7:15 pm

Re: Threads test incl. Stockfish 7

Post by Jouni » Thu Jan 14, 2016 5:14 pm

Definitely. SF7 got +62 in selfplay (framework) and +58 against other engines (IPON). "usually exaggerated significantly" seems to be history :) .
Jouni

APassionForCriminalJustic
Posts: 415
Joined: Sat May 24, 2014 7:16 am

Re: Threads test incl. Stockfish 7

Post by APassionForCriminalJustic » Thu Jan 14, 2016 6:06 pm

Jouni wrote:Definitely. SF7 got +62 in selfplay (framework) and +58 against other engines (IPON). "usually exaggerated significantly" seems to be history :) .
If you really believe that Stockfish 7 is like 100+ Elo over Stockfish 6 with a lot of cores then you are living in lala land. At standard match play I would seriously doubt 100+ Elo. Lazy smp will probably fail in the end given that it is a belief amongst some of the experts that it is not a very good smp implementation. Lazy didn't do well for TCEC nor was it successful in Clemens Keck's latest tournament versus Komodo. Who really cares about bullet games where lazy apparently shines. Nobody cares about these fast, pointless games.

Dann Corbit
Posts: 10112
Joined: Wed Mar 08, 2006 7:57 pm
Location: Redmond, WA USA
Contact:

Re: Threads test incl. Stockfish 7

Post by Dann Corbit » Thu Jan 14, 2016 6:37 pm

I am really keen to see CCRL 40/40 results for many threads for both SF and Crafty. They both have shiny new threading implementations with high hopes.

The only thing that will reveal the truth is measurement. Someone has a sig about theory verses practice. Let me say this about that. You gotta measure.
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.

User avatar
mhull
Posts: 12440
Joined: Wed Mar 08, 2006 8:02 pm
Location: Dallas, Texas
Full name: Matthew Hull

Re: Threads test incl. Stockfish 7

Post by mhull » Thu Jan 14, 2016 6:44 pm

Dann Corbit wrote:I am really keen to see CCRL 40/40 results for many threads for both SF and Crafty. They both have shiny new threading implementations with high hopes.
CCRL has an 8-engine match going which includes Stockfish:
http://www.talkchess.com/forum/viewtopi ... 421#654421

And an 8-engine swiss of mass proportions including Crafty 25.0:
http://www.talkchess.com/forum/viewtopi ... 099#653099

But Crafty won't get truly interesting, parallel-wise until Bob completes his conversion to DTS later this year.
Dann Corbit wrote: The only thing that will reveal the truth is measurement. Someone has a sig about theory verses practice. Let me say this about that. You gotta measure.
...theory versus practice... ;)
Matthew Hull

Post Reply