lazy smp questions

Discussion of chess software programming and technical issues.

Moderator: Ras

User avatar
cdani
Posts: 2204
Joined: Sat Jan 18, 2014 10:24 am
Location: Andorra

Re: lazy smp questions

Post by cdani »

mbootsector wrote: Ok, then the results are more interesting. :)
Does the machine have two E5-2670 cpus? Intels pages show that E5-2670 has only 8 cores.
Exact! And is not a virtual server. It has Windows installed in the hard disk. Will be nice to know the results of someone with permanent access to one of these servers :-)
zullil
Posts: 6442
Joined: Tue Jan 09, 2007 12:31 am
Location: PA USA
Full name: Louis Zulli

Re: lazy smp questions

Post by zullil »

cdani wrote: There are not a lot of games, but seems probable that with more time or with more threads lazy mp is better than stockfish standard.
That would surprise me. If I understood your post, your testing was done at very short time control---less than a minute for all moves.
bob
Posts: 20943
Joined: Mon Feb 27, 2006 7:30 pm
Location: Birmingham, AL

Re: lazy smp questions

Post by bob »

zullil wrote:
cdani wrote: There are not a lot of games, but seems probable that with more time or with more threads lazy mp is better than stockfish standard.
That would surprise me. If I understood your post, your testing was done at very short time control---less than a minute for all moves.
Would surprise me too. Lazy SMP gets worse with more threads, not better...
Joost Buijs
Posts: 1635
Joined: Thu Jul 16, 2009 10:47 am
Location: Almere, The Netherlands

Re: lazy smp questions

Post by Joost Buijs »

YBWC and most other SMP schemes need a certain minimum depth before it catches on.
I can imagine that with very short time controls and very small depths a lazy SMP scheme has an advantage.
Not that anybody wants to play games with such short time controls in practice.
User avatar
cdani
Posts: 2204
Joined: Sat Jan 18, 2014 10:24 am
Location: Andorra

Re: lazy smp questions

Post by cdani »

zullil wrote:
cdani wrote: There are not a lot of games, but seems probable that with more time or with more threads lazy mp is better than stockfish standard.
That would surprise me. If I understood your post, your testing was done at very short time control---less than a minute for all moves.
I understand. I hope someone is able to do a longer test. I will try to do it also.
zullil
Posts: 6442
Joined: Tue Jan 09, 2007 12:31 am
Location: PA USA
Full name: Louis Zulli

Re: lazy smp questions

Post by zullil »

cdani wrote:
zullil wrote:
cdani wrote: There are not a lot of games, but seems probable that with more time or with more threads lazy mp is better than stockfish standard.
That would surprise me. If I understood your post, your testing was done at very short time control---less than a minute for all moves.
I understand. I hope someone is able to do a longer test. I will try to do it also.
Here are the results of a 50 game match. Time control is 300+3. Each engine using 20 threads on a 20 core machine. Hash = 4 GB. Standard Stockfish testing book: 2moves_v1.pgn

Code: Select all

Score of SF-master vs SF-lazySMP: 6 - 5 - 39  [0.510] 50
ELO difference: 7
Finished match
So, no conclusion---too few games. This took almost 12 hours.

zullil
Posts: 6442
Joined: Tue Jan 09, 2007 12:31 am
Location: PA USA
Full name: Louis Zulli

Re: lazy smp questions

Post by zullil »

User avatar
cdani
Posts: 2204
Joined: Sat Jan 18, 2014 10:24 am
Location: Andorra

Re: lazy smp questions

Post by cdani »

Two more tests, with not much cores but with more time and more games:
www.andscacs.com/stockfish/stockfish_la ... ames_2.zip

Code: Select all

rst_master_lsmpv2_100.1_6t_I75820K.pgn  (100 seconds + 0.1 added for move)
1   st_master    +2  +37/=279/-35 50.28%  176.5/351
2   st_lsmpv2    -2  +35/=279/-37 49.72%  174.5/351

rst_normal_lzmpv2_200.1_7t_FX8350.pgn  (200 seconds + 0.1 added for move)
1   st_modern          +11  +53/=400/-37 51.63%  253.0/490
2   st_modern_lzmpv2   -11  +37/=400/-53 48.37%  237.0/490
User avatar
hgm
Posts: 28354
Joined: Fri Mar 10, 2006 10:06 am
Location: Amsterdam
Full name: H G Muller

Re: lazy smp questions

Post by hgm »

It is a bit wasteful to determine effectivity of an SMP implementation (or in fact any speed-modifying change) by playing games. Measuring time-to-depth for a few hundred representative test positions should be enough to get a reliable impression. If the implementation would be 10% faster on average, it would very clearly stick out, and your accuracy could easily be good enough to distinguish 10% speedup from 9% speedup. 10% speedup would mean only 10 Elo improvement, i.e. about 1.5% score improvement, which would mean 3200 full games (~200,000 positions) to even convincingly (95% confidence) see there is an improvement. Now guess how many full games you need to see the difference between 9% and 10% improvement...
User avatar
hgm
Posts: 28354
Joined: Fri Mar 10, 2006 10:06 am
Location: Amsterdam
Full name: H G Muller

Re: lazy smp questions

Post by hgm »

It is a bit wasteful to determine effectivity of an SMP implementation (or in fact any speed-modifying change) by playing games. Measuring time-to-depth for a few hundred representative test positions should be enough to get a reliable impression. If the implementation would be 10% faster on average, it would very clearly stick out, and your accuracy could easily be good enough to distinguish 10% speedup from 9% speedup. 10% speedup would mean only 10 Elo improvement, i.e. about 1.5% score improvement, which would mean 3200 full games (~200,000 positions) to even convincingly (95% confidence) see there is an improvement. Now guess how many full games you need to see the difference between 9% and 10% improvement...