something weird

Discussion of anything and everything relating to chess playing software and machines.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
stavros
Posts: 158
Joined: Tue Dec 02, 2014 12:29 am

something weird

Post by stavros » Tue Dec 31, 2019 10:28 pm

hi. i have made some tests and observed this:
old stockfish 9 reached depth 32 at 45 sec average
26/10/19 sf dv reached 32 depth at 60+ sec average
so sf 9 is faster but in tests is far weaker vs sf dv
is there an explanation?

Dann Corbit
Posts: 11216
Joined: Wed Mar 08, 2006 7:57 pm
Location: Redmond, WA USA
Contact:

Re: something weird

Post by Dann Corbit » Tue Dec 31, 2019 10:37 pm

YBW search verses Lazy SMP search.
YBW is faster in time to ply.
Lazy SMP is better in Elo strength.
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.

Raphexon
Posts: 261
Joined: Sun Mar 17, 2019 11:00 am
Full name: Henk Drost

Re: something weird

Post by Raphexon » Tue Dec 31, 2019 10:41 pm

Dann Corbit wrote:
Tue Dec 31, 2019 10:37 pm
YBW search verses Lazy SMP search.
YBW is faster in time to ply.
Lazy SMP is better in Elo strength.
SF9 uses Lazy SMP.
I think some dev version of SF6 started using Lazy SMP*.
SF6 was the last YWB-fish
Last edited by Raphexon on Tue Dec 31, 2019 10:49 pm, edited 1 time in total.

Dann Corbit
Posts: 11216
Joined: Wed Mar 08, 2006 7:57 pm
Location: Redmond, WA USA
Contact:

Re: something weird

Post by Dann Corbit » Tue Dec 31, 2019 10:47 pm

You are right, I looked it up. SF7 changed to Lazy SMP.

In that case, it is probably changes to the extensions and pruning rules.

At any rate, you cannot directly correlate depth to Elo, except to say that deeper searches are generally better than shallower searches by the same engine.
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.

User avatar
Ovyron
Posts: 4341
Joined: Tue Jul 03, 2007 2:30 am

Re: something weird

Post by Ovyron » Tue Dec 31, 2019 10:50 pm

The general explanation is that Depths of different engines mean different things, as an engine evolves it will stop meaning the same as old versions. For instance if a new evaluation term is introduced and it's very heavy, every cycle that uses it will make the iteration take longer, so depth will decrease, even though the engine will play stronger (Crystal is an example of this, with much thicker Depths than Stockfish.) If an evaluation term becomes deprecated and it is found that it's not worth the extra time used and it's removed, the engine will get deeper depth faster (and will play worse in some positions where that term was useful. But if of every 100 affected positions, it becomes dumber in 49 but reaches bigger Depth and plays a better move in 51, it's a net gain.)

I still remember doing some Depth 10 match between Pro Deo and Houdini 5 or some version and was happy to see Pro Deo absolutely DESTROY Houdini. Until I realized it meant nothing (you can actually make a personality of Pro Deo that is much stronger than default at fixed depth, you just make it advance through the iterations very slowly), because of this.

bob
Posts: 20916
Joined: Mon Feb 27, 2006 6:30 pm
Location: Birmingham, AL

Re: something weird

Post by bob » Wed Jan 01, 2020 12:31 am

Dann Corbit wrote:
Tue Dec 31, 2019 10:37 pm
YBW search verses Lazy SMP search.
YBW is faster in time to ply.
Lazy SMP is better in Elo strength.
I'm not sure that is the truth, the whole truth, and nothing but the truth.

I don't believe lazy SMP is inherently stronger than YBW and its derivatives (such as DTS). It does seem to scale better, in that adding additional cores/threads improves playing skill, where the YBW/DTS algorithm encounters difficulty at the 32-64 core level due to lock contention and the shared data structures necessary to implement YBW/DTS.

Given a 4-cpu system, I would be utterly amazed if lazy SMP was better. In fact, I would be utterly amazed if it were even close to a good YBW/DTS implementation. On a 128 core system, I would expect lazy SMP to be significantly stronger due to the lock contention / shared memory issues I mentioned. Somewhere there is a crossover point where lazy SMP gains the advantage. It is inherently inferior as a search algorithm compared to YBW, until you reach the crossover and the SMP overhead in YBW causes issues. I have seen implementations that would play worse with 32 cores than with 16, for example. Lazy SMP doesn't seem to suffer from that issue. It just scales significantly worse until you reach the magic crossover point.

Depth is a completely different issue. depth means different things to different programs. The old "King" engine would often 2-3 minutes on a 2 ply search, and uncork a 10-15 move long PV. LOTS of extensions. If you did a depth=2 test against other programs it would blow them away, because it would take 2-3 minutes per move while the opponents would take a fraction of a second (back then).

User avatar
Ovyron
Posts: 4341
Joined: Tue Jul 03, 2007 2:30 am

Re: something weird

Post by Ovyron » Wed Jan 01, 2020 3:03 am

If those are the extremes now I wonder how high a Depth does Stockfish need to play at the level of The King's Depth 2. Probably the extreme is some Komodo Personality made to reach high depth, I've seen it reach Depth 60 in middle game positions in under a minute.

corres
Posts: 3062
Joined: Wed Nov 18, 2015 10:41 am
Location: hungary

Re: something weird

Post by corres » Thu Jan 02, 2020 4:55 pm

Ovyron wrote:
Tue Dec 31, 2019 10:50 pm
The general explanation is that Depths of different engines mean different things, as an engine evolves it will stop meaning the same as old versions. For instance if a new evaluation term is introduced and it's very heavy, every cycle that uses it will make the iteration take longer, so depth will decrease, even though the engine will play stronger (Crystal is an example of this, with much thicker Depths than Stockfish.)
...
At least the engine will play stronger in the sense of the new evaluation term.
In general Crystal plays weaker than Stockfish.dev but it finds mate and solving of puzzles more faster than Stockfish.dev.

stavros
Posts: 158
Joined: Tue Dec 02, 2014 12:29 am

Re: something weird

Post by stavros » Sun Jan 05, 2020 8:37 pm

hello friends
making something experiments i found something paranormal fhenomenom
my device has 8 cores with 5200kn/sec with time to depth 33d/120secs average ,hash at 512mb
then i reduced the number of cores to 2 and i found this:2090kn/sec with time to depth 33d at 90 secs!!! 512 hash also.
tested up to 20 times ,with reduced cores lower speed but faster time of depth?
anyone to explain this mystery?

ps.i used droidfish latest sf dv arm8 android 9

mwyoung
Posts: 2114
Joined: Wed May 12, 2010 8:00 pm

Re: something weird

Post by mwyoung » Sun Jan 05, 2020 10:19 pm

stavros wrote:
Sun Jan 05, 2020 8:37 pm
hello friends
making something experiments i found something paranormal fhenomenom
my device has 8 cores with 5200kn/sec with time to depth 33d/120secs average ,hash at 512mb
then i reduced the number of cores to 2 and i found this:2090kn/sec with time to depth 33d at 90 secs!!! 512 hash also.
tested up to 20 times ,with reduced cores lower speed but faster time of depth?
anyone to explain this mystery?

ps.i used droidfish latest sf dv arm8 android 9
There is no mystery here. I have explained this many times. And this is why you can not use NPS to judge the speed of the engine. When using different core count. If you want to know the true difference in speed you must use time to depth. And this only gets worse as the core count gets higher.

What is your CPU? Your scaling is very poor.... And your Speed at 5200kn/s is very slow for 8 Cores. My phone is almost as fast with 8 cores.

If you are running a phone. Sometimes not all cores are clocked at the same speed.
Professing themselves to be wise, they became fools,
take on me. Foes 0.

Post Reply