fail soft vs fail hard

Discussion of chess software programming and technical issues.

Moderators: hgm, chrisw, Rebel

cyberfish

fail soft vs fail hard

Post by cyberfish »

I have always been wondering, what are the advantages to failing hard?

Wouldn't fail soft always reduce the number of nodes to search?
User avatar
hgm
Posts: 28205
Joined: Fri Mar 10, 2006 10:06 am
Location: Amsterdam
Full name: H G Muller

Re: fail soft vs fail hard

Post by hgm »

I think failing soft is always superior from a performance point of view. (If you use the scores in the hash table for cutoffs, that is. Otherwise it is strictly equivalent.)

The only advantage I see is that with fail hard you will catch any bugs in your alpha-beta implementation immediately, as any cutoff will only be marginal: you will never have scores above beta, the cutoffs all occur by a score equal to beta. This can be important in implementations where setting the search window is not trivial. Such as what I do when using a delayed-loss bonus.
AndrewShort

Re: fail soft vs fail hard

Post by AndrewShort »

cyberfish wrote:I have always been wondering, what are the advantages to failing hard?

Wouldn't fail soft always reduce the number of nodes to search?
-fail soft requires an extra local variable
-fail hard is simpler
-however, with hashing, fail soft should result in fewer nodes searched
-fail soft also allows things like null move mate threat extension, should you want to implement that
bob
Posts: 20943
Joined: Mon Feb 27, 2006 7:30 pm
Location: Birmingham, AL

Re: fail soft vs fail hard

Post by bob »

hgm wrote:I think failing soft is always superior from a performance point of view. (If you use the scores in the hash table for cutoffs, that is. Otherwise it is strictly equivalent.)

The only advantage I see is that with fail hard you will catch any bugs in your alpha-beta implementation immediately, as any cutoff will only be marginal: you will never have scores above beta, the cutoffs all occur by a score equal to beta. This can be important in implementations where setting the search window is not trivial. Such as what I do when using a delayed-loss bonus.
I would add that it probably depends on your basic search approach. If you use PVS, so that almost 100% of the searches are with a null-window, fail-soft offers nothing over fail-hard. The only case I have found where fail-soft is important is in mtd(f) where fail-soft gives you a decent "hint" about how far off the root null-window actually is, so that you can converge faster.

I have tried both in Crafty and found no difference that could be measured... But then I have been using PVS since around 1980 or so...
User avatar
ilari
Posts: 750
Joined: Mon Mar 27, 2006 7:45 pm
Location: Finland

Re: fail soft vs fail hard

Post by ilari »

bob wrote:I have tried both in Crafty and found no difference that could be measured...
Same thing with Sloppy, so I erred on the safe side and went with fail-hard.
User avatar
hgm
Posts: 28205
Joined: Fri Mar 10, 2006 10:06 am
Location: Amsterdam
Full name: H G Muller

Re: fail soft vs fail hard

Post by hgm »

What is safe about going for the solution that performs at best equal?
bob
Posts: 20943
Joined: Mon Feb 27, 2006 7:30 pm
Location: Birmingham, AL

Re: fail soft vs fail hard

Post by bob »

hgm wrote:What is safe about going for the solution that performs at best equal?
It is simpler.
User avatar
michiguel
Posts: 6401
Joined: Thu Mar 09, 2006 8:30 pm
Location: Chicago, Illinois, USA

Re: fail soft vs fail hard

Post by michiguel »

hgm wrote:What is safe about going for the solution that performs at best equal?
Simplicity of execution ---> less chances for bugs or unintended behaviors. Unless there is a *demonstration* that simplicity pays a performance price. However, I did not follow my own advice :-)

We had a discussion some time ago, where I show that having fail hard in quiescence eliminated a strange behavior in an endgame position (that is because qsearch eliminates moves by nature). We discussed in length that the cause may have been a set of reasons, including a wrong way to adjust alpha and beta. I am not sure we agreed on everything, but fail soft in quiescence made things worst.

IMHO, I think that if PVS is used, a beginner will be better off using fail hard, or have a compiler switch to do one or the other and test. I did not do that. I am using fail hard in Qsearch() and fail soft in search() for now.

Miguel
User avatar
hgm
Posts: 28205
Joined: Fri Mar 10, 2006 10:06 am
Location: Amsterdam
Full name: H G Muller

Re: fail soft vs fail hard

Post by hgm »

Although this might be true, it seems to me that this kind of simplification would be infinitesimal on the scale of any program that is significantly larger than micro-Max. Which, btw, uses fail soft...

The gains in terms of simplicity that you would reap by not using PVS, but simple alpha-beta, is far larger.
Pablo Vazquez
Posts: 154
Joined: Thu May 31, 2007 9:05 pm
Location: Madrid, Spain

Re: fail soft vs fail hard

Post by Pablo Vazquez »

But if you do fail hard in qsearch(), you are going to return a score in the range of [alpha, beta]. And then, doing fail soft in search() would be the same as fail hard, or am i missing something?