Lazy eval

[rvida]

But with margins of around half a pawn, we only get about a 2% speedup (roughly 4% more nps but a 2% increase in nodes) and our quality clearly suffers too much for that. Can you check what speedup you get, and with what margins?

This is clearly impossible. You must have an error in implementation.
The usual speed saving is 10% for 2-3 pawns margin. For under 100cp margin you should get well over 15%.

Well, Critter gets about 10% for a 60 cp margin, so not all programs get as much as you say. But our figures are indeed suspiciously low. Still, it's possible that other things we do are so similar to lazy eval that this could be the explanation. We'll have to investigate further.

Not sure where did you get the quoted 60cp margin. Critter uses 150 cp lazy margin in qsearch and 300 cp margin at interior nodes. Also, when the absolute value of positional component from ply-1 eval is greater than 150 cp, lazy eval is not used.

[Houdini]

Also, when the absolute value of positional component from ply-1 eval is greater than 150 cp, lazy eval is not used.

What a coincidence, Houdini does exactly the same...
LOL.

[lkaufman]

I note that SF rejected lazy eval. So at least one other top program failed to demonstrate a benefit from it. Why didn't it help Stockfish? I'm sure they would not have rejected a 10% nearly-free speedup.

For a very selective search you may want to have accurate eval info to push pruning to the limits. In case of Crafty, where the pruning is far lower I guess given the 300 ELO gap or so, my guess is the speed up plays a factor more than accurate eval. Another point are the spikes in king evaluations, present in SF and, as you mentioned also in Komodo, these are filtered out with lazy eval, but play an important role for pruning decision.

I still should have the SF lazy eval patch somewhere, I can post it if someone is interested in experimenting with this.

Do you remember how much of a NPS gain you got from lazy eval? If not we could test it with your patch.

[Milos]

Also, when the absolute value of positional component from ply-1 eval is greater than 150 cp, lazy eval is not used.

What a coincidence, Houdini does exactly the same...
LOL.

If you use a pure positional value that's a bit too restrictive. I would lower the margin for that condition and use positional-PST instead of just positional.
Also the margin should be dependent on if the position is quiet or not.

[Houdini]

Also, when the absolute value of positional component from ply-1 eval is greater than 150 cp, lazy eval is not used.

What a coincidence, Houdini does exactly the same...
LOL.

If you use a pure positional value that's a bit too restrictive. I would lower the margin for that condition and use positional-PST instead of just positional.
Also the margin should be dependent on if the position is quiet or not.

You may or may not be right, but that wasn't my point.
It's just funny to see how much of Houdini ideas have already made it into other engines...

[lkaufman]

But with margins of around half a pawn, we only get about a 2% speedup (roughly 4% more nps but a 2% increase in nodes) and our quality clearly suffers too much for that. Can you check what speedup you get, and with what margins?

This is clearly impossible. You must have an error in implementation.
The usual speed saving is 10% for 2-3 pawns margin. For under 100cp margin you should get well over 15%.

Well, Critter gets about 10% for a 60 cp margin, so not all programs get as much as you say. But our figures are indeed suspiciously low. Still, it's possible that other things we do are so similar to lazy eval that this could be the explanation. We'll have to investigate further.

Not sure where did you get the quoted 60cp margin. Critter uses 150 cp lazy margin in qsearch and 300 cp margin at interior nodes. Also, when the absolute value of positional component from ply-1 eval is greater than 150 cp, lazy eval is not used.

Don quoted the value of 150 in qsearch for Critter, which I took to mean 150/256 or about 0.6 since you use 256 for a pawn. Was this 150 actually in centipawns, i.e. 384 on your scale? If so it is amazing that you get 10% for lazy with such a large margin.

[jdart]

I've always used it. I use a pretty large margin for evaluating a score lazily - i.e. it has to be far above beta, or below alpha, to skip the rest of the eval. Once in a while I turn on a debug mode that prints out the FENs of any positions that would have been inaccurately scored with lazy eval - in other words, would have caused an early cutoff but if the full eval was done, the score would have been below beta (for a high cutoff) or above alpha (for a low cutoff). Usually that's a small number, so I am being conservative and not accepting a lot of error.

As noted, the problem is king safety. For me that's an expensive calculation, so I'd like to avoid it with lazy eval. But the king safety scores can be relatively large. Hence the need for a high margin, which reduced the effectiveness of lazy eval.

--Jon

[Milos]

You may or may not be right, but that wasn't my point.
It's just funny to see how much of Houdini ideas have already made it into other engines...

This particular idea is something that is below 5Elo and practically unmeasurable.
I guess some ppl copy ideas mechanically following the principle if it doesn't hurt it helps. Others however, require to see measurable benefit.
Different ppl, different approaches .

P.S. I wouldn't be too worried. Only 2 ppl capable and willing to RE for ideas (and with their own strong engines) have done it so far . Others just go around and ask questions .
P.P.S You also made it too easy since everyone has an exact source of Houdini beta .

[Houdini]

You may or may not be right, but that wasn't my point.
It's just funny to see how much of Houdini ideas have already made it into other engines...

This particular idea is something that is below 5Elo and practically unmeasurable.
I guess some ppl copy ideas mechanically following the principle if it doesn't hurt it helps. Others however, require to see measurable benefit.
Different ppl, different approaches .

P.S. I wouldn't be too worried. Only 2 ppl capable and willing to RE for ideas (and with their own strong engines) have done it so far . Others just go around and ask questions .
P.P.S You also made it too easy since everyone has an exact source of Houdini beta .

I'm not worried. Like I said before, imitation is the sincerest form of flattery.

I just notice that Houdini's specific implementation of an original idea is being presented by an acclaimed author, without giving any credit.
Unless this has been in Critter from before version 1.0 (Richard RE'd Houdini 1.5 between Critter 0.90 and 1.0), and we've just ended up by a great coincidence using exactly the same code?!

Robert

[Rebel]

Not sure where did you get the quoted 60cp margin. Critter uses 150 cp lazy margin in qsearch and 300 cp margin at interior nodes. Also, when the absolute value of positional component from ply-1 eval is greater than 150 cp, lazy eval is not used.

My LE-margin is 0.50 for the leafs and QS.

Here is what I do.

ISCORE = incremental update score of material + PST's
SCORE = score of EVAL

RDx = remaining depth

In EVAL I stack (ISCORE-SCORE) + LE-margin (0.50)

Level=3 Ply

1. e4 e5 2. Nf3

Ar RD0 (Nf3) before make_move:

1. Update ISCORE
2. SCORE = ISCORE + stacked score of RD1 (e5)
3. if (alpha >= SCORE) allow LE and prune; else EVAL

Same for the whole QS.

That's it except for the usual troublemakers so I exclude LE when:

1. King in Check
2. Move gives a check (you probably need to write some new code)
3. Extreme cases like an unstoppable pawn (+800) at RD1, I stack those also in EVAL.