LMR Research

Discussion of chess software programming and technical issues.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
bob
Posts: 20638
Joined: Mon Feb 27, 2006 6:30 pm
Location: Birmingham, AL

Re: LMR Research

Post by bob » Wed Jul 04, 2012 9:30 pm

ZirconiumX wrote:
bob wrote:
lucasart wrote:
ZirconiumX wrote:There are some programs which research after LMR if value >= beta, some if value > alpha, and some don't research at all.

FF currently uses value >= beta, but I haven't tested this. This is why I ask if anyone else *has*.

Something tells me that value > alpha is best, because it allows Cut-Nodes AND PV-Nodes, rather than either only Cut-Nodes (value >= beta), or nothing (no research).

Matthew:out
Thanks Matt. You're right!!

I replaced >= beta by > alpha and DiscoCheck scored over 54% after 2000 games against itself (easily significant).

Bob: yes there are much fewer PV nodes, but triggering a full window research in those realatively fewer nodes is much costlier, so overall the difference in node count is actually quite significant (ie. >alpha is slower in time to depth, but more accurate, in the end tha accuracy wins for me).

One case where >= beta makes sense is if your engine doesnt reduce at PV nodes, of course (in which case >alpha and >= beta will produce the same result).
You should not draw conclusions on a few test positions, rather over complete games, where the difference is not so big. >alpha is still the right answer, because any score > alpha when you are expecting all moves to fail low indicates that something is not right. And most likely it will still fail low on the re-search when the depth returns to normal, and that is a LOT cheaper than accepting that fail-high (or actual score from a search that is too shallow) and then triggering a window-widening search, only to discover it really doesn't fail high like you thought...

IE it is better to just verify that the fail-high move is really going to fail high, rather than having to search ALL moves at this ply to see if that single move will fail high normally...
Bob,
Yes, I know that you prefer a gauntlet of engines over games against itself, but please, where did Lucas mention test positions?!?!

Lucas,
Glad to help, that's 2 improvements I've made to DiscoCheck. According to Sigma Chess, that means you've got about +30 ELO because of that.

Don,
That is an interesting idea. Maybe you could have a variable margin over depth, the idea being that at a higher depth, you may miss more. But then you pay the price of speed. Maybe that is something for you to try, Lucas? (*hint*)

Matthew:out
I didn't say "test positions". I said "positions". 2000 games is a marginal test with a significant error bar. Probably too big to measure which of the two is better.

>alpha is the only thing that makes any sense to me, otherwise you invite errors and search instability into the equation... You don't want to accept value > alpha unless it is with a proper search to the correct depth...

syzygy
Posts: 4457
Joined: Tue Feb 28, 2012 10:56 pm

Re: LMR Research

Post by syzygy » Wed Jul 04, 2012 10:32 pm

bob wrote:You should not draw conclusions on a few test positions, rather over complete games, where the difference is not so big.
bob wrote:I didn't say "test positions". I said "positions".

diep
Posts: 1780
Joined: Thu Mar 09, 2006 10:54 pm
Location: The Netherlands
Contact:

Re: LMR Research

Post by diep » Wed Jul 04, 2012 11:21 pm

syzygy wrote:
bob wrote:You should not draw conclusions on a few test positions, rather over complete games, where the difference is not so big.
bob wrote:I didn't say "test positions". I said "positions".
In the context it's obvious what Bob meant.

User avatar
marcelk
Posts: 348
Joined: Fri Feb 26, 2010 11:21 pm
Contact:

Re: LMR Research

Post by marcelk » Wed Jul 04, 2012 11:26 pm

syzygy wrote:
bob wrote:You should not draw conclusions on a few test positions, rather over complete games, where the difference is not so big.
bob wrote:I didn't say "test positions". I said "positions".
He wrote "test positions", but said "positions" when typing the words. Learn to read and master some basic English vocabulary!

But seriously, I hereby coin Marcel's Law of Computer Chess Communications (MLCCC):
MLCCC wrote:As in a computer chess discussion the age variance increases,
the probability of a dictionary reference approaches 1.
"

User avatar
lucasart
Posts: 3041
Joined: Mon May 31, 2010 11:29 am
Full name: lucasart
Contact:

Re: LMR Research

Post by lucasart » Thu Jul 05, 2012 4:16 am

bob wrote: I didn't say "test positions". I said "positions". 2000 games is a marginal test with a significant error bar. Probably too big to measure which of the two is better.
Wrong!
Of course if the score is 50.1% after 200 games, it's not significant, but the 54.5% I got *is* significant. Just do tha math to convince yourself !

User avatar
lucasart
Posts: 3041
Joined: Mon May 31, 2010 11:29 am
Full name: lucasart
Contact:

Re: LMR Research

Post by lucasart » Thu Jul 05, 2012 1:39 pm

lucasart wrote:
bob wrote: I didn't say "test positions". I said "positions". 2000 games is a marginal test with a significant error bar. Probably too big to measure which of the two is better.
Wrong!
Of course if the score is 50.1% after 200 games, it's not significant, but the 54.5% I got *is* significant. Just do tha math to convince yourself !
*typo* I meant 2000 games of course

ZirconiumX
Posts: 1327
Joined: Sun Jul 17, 2011 9:14 am

Re: LMR Research

Post by ZirconiumX » Thu Jul 05, 2012 2:51 pm

lucasart wrote:
bob wrote: I didn't say "test positions". I said "positions". 2000 games is a marginal test with a significant error bar. Probably too big to measure which of the two is better.
Wrong!
Of course if the score is 50.1% after 2000 games, it's not significant, but the 54.5% I got *is* significant. Just do the math to convince yourself!
We need Jesus Munoz's magic tool for this.

Matthew:out
Some believe in the almighty dollar.

I believe in the almighty printf statement.

User avatar
lucasart
Posts: 3041
Joined: Mon May 31, 2010 11:29 am
Full name: lucasart
Contact:

Re: LMR Research

Post by lucasart » Thu Jul 05, 2012 2:58 pm

ZirconiumX wrote:
lucasart wrote:
bob wrote: I didn't say "test positions". I said "positions". 2000 games is a marginal test with a significant error bar. Probably too big to measure which of the two is better.
Wrong!
Of course if the score is 50.1% after 2000 games, it's not significant, but the 54.5% I got *is* significant. Just do the math to convince yourself!
We need Jesus Munoz's magic tool for this.

Matthew:out
no need, a few very elementary calculations: empirical mean and stdev and gaussian distribution function. i do that in a spreadsheet in less than 30 sec

ZirconiumX
Posts: 1327
Joined: Sun Jul 17, 2011 9:14 am

Re: LMR Research

Post by ZirconiumX » Thu Jul 05, 2012 3:01 pm

lucasart wrote:
ZirconiumX wrote:
lucasart wrote:
bob wrote: I didn't say "test positions". I said "positions". 2000 games is a marginal test with a significant error bar. Probably too big to measure which of the two is better.
Wrong!
Of course if the score is 50.1% after 2000 games, it's not significant, but the 54.5% I got *is* significant. Just do the math to convince yourself!
We need Jesus Munoz's magic tool for this.

Matthew:out
no need, a few very elementary calculations: empirical mean and stdev and gaussian distribution function. i do that in a spreadsheet in less than 30 sec
And for those who can't?

Matthew:out
Some believe in the almighty dollar.

I believe in the almighty printf statement.

bob
Posts: 20638
Joined: Mon Feb 27, 2006 6:30 pm
Location: Birmingham, AL

Re: LMR Research

Post by bob » Thu Jul 05, 2012 3:09 pm

lucasart wrote:
bob wrote: I didn't say "test positions". I said "positions". 2000 games is a marginal test with a significant error bar. Probably too big to measure which of the two is better.
Wrong!
Of course if the score is 50.1% after 200 games, it's not significant, but the 54.5% I got *is* significant. Just do tha math to convince yourself !
I am not talking about the result of the change. I was talking about your statement about affecting the size of the tree due to researches and where they are done. For me, first rule of investigating a change is to understand how it affects the tree. If you play very fast games, and you are changing the shape of the tree significantly, the results can easily be skewed for, or against the change, where the results might be completely different at a longer time control. In fast games, reduced searches often drop right into q-search, which is cheap. In long games, they don't...

Post Reply