| View previous topic :: View next topic |
| Author |
Message |
Steve Mulligan
Joined: 20 Jul 2011
Posts: 114
Location: Ottawa, Canada
|
 Posted: Sun Feb 05, 2012 10:09 pm Post subject: Null Move Help |
  |
|
I'm trying to implement null move pruning and I'm not sure how to test it. I want to determine if my simple implementation is actually working, or if it's pruning too much.
eg, null move off I get f4f5 d6f6 f2g1 (score is 56)
and with null move on the best moves are e4e5 d6d5 f2g1 (score is 48)
Is it normal to see entirely different lines with scores that are worse? Or should null move on return the same score and best moves more often than not? |
|
| Back to top |
|
 |
Sven Schüle
Joined: 15 May 2008
Posts: 2240
Location: Berlin, Germany
|
| Posted: Sun Feb 05, 2012 10:36 pm Post subject: Re: Null Move Help |
|
|
| stevemulligan wrote: |
I'm trying to implement null move pruning and I'm not sure how to test it. I want to determine if my simple implementation is actually working, or if it's pruning too much.
eg, null move off I get f4f5 d6f6 f2g1 (score is 56)
and with null move on the best moves are e4e5 d6d5 f2g1 (score is 48)
Is it normal to see entirely different lines with scores that are worse? Or should null move on return the same score and best moves more often than not? |
Depends on your testing conditions. When using time control, correctly doing nullmove will let your engine search deeper, and so it will eventually choose a different best move. So you might want to test with fixed depth and compare whether nullmove keeps your final PV intact, which should be the goal since the main idea of nullmove pruning is to reduce the tree size. In this case, not matching PVs can be caused by
a) a bug,
b) a zugzwang situation that you don't handle, or
c) some hash table effect.
If you use nullmove also for other purposes, e.g. to detect threats, then it could have a regular impact on the PV.
Sven |
|
| Back to top |
|
|
Robert Purves
Joined: 15 Feb 2010
Posts: 155
Location: New Zealand
|
| Posted: Sun Feb 05, 2012 10:38 pm Post subject: Re: Null Move Help |
|
|
NMP, like most modifications to an engine, changes the search tree and so it is entirely normal to get a different score and PV.
NMP should give you a big improvement in time-to-depth (or nodes at given depth). In my engine NMP reduces those metrics to a half at low depths and a quarter in deep searches.
In self-play NMP should be worth roughly 100 Elo, easily confirmed in a fast match with only a few hundred games.
Robert P. |
|
| Back to top |
|
|
Steve Mulligan
Joined: 20 Jul 2011
Posts: 114
Location: Ottawa, Canada
|
| Posted: Sat Feb 11, 2012 9:19 am Post subject: Re: Null Move Help |
|
|
I setup cutechess-cli for the first time. What are good time control settings for self-play testing? I read somewhere I need like 1000 rounds to get an accurate ELO, is that true for self play as well?
This is what I'm using now (40 moves in 60 seconds):
cutechess-cli -engine conf=Pwned1 -engine conf=Pwned2 -each tc=40/60 -rounds 1000 -pgnout selfplay.pgn
engines.json
| Code: |
[
{
"name": "Pwned1",
"command": "pwned_console.exe",
"protocol": "xboard",
"workingDirectory": "C:\\Source\\pwned\\pwned_console\\bin\\Debug",
"initStrings": ["nullmove"]
},
{
"name": "Pwned2",
"command": "pwned_console.exe",
"protocol": "xboard",
"workingDirectory": "C:\\Source\\pwned\\pwned_console\\bin\\Debug",
"initStrings": []
}
]
|
(nullmove is the engine command to enable null move pruning/ordering) |
|
| Back to top |
|
|
Lucas Braesch
Joined: 31 May 2010
Posts: 1730
|
| Posted: Sat Feb 11, 2012 11:23 am Post subject: Re: Null Move Help |
|
|
| stevemulligan wrote: |
| I setup cutechess-cli for the first time. What are good time control settings for self-play testing? I read somewhere I need like 1000 rounds to get an accurate ELO, is that true for self play as well? |
Depends what you're testing and how fast your engine is. I use 6"+0.1", which is very fast, although I wouldn't recommend it for an engine that isn't fast and mature enough |
|
| Back to top |
|
|
Steve Mulligan
Joined: 20 Jul 2011
Posts: 114
Location: Ottawa, Canada
|
| Posted: Sat Feb 11, 2012 8:06 pm Post subject: Re: Null Move Help |
|
|
| Sorry I'm not that familiar with terms yet. What does 6"+0.1" look like in xboard syntax? |
|
| Back to top |
|
|
Tony Mokonen
Joined: 12 Mar 2006
Posts: 545
Location: Vancouver
|
| Posted: Sun Feb 12, 2012 1:01 am Post subject: Re: Null Move Help |
|
|
6 second base, with a tenth of a second (100 millisecond) increment.
level 0 0:6 0.1
is what Arena gives me if I set a 6 second base, and 100 millisecond increment. |
|
| Back to top |
|
|
Lucas Braesch
Joined: 31 May 2010
Posts: 1730
|
| Posted: Sun Feb 12, 2012 1:14 am Post subject: Re: Null Move Help |
|
|
| tmokonen wrote: |
6 second base, with a tenth of a second (100 millisecond) increment.
level 0 0:6 0.1
is what Arena gives me if I set a 6 second base, and 100 millisecond increment. |
using a GUI like Arena for such testing is a very bad idea. You can be sure there will be a ton of time losses that are 100% attributable to Arena, because it's way to slow to handle such time controls. Besides, you're not taking advantage of multiple cpus. I have a duo core and my program is single threaded so with cutechess-cli I run 2 games in parralel. For fast and parralel testing, cutechess-cli is beats the crap out of any GUI (precisely because it is not a *G*UI), and it also works easily with shell or python scripting. A great example is the combinaison of cutechess-cli and CLOP |
|
| Back to top |
|
|
Patrik Karlsson
Joined: 18 Jan 2009
Posts: 33
Location: Sweden
|
| Posted: Sun Feb 12, 2012 9:06 am Post subject: Re: Null Move Help |
|
|
| lucasart wrote: |
| tmokonen wrote: |
6 second base, with a tenth of a second (100 millisecond) increment.
level 0 0:6 0.1
is what Arena gives me if I set a 6 second base, and 100 millisecond increment. |
using a GUI like Arena for such testing is a very bad idea. You can be sure there will be a ton of time losses that are 100% attributable to Arena, because it's way to slow to handle such time controls. Besides, you're not taking advantage of multiple cpus. I have a duo core and my program is single threaded so with cutechess-cli I run 2 games in parralel. For fast and parralel testing, cutechess-cli is beats the crap out of any GUI (precisely because it is not a *G*UI), and it also works easily with shell or python scripting. A great example is the combinaison of cutechess-cli and CLOP |
Oh dear, I think Tony will explode this time. He didn't make a point of using Arena. He was just explaining what the time settings meant. |
|
| Back to top |
|
|
Tony Mokonen
Joined: 12 Mar 2006
Posts: 545
Location: Vancouver
|
| Posted: Sun Feb 12, 2012 9:59 am Post subject: Re: Null Move Help |
|
|
| elpapa wrote: |
Oh dear, I think Tony will explode this time. He didn't make a point of using Arena. He was just explaining what the time settings meant. |
Yeah, what he said.... 'twas a copy and paste from an output log, nothing more. Lucas does have a point with using command line tools, though. Haven't tried CLOP yet for myself, but it looks like an interesting concept. Maybe I could use it to try tune material values for variants. |
|
| Back to top |
|
|
|
FAQ
Search
Memberlist
Usergroups
Register
Profile
Log in to check your private messages
Log in
