From the help-file:
Arena.exe /fcp SOS /scp Crafty /lpf C:\Arena\123test.EPD /lpi 2 /mg 4 /inc 1 /tc 01:00
Starts Arena with first engine SOS, second engine Crafty for a match for 4 games with the second position of the specified EPD file in Fischer-clock-mode 1/1.
=====
Does anyone know how to set the level to fixed depth?
Arena question
Moderators: hgm, Rebel, chrisw
-
- Posts: 27811
- Joined: Fri Mar 10, 2006 10:06 am
- Location: Amsterdam
- Full name: H G Muller
Re: Arena question
Amazing! Exactly the same command-line arguments would have to be used with WinBoard, to do the same thing! Perhaps /searchDepth D, which is what you would have to add in WinBoard to do as you ask, would work here too?
-
- Posts: 6997
- Joined: Thu Aug 18, 2011 12:04 pm
- Full name: Ed Schröder
Re: Arena question
I tried various logical options including yours, to no avail. Then looked into the executable and unfortunately a command-line option for fixed depth does not exist. Will have a look at Winboard then
-
- Posts: 20943
- Joined: Mon Feb 27, 2006 7:30 pm
- Location: Birmingham, AL
Re: Arena question
Obvious question: "Why fixed depth?" Are you SURE that the two programs have equivalent "plies"? Identical extensions and reductions and such, so that this is not a handicap match where the handicap is not obvious???Rebel wrote:I tried various logical options including yours, to no avail. Then looked into the executable and unfortunately a command-line option for fixed depth does not exist. Will have a look at Winboard then
-
- Posts: 6997
- Joined: Thu Aug 18, 2011 12:04 pm
- Full name: Ed Schröder
Re: Arena question
I am using cutechess-cli in the meantime. To test my eval I run fixed depth self-play matches controlled by a depth based flexible parameter. For instance when I set the following parameter [PLY = 8] as a base then:bob wrote:Obvious question: "Why fixed depth?" Are you SURE that the two programs have equivalent "plies"? Identical extensions and reductions and such, so that this is not a handicap match where the handicap is not obvious???Rebel wrote:I tried various logical options including yours, to no avail. Then looked into the executable and unfortunately a command-line option for fixed depth does not exist. Will have a look at Winboard then
1. midgame positions will be done 8 plies
2. midgame (minus queens) 10 plies
3. normal endgame 11 plies
4. rook endings 12 plies
5. minor endings 13 plies
6. pawn endings 14 plies
7. queen endings 10 plies
When I raise the parameter to 9 all depths increase with one and vice versa. When I have a handful of improvements I try them with regular time control.
I am indirectly using the system also to measure the impact of search changes. Say you add a static reduction which gives you a 10% speed-up but loses 20 elo points, it's likely a bad idea. Getting insight in the mysterious ways of the search is the keyword here.
-
- Posts: 20943
- Joined: Mon Feb 27, 2006 7:30 pm
- Location: Birmingham, AL
Re: Arena question
I don't see how you can conclude the latter. If you make the search 10% faster, at fixed depth you see absolutely no benefit. Take a simple example. Search to 10 plies and stop. You now decide to try more aggressive pruning, and you notice that the search is now 2x faster. But at a fixed depth of 10, it may well lose a little due to the pruning. But the extra ply (which you will never see in a fixed depth test) might gain more than the pruning loss. IF you had been using a fixed time limit instead so that the 2x faster would turn into one extra ply.Rebel wrote:I am using cutechess-cli in the meantime. To test my eval I run fixed depth self-play matches controlled by a depth based flexible parameter. For instance when I set the following parameter [PLY = 8] as a base then:bob wrote:Obvious question: "Why fixed depth?" Are you SURE that the two programs have equivalent "plies"? Identical extensions and reductions and such, so that this is not a handicap match where the handicap is not obvious???Rebel wrote:I tried various logical options including yours, to no avail. Then looked into the executable and unfortunately a command-line option for fixed depth does not exist. Will have a look at Winboard then
1. midgame positions will be done 8 plies
2. midgame (minus queens) 10 plies
3. normal endgame 11 plies
4. rook endings 12 plies
5. minor endings 13 plies
6. pawn endings 14 plies
7. queen endings 10 plies
When I raise the parameter to 9 all depths increase with one and vice versa. When I have a handful of improvements I try them with regular time control.
I am indirectly using the system also to measure the impact of search changes. Say you add a static reduction which gives you a 10% speed-up but loses 20 elo points, it's likely a bad idea. Getting insight in the mysterious ways of the search is the keyword here.
In some cases, fixed nodes or fixed depth might be useful, but I would never use 'em to decide whether a change is good or bad if it introduces more error AND more depth at the same time... You miss the "more depth" and only see the "more error" and reach the wrong conclusion...
YMMV of course...
-
- Posts: 5566
- Joined: Tue Feb 28, 2012 11:56 pm
Re: Arena question
So what if Ed simply wants to measure the "more error" part isolated from the "more depth" part? Fixed depth is the way to do that. Ed is not saying that all his tests are fixed depth.bob wrote:You miss the "more depth" and only see the "more error" and reach the wrong conclusion...
-
- Posts: 20943
- Joined: Mon Feb 27, 2006 7:30 pm
- Location: Birmingham, AL
Re: Arena question
I thought it rather obvious. If you do fixed-depth, null-move is a HUGE loser. Ever think about that???syzygy wrote:So what if Ed simply wants to measure the "more error" part isolated from the "more depth" part? Fixed depth is the way to do that. Ed is not saying that all his tests are fixed depth.bob wrote:You miss the "more depth" and only see the "more error" and reach the wrong conclusion...