Arena question

[Rebel]

From the help-file:

Arena.exe /fcp SOS /scp Crafty /lpf C:\Arena\123test.EPD /lpi 2 /mg 4 /inc 1 /tc 01:00

Starts Arena with first engine SOS, second engine Crafty for a match for 4 games with the second position of the specified EPD file in Fischer-clock-mode 1/1.

=====

Does anyone know how to set the level to fixed depth?

[hgm]

Amazing! Exactly the same command-line arguments would have to be used with WinBoard, to do the same thing! Perhaps /searchDepth D, which is what you would have to add in WinBoard to do as you ask, would work here too?

[Rebel]

I tried various logical options including yours, to no avail. Then looked into the executable and unfortunately a command-line option for fixed depth does not exist. Will have a look at Winboard then

[bob]

I tried various logical options including yours, to no avail. Then looked into the executable and unfortunately a command-line option for fixed depth does not exist. Will have a look at Winboard then

Obvious question: "Why fixed depth?" Are you SURE that the two programs have equivalent "plies"? Identical extensions and reductions and such, so that this is not a handicap match where the handicap is not obvious???

[Rebel]

I tried various logical options including yours, to no avail. Then looked into the executable and unfortunately a command-line option for fixed depth does not exist. Will have a look at Winboard then

Obvious question: "Why fixed depth?" Are you SURE that the two programs have equivalent "plies"? Identical extensions and reductions and such, so that this is not a handicap match where the handicap is not obvious???

I am using cutechess-cli in the meantime. To test my eval I run fixed depth self-play matches controlled by a depth based flexible parameter. For instance when I set the following parameter [PLY = 8] as a base then:

1. midgame positions will be done 8 plies
2. midgame (minus queens) 10 plies
3. normal endgame 11 plies
4. rook endings 12 plies
5. minor endings 13 plies
6. pawn endings 14 plies
7. queen endings 10 plies

When I raise the parameter to 9 all depths increase with one and vice versa. When I have a handful of improvements I try them with regular time control.

I am indirectly using the system also to measure the impact of search changes. Say you add a static reduction which gives you a 10% speed-up but loses 20 elo points, it's likely a bad idea. Getting insight in the mysterious ways of the search is the keyword here.

[bob]

I tried various logical options including yours, to no avail. Then looked into the executable and unfortunately a command-line option for fixed depth does not exist. Will have a look at Winboard then

Obvious question: "Why fixed depth?" Are you SURE that the two programs have equivalent "plies"? Identical extensions and reductions and such, so that this is not a handicap match where the handicap is not obvious???

I am using cutechess-cli in the meantime. To test my eval I run fixed depth self-play matches controlled by a depth based flexible parameter. For instance when I set the following parameter [PLY = 8] as a base then:

1. midgame positions will be done 8 plies
2. midgame (minus queens) 10 plies
3. normal endgame 11 plies
4. rook endings 12 plies
5. minor endings 13 plies
6. pawn endings 14 plies
7. queen endings 10 plies

When I raise the parameter to 9 all depths increase with one and vice versa. When I have a handful of improvements I try them with regular time control.

I am indirectly using the system also to measure the impact of search changes. Say you add a static reduction which gives you a 10% speed-up but loses 20 elo points, it's likely a bad idea. Getting insight in the mysterious ways of the search is the keyword here.

I don't see how you can conclude the latter. If you make the search 10% faster, at fixed depth you see absolutely no benefit. Take a simple example. Search to 10 plies and stop. You now decide to try more aggressive pruning, and you notice that the search is now 2x faster. But at a fixed depth of 10, it may well lose a little due to the pruning. But the extra ply (which you will never see in a fixed depth test) might gain more than the pruning loss. IF you had been using a fixed time limit instead so that the 2x faster would turn into one extra ply.

In some cases, fixed nodes or fixed depth might be useful, but I would never use 'em to decide whether a change is good or bad if it introduces more error AND more depth at the same time... You miss the "more depth" and only see the "more error" and reach the wrong conclusion...

YMMV of course...

[syzygy]

You miss the "more depth" and only see the "more error" and reach the wrong conclusion...

So what if Ed simply wants to measure the "more error" part isolated from the "more depth" part? Fixed depth is the way to do that. Ed is not saying that all his tests are fixed depth.

[bob]

You miss the "more depth" and only see the "more error" and reach the wrong conclusion...

So what if Ed simply wants to measure the "more error" part isolated from the "more depth" part? Fixed depth is the way to do that. Ed is not saying that all his tests are fixed depth.

I thought it rather obvious. If you do fixed-depth, null-move is a HUGE loser. Ever think about that???