Crafty problem with cutechess-cli

[Matthias Gemuh]

It seems to me that "Log off" should be the default setting in Crafty.

in every engine (not just Crafty)
My engine logs to 8 always-overwrite files.
I will reduce from 8 to 2 soon.

.

[Gian-Carlo Pascutto]

Reducing the number of logfiles isn't going to help you. The problem is the speed penalty in writing them out.

[bob]

There are some "bugs" that are not worth the effort of fixing. In a program of nearly 50K lines of code, that is under development every day, making sure that "new" initializes every last thing that can be modified internally is a non-trivial task.

It should be very trivial if the program is designed properly.

What a silly statement. I add a new feature for testing. I now need to add code to make sure it gets re-set on a new command. I change the code. I have to change the new code. If you do something "trivial" enough times, the total time suddenly becomes non-trivial. Most people don't complain when their compiler or other large application won't just sit around waiting on the next program to compile, but instead has to be re-loaded into memory for execution.

But there are other cases of things you are not even considering. take a cluster run, where I want to vary some value over a range, so I add some code in option.c to handle a "scale" command (this code is actually in there right now, but does nothing for released versions. If I enter scale 50, that means to take the default value (or values) and reduce them to 50% of their original value, and then play a game. A "new" is problematic here, I need to restore the values to their original state. Or I can just restart the engine.

Don't put something down until you have considered all angles. Things are not always as cut-and-dried as you might assume.

[bob]

There are some "bugs" that are not worth the effort of fixing. In a program of nearly 50K lines of code, that is under development every day, making sure that "new" initializes every last thing that can be modified internally is a non-trivial task.

It should be very trivial if the program is designed properly.

That does not apply to a long term project like Crafty, with its countless releases. Are you suggesting that Crafty 8.11 should have been designed in its time and C language standards to look like today's Crafty 23.0 or even better ?

No, I'm suggesting Crafty in general should have been designed with some function that resets its state to the way it was before playing any games (but after reading options such as "do not create log files"). This is very simple stuff, that could be done before there was anything resembling a C language standard. Basic software engineering.

Read what you are writing. How can I write code to "reset values to their original state" until there is an original state to start with? Each new scoring term that gets modified during a search needs to be reset. If you want to spend the time doing this for every thing you add, that's fine by me. I simply prefer the one execution - one game model myself.

[bob]

Reducing the number of logfiles isn't going to help you. The problem is the speed penalty in writing them out.

I can't even measure the cost for this it is so small, so long as the "noise value" is set to something reasonable. I try to use noise X where X is the typical nps I see, so that for the first second, no log output at all.

However, remember that Crafty is not a commercial project. Crafty is a "Bob's project" and for me, the logs are beyond just useful, they are critical. I compile and execute on all sorts of machines, and don't want to have to remember to turn logging on so that I don't miss something important.

[Matthias Gemuh]

Reducing the number of logfiles isn't going to help you. The problem is the speed penalty in writing them out.

I am not reducing to help my engine.
It is to comply with users out there who hate seeing too many log files.

Matthias.

[Zach Wegner]

What a silly statement. I add a new feature for testing. I now need to add code to make sure it gets re-set on a new command. I change the code. I have to change the new code. If you do something "trivial" enough times, the total time suddenly becomes non-trivial. Most people don't complain when their compiler or other large application won't just sit around waiting on the next program to compile, but instead has to be re-loaded into memory for execution.

Most people would complain if their compiler didn't allow this:

Code: Select all

cc *.c

but instead required:

Code: Select all

cc a.c
cc b.c
cc c.c
...
cc *.o

But there are other cases of things you are not even considering. take a cluster run, where I want to vary some value over a range, so I add some code in option.c to handle a "scale" command (this code is actually in there right now, but does nothing for released versions. If I enter scale 50, that means to take the default value (or values) and reduce them to 50% of their original value, and then play a game. A "new" is problematic here, I need to restore the values to their original state. Or I can just restart the engine.

Sorry, but this sounds to me like a bad design. If you run "scale 50", it divides every evaluation parameter by 2? A much better way would be to simply divide by 2 at the end of the evaluation. But even then, I have a function that takes a set of evaluation parameters and makes a copy, so that you can restore it later. This was created for tuning parameters. Very trivial code.

In general, if you have data, you either initialize it explicitly in the instantiation, or you calculate its value in a function. Values that are modified during a game should obviously be initialized in a function. Values that are constant throughout a game can be initialized either way.

Don't put something down until you have considered all angles. Things are not always as cut-and-dried as you might assume.

I have. I honestly can't think of one good reason why a well-designed chess program would need to restart every game (unless you are HG and need to minimize characters).

[bob]

What a silly statement. I add a new feature for testing. I now need to add code to make sure it gets re-set on a new command. I change the code. I have to change the new code. If you do something "trivial" enough times, the total time suddenly becomes non-trivial. Most people don't complain when their compiler or other large application won't just sit around waiting on the next program to compile, but instead has to be re-loaded into memory for execution.

Most people would complain if their compiler didn't allow this:

Code: Select all

cc *.c

but instead required:

Code: Select all

cc a.c
cc b.c
cc c.c
...
cc *.o

Not most. Most would recognize that for exactly what it represents, a very _sloppy_ way of compiling. Do you really want to compile every source file, even if just one was modified? So we use make.

Which is my point. During development, I try to use my time as efficiently as possible, because it is a limited resource. Remembering to restore every value to its starting point requires effort that can be used elsewhere. And I have elected to do so. I'm not aware of any issues in Crafty with the "new" command. I simply pointed out that _I_ prefer to do all testing of _all_ engines without depending on this so that I don't introduce a little unexpected bias to the results.

But there are other cases of things you are not even considering. take a cluster run, where I want to vary some value over a range, so I add some code in option.c to handle a "scale" command (this code is actually in there right now, but does nothing for released versions. If I enter scale 50, that means to take the default value (or values) and reduce them to 50% of their original value, and then play a game. A "new" is problematic here, I need to restore the values to their original state. Or I can just restart the engine.

Sorry, but this sounds to me like a bad design. If you run "scale 50", it divides every evaluation parameter by 2? A much better way would be to simply divide by 2 at the end of the evaluation. But even then, I have a function that takes a set of evaluation parameters and makes a copy, so that you can restore it later. This was created for tuning parameters. Very trivial code.

In general, if you have data, you either initialize it explicitly in the instantiation, or you calculate its value in a function. Values that are modified during a game should obviously be initialized in a function.

Idea occurs in no programming book I own, and shouldn't. There are reasons for run-time initialization vs compile-time initialization. And they don't have anything to do with whether the value is modified during execution or not. I suppose I could take the easy way out, and when I get a "new" command, I just do an

exec("./crafty");

and be done with it without anyone being the wiser.

Values that are constant throughout a game can be initialized either way.

Don't put something down until you have considered all angles. Things are not always as cut-and-dried as you might assume.

I have. I honestly can't think of one good reason why a well-designed chess program would need to restart every game (unless you are HG and need to minimize characters).

[Zach Wegner]

Which is my point. During development, I try to use my time as efficiently as possible, because it is a limited resource. Remembering to restore every value to its starting point requires effort that can be used elsewhere. And I have elected to do so. I'm not aware of any issues in Crafty with the "new" command. I simply pointed out that _I_ prefer to do all testing of _all_ engines without depending on this so that I don't introduce a little unexpected bias to the results.

Perhaps you should re-read the first post in this thread then.

Idea occurs in no programming book I own, and shouldn't. There are reasons for run-time initialization vs compile-time initialization. And they don't have anything to do with whether the value is modified during execution or not.

Sure there are. Say your board representation is based on an array int board[64]; (ignore the fact that this is bad design in itself). Do you want to instantiate it at compile time, or have a function to set up the initial position? If you do the former, then you're going to need to duplicate the data to do the latter. Much better is to simply call said function to initialize it in the first place.

Likewise, if you have a variable, say "use_log", that should persist across games, you probably don't want to initialize it in the same function you use to start a game. Set a default either at compile time or at startup and don't touch it until the user says so. Doing otherwise is simply a bug.

[xsadar]

What a silly statement. I add a new feature for testing. I now need to add code to make sure it gets re-set on a new command. I change the code. I have to change the new code. If you do something "trivial" enough times, the total time suddenly becomes non-trivial. Most people don't complain when their compiler or other large application won't just sit around waiting on the next program to compile, but instead has to be re-loaded into memory for execution.

Most people would complain if their compiler didn't allow this:

Code: Select all

cc *.c

but instead required:

Code: Select all

cc a.c
cc b.c
cc c.c
...
cc *.o

I wouldn't complain. I wouldn't have to change anything in any makefile I've ever written (assuming the -c option which you left out is used the same as it is now).
Also, from my understanding of GNU's gcc, the program you run is not the actual compiler, but invokes the appropriate compiler (and/or preprocessor, assembler, linker, etc) exactly once for each output file, which would be akin to invoking the chess engine exactly once for each game.

But there are other cases of things you are not even considering. take a cluster run, where I want to vary some value over a range, so I add some code in option.c to handle a "scale" command (this code is actually in there right now, but does nothing for released versions. If I enter scale 50, that means to take the default value (or values) and reduce them to 50% of their original value, and then play a game. A "new" is problematic here, I need to restore the values to their original state. Or I can just restart the engine.

Sorry, but this sounds to me like a bad design. If you run "scale 50", it divides every evaluation parameter by 2? A much better way would be to simply divide by 2 at the end of the evaluation. But even then, I have a function that takes a set of evaluation parameters and makes a copy, so that you can restore it later. This was created for tuning parameters. Very trivial code.

In general, if you have data, you either initialize it explicitly in the instantiation, or you calculate its value in a function. Values that are modified during a game should obviously be initialized in a function. Values that are constant throughout a game can be initialized either way.

Don't put something down until you have considered all angles. Things are not always as cut-and-dried as you might assume.

I have. I honestly can't think of one good reason why a well-designed chess program would need to restart every game (unless you are HG and need to minimize characters).