UCI Engine Tuning

[stevenaaus]

The various options supported by engines varies alot. What are the best sites for some general purpose doco ? Sorry for being such a noob.

And it's obviously an important part of engine testing and creating a level playing field.... Comments ??

[Michel]

I think this poll is misguided. xboard and uci engines both have options
with a more or less 1-1 mapping between them.

But the UCI/xboard options are for users. They can not expose all the internal parameters of an engine. If they did there would be far too many of them.

I do tuning with special options tune1,tune2,.... whose meaning varies depending on what I want to tune.

[hgm]

"Best" is an ill-defined concept. There are people that would consider a protocol "best" because it adhers to some elegant mathematical concept, like being stateless, even if it would be absolutely inadequate to do anything useful at all. (Such people tend to prefer UCI. ) I belong to the more pragmatic group, that puts capabilities before efficiency, and efficiency before beauty.

Based on those creteria WinBoard proctocol is objectively highly superior to UCI, a matter that can be decided by a poll in the same sense a poll can decide whether 1+1 equals 2 or 3: basically it is just a circumspect way to ask: "how many of you are idiots?".

Some of the things that UCI fails to do:

*) provide unambiguos specifications for itself.
*) allow engines to offer or accept draws.
*) make the engine aware of the upcoming session of the TC (e.g. how long should it best think on "movestogo 1 wtime 300000 btime 300000")?
*) let the GUI know if a string option is a file, a path, or just text.
*) allow advanced time-controls in node-based play.

WinBoard protocol in general is more versatile at handling engine-defined options.

[stevenaaus]

"Best" is an ill-defined concept.

Spoken like a true Vulcan. But I'm afraid, Mr Spock, that "Best" is a very well defined concept. It's a personal preference for one thing above others. What i'm not disputing is that people's opinions will differ

Thanks for your input - you're a wealth of knowledge HGM. Looking around i'd seen some people still prefer Xboard, but in my protocol programming, i've found it a little ... obtuse. Perhaps i do things wrong, though everything seems to work.

Mainly I'm just looking for some generic documentation for Scid vs. PC's UCI Config window, which i've tidied up a little this week. UCI options are now configurable from the Tournament window, and Pascal's wild config widget has been tamed somewhat. I know i've been critical of his code in places, but Pascal wrote all of Scid's UCI support, so kudos to him.

[Don]

"Best" is an ill-defined concept. There are people that would consider a protocol "best" because it adhers to some elegant mathematical concept, like being stateless, even if it would be absolutely inadequate to do anything useful at all. (Such people tend to prefer UCI. ) I belong to the more pragmatic group, that puts capabilities before efficiency, and efficiency before beauty.

I think stateless is more than just beauty. Suppose I wanted to build a user interface that didn't JUST play a game from start to finish? Maybe it is designed to allow a single executable to play a simultaneous exhibition? (Not that we would ever do that, but anything is possible.) Sure it's possible with either protocol, but with UCI the abstraction for that is more natural.

The reason stateless is more elegant is that it doesn't assume anything about what it is being used for. Winboard assumes you are playing a game from start to finish and requires both the engine and the interface to track the same state and stay synchronized.

But truth be told - I actually think this is rather a dumb thing to argue about because they are both more than sufficient and both do a good job. You can easily simulate statelessness using the winboard protocol by just making a bunch of move. I assume there is a command for making many moves simultaneously.

I will say that in the real world UCI is much easier to implement and it was specifically designed to let the chess author focus on playing chess. That can be a good thing or a bad thing according to your point of view.

WinBoard gives the program author a bit more control over things but at a price, and those things are not things that really make a big difference. For instance I couldn't care less if Komodo doesn't know how to resign, I think that should be a setting in the user interface which can be customized for each program.

Almost every argument in favor of one over the other is pretty hollow as you can simulate just about any behavior in either (such as simulating stateless behavior in winboard.) One argument against UCI is that it's easiness for implementing pondering also takes flexibility away from the program author. This is nonsense too because you can still ignore the mechanism and implement your own way if you enjoy the extra complexity. However I don't know of any program that implements ponder in a non-standard way. You give the program the ponder move and you start searching it. Standard for all programs (as far as I know.)

I just looked at the IPON list and the top 7 programs all use the UCI protocol and the ones below it I don't know for sure, but I think 8 or 9 of the top 10 do. There may be one or two of those programs that also use xboard in addition to UCI, I think Spike is one of these if I remember correctly.

But the point is that UCI could not be as inferior as you say that the very best programmers are too stupid to know better.

Based on those creteria WinBoard proctocol is objectively highly superior to UCI, a matter that can be decided by a poll in the same sense a poll can decide whether 1+1 equals 2 or 3: basically it is just a circumspect way to ask: "how many of you are idiots?".

Some of the things that UCI fails to do:

I have a comment or two on your list

*) provide unambiguos specifications for itself.
*) allow engines to offer or accept draws.

who cares?

*) make the engine aware of the upcoming session of the TC (e.g. how long should it best think on "movestogo 1 wtime 300000 btime 300000")?

This is not a big deal but I admit that it might be useful. I doubt that many people really care or that it will add more than 1 or 2 ELO to the playing strength but it is information that UCI does not provide.

*) let the GUI know if a string option is a file, a path, or just text.

That is an annoyance and of all things you mentioned that is only one that I personally even care about.

*) allow advanced time-controls in node-based play.

winoard has a few more features that nobody cares about but that is not what defines the quality of a protocol. I you added 1000 features that UCI does not have would that make everyone want to spend the next 10 years working on the protocol instead of actually improving the engine?

WinBoard protocol in general is more versatile at handling engine-defined options.

You forget xml support. Add that to winboard and you will have a winner. All data passed back and forth should be in XML so that it is buzzword compliant.

[hgm]

I think stateless is more than just beauty. Suppose I wanted to build a user interface that didn't JUST play a game from start to finish? Maybe it is designed to allow a single executable to play a simultaneous exhibition? (Not that we would ever do that, but anything is possible.) Sure it's possible with either protocol, but with UCI the abstraction for that is more natural.

You talk about statelessness like it is an asset, while in fact it is a limitation: WinBoard protocol could be made stateless by adding only a single line to the specs, which does not specify something you must do, but something you can refrain from doing: have the engine remember the game state after it has performed a search.

The example you give in fact would have exactly the same solution in UCI and WB protocol:

UCI:
position fen POSITION moves MOVE1 MOVE2 MOVE3 ...
go wtime XXX btime YYY

WB:
new
setboard POSITION
MOVE1
MOVE2
MOVE3
...
time XXX
otim YYY
go

OK, so WB protocol is more consistent in the use of linefeeds than UCI, but otherwise the sequence of commands is virtually identical. So how is that "more natural" in UCI? The task requires loading a new state in the engine before every search, and thus doing so is natural.

The difference occurs in applications where there is no need to reload a state, because both parties can know it, like in a regular game. In that case you have to resend the entire game in UCI just to reconfirm it. Now that is what I would call unnatural.

The reason stateless is more elegant is that it doesn't assume anything about what it is being used for. Winboard assumes you are playing a game from start to finish and requires both the engine and the interface to track the same state and stay synchronized.

WB protocol assumes no such thing. It just specifies the state of the engine after a search in such a way that it could be exploited to your advantage in a very common case. Absolutely nothing is lost by that: if it is not the state you want you reload the state you do want from scratch. In UCI the engine state after a search is undefined, so you are always forced to do it, even when it is unnatural.

But truth be told - I actually think this is rather a dumb thing to argue about because they are both more than sufficient and both do a good job. You can easily simulate statelessness using the winboard protocol by just making a bunch of move. I assume there is a command for making many moves simultaneously.

I will say that in the real world UCI is much easier to implement and it was specifically designed to let the chess author focus on playing chess. That can be a good thing or a bad thing according to your point of view.

I doubt that very much. For one, to work efficiently UCI engines have to figure out if the position you load into them is the same as that they were actually left in, or whether it is from a completely different game, and it had better clear its internal tables. From the point of view of the engine programmer the main difference is that WB engines in some situations move spontaneously, without waiting for a 'go' command. And programming this is totally trivial.

[Michel]

I think this discussion is completely misdirected.

The poll is about _options_ in UCI versus xboard. Not about the general merits of the two protocols.

The poll also asks a secondary question: how should options be used for _engine tuning_?.

The correct answer to the primary question is of course "neutral". There is very little difference in the handling of options between xboard and UCI. Unfortunately "neutral" was not provided as a possible answer.

The second question has nothing to do with the primary question.

Since it is impossible to answer two questions in one poll, I propose that
the poll be closed and replaced by a properly worded one.

[Michel]

I wrote

_engine tuning_

Well the _subject_ of this thread is about _UCI engine tuning_ whereas the actual question asks about _engine testing_.

Since testing and tuning are different things this is another inconsistency.

I think it is best to close this poll.

[hgm]

Well, talking about engine options, there definitely seems to be a problem in SCID vs. PC:

when I load Fairy-Max, (through the Tools -> Analysis Engines -> New... dialog) the "Configure" button for the engine options stays grayed out, so I can't configure any of its options...

[Don]

I think this discussion is completely misdirected.

And I'm arguing with someone who could never be objective anyway. It is like asking Bjarne Stroustrup which is better, C or C++ or getting a kid to admit that your Dad is better than his Dad.

The poll is about _options_ in UCI versus xboard. Not about the general merits of the two protocols.

The poll also asks a secondary question: how should options be used for _engine tuning_?.

The correct answer to the primary question is of course "neutral". There is very little difference in the handling of options between xboard and UCI. Unfortunately "neutral" was not provided as a possible answer.

The second question has nothing to do with the primary question.

Since it is impossible to answer two questions in one poll, I propose that
the poll be closed and replaced by a properly worded one.

I think my Dad is better than his Dad.