UCI Engine Tuning

[hgm]

And I'm arguing with someone who could never be objective anyway. It is like asking Bjarne Stroustrup which is better, C or C++ or getting a kid to admit that your Dad is better than his Dad.

That is a low blow! I think I completely objectively showed that what you said could be done in a "more natural" way in UCI is in fact done in exactly the same way in WB protocol. There is a 1-1 mapping of all keywords. What more do you want as objective proof that your "more natural" claim has no basis at all?

Or did you mean that it is more natural to use UCI only for simultaneous exhibtions, rather than playing games? (In that case I might even agree with you!)

So far your only arguments have been of the kind "Yes,it is true that UCI cannot do this, but that does not count, because I don't want to do that anyway". How is that supposed to convince an objective person of anything?

[mar]

"Best" is an ill-defined concept. There are people that would consider a protocol "best" because it adhers to some elegant mathematical concept, like being stateless, even if it would be absolutely inadequate to do anything useful at all. (Such people tend to prefer UCI. ) I belong to the more pragmatic group, that puts capabilities before efficiency, and efficiency before beauty.

Based on those creteria WinBoard proctocol is objectively highly superior to UCI, a matter that can be decided by a poll in the same sense a poll can decide whether 1+1 equals 2 or 3: basically it is just a circumspect way to ask: "how many of you are idiots?".

Some of the things that UCI fails to do:

*) provide unambiguos specifications for itself.
*) allow engines to offer or accept draws.
*) make the engine aware of the upcoming session of the TC (e.g. how long should it best think on "movestogo 1 wtime 300000 btime 300000")?
*) let the GUI know if a string option is a file, a path, or just text.
*) allow advanced time-controls in node-based play.

WinBoard protocol in general is more versatile at handling engine-defined options.

You forgot to mention one thing that chess engine communication protocol can't do:

*) handle mate scores

[hgm]

That was not my task, that was yours! Still leading by 5-1, though, despite the fact that you "saved the honor".

But to be serious, it seems this is more a GUI implementation matter than a protocol matter. I was told Arena does display mate scores even for WB engines.

[mar]

That was not my task, that was yours! Still leading by 5-1, though, despite the fact that you "saved the honor".

But to be serious, it seems this is more a GUI implementation matter than a protocol matter. I was told Arena does display mate scores even for WB engines.

That's fair I even forgot to mention that UCI doesn't support variants. In fact I thought this would be your point #1

I just checked and indeed, Arena does display mate scores. But I wasn't expecting you to speak in favor of Arena Anyway how can that be when winboard engines can return any scores they wish? Perhaps the good old WB engines use a matescore convention which Arena supports? I don't want to shift the discussion off topic, just wondering...

[hgm]

But I wasn't expecting you to speak in favor of Arena

That is because, despite evil suggestions to the contrary, I am actually very objective!

I was told Arena assumes the engine will use the EPD standard for scoring, which encodes checkmate as 32767 cP. So it probably won't work for all engines. Of course you could provide GUI options to specify the mate score for all engines separately, but I don't think that would be a good policy.

Anyway, I think we can objectively agree that not enforcing uniformity of mate scores, although a black mark, is at a totally different level than, say, not supporting draw offers. Mate scores are a cosmetic feature. Not handling draw offers or resigns changes the game, and can even change the result.

[Don]

And I'm arguing with someone who could never be objective anyway. It is like asking Bjarne Stroustrup which is better, C or C++ or getting a kid to admit that your Dad is better than his Dad.

That is a low blow! I think I completely objectively showed that what you said could be done in a "more natural" way in UCI is in fact done in exactly the same way in WB protocol. There is a 1-1 mapping of all keywords. What more do you want as objective proof that your "more natural" claim has no basis at all?

Or did you mean that it is more natural to use UCI only for simultaneous exhibtions, rather than playing games? (In that case I might even agree with you!)

So far your only arguments have been of the kind "Yes,it is true that UCI cannot do this, but that does not count, because I don't want to do that anyway". How is that supposed to convince an objective person of anything?

[/quote]
That has not been my "only" argument but it seems like s your only argument. Your criteria (and really the only thing you can talk about) seems to be the total number of obscure features that a protocol has and my criteria is the basic soundness of the protocol and the ease of use.

If you look at the XML vs JSON debate you will get a taste of what I'm saying. XML is more comprehensive and does more, but people are moving away from XML and towards JSON in droves because it's conceptually simpler and more elegant.

UCI is easier to implement and you just keep saying, "no it isn't" but saying it over and over does not make it so. WinBoard is NOT easier to implement, especially with respect to pondering. The control logic for pondering has to be built into the chess engine with winboard but with UCI this is already handled by the protocol. You can try to blow that off as not important with some hand waving and I'm sure you will try, but you are not fooling most people, just look at the poll results.

By the way, I don't think UCI is superior in every possible way - I think both protocols have some good points and some ugly warts. That is my assessment and I hope I'm being objective - but you are clearly not objective even though you believe you are - over the past 6 months almost all your posts seem to me to be about UCI bashing and winboard advocacy and diggs. How do you expect anyone to think you are objective on this?

And I still say my Dad is better than your Dad.

[mar]

But I wasn't expecting you to speak in favor of Arena

That is because, despite evil suggestions to the contrary, I am actually very objective!

I was told Arena assumes the engine will use the EPD standard for scoring, which encodes checkmate as 32767 cP. So it probably won't work for all engines. Of course you could provide GUI options to specify the mate score for all engines separately, but I don't think that would be a good policy.

Anyway, I think we can objectively agree that not enforcing uniformity of mate scores, although a black mark, is at a totally different level than, say, not supporting draw offers. Mate scores are a cosmetic feature. Not handling draw offers or resigns changes the game, and can even change the result.

Sounds interesting. Wouldn't it be a good idea to support EPD mate scores in WinBoard as well? Perhaps as a GUI option?
I agree that mate scores are a cosmetic feature, however when you analyze a position it becomes annoying for the user to decode engine score whatever that might be. Not supporting draw offer/resign would normally be a serious problem but since this only affects engines it's difficult to say. It can change the outcome as you mentioned.
I'd rather have the resign option than offer draw option. Hard to say when to offer/accept draw. When score drops to zero after x moves played? Rather than that I'd like if UCI engines could remember the last position searched and that the GUI would send something like position current moves nnnn. Sending position startpos moves ...huge list... is something that wastes IO and CPU time required to parse and make all the moves (although negligible and only a potential problem in lightning)

[michiguel]

But I wasn't expecting you to speak in favor of Arena

That is because, despite evil suggestions to the contrary, I am actually very objective!

I was told Arena assumes the engine will use the EPD standard for scoring, which encodes checkmate as 32767 cP. So it probably won't work for all engines. Of course you could provide GUI options to specify the mate score for all engines separately, but I don't think that would be a good policy.

I was saying this for a long time! There is already a standard, let's use it!
I suggest wb/xb sees is a score is between 32767 and say, 32567, and it it is, convert it to a mate score. 99.9% of the cases it will be correct.
If the engine has another scale, most likely nothing would change.
Miguel

Anyway, I think we can objectively agree that not enforcing uniformity of mate scores, although a black mark, is at a totally different level than, say, not supporting draw offers. Mate scores are a cosmetic feature. Not handling draw offers or resigns changes the game, and can even change the result.

[Don]

But I wasn't expecting you to speak in favor of Arena

That is because, despite evil suggestions to the contrary, I am actually very objective!

I think you are generally very objective and logical, just not about this issue.

I was told Arena assumes the engine will use the EPD standard for scoring, which encodes checkmate as 32767 cP. So it probably won't work for all engines. Of course you could provide GUI options to specify the mate score for all engines separately, but I don't think that would be a good policy.

Anyway, I think we can objectively agree that not enforcing uniformity of mate scores, although a black mark, is at a totally different level than, say, not supporting draw offers. Mate scores are a cosmetic feature. Not handling draw offers or resigns changes the game, and can even change the result.

Not necessarily. I would be really annoyed if the user interface never announced mate correctly, cosmetic or not. Accepting a draw, offering a draw, or resigning - implement that and your program will get a worse result because programs are too stupid to make the right decisions here.

In an idealistic sense however you are right - it's missing from UCI - each player should have the right to make or accept a draw or to resign. I'm not losing any sleep over this missing feature but I suppose it has a lot to do with the temperament of the engine author. If UCI provided it I wold not implement it (or the default would always be NO.)

In my view the biggest wart with UCI is the "file" thing you mentioned. Also, I wish there was a floating point data type. I can get around that by telling Larry that the parameter we are tuning is in centi-units or some other, but this is annoying.

I would never do this because it was fragment the computer chess community even more, but the whole thing should be scrapped and rethought from scratch.

[michiguel]

"Best" is an ill-defined concept. There are people that would consider a protocol "best" because it adhers to some elegant mathematical concept, like being stateless, even if it would be absolutely inadequate to do anything useful at all. (Such people tend to prefer UCI. ) I belong to the more pragmatic group, that puts capabilities before efficiency, and efficiency before beauty.

I think stateless is more than just beauty. Suppose I wanted to build a user interface that didn't JUST play a game from start to finish? Maybe it is designed to allow a single executable to play a simultaneous exhibition? (Not that we would ever do that, but anything is possible.) Sure it's possible with either protocol, but with UCI the abstraction for that is more natural.

The reason stateless is more elegant is that it doesn't assume anything about what it is being used for. Winboard assumes you are playing a game from start to finish and requires both the engine and the interface to track the same state and stay synchronized.

But truth be told - I actually think this is rather a dumb thing to argue about because they are both more than sufficient and both do a good job. You can easily simulate statelessness using the winboard protocol by just making a bunch of move. I assume there is a command for making many moves simultaneously.

I will say that in the real world UCI is much easier to implement and it was specifically designed to let the chess author focus on playing chess. That can be a good thing or a bad thing according to your point of view.

WinBoard gives the program author a bit more control over things but at a price, and those things are not things that really make a big difference. For instance I couldn't care less if Komodo doesn't know how to resign, I think that should be a setting in the user interface which can be customized for each program.

Almost every argument in favor of one over the other is pretty hollow as you can simulate just about any behavior in either (such as simulating stateless behavior in winboard.) One argument against UCI is that it's easiness for implementing pondering also takes flexibility away from the program author. This is nonsense too because you can still ignore the mechanism and implement your own way if you enjoy the extra complexity. However I don't know of any program that implements ponder in a non-standard way. You give the program the ponder move and you start searching it. Standard for all programs (as far as I know.)

I just looked at the IPON list and the top 7 programs all use the UCI protocol and the ones below it I don't know for sure, but I think 8 or 9 of the top 10 do. There may be one or two of those programs that also use xboard in addition to UCI, I think Spike is one of these if I remember correctly.

But the point is that UCI could not be as inferior as you say that the very best programmers are too stupid to know better.

Based on those creteria WinBoard proctocol is objectively highly superior to UCI, a matter that can be decided by a poll in the same sense a poll can decide whether 1+1 equals 2 or 3: basically it is just a circumspect way to ask: "how many of you are idiots?".

Some of the things that UCI fails to do:

I have a comment or two on your list

*) provide unambiguos specifications for itself.
*) allow engines to offer or accept draws.

who cares?

*) make the engine aware of the upcoming session of the TC (e.g. how long should it best think on "movestogo 1 wtime 300000 btime 300000")?

This is not a big deal but I admit that it might be useful. I doubt that many people really care or that it will add more than 1 or 2 ELO to the playing strength but it is information that UCI does not provide.

*) let the GUI know if a string option is a file, a path, or just text.

That is an annoyance and of all things you mentioned that is only one that I personally even care about.

*) allow advanced time-controls in node-based play.

winoard has a few more features that nobody cares about but that is not what defines the quality of a protocol. I you added 1000 features that UCI does not have would that make everyone want to spend the next 10 years working on the protocol instead of actually improving the engine?

WinBoard protocol in general is more versatile at handling engine-defined options.

You forget xml support. Add that to winboard and you will have a winner. All data passed back and forth should be in XML so that it is buzzword compliant.

Both protocols follow a different philosophy. In both there are "technical" issues (like resolving string or file names etc.) and design issues. Winboard is more faithful to its philosophical design than UCI. UCI was supposed to be stateless, despite some of its fans denied it in previous discussions, but has some hybrid things that make it a bit clumsy. For instance, time management. It is is truly stateless, it should receive a command that says "Panic time is 2 minutes, do not ever exceed that, suggested time: 10 seconds". In that way, the GUI will make all the decisions regarding time management and make the engine stateless.

A similar issues occurs with the hash tables. Engines guess if the position belongs to a game or not in order to keep the previous information (tables, killers etc.), but that is not truly stateless, and the protocol was modified to acknowledge this, with a kludge.

UCI is very well suited for a engine used as an analysis tool, but winboard's design fits better an actual game.

Yes, UCI may sound easier, but that is because it relieves responsibilities to the GUI. At one point I thought I could separate roles into two executables 1) a polyglot-like adaptor who handles lots of things including different protocols, books, TBs, and 2) a uci-like engine. That is, basically, splitting my current engine in two. But if I ever do that, I wouldn't use UCI and I would write my own. It would look very similar to UCI, but it should be truly stateless, and if it remembers any part of a state, it should be explicitly told what to remember.

Miguel