Formalizing the Universal Chess Interface

[Ras]

Perhaps not, but it is totally needless code complexity.

Code complexity, yes, but not needless. Since race conditions are nasty to debug, and their absence cannot be proven through testing, I prefer a design that will never lose messages simply because it doesn't discard any to begin with. Even if it's not strictly in line with the protocol definition, as long as it doesn't introduce new problems at functional level.

[expositor]

Well, then it is a completely pointless exercise.

Chess programming is a hobby – everything we do here is completely pointless.

That said, I'm not sure what your purpose is in belittling a project that I find interesting.

People are not going to abandon UCI for any new protocol you invent. Not in a million years.

We'll find out!

If you try to present this as UCI, and thereby mislead engine authors into thinking they can follow your specs to make their engine interoperable with true UCI software, you should not be surprised if others (and I in particular) will warn them against such practice.

I'll take that as a yes for the question Should this formalization of UCI be given [its own] name? listed under the Outstanding Issues question. I have no intention or desire to mislead anyone.

I'm a little surprised at the negativity of the reception and the readiness to promise to be my opponent.

———

1. One of the strong points of UCI is that it isn't amended all the time, resulting in a stable ecosystem instead of fragmentation. UCI engines simply tend to work. I don't see the need for a mostly identical but slightly incompatible UCI version.

That's fair! Any refinement of UCI is not particularly useful for those who already have their engine working with the clients they care about.

2. I don't like the state diagrams. That's a possible the implementation of a protocol - the protocol definition should not be concerned with implementation. It's only relevant what goes back and forth "on the line".

If it's not helpful, feel free to ignore the state diagram! I was very, very careful to restrict the specification to governing only the messages that the client and engine send to each other. Anything have to do with meaning or semantics or the internal operation of clients and engines is given as a recommendation within a comment (light grey text) and is not part of the specification proper.

3. I find the original spec easier to understand.

If you can put on a finger on why, that would help me know how I can improve the draft.

4. What would be good would be the old spec, just with some more clarifying comments on frequently asked questions and hints for how the ecosystem has developed in reality. For example, most UCI engines including Shredder cannot deal with parameters after searchmoves in go if searchmoves isn't the last parameter. Similarly, the PV should probably be the last part of the info message during search. Everyone does it that way already, but that should be documented.

Yes! agreed. Both of these are included in the draft.

[Ras]

That's fair! Any refinement of UCI is not particularly useful for those who already have their engine working with the clients they care about.

It's not useful for client authors, either, because what engines would they target with a slightly incompatible protocol? And for new engine authors, which clients would they target? In both cases: there are none, and I don't see any disruptive feature with any hope to beat that network effect. On top of that, the success of a protocol is rarely determined by what exactly goes over the line, and much more by the ecosystem.

The CECP ecosystem serves as cautionary tale here. The protocol got amended with tons of new features in theory, but the actual support was spotty at best in the wild. The result is that you can rely only on the basic features that have always been there, and even then.

With UCI, it's not perfect, but the strongest point is that it doesn't get amended - which is a point you'd destroy with a slightly incompatible version. That is, if that got any traction, which it will not.

If it's not helpful, feel free to ignore the state diagram!

Then it is bloat that shouldn't even be there.

If you can put on a finger on why, that would help me know how I can improve the draft.

Too much bloat. The original spec just goes to the meat. You could ofc have formalisation stuff that nobody cares about, but then just tuck it away in some appendix that is not meant to even being read. Or even better, cut it out.

My suggestion: stay in line with what UCI already is, avoid the https://xkcd.com/927/ trap, just concentrate on clarifying ambigous parts and document common interpretations such as searchmoves / pv order. That would have value. "Improving" the protocol while still calling it "UCI" has no value. Whatever you think would be "nicer" to have over the line has no value compared to the value of the existing ecosystem. That's the harsh reality.

[expositor]

It's not useful for client authors, either, because what engines would they target with a slightly incompatible protocol?

I am fairly certain that, given the current draft, a conforming client would work with most (if not all) legacy engines.

I don't see any disruptive feature with any hope to beat that network effect.

The goal of the endeavour is to provide a guarantee: if your engine conforms to the formal UCI specification (or whatever it'd be called), then it is guaranteed that your engine will work with any conforming client.

No such guarantee exists currently. Every time I've used a different client, a new edge case has caused problems, and I'm not the only one this has happened to. And there's no way to know whether your engine implements UCI correctly, but my hope is to provide a set of conformance tests that will diagnose any issues. To write tests, you first need a specification.

Then it is bloat that shouldn't even be there.

I meant if it's not helpful to you, feel free to ignore the state diagram. I happen to find it quite helpful ^_^

But I can omit it from future drafts.

Too much bloat. The original spec just goes to the meat. You could ofc have formalisation stuff that nobody cares about, but then just tuck it away in some appendix that is not meant to even being read. Or even better, cut it out.

I don't understand how it would qualify as a specification, then, rather than an informal tutorial or "getting started" guide.

An analogy: many people believe that C is a simple language, but it isn't at all – this is what's needed to define C, and this is what compilers implement. It's been my experience that people who don't consult documentation* write programs riddled with undefined behavior, and this has caused some nasty bugs. Anyway, although C is usually straightforward, there are tricky edge cases, and having an exhaustive resource is handy. I thought UCI could use something similar, because it doesn't answer questions like these:

• If an engine sends a bestmove message with a bad value, does it have to send another bestmove message to indicate that it's done searching? Is it allowed to send another bestmove message? Or should the client simply kill the engine for sending a bad message?
• How large can numeric arguments be? When can they be positive, negative, or zero?
• When more than one of "movetime", "depth", and "nodes" are given in a single go command, do all quantities need to be met for the engine to halt, or should the engine halt as soon as any of the quantities are met? Is the client even allowed to send more than one?

And yes, some people may not care about formalization stuff, but some people do (such as myself).

*My favorite resource is https://en.cppreference.com, which is excellent! and which I'd highly recommend.

My suggestion: stay in line with what UCI already is, avoid the https://xkcd.com/927/ trap, just concentrate on clarifying ambigous parts and document common interpretations such as searchmoves / pv order. That would have value.

You've just described my goal! The list of differences is on page 15, under Revision Notes. There are only three notable differences:

• the engine may send a bestmove message during an infinite search (so that it can communicate, when it has reached its depth limit, that it will make no further progress)
• the client is not required to send a position before sending go – the engine assumes the last given position, or the starting position if no position has been given (this matches the behavior of Stockfish and countless other engines)
• when the client sends a stop message it must wait for a bestmove message before preceding (given feedback I've received, I think this will be changed in the next draft)

I'm beginning to feel that no one bothered to read the document before critiquing its (imagined) contents.

[JoAnnP38]

I don't see any disruptive feature with any hope to beat that network effect.

The goal of the endeavour is to provide a guarantee: if your engine conforms to the formal UCI specification (or whatever it'd be called), then it is guaranteed that your engine will work with any conforming client.

As it exists there is no way to look at the specification and know whether an engine/client is conforming. I think the idea to tighten up the spec so there is no ambiguity is a worthy goal.

Then it is bloat that shouldn't even be there.

I meant if it's not helpful to you, feel free to ignore the state diagram. I happen to find it quite helpful ^_^

But I can omit it from future drafts.

Please leave it in. If you need a TLDR section, then provide that as an opening section.

I'm beginning to feel that no one bothered to read the document before critiquing its (imagined) contents.

Maybe less words and more charts and diagrams. Perhaps a fully exploded grammar for the commands/responses would be nice as well. BNF would work well there.

[JacquesRW]

No such guarantee exists currently. Every time I've used a different client, a new edge case has caused problems, and I'm not the only one this has happened to.

Any chance you could detail some specific examples? I think aside from the ongoing debate about this project it would be quite nice to compile a list of common problems/edge cases for beginners. One I remember distinctly was being unsure whether to use an ‘n’ or ‘k’ for knight promotions (which you have explicitly stated in your spec, thanks).

[clayt]

Hey, I just wanted to chime in and say thank you for making this!
Writing the UCI part of a chess engine was by far the most inconvenient step of the process, since it revolved around trying to divine the "true" meaning of the original UCI write-up. This would have been a big help for me about a year ago.

[Fulvio]

Honestly, I didn't find it very formal.
Compared for example with the definition of something more complex, like HTTP:
https://datatracker.ietf.org/doc/html/rfc1945

I thought UCI could use something similar, because it doesn't answer questions like these:

• If an engine sends a bestmove message with a bad value, does it have to send another bestmove message to indicate that it's done searching? Is it allowed to send another bestmove message? Or should the client simply kill the engine for sending a bad message?
• How large can numeric arguments be? When can they be positive, negative, or zero?
• When more than one of "movetime", "depth", and "nodes" are given in a single go command, do all quantities need to be met for the engine to halt, or should the engine halt as soon as any of the quantities are met? Is the client even allowed to send more than one?

This is like asking why the HTTP protocol does not define what to do if threre are grammatical errors in a web page. Should the browser fix it? Should the server send another response?
If the engine sends an illegal move maybe it will lose the game. Or maybe the current position will be sent again with the updated time. Or maybe something else. It depends on the circumstances and there is no right behavior that needs to be specified.

• When more than one of "movetime", "depth", and "nodes" are given in a single go command, do all quantities need to be met for the engine to halt, or should the engine halt as soon as any of the quantities are met? Is the client even allowed to send more than one?

It is impossible to guarantee that all limits are met, a fast computer may reach the desired depth in less than movetime. A slower one maybe not.
The engine should just stop thinking when it reaches the first limit.

Let's recap it simply.
The engine enters "search mode" after receiving "go" and return to "wait mode" just before sending "bestmove".
In "search mode" the direction of communication is reversed. The engine repeatedly sends "info messages" and the client is only allowed to send "stop" to ask the engine to return to "wait mode".

Code: Select all

--> go
<-- info
<-- info
....
[--> stop]
[<-- info]
<-- bestmove

The "go" message have diffent options with default values:

• searchmoves: default is all the moves
• ponder: false
• wtime/btime: infinite
• winc/binc: 0
• movestogo: 0
• depth: infinite
• nodes: infinite
• movetime: infinite
• infinite: false [this should have been called auto_exit_search_mode]

So "go depth 30 movetime 20000 infinite" means:
enter "search mode", search at most to depth 30 or for 20 seconds, stay in "search mode" and send "best move" only after receiving "stop".

[hgm]

If an engine sends a bestmove message with a bad value, does it have to send another bestmove message to indicate that it's done searching? Is it allowed to send another bestmove message? Or should the client simply kill the engine for sending a bad message?

This is not a matter of protocol, but of client design. Full covarage of error handling is a logical impossibility for any protocol: by prescribing what to do in case of 'error' you in fact extended the protocol with new rules, and these rules could be violated too in new ways.

Engines should not send invalid commands, period. If they do, they are not complient, and they are completely at the mercy of the client. If they receive invalid commands, they can also do whatever they feel would be appropriate. Non-complient clients do not qualify for any mercy either.

You've just described my goal! The list of differences is on page 15, under Revision Notes. There are only three notable differences:

• the engine may send a bestmove message during an infinite search (so that it can communicate, when it has reached its depth limit, that it will make no further progress)
• the client is not required to send a position before sending go – the engine assumes the last given position, or the starting position if no position has been given (this matches the behavior of Stockfish and countless other engines)
• when the client sends a stop message it must wait for a bestmove message before preceding (given feedback I've received, I think this will be changed in the next draft)

You missed the point. The recommendation was that there should not be any difference. Just clarification and recommendations.

That doesn't exclude that what you propose would make a better protocol. As far as I am concerned almost anything would be better than UCI. But designing new protocols is something completely different from improving the specification of an existing one. Just call it NCI for New Chess Interface, and you would not hear any objections from me.

[hgm]

Let's recap it simply.
The engine enters "search mode" after receiving "go" and return to "wait mode" just before sending "bestmove".
In "search mode" the direction of communication is reversed. The engine repeatedly sends "info messages" and the client is only allowed to send "stop" to ask the engine to return to "wait mode".

Code: Select all

--> go
<-- info
<-- info
....
[--> stop]
[<-- info]
<-- bestmove

There is a problem with that definition, because it delimits the "search mode" (which the specs call "calculating") by communication events in opposit directions. This can lead to race conditions. A specification that would cover existing practice would say:

A 'go' command will bring the engine in the "calculating" state, during which it is not allowed to send it any other commands than 'stop', 'ponderhit', 'debug' and 'isready' (and 'quit'?). This limitation lasts until the 'stop' command or the 'bestmove' command (whichever comes earlier).

And as for stating the obvious:

An engine should execute allowed commands in the order it receives those. It should not start executing a command before execution of all previous commands has been completed, but should never delay execution any further than that. Commands that are not allowed when their turn for execution comes up should be ignored. Each of the other commands should be executed as per their specification. It is not allowed to ignore commands just because the engine feels their purpose has already been accomplished.

A 'go' command is considered executed as soon as the engine has started up calculation of the new move. Processing of subsequent commands must start immediately after that, and not wait for this calculation to finish. A 'stop' command is considered executed when the engine would be ready to initiate calculation of a new move.