Ugly UCI

[hgm]

I increase the age/date counter when a search starts on a new root position.

Also in analyze mode?

What I think is quite elegant in the uci design is that with just one pair of simple commands ("position" and "go") an uci engine can already make moves in a game, accept take back, setup and play from a new position, swap sides, accept new time controls, reset and start a new game and analyse random positions.

But it is a fraud: they are not simple commands at all. They are multiple commands written on the same line. They are exactly the same commands as in CECP. "positon startpos" = "new", "position fen" = "setboard", "moves" = "force", "wtime/btime" = "time/otim".

The lack of a "level" command as an advance warning of what to expect is actually a serious defect of UCI. With "wtime 1000 movestogo 1", an UCI engine has no idea whether it will get 1000 or 2 sec of time for the next session, and thus how important it is to not use up all time to the last msec.

[mvk]

I increase the age/date counter when a search starts on a new root position.

Also in analyze mode?

It makes no distinction, so the newly generated nodes get precedence over the older ones in the hash table. I haven't decided yet if I want to refresh the table-stored age when there is a useful hit. Currently I don't that, but I might change my mind about that.

[hgm]

I also used to do that in Shokidoki, but when I started to use it heavily for analyzing, it caused very undesirable behavior, losing the results of the deep main line after I investigated a side branch. After some thinking I came to the conclusion that an interactive analysis session is basically one big search, partly hand-guided, and that nothing can be considered useless. So it replaces purely by depth.

[mvk]

What I think is quite elegant in the uci design is that with just one pair of simple commands ("position" and "go") an uci engine can already make moves in a game, accept take back, setup and play from a new position, swap sides, accept new time controls, reset and start a new game and analyse random positions.

But it is a fraud: they are not simple commands at all. They are multiple commands written on the same line. They are exactly the same commands as in CECP. "positon startpos" = "new", "position fen" = "setboard", "moves" = "force", "wtime/btime" = "time/otim".

I wanted to say that the single "position\ngo" command is much simpler, but I was waiting for the best opportunity.

uci.html is 26kB, xboard.html is 96kB. Both are equally useful documentation for their standard. Where does the size difference come from? Not only the resign and draw commands. It is also unneeded complexity: multiple ways to do the same thing. It is things like this: when I go back in xboard, I don't receive "force" and "setboard" with all the things you mention. I get "undo". Undo is a redundant command and yet it is mandatory to implement to get full functionality. Or if I retract a move while playing, I don't get 2 times "undo", I get ... "remove"! Such convoluted things are the reason that when you start with uci you have full functionality quicker and can focus on search and evaluation sooner. And your users can run under any GUI of their choosing ofcourse. One more such thing: you can send "feature setboard=1", but as long as the GUI is allowed to send "rejected setboard", you must still support "edit" and all its drama. But some GUIs need "setboard" and won't do "edit", so you must still have both. Argh. With uci the gui has no such weird freedoms and that makes life easier for everyone. If you intend to give the user the full experience, you must implement the whole standard, not just the subset that lets you play a single game.

The lack of a "level" command as an advance warning of what to expect is actually a serious defect of UCI. With "wtime 1000 movestogo 1", an UCI engine has no idea whether it will get 1000 or 2 sec of time for the next session, and thus how important it is to not use up all time to the last msec.

That is a constructed scenario which clearly didn't hinder adoption. Or at least, the xboard engines on the rating lists seem to have a really hard time exploiting such "defect" from their poorly uninformed uci colleagues and the forums are not overflowing with user complaints about this either. When I look at simple.c/kingslayer as a reference implementation on how to respond to "level" and "go", it is also only concerned about the next time control and not about the tightest, unless I misread its code somehow.

[mcostalba]

but I dont like UCI. In each move, the GUI send something like:

position startpos moves e2e4 e7e6 g1f3 d7d5 e4d5 e6d5 d2d4 b8c6 c2c4 f8b4 b1c3

For my undestanding, this is very ugly.

For my taste this is the most powerful and deep idea that the author of UCI got. It is amazing to me how this single assumption is able to get rid of tons of cruft and special cases: from software engeneering point of view this is pure brillancy.

For the amateurish programmer simple == what seems simple for human point of view

For the talented programmer simple == concise, self-contained, robust, consistent, general

[Ferdy]

Hello,

I am near to release my engine. At the moment it goes good with xboard, but I have implemented a draft uci handle module, but I dont like UCI. In each move, the GUI send something like:

position startpos moves e2e4 e7e6 g1f3 d7d5 e4d5 e6d5 d2d4 b8c6 c2c4 f8b4 b1c3

For my undestanding, this is very ugly. I dont clear hast tables between moves: my engine can transpose from one move to onother utilizing hash entries of previous searches. Also I store things like previus scores. The above strings force my to set a new position and make all moves in internal board without doing searches. Also I must clear the hash table, because if the move is a takeback, the age variable in the hash is newer and searhes will not going to overwrite.

Also, if I dont store scores, I will not able to calculate learning values for my book handling (By the way, I would like to handle the book and learning values, and not the GUI).

There are also other issues related to this way to handle UCI. At first sign, I really dont like the protocol.

Could someone point a correct way to handle this issues?
are you like me: dont like UCI? or am I the only that see bad things in it.
why is it so popular in top engines?

Best regards.
Fermin

Let me describe the basic part.
Let's start with position startpos, say you are white the gui will send position startpos then go ..., now you will search the best move and returns it say
bestmove e2e4. From that your move history has e2e4. You can also store its score. Now the gui will asks a move from your opp and opp returns a move say e7e6. The gui will send you position startpos moves e2e4 e7e6. What I did is to process only the last move in that string in that case I will look at move e7e6. Question, what if the gui has sent
Position startpos moves e2e4 b8c6 d2d4 e7e6?
To check that you are in sync with the moves sent by gui count the moves sent by gui, in this case it is 4 moves. When you look at your move history you only have 1 move, and expect to receive 2 moves!! Now check if you receive the ucinewgame command, the position you are receiving might be a new position not related to the previous position which in this case you have to start over recording your move history.
When playing games you usually receive 2 moves from the gui in this case.
So we continue, you receive position startpos moves e2e4 e7e6, look only at move e7e6 if legal you may record it in your move history and when you receive a go command then search a bestmove say d2d4. You also can record that move in your move history. So your history would look, e2e4 e7e6 d2d4.
Expect the gui will send 4 moves to you next time, the 4th move is the move from your opp and the first 3 moves is in your move history.

Learning is not a problem, the gui will not send you result command but you have the move history score plus the number of moves sent by gui. Take fritz interface for example. Near the end of game you will receive the typical position startpos moves .... More moves say 60 from both sides. Then suddenly the fritz will send position startpos moves g1f3. Why only 1 move my move history has more than 1? That signals to you that you are in a new game. That is fritz interface from chessbase, at least version 13 which I have. I already sent an email to support ucinewgame command. I don't know if this is supported in fritz 15 now.
Other interface like arena and winboard they send ucinewgame command that would tell you that you should be ready for a new game and have to clear your move history after doing your learning stuff and other things.

One reason I use uci protocol is because of IDEA from aquarium gui. An amazing software that could record the analysis tree of your engine and compare the analysis tree of other engine. Side by side you can see the score of Sf for example and the score of your engine from a given position. Also because I want to see the beautiful notation with text and variation from chessbase interfaces, namely fritz and cb12. Also want to see the automatic analysis output using fritz with my uci engine, you may use blunder check, full game analysis and deep position analysis.

Another reason is I could test my engine in a fast TC like 5s + 100 ms with cutechess for example and could compare results with other uci engine at this fast TC. Your development could also be fast when using fast TC, but you have to take care of long TC as well, but at least you have a record of its performance in fast TC.

[elcabesa]

my engine use the uci protocol. I think it's really easy to implement has all the relevant command to create a chess engine.

it's indeed a stateless protocol. each time it want the engine to think on a position it tell the engine: "this is the full game ( position startpos moves ....) and now you can GO".

it has the ucinewgame command to inform the engine that from now this is a now game and not much more.

now it's up to you to write a statefull or stateless engine. Most of UI engines are a mix of stateless and statefull;
Stateless: they don't remember the game and relies and the protocl to know the position and the history.
Statefull: the engines doesn't clear the hash and history killer between GO commands to get full advantage of the fact thea I have already good data in my memory

I think a programmer could also implement some form of statefullness in his engine. Uci doesn't mandate it, he is stateless, but the programmer can do whatever he wants.


bye

[mar]

Heck, they can't agree on mate scores, or even the evaluation sign, and to solve that as a user you have to tick the right box.

Yes, mate scores...

The GUI can actually detect and solve both problems at once:
- maintain say md5 hash of engine binaries, associated with mate score & POV
- if not found, let the engine analyze a position where black mates in one; this way you can deduce both mate score and POV

Is there a GUI out there that already does something like that?

[hgm]

uci.html is 26kB, xboard.html is 96kB. Both are equally useful documentation for their standard. Where does the size difference come from?

The CECP specs to which you refer are extremely poor and verbose. They address many things that have nothing to do with the protocol. Like XBoard peculiarities, and how you have to use sdin/stdout in general to have an engine communicate with a GUI (buffering, etc.). I am pretty sure I could be describe CECP in 10KB of HTML.

Not only the resign and draw commands. It is also unneeded complexity: multiple ways to do the same thing. It is things like this: when I go back in xboard, I don't receive "force" and "setboard" with all the things you mention. I get "undo". Undo is a redundant command and yet it is mandatory to implement to get full functionality. Or if I retract a move while playing, I don't get 2 times "undo", I get ... "remove"!

The latter is pretty essential, as after the first "undo" the engine would immediately move again.

I agree that "undo" is troublesome, and in fact all my engines implement it internally in the UCI way: setup the board with the remembered FEN, and replay all moves.
I guess the point here is that CECP is actually two communication protocols in one: a protocol for engine-GUI communication and an interface for running engines from the command line. The "undo"/"remove" command belong to the latter; you would not think it so redundant when you want to take back a move and could only do it by retyping the entire game...

A good addition to CECP would be "feature undo=0", which would inform the GUI that the engine does not support "undo"/"remove", so that the latter could use the "new"/"force"/moves method to effectuate a takeback. Then the engine author that does not want to implement command-line usage would not have to implement that internally.

Actually I think it would be better if GUI's would use this method for take-back by default. Deprecate the "undo" and "remove" commands for GUI-engine communication.

Such convoluted things are the reason that when you start with uci you have full functionality quicker and can focus on search and evaluation sooner. And your users can run under any GUI of their choosing ofcourse. One more such thing: you can send "feature setboard=1", but as long as the GUI is allowed to send "rejected setboard", you must still support "edit" and all its drama.

I don't know any engine that does this, and they all seem to get away with it, so this seems mostly a hypothetical problem. Note that no one forces you to rely on optional protocol extensions. You can stick to the default, and you would not run the risk anything ever getting rejected.

But some GUIs need "setboard" and won't do "edit", so you must still have both. Argh.

That simply counts as a broken GUI. The specs allow a GUI to reject the optional v2 protocol extensions, but it does not allow (and has no mechanism for) the GUI rejecting the default protocol. A GUI that would do so is non-compliant.

With uci the gui has no such weird freedoms and that makes life easier for everyone. If you intend to give the user the full experience, you must implement the whole standard, not just the subset that lets you play a single game.

I agree that it was a bad mistake to add features setboard and san to CECP. In practice a situation has developed where v2 engines can assume feature setboard will never be rejected, and either don't implement setting up of positions at all, or provide setboard as the only method. Note that in UCI you can also still play games without implementing "position fen", and rely on "position startpos".

That is a constructed scenario which clearly didn't hinder adoption.

Apparently adoption doesn't mean a thing, if you can get adopted without supporting draw handling. This is in general quite common; in many areas the worst system dominates the market. (VCRs, PC operating systems...)

[hgm]

Heck, they can't agree on mate scores, or even the evaluation sign, and to solve that as a user you have to tick the right box.

Yes, mate scores...

The GUI can actually detect and solve both problems at once:
- maintain say md5 hash of engine binaries, associated with mate score & POV
- if not found, let the engine analyze a position where black mates in one; this way you can deduce both mate score and POV

Is there a GUI out there that already does something like that?

Better is to define a standard, stick to it, and leave the engines that do not want to comply hangout to dry. UCI GUIs in general also don't have special handling of engines that would print mate scores as score cp 32767 instead of score mate 1.

In XBoard all scores of the form (+/-) 100000+N are recognized as mate(d) in N, and displayed as #N in the engine output.