UCI protocol issue

[hgm]

Some people be playing round robin matches between many players - so it cannot just always be one of the 2 players. I don't how you handle matches in xboard between 2 random programs - you tell me how it's handled. If I create a "foobar chess" engine and talk 10 friends into creating their own and talk you into supporting it with xboard, what happens when a round robin is played?

Currently I haven't implemented 3rd-party referee engine yet, so the only way to do it would be to play with legality testing off, with all the obvious disadvantages that brings.

But the plan is to allow XBoard to connect to 3 engines in stead of 2, where the third engine is fixed at startup, or at least not under control of any tournament manager. This third engine (when specified), would then be used as the referee for every tourney game. The other two engines would be loaded according to the tourney schedule, and play each other, while the third engine is looking over their shoulder for any misbehavior.

This is precisely my suggestion in the case where you just don't have the time or inclination to build in independent support for that game. It's like the old days, when I did Rexchess it was both engine and interface.

Well, I did not suggest to put engine and interface in the same process or program. If anything, I want them separated more strictly than usual. I want the interface to only involve itself with display, user input, recording the game and passing on moves between players and referee. Everything that has to do with the game rules should go into an engine (which can be a player or a referee).

Biggest problem is notation, btw. To write SAN requires rule knowledge. So an ignorant GUI could not generate or parse it. So we would need new protocol for that. Whenever a move is sent for legality checking to the referee, the referee could respond either by "Illegal move" (existing protocol) or "Legal move LONGALGEBRAICFORM SANFORM" (new protocol). Then you also would not have to use 'ping' to detect that there was no complaint. The referee should understand both SAN and long algebraic, so it could be used to do the conversion either way.

The referee could also send board updates (as FEN) after receiving a move, to implement side effects not obvious from the move notation (like colorflipping of the sandwiched pieces in Reversi).

You are not that far from a universal system like zillions of games. As it is now you have to do some programming work to support a new game - like you did with my schess program. But ideally you should not have to set up a new games - although there must be some constraints - you obvious cannot play monopoly or poker with your system, but any two player game of perfect information played on an NxN board with fairly well defined piece movements should be possible.

Indeed. Apart from the trust problem this already works. The WinBoard-Nebiyu package on my website bundles the WinBoard 'Alien Edition' with Daniel Shawul's Nebiyu engine, and together they can be used to play many non-Chesslike games that are played on a board of squares. Amongst those are Checkers / International Draughts, Amazons, Go, Ultima, Reversi, Amazons (plus many genuine Chess variants). Works like a charm. What is not possible is hexagonal boards (I am still brooding if this couldn't be done by offsetting ranks half a square from each other horizontally).

You should be able to load up the engine and start playing if the game fits the constraints. The engines tell you how a piece should be physically moved, what the notation looks like and so on. You need to cover piece drops and perhaps a few other things we are not used to in western chess but it should be possible - without looking like a big hack. One wrinkle of course is piece graphics but those could be user supplied/contributed.

Piece drops were already supported by XBoard even before I got involved with it. (Crazyhouse was one of the supported variants, because FICS and ICC feature it.) I just added the interface to make them graphically (displaying the captured pieces next to the board, so the user can grab them from there). Nebiyu also supports a number of variants with piece drops (amongst which Shogi and mini-Shogi).

[bob]

To be brutally honest, both winboard and uci protocols make a huge number of assumptions about how a chess program works. From a purely idealistic point of view that is pretty dicey. Three examples are the assumptions in the protocol (and GUI's use this to do what they do) that ALL programs iterate, all programs produce a "score" as a by-product of how they select a move and all programs work their way down a move list.

But it's easy to imagine someone building a monte carlo searcher that doesn't have some of this "classical" behavior. It could be made to work of course as a hack, but it would really look strange in a modern GUI that is built around all these assumptions.

Actually the protocols are only weakly dependent on these assumptions. The concept of a PV is really a game-theoretical one. Even a Monte-Carlo program constructs some kind of a PV. As to iterating, the depth counter could simply be taken to mean the length of the PV it has currently determined. Time would be universally valid, and for node count you could substitute any number that seems relevant for the algorithm (e.g. number of games played in MC).

Have you ever tried to run Fairygen under WinBoard? This is a good example of how well WB protocol can be used with a non-conventional engine (in this case a table-base generator rather than an alpha-beta searcher). In stead of PVs it prints a list of possible moves, and their DTC in the Engine-Output window. If I wasn't so lazy I could have even accompanied each of those with a PV.

I don't see the assumptions about a score and such. Nothing dictates that I send anything other than a move. The "." status check has always been optional so far as I know. I think the ideal protocol, of which winboard is a pretty good example, is a protocol that explicitly defines what is necessary to play the game (moves to / from engine and time remaining) and then provides whatever other information might be useful (rating, name etc). UCI, for example, tramples all over a normal console-mode engine, interfering with pondering and such, trying to manage time and all the other stuff.

The option menus/buttons/knobs are nice, but that's fluff when the goal is to use the gui to play a game.

[Don]

Biggest problem is notation, btw.

With a protocol that uses a 3rd party arbiter engine you should have 2 output formats, one of them is "raw" and the other is "formal." For almost everything you use the raw format which is expressive enough to decode any move even if you don't understand the rules. For example long algebraic is ALMOST expressive enough to do this but not quite for western chess.

For display purpose you have the FORMAL notation - which falls back to the same RAW format if it has not been implemented. In this way you do not have to do anything - whether the arbiter can handle it or not doesn't prevent the GUI from working.

So all program pretend they support the formal notation, even if they don't.

[hgm]

The problem arises, however, when you want to feed a game in formal notation (e.g. from a PGN file loaded by the GUI) to an engine that does not truly understand it. Requiring all engines to be able to decode SAN is a bit hard on the engine programmers. But some entity would have to be able to convert the formal notation to the raw notation, which can be trivially understood. And this does require knowledge of the game rules, so I want to keep it out of the GUI. The referee engine would thus be the best choice, but that means the referee engine should do some things that are not needed in a normal engine. (In particular, echo moves that are sent to it, in one format or another, and understand both the formal and the raw format on input.)

Playing engines would then only have to understand the raw format. In the WinBoard Alien Edition we use a comma-separated list of long-algebraic moves, (to allow moving multiple pieces per turn, or a single piece making several steps). Castling and e.p. capture are still implied, however, you don't have to use a multi-leg move to specify those.

Games like Checkers, Go or Ultima, which do not use replacement capture, could in theory be handled by long algebraic as if they did, letting the moved piece make multiple steps visiting all squares where pieces should disappear. (So e.p. capture exf6 could be, say, e5d5,d5f6, to obliterate the victim Pawn. My ChessLive! web viewer actually does it that way.)

This does not work for games like Reversi, where pieces are converted to the other color. Although you could drop pieces on top of the squares you want to color-flip.

An alternative for the raw format would be to send a FEN of the position after the move. That would capture any side effects automatically.

[Don]

The problem arises, however, when you want to feed a game in formal notation (e.g. from a PGN file loaded by the GUI) to an engine that does not truly understand it. Requiring all engines to be able to decode SAN is a bit hard on the engine programmers. But some entity would have to be able to convert the formal notation to the raw notation, which can be trivially understood. And this does require knowledge of the game rules, so I want to keep it out of the GUI. The referee engine would thus be the best choice, but that means the referee engine should do some things that are not needed in a normal engine. (In particular, echo moves that are sent to it, in one format or another, and understand both the formal and the raw format on input.)

You don't require ANY engines to be able to decode the formal notation (SAN.) It is purely optional for any engine. But every engine is require to at least PRETEND it supports formal notation. In this way any engine can be the referee, even if it doesn't actually know how to support formal notation.

Now you will always prefer to use an engine that knows formal notation as the referee engine if one is available, but it will still work if it is not. Of course the only engine that will ever be asked to send anything in formal notation is the referee program. If it cannot, it sends raw notation. No muss, no fuss.

Playing engines would then only have to understand the raw format.

Yes, that is exactly my point. In fact not even the referee with be required to understand formal notation although it will always be more desirable if it does.

In the WinBoard Alien Edition we use a comma-separated list of long-algebraic moves, (to allow moving multiple pieces per turn, or a single piece making several steps). Castling and e.p. capture are still implied, however, you don't have to use a multi-leg move to specify those.

Games like Checkers, Go or Ultima, which do not use replacement capture, could in theory be handled by long algebraic as if they did, letting the moved piece make multiple steps visiting all squares where pieces should disappear. (So e.p. capture exf6 could be, say, e5d5,d5f6, to obliterate the victim Pawn. My ChessLive! web viewer actually does it that way.)

This does not work for games like Reversi, where pieces are converted to the other color. Although you could drop pieces on top of the squares you want to color-flip.

An alternative for the raw format would be to send a FEN of the position after the move. That would capture any side effects automatically.

Except that fen has to be defined for each game type. If you want to have a kind of universal GUI that doesn't require explicit programming you must have some configuration file or else the engines must pass enough information on startup to handle the game in question.

[hgm]

Now you will always prefer to use an engine that knows formal notation as the referee engine if one is available, but it will still work if it is not.

I think you focus too much on playing engine-engine. Loading an externally provided file in formal notation (as they tend to be distributed) would not work at all.

Of course the only engine that will ever be asked to send anything in formal notation is the referee program. If it cannot, it sends raw notation. No muss, no fuss.

Indeed, that is a good scheme.

Except that fen has to be defined for each game type. If you want to have a kind of universal GUI that doesn't require explicit programming you must have some configuration file or else the engines must pass enough information on startup to handle the game in question.

A FEN is a quite general representation of what the GUI does graphically: display a board of squares with (possibly) a piece of a certain type and color on each square. If that is all the GUI can do, the information can be fully specified by a FEN. The only thing that has to be defined is the mapping of letters on piece images. In XBoard (the standard edition) this is done by a pieceToCharTable option, which contains the pieces in 'canonical order' (i.e. there is a fixed GUI-piecetype-to-string-position mapping), and which then contains the one-letter piece-ID to be used for that piece in this particular game in the position reserved for that piece.

Engines can send a 'setup' command to the GUI in response to the 'variant NAME' command, containing a FEN for the starting position, and optionally a pieceToCharTable string telling the GUI how to interpret the FEN (and which pieces are allowed to participate, if not all are present in the initial position). Currently XBoard only pays attention to the setup command in 'variant fairy' (a 'catch-all' variant, where the engine dicides what is played), or when the user has specified a board-width override so that the default position know by XBoard for that variant no longer fits the board. This is how Fairy-Max implements the different flavors of 'Chess with Different Armies': you can select the armies through an engine-defined option in XBoard's Engine Settings dialog, and when you then select 'fairy' from the 'New Variant' menu, the engine will configure XBoard through the 'setup' command to play with the selected armies.

This all works very satisfactorily, without any special problems. There is a refinement for drop games, where there exist extended FEN standards (bFEN and SFEN) that allow you to specify if a piece (like a Queen in Bughouse) is a primordial Queen, or a promoted Pawn that should demote when it goes to the hand on capture). To cater for such games XBoard has organized its piece types in pairs (base piece, promoted piece). But that is just a refinement that can also be applied quite generally.

In the standard edition of XBoard the setup command is used only to define initial position and one-letter piece ID to be used in FEN and SAN, and is ignored when received in a game that is already in progress (or when the user had already set up a position himself). But in the 'Alien Edition' there is a catch-all 'variant alien' that would max the GUI obey the setup command at any time, for updating the board after moves, to be used for games that have moves which cause board mutations that are not obvious from the move notation as (from, to) pair.

Using a 'raw' notation that fully specifies the board notation (like e1g1,h1f1 for castling, and d5e5,e5e6 for e.p. capture) is nice in engine-engine games, where both engines use the raw notation, but it would be cumbersome in human-engine games (which is often the primary application of one-of-a-kind engines). You want the Rook to move automatically when the user drags the King for entering a castling move, and the e.p. victim to disappear automatically. And in Checkers you want the jumped-over chips to disappear without explicit user action to make that happen. So the user manipulation would only lead to a single (from-square, to-square) pair, and the system must be able to implement any implied side effects. Sending FENs for board update would be one way to do that. (The other is to allow the engine to finish a user move by simulating mouse-click events on specified squares. This could also be used on ordinary moves to implement 'one-click moving', in cases where a piece has only a single legal move, and you click on the from-square.)