UCI protocol issue

[kbhearn]

You don't consider it strange that the protocol specs make an explicit provision for what the engine must do in a situation that according to the same specs would be (implicitly) forbidden to create? That is what I would call 'inconsistent'. Consistent is to either assume that the GUI should send such positions to the engine (to inform it of the way the game ended, so the engine can use it as a trigger for its learn code, say), and the engine should respond with 'bestmove 0000' (which I agree the specs do not say), or to assume that it is indeed a GUI bug, and the 0000 must be intended for something else (but what?).

perhaps in a refutation line? where a null move would actually make sense.

[jdart]

It is not rocket science to do move legality and end of game detection. I haven't had a move generator bug in ages. End of game is trickier and you have to be careful with the 50-move draw rule in particular. But testing can take care of that.

I think using a reference engine for this is overkill. Just do it in the GUI.

--Jon

[hgm]

It is not rocket science to do move legality and end of game detection. I haven't had a move generator bug in ages. End of game is trickier and you have to be careful with the 50-move draw rule in particular. But testing can take care of that.

I think using a reference engine for this is overkill. Just do it in the GUI.

You overlook the fact that most people that write a new engine would not have access to the GUI. Even if the GUI is open source, it would require about 10 times as much work to get familiar with it to the point where they could make this kind of a change without breaking anything as it would to build the entire engine.

To get it right in their own engine is not anymore rocket science than to get it right in the GUI. (Often easier, since the engine can do perft etc.) And they have to get it right there eventually anyway. And once they get it right they can use their own engine as a referee engine for zero extra effort. And only very little effort (at the level of the protocol driver, for which sample code will float around) to support target-square highlighting next to legality checking.

What you suggest (everyone having to change the GUI) would also create an enormous logistical problem with forking of the GUI. It is just not a practical solution.

[hgm]

To be brutally honest, both winboard and uci protocols make a huge number of assumptions about how a chess program works. From a purely idealistic point of view that is pretty dicey. Three examples are the assumptions in the protocol (and GUI's use this to do what they do) that ALL programs iterate, all programs produce a "score" as a by-product of how they select a move and all programs work their way down a move list.

But it's easy to imagine someone building a monte carlo searcher that doesn't have some of this "classical" behavior. It could be made to work of course as a hack, but it would really look strange in a modern GUI that is built around all these assumptions.

Actually the protocols are only weakly dependent on these assumptions. The concept of a PV is really a game-theoretical one. Even a Monte-Carlo program constructs some kind of a PV. As to iterating, the depth counter could simply be taken to mean the length of the PV it has currently determined. Time would be universally valid, and for node count you could substitute any number that seems relevant for the algorithm (e.g. number of games played in MC).

Have you ever tried to run Fairygen under WinBoard? This is a good example of how well WB protocol can be used with a non-conventional engine (in this case a table-base generator rather than an alpha-beta searcher). In stead of PVs it prints a list of possible moves, and their DTC in the Engine-Output window. If I wasn't so lazy I could have even accompanied each of those with a PV.

[jdart]

You misunderstand - and maybe I misunderstand you.

All I am saying is, the IMO GUI has to do it even if the engine does, too.

--Jon

[BubbaTough]

You misunderstand - and maybe I misunderstand you.

All I am saying is, the IMO GUI has to do it even if the engine does, too.

--Jon

Well, I don't know what is right or wrong, but I would prefer it if GUIs did not ask engines to request moves (or move pondering) in situations where there are no moves to make. I guess that means I agree with Jon and Don, and probably most other engine writers.

-Sam

[hgm]

All I am saying is, the IMO GUI has to do it even if the engine does, too.

But why? If the engine already does it, the GUI can ask the engine. That is what engines are for, to be consulted by the GUI. Why complicate the GUI by letting it do things in which engines already are expert?

[Don]

All I am saying is, the IMO GUI has to do it even if the engine does, too.

But why? If the engine already does it, the GUI can ask the engine. That is what engines are for, to be consulted by the GUI. Why complicate the GUI by letting it do things in which engines already are expert?

Some of this comes down to design issues, preferences and trade-offs. I'm not saying it's WRONG to ask the engine, but there are factors both pro and con.

First issue is one of trust. In the highly competitive world of computer chess where cheating and unethical behavior is rampant, it would make many people uneasy to "trust" an engine. I would certainly never ask an engine if it though it was winning and then stop the game based on what it says. But even in a model where all are honest you have to consider whether the engines in question have performance or reliability issues.

Another issue is division of effort. For a GUI maker it is natural to want to make his own life as easy as possible, but extra protocol is a burden on every program author who designs an engine to work with your new ches GUI that knows almost nothing about chess. Let's see how that works out: One GUI you design with such a protocol, 4000+ chess engines - hmm, not a good tradeoff and a barrier to engine authors for the game in question.

Hot shot programmers like you and I do not always consider the needs of entry level programmers. There are people building engines who are barely capable of succeeding but eventually manage. Some of these people will improve with time and experience. But some of them will never get started if the protocol is not made as simple as possible. They will get frustrated and quit. I consider it partly my duty to nurture these people.

Now imagine building an automated tester with your protocol - one that can test hundreds of engines. Each engine reports when it "thinks" it can claim a draw, when it declares it has one the game and if the opponent makes a move it believes is illegal. How does that work?

The answer is that it CAN work with a lot of extra protocol. Each engine has to "bless" the moves of the other engine, have a means to disagree with it, agree on endgame status and otherwise. But yes, it can be made to work. With some creativity you can make it relatively simple - but not if you are too lazy to do so - easy to put more and more burden on the engine author.

As I said previously, when working on a project where you provide a graphical board for any conceivable game, some of which has not even been written, it's a practical solution to get something up and running quickly. But in my view it will always be a compromise.

A far better way to do this is to define an interface that is consistent from game to game. Call it an API - a set of calls to get a list of legal moves in some raw format, a way to covert these moves to some "official" notation (which could be same format), a call to get end of game status, etc .... whatever is needed to support all reasonable chess-like games.

Until such time that you can build a proper API with native code, you can take a trusted game playing program, fork a copy of it, and use it to fill the functions of the API. In this way you have the best of all worlds and as long as you have a single program that plays the game in question you can build your GUI correctly. You don't have to trust whatever program happens to be playing at the time, just one that you have appointed as the referee.

The only changes required to the protocol are minor, a call to produce a list of legal moves, etc .....

Later, if the game starts accumulating engines and becomes popular and someone is willing to do the work you can bake this into the GUI so that you don't have such a fragile system.

Don

[hgm]

But we were talking about a third-party referee engine, not one of the playing engines. Even in this highly competitive business there must be some engine you can trust to not cheat in reporting illegal moves, and correctly flagging 3-fold reps, 50-move draws and checkmates. Then you simply use that one for referee.

If you are really in a situation that there is only a single engine for the game at hand, you can always use another copy of that engine as referee, or play without referee. (If the GUI supports such a 'legality-testing off' option.) You won't catch any move bugs that way, of course. But if you think writing an independent move generator for checking this is the most efficient way to debug, it would still be easier for most people to write that as part of a second engine (that doesn't need a search, and can use a slow but straightforward algorithm), then having to implement it in a GUI.

The way it would work in detail is that all moves that come in from an engine go to the referee first. If the referee does not complain, it then also goes to the opponent. If it does, the game is forfeited by illegal move, and the playing engines get that result reported to them. If the move is legal, but cause game end, the referee engine will respond with a game-end command (win, loss or draw, + reason), and the game is terminated as well (after the opponent receives the move in force mode, both receive the result). This all works with existing protocol.

You can try to be smart, and save some time by immediately relaying the move to the opponent without waiting for the referee result. You would then need some kind of a flag to postpone processing of moves that come in before the referee had blessed the previous move. Presumably this should not happen often, as engines should take more time to think up a move than a referee takes to check its legality. It is slightly more complex, but not terribly so.

[Don]

But we were talking about a third-party referee engine, not one of the playing engines. Even in this highly competitive business there must be some engine you can trust to not cheat in reporting illegal moves, and correctly flagging 3-fold reps, 50-move draws and checkmates. Then you simply use that one for referee.

Some people be playing round robin matches between many players - so it cannot just always be one of the 2 players. I don't how you handle matches in xboard between 2 random programs - you tell me how it's handled. If I create a "foobar chess" engine and talk 10 friends into creating their own and talk you into supporting it with xboard, what happens when a round robin is played?

If you are really in a situation that there is only a single engine for the game at hand, you can always use another copy of that engine as referee, or play without referee.

This is precisely my suggestion in the case where you just don't have the time or inclination to build in independent support for that game. It's like the old days, when I did Rexchess it was both engine and interface.

(If the GUI supports such a 'legality-testing off' option.) You won't catch any move bugs that way, of course. But if you think writing an independent move generator for checking this is the most efficient way to debug, it would still be easier for most people to write that as part of a second engine (that doesn't need a search, and can use a slow but straightforward algorithm), then having to implement it in a GUI.

The idea is to have a trusted arbiter, not just any program that half way knows the rules. If you have only 1 playing program that becomes the trusted arbiter, even if it's buggy but if you have more than one you determine which is the best candidate for this job. The arbiter is always a 3rd running program even if a copy of itself is playing a game.

The way it would work in detail is that all moves that come in from an engine go to the referee first. If the referee does not complain, it then also goes to the opponent. If it does, the game is forfeited by illegal move, and the playing engines get that result reported to them. If the move is legal, but cause game end, the referee engine will respond with a game-end command (win, loss or draw, + reason), and the game is terminated as well (after the opponent receives the move in force mode, both receive the result). This all works with existing protocol.

You can try to be smart, and save some time by immediately relaying the move to the opponent without waiting for the referee result. You would then need some kind of a flag to postpone processing of moves that come in before the referee had blessed the previous move. Presumably this should not happen often, as engines should take more time to think up a move than a referee takes to check its legality. It is slightly more complex, but not terribly so.

You are not that far from a universal system like zillions of games. As it is now you have to do some programming work to support a new game - like you did with my schess program. But ideally you should not have to set up a new games - although there must be some constraints - you obvious cannot play monopoly or poker with your system, but any two player game of perfect information played on an NxN board with fairly well defined piece movements should be possible. You should be able to load up the engine and start playing if the game fits the constraints. The engines tell you how a piece should be physically moved, what the notation looks like and so on. You need to cover piece drops and perhaps a few other things we are not used to in western chess but it should be possible - without looking like a big hack. One wrinkle of course is piece graphics but those could be user supplied/contributed.