Some more WB protocol extensions

[hgm]

Polling for something to happen is always bad.

Polling to measure the progress of something is not necessarily bad.

Would you like your wall clock to emit a beep every second or do you prefer to look at it when you need to know the time?

I think this is a false analogy. I decide when to look at the GUI display anyway. I'd much prefer the GUI to show me instantly when the engine changes to serching a new move, than having to suffer a delay because of asynchronous polling.

If we want something user-configurable here, I think Bob's approach would be much better. We could add a parameter to the post command, or perhaps even two, to control the stream of thinking output: specify a minimum and a maximum time between reports, in centi-seconds. The engine reports the first real event (someting else then a mere node-count change) after the minimum time has expired since the last report, and if no such event occurred after the maximum period it sends a new line to update the thinking output.

[hgm]

By the way, there is and epd2wb utility written by B. Moreland and modified by T. Mayer that does that.

But I bet you that it sends neither 'reset' nor 'memory' to the engine... The point is that we have to decide now which of thetwo it should choose.

That is why I think reset or clear will be useful when the user really want to start from zero.

Well, you certainly have a point, and you nearly have me convinced. But I consider the practical side of this too. By offering the protocol user only a package deal, he wil be more likely to buy the entire package, just to get only one of the parts he really wants. And the package can be offered at a discount, as it is easier to implement only a single command and a single enabling feature, then two independent ones. So I am still in doubt. It seems that in the past people have already resorted to kludges such as letting setboard clear the hash table to get the epd suites running, which IMO is far worse than abusing the memory command a little.

[bob]

I was simply objecting to the use of the word "hideous" for a command that allows the GUI to ask information from the engine.

There is no need to use such charged words when discussing a communication protocol.

I agree with HGM. Polling is a poor way of doing this. So rather than "hideous" how about "poor approach" or "crappy approach" or "antiquated approach" or "sophmoric approach" or something equally descriptive. Polling is never a good design. It wasn't a good design in MSDos when it polled a disk drive over and over to ask "are you done, yet?". It is not really a good design today when we ask "is there a move to read". We do this because there is no compatible way in windows to avoid this. In unix, it is trivial to use asynchio. One can always eliminate the polling by using a second thread, and then some sort of signaling mechanism to tell the search when you have a new move, etc. And that is a better design as well. In short, polling is a poor solution to a simple problem.

I have a second thread that deals with the "polls". That is still a "poll" from the GUI. I do not know then what you mean by poll because it is not eliminated.

What do you do between polls in the second thread? When I used that approach in Cray Blitz, the second thread just did a normal read() and blocked, while the first thread searched/pondered like mad. When the read() was satisfied, the "read thread" would signal the "search thread" that there was something that required attention.

To poll means to repeatedly ask "do you have a move?" <NO> "do you have a move NOW?" NO. What about NOW? NO. Etc. A read isn't a "poll" it is a blocking operation that causes that thread to stop executing until data is available, rather than sitting in a tight loop asking "is there something to read?"

If you want to see something every N seconds, then we should define a mechanism to make that happen _automatically_.

The user may want to know only once what is going on and do it manually, so being spammed every N seconds just in case the user wants to know something is hideous

That is OK. Again, that is not polling. Polling is something done at regular intervals, such as how xboard sends that "." every so often to get the status information. Sending a "." when a user clicks a box on the GUI is OK, as that is something that would only happen when clocked, not like clockwork, even when nothing has changed...

Crafty has a "burp" parameter that tells it how often to update the current display so that we can tell it is alive in tournament play when ICGA rules do not allow you to hit a key to ask "are you alive".

That is fine, but ICGA tournaments are a small fraction of the universe of CC. I think you agree

No disagreement at all. I just pointed out that sending some sort of status info is something a program can do at the appropriate intervals. For example, the famous "Nh4 21/40" type output. If you want that, the program ought to send it each time it searches a new root move, rather than constantly being asked "got anything new yet?" "got anything new yet?" The program knows when it has something new and interesting to display.

So a solution would be to either (a) let the GUI tell the engine how often to provide some sort of update or (b) let the user tell the engine how frequently he wants to see information (about all that this shows is total nodes and current move being searched, most engines will tell the gui when the score or PV changes or there is a fail-high or fail-low automatically. I think the entire idea could be dropped and just let the engine tell the GUI about current state whenever it feels like it, or not at all.

If the engine is capable to relay information when it is asked, I do not see the problem.

Miguel

We are not quite on the same page. Nothing wrong with the engine being able to respond when asked. But to ask every 2-3 seconds is a waste of time and introduces overhead.

[hgm]

I think if we're going to standardize thinking output, we should do it properly. This is one area where UCI excels. Though the syntax is ugly, the standardization is very nice. Xboard has a problem here due to backward compatibility. I think in this case, abandoning backward compatibility with old GUIs would be worth it for better thinking output.

Backward compatibility is almost sacred. As I see it, the main advantage of sacrificing it is that you slice up the info over several parts, so that only the changed part has to be resent. But what does this actually save? In practice considered move, time, nodes, tbhits all change together. Only score, depth, PV and nr of root moves might be durable. (Come to think of it, the number of root moves is bit of a questionable thing to send anyway. Wouldn't the GUI know this by itself?) But part of the potential gain of leaving out some stuff is lost, because you now have to specify what you are sending, and use a more verbose format than just integers.

I think sending level in the middle of a game is just going to cause bugs, even if the behavior is well defined. I don't see it as being too useful anyways.

That you don't see it is useful, doesn't mean that everyone thinks it is not useful. I see many applications for this, and will probably be using it myself.

[bob]

For the record, in the world of non-windows, which is a sizable world, CPU time is problematic. There is no standard on obtaining this information. Some systems return total time for all threads, some return the time for the specific thread making the request. In Crafty, I am unable to report this time as I removed that code completely. I can report elapsed time only, as that is all I have. I don't particularly like the idea of CPU time anyway. A parallel program can use less than N*elapsed_time seconds (where it is using N threads) for several reasons. Other threads are preempting the chess engine. The chess engine is doing EGTB accesses that block for I/O and consume no CPU, the parallel search uses blocking for whatever reason, such as thread synchronization, MUTEX locks, etc. The "cpu load" doesn't really give a lot of information by itself.

The reason I proposed this is that there was demand for having the engine report its CPU load. All the difficulties you mention would also apply if the engine would have to calculate this load by itself by dividing CPU-time by wall-clock time. So perhaps it is good to split up the debate into two sub-issues:

1) would it be better to have the engine calculate its CPU load and sent it to the GUI as a separate parameter, or should it simply send the raw CPU time, and let the GUI do the math? I would aim for the latter.

2) Is any of this feasible? That you took out the code, is not the same as it being impossible... If it is not feasible under non-Windows OS, my take would be then just to "cheat" and report wall-clock time (or wallClockTime * nrOfThreads), so that a GUI would report 100% CPU usage. Not much harm done, I would say. If some engines _can_ meaningfully report this quantity in _some_ situations, the protocol should provide a way to do it. Engines that for som reason cannot or don't want to comply, can simply sent some default. There are plenty of WinBoard engines that always send 0 for the nod count, for instance.

All I can say is that this caused much grief in early crafty development. Even versions of windows were not consistent in how you grab CPU time and whether it is per thread or per thread group. That's why I took it out. We kept seeing oddball cpu loading because different systems made obtaining total CPU time a portability issue and also a moving target as to how it was done.

Some programs use processes rather than threads. That requires a different way of measuring time than a program that uses threads. And when you can do either on windows or on unix, that is 4x the complexity. I decided that was not worth the effort. I originally did it so that I would notice poor CPU usage back in the Cray days indicating that something was running on my box that should not be there.

Probably an unsafe assumption. To break this, one needs to (a) protect the best move when you store an entry that is UPPER (fail low, no best move) by keeping the best move for this position from the position you are replacing (assuming the two positions are the same of course) and then (b) doing hash probes to fetch the PV to display it. That would allow a longer PV.

OK, so you are explaing us how engines could foul it up. But it takes a conscious action of the programmer to make the engine emit the + or - fail indications, and if he takes care to program that, he might as well put in code to suppress the PV. My point was that a long PV is not _needed_ in these situations, so there is no downside to requiring that these symbols go behind the PV. You know if you failed high, and can then print the failing move, suffixed with a + (or ++, or whatever we settle on). If you fail low, you can just print a -, and no move.

[/quote]

I wasn't suggesting this is a good idea. My fail high or fail low PVs are a single move (I ought to display the second move in the PV as that is good, but that is a feature for later as it would be pretty useful to know what actually refutes the move we thought was best until this iteration proved it bad.) But I have seen programs that do this. Where the fail high has a long PV associated with it. It would take a conscious effort on their part to turn this off.

[bob]

agree with HGM. Polling is a poor way of doing this. So rather than "hideous" how about "poor approach" or "crappy approach" or "antiquated approach" or "sophmoric approach" or something equally descriptive. Polling is never a good design.

Polling for something to happen is always bad.

Polling to measure the progress of something is not necessarily bad.

Would you like your wall clock to emit a beep every second or do you prefer to look at it when you need to know the time?

That's the point. When I want to know, I will ask. At other times, I will asked to be notified. But if I have a good alarm clock with severa settable alarms, I will _never_ "poll" the clock, I will be waiting for a beep while doing useful stuff that does not include checking the clock every so often. I told the clock when I wanted to do something, it is the best at knowing when that time comes and to inform me about it without me asking.

[bob]

I think if we're going to standardize thinking output, we should do it properly. This is one area where UCI excels. Though the syntax is ugly, the standardization is very nice. Xboard has a problem here due to backward compatibility. I think in this case, abandoning backward compatibility with old GUIs would be worth it for better thinking output.

Backward compatibility is almost sacred. As I see it, the main advantage of sacrificing it is that you slice up the info over several parts, so that only the changed part has to be resent. But what does this actually save? In practice considered move, time, nodes, tbhits all change together. Only score, depth, PV and nr of root moves might be durable. (Come to think of it, the number of root moves is bit of a questionable thing to send anyway. Wouldn't the GUI know this by itself?) But part of the potential gain of leaving out some stuff is lost, because you now have to specify what you are sending, and use a more verbose format than just integers.

I think sending level in the middle of a game is just going to cause bugs, even if the behavior is well defined. I don't see it as being too useful anyways.

That you don't see it is useful, doesn't mean that everyone thinks it is not useful. I see many applications for this, and will probably be using it myself.

That's what Intel said years ago too. God how I hate just having eax, ebx, ecx and edx until we get to 64 bit processors with the extra registers. There must come a point in the life of every design where things have changed enough that it is time for a "clean sheet of paper".

[hgm]

This is why I say 'almost'. But there must be some obvious and major advantage to outweigh the massive nuisance it will cause to drop backward compatibility, otherwise the community will simply not accept it. To stick to your Intel metaphore: there have been many CPUs that technically and architecturally were superior to i386. Commercially they were all dismal failures compared to i386. Even Intels own 64-bit architecture went that way, because it was not as backward compatible with i386 as AMD's x64 design.

But we are digressing, please let us stay focused on the protocol proposal.

I figure that the multipv proposal is OK, and the xlevel not too bad?

[Michel]

If we want something user-configurable here, I think Bob's approach would be much better. We could add a parameter to the post command,

Well I really don't see the difference between

(1) Sending a command to the engine to make it send a node count update every second.

(2) Asking the engine for a node count update every second?

Wouldn't you agree that these are functionally completely equivalent....?

I don't see why you call (2) bad and (1) good.

[hgm]

1) no unnecessary traffic from GUI to Engine
2) the engine does not have to snd it exactly every N sec, but can send when it has something interesting, and start counting from there.