WB protocol extension: thinking output

[mar]

I do not like that. First, if a score should be interpreted as "mate in", I think it should be better to use the EPD standard. Why use a different one?

For the sake of simplicity and full backward compatibility. The change for existing engines to comply with 1m+n/-1m-n is one line of code.

The only engine that uses very high mate score that I'm aware of is ExChess (100000-n). I don't think there would be any score conflicts, 1M is large enough.

[michiguel]

Well, I don't know. What is readable about this:

Code: Select all

 6    -11       38      42940 g1h3 d7d5 d2d4 c8f5 b1c3 b8c6
 6     -9       54      63384 g1f3 d7d5 d2d4 b8c6 b1c3 c8f5
 6     -1       67      81000 a2a4 d7d5 d2d4 b8c6 b1c3 c8f5
 6      0       84     106629 c2c3 d7d5 d2d4 b8c6 c1f4 c8f5
 6      3      105     141781 f2f4 c7c5 b1c3 b8c6 g1f3 d7d5
 6      7      130     186968 b1c3 d7d5 f2f4 b8c6 g1f3 c8f5
 7      8      203     336052 b1c3 d7d5 g1f3 b8c6 d2d4 c8f5 c1f4
 7     11      317     580601 g1h3 c7c5 d2d3 d7d5 c1e3 d8a5 b1c3 c8f5
 7     17      400     752332 a2a4 c7c5 f2f4 d7d5 b1c3 b8c6 g1f3
 7     24      554    1083815 d2d3 c7c5 c1e3 b8c6 c2c3 d8b6 d1c1

it doesn't say

depth=7, score=24, time=554, nodes=1083812, pv=d2d3 ...

The latest proposal would just be more of the same, and definitely be the closest possible to the existing protocol. Only the requirement about the tab character is new, and I think it would be an almost fail-proof way to recognize one is dealing with the new format, and not accidentally interpreting some line number. It doesn't affect readability in any way, as the tab just shows up as whitespace.

The difference is that those (depth, score, time, nodes, pv) are basic fundamental items that are always present (fail low being an exception for PVs). Others are optional, there could be many, and the list could be growing. For instance, if I do not compute selective depth, why should I send something? should I send a zero? If I only compute tbhits it could be nicer to just send {tbhits=195} rather than {0,0,0,195,0,0}. The latter suffers from rigidity.

Miguel

[michiguel]

I do not like that. First, if a score should be interpreted as "mate in", I think it should be better to use the EPD standard. Why use a different one?

1) Better readability. Both in the protocol and in interfaces that do not support the standard.

How long would it take you to understand 327.41 or 32741 means 'mate in 14'? Now compare that to 1000.14 or 100014. Not only did you have to subtract in the former case, but most people would not even know eher to subtract from...

2) The EPD standard is for EPDs, and WB Thinking Output is not an EPD.

3) Larger range. The EPD standard would get into trouble if DTM > 32768.

Still, since most engines do not use that standard, using any kind of number may produce weird results. Some engines may use a "close" number for mate, which will result in a wrong interpretation. For that reason, and to guarantee that that "mate in __" will have the proper interpretation, a separate "mate in X" should be sent by the engine. In this respect, what UCI is doing is a good idea IMHO.

Although this seems correct in principle, it also seems completely hypothetical. Is here an actual example known of any engine that uses mate scores above 1,000,000 cP? Or even above 100,000 cP? (Yes: Joker. But that already uses this method! ) Writing a sufficiently large number in practice is as good for recognizability as using a non-numeric keyword, and has the advantage that it is compatible with the existing standard for scores.

I believe different concepts should be kept separate and clear. Mate in X, and scores mean different things and fudging it into a single number is not good. Sending "#+12" or mate +12 etc. spell things out, regardless of how the engine manages internally. In other words, if the engine author wants to adopt these improvements, it just adds something into the proposed pv line, rather than touching code for scores. I mentioned the EPD standard because some engines could have output for epd already. Having mate scores different for different established standards is annoying. Much better is to spell it out for the GUI.

Miguel

[hgm]

You are still missing the crux on the mate scores. Of course the engine should not alter anything in the mate scoring it uses internally in its search. In all cases he "just adds something in the proposed PV line". E.g. your proposal:

Code: Select all

if(score > MATESCORE-100) printf("mate +%d", (MATESCORE-score+1)/2); else

while in my proposal

Code: Select all

if(score > MATESCORE-100) printf("%d", 100000 + (MATESCORE-score+1)/2); else

Mentioning the EPDs is also a non-argument. EPDs do not use #2 or "mate +2", so you your proposal differs just as much from the EPD standard as mine.

So the only difference remaining is that your proposal would totally break backward compatibility. So that additional mechanisms would have to be programmed to negociate with the GUI if it could be used. And the engine would have to be able to fall back on the old method if it the GUI did not support the new one, in which case the user would also not have the benefit of more easilaby interpretable mate scores.

As for the thinking output, your arguments also seem very shaky:
1) There is no need for an engine to count nodes, and many engines indeed print 0 in the time and node field (e.g. Pulsar). So this is an already accepted method in the existing protocol.

2) If we accept the idea that engines must calculate quantities like 'nodes' just because the protocol requires them to print those, keeping track of seldep is not intrinsically more complicated than counting nodes. Any engine could do it. Printing nodes/sec should never be a problem; engines that do not keep track of it separately can always divide nodes by time, both of which they already printed in the standard output.

3) The proposed system is not as rigid as you imagine, because it is not mandatory to print all extra infos, but you can clip off those at the end of the priority list if you don't want them. If you only want to print tbhits, you would print

Code: Select all

195              (compare "tbhits=195")

(Note that I have already abandoned the idea of using enclosing {}.) If you want to print only seldepth:

Code: Select all

27 0             (compare "seldepth=27")

If you want to print only knps:

Code: Select all

0 1528 0    (compare "knps=1528")

So the 'overhead' is typically quite small, even when you only want to print a single item. Even in the last case above it was only 4 extra characters, while the "knps=" prefix is already 5. And there is actually little excuse for not printing seldepth: any engine could calculate that, if it wanted. Printing 0 for seldepth or nodes is just laziness. Only the required extra 0 for tbhits in engines that do not use EGT should really be counted as overhead.

But I defined tbhits as highest priority, because IMO it is the only important number amongst the stuff UCI engines send. If I did not want to print tbhits in my engine, I would not bother printing any "extended infos" at all, and just stick to the old standard. Who cares about seldepth or knps?

I think compactness carries a lot of weight here. About 99% of all engine->GUI traffic is Thinking Output in WB protocol. I don't want it to become excessively verbose. It would IMO be extremely annoying to see tbhits=, seldepth=, knps=, cpuload= repeated at nauseum in every line of Thinking Output for the entire game.

It would be even more annoying if people using interfaces that do not implement the extended standard see that info push the entire PV out of view. They would have no choice but to switch that info off (hoping the engine at least supports an option for that). While a prefix to the PV like "27 1528 0 " to the PV to see seldep and and knps would be quite bearable, definitely much less intrusive than "seldepth=27, knps=1528 ". Compactness also matters there.

On a single line, printing "name=" keys might seem a good idea, but in a table, seeing it repeated in every line is actually quite annoying. Columnar output is much more readable.

[Kempelen]

I would suggest don't parse moves with are enclosed in {}, like comments. Sometimes I use {move} before print PV to show "debug" moves, ponder (useful when see in console), the easy move, etc.

[hgm]

WinBoard uses the same parser on the PVs as it uses on PGN. So it would automatically skip anything enclosed in {}, because it considers it comment.

Btw, I thought the recommendation of WB protocol is to print the ponder move before the PV in paretheses. (This is not PGN compatible, but WinBoard tests for that special case when it starts to parse the PV.)

[mvk]

Again, the feature command makes it possible to communicate about what is and is not supported and act accordingly. It seems like a mistake to not use it.

Exactly my sentiment.

I don't like the new and ad-hoc/implicit syntax between {}. We already have JSON for structured data. And explicit is better than implicit.

I don't like using ASCII control characters (TAB). Makefiles have thought us that lesson.

Why not adopt JSON for extensions. Or allow the engine to output UCI output in XBoard mode. All guarded by 'feature' of course. The less hackery type of creativity and more adopting standards, the better.

[Greg Strong]

I'm not sure I understand CPU load. Engines use all the CPU available don't they?

I must admit I don't really understand that either. Even more so 'hashfull'; IMO the hash should always quickly fill up to 100%.

But these are things UCI engines print, and which other UCI supporting GUIs thus show, so that people expect XBoard to show them too. Which is only possible if the Polyglot adapter has a way to transmit them to XBoard in XBoard protocol.

Got it. You said values 0-99. Shouldn't that be 0-100? Incidentally, I would expect every chess program on the planet to report 100 for CPU all the time unless I'm missing something (which I probably am.)

[hgm]

Well, I thought it would be better to exclude 100, to exclude that it could be 3 characters. If it is 100, just send 99...

Engines don´t need to always use 100%. Other programs could steal CPU time from them. I guess this is the point of having this info.

[bob]

I'm not sure I understand CPU load. Engines use all the CPU available don't they?

I must admit I don't really understand that either. Even more so 'hashfull'; IMO the hash should always quickly fill up to 100%.

But these are things UCI engines print, and which other UCI supporting GUIs thus show, so that people expect XBoard to show them too. Which is only possible if the Polyglot adapter has a way to transmit them to XBoard in XBoard protocol.

I don't even think there is a consensus on what "hash full" means. For example, only positions from current search (group of iterations) with same "age" value? Or number of entries with ANY age (old searches + current). The latter seems more common, and it ought to hit 99% at least and then "stick".

I do think cpu % can be useful. I added that to my old program (Cray Blitz) as a safety factor. I once lost a game in an ACM event because someone walked into the lab where the Cray was located and started running something, not knowing it was scheduled for dedicated use (by me) for that evening. I updated cpu% frequently and when it dropped below 99% I started looking to see what was going on. Not very informative otherwise.