Authors of WinBoard SMP engines, take note!

[Zach Wegner]

It seems in the above engine output, PV lines are sent repeatedly: each time one of the four PV lines is increased in depth from 2 to 3, not only the new line is sent, but also a repetition of all the other PVs, which are still at depth 2.

This is not what the engine should do. Even if WinBoard would display such lines, it would needlessly eat up space in the engine-output window, as the lines are already there exactly as repeated. WinBoard reacts by clearing the engine-output window when it receives a PV with lower depth as the one it already has, as it assumes a new search has started.

In the end the 4 d=3 lines should have been displayed, though, which is the intended effect. But you would have lost all d=2 info, which is unintended. And if the search would have been continued with d=4, all d=4 lines would have immediately been cleared away by the following (repeated) d=3 lines, untill all d=4 lines are available.

A WinBoard engine running in multi-PV mode should not repeat lower-depth lines in its thinking output, but only send the PVs that are truly new.

I disagree. The engine searches multiple moves, and whenever any new PV comes in, it must re-rank the moves (depending on the algorithm of course). I don't see any need to display old multi-PV info, but the GUI should update the display whenever it gets new information with all the PVs in order. I don't think this can be accurately replicated with just a polyglot translation layer. The display for multi PV should pretty much work like clearing the display whenever the X lines are sent.

[hgm]

The way you propose it, the output would get cluttered with duplicate line, and the previous iterations would get erased or disappear from view very quickly.

Normally engines send their PV after the iteration completes, and if they to that in the sorted order, there will be a neat list of successively deeper PVs in the engine-output window, each depth sorted by score. For people that want to see information during an iteration, such information is necessarily incomplete. The order will be meaningless because not all PVs will be available at the same depth, and different depths canot be compared well. So having the engine spew out every PV for which it gets an exact score seems good enough and avoids duplication, while keeping old relevant information in view. The user can see the scores, and decide which is best. After the iteration finishes, the engine can make the final ranking, and send it again. This is what Joker does in single PV mode anyway: It sends a PV as it gets it, and then sends the same PV as the iteration is finished.

[bob]

The way you propose it, the output would get cluttered with duplicate line, and the previous iterations would get erased or disappear from view very quickly.

Normally engines send their PV after the iteration completes, and if they to that in the sorted order, there will be a neat list of successively deeper PVs in the engine-output window, each depth sorted by score. For people that want to see information during an iteration, such information is necessarily incomplete. The order will be meaningless because not all PVs will be available at the same depth, and different depths canot be compared well. So having the engine spew out every PV for which it gets an exact score seems good enough and avoids duplication, while keeping old relevant information in view. The user can see the scores, and decide which is best. After the iteration finishes, the engine can make the final ranking, and send it again. This is what Joker does in single PV mode anyway: It sends a PV as it gets it, and then sends the same PV as the iteration is finished.

The best idea is a more specific protocol... So that you send a depth, an eval, and a PV, but in a standard form, so that the GUI can then manage this. The GUI would rank moves by depth, and within depth, by eval, both in descending order. Then it would not matter whether lines were repeated or not, the GUI could discover this with a simple strcmp() for the PV, and if the depth and eval are the same, get rid of the duplicates...

[hgm]

The best idea is a more specific protocol... So that you send a depth, an eval, and a PV, but in a standard form, so that the GUI can then manage this.

But this is already the case. WB protocol exactly prescribes the format of the thinking output.

But I don't see much benefit in juggling lines around when their ranking changes. That would just make it more difficult to read the output, as the line you were reading would suddenly jump to another location. I also dislike the idea of the GUI disguising the thought process of the engine, by re-ordering its output.

All ways of doing this have their disadvatages, and I think the disadvantage of the lines not being ordered by score during an iteraton (as opposed to when the iteration completes) is the least of most evils, and something everyone should be able to live with.

[bob]

The best idea is a more specific protocol... So that you send a depth, an eval, and a PV, but in a standard form, so that the GUI can then manage this.

But this is already the case. WB protocol exactly prescribes the format of the thinking output.

But I don't see much benefit in juggling lines around when their ranking changes. That would just make it more difficult to read the output, as the line you were reading would suddenly jump to another location. I also dislike the idea of the GUI disguising the thought process of the engine, by re-ordering its output.

All ways of doing this have their disadvatages, and I think the disadvantage of the lines not being ordered by score during an iteraton (as opposed to when the iteration completes) is the least of most evils, and something everyone should be able to live with.

Why would you not want the "best move" ordered at the top? You can then keep the older search results visible (if you want) but at the very least, the first one is the best. This would most likely be the normal case anyway, except for those that fudge and use results from older searches as well (which is wrong, but that is another story).

[hgm]

Why would you not want the "best move" ordered at the top?

Because that would obscure the fact that the engine might not have found it last, for one. You can always see which had the best score: the score is printed. But the order of the PVs is the only thing that contains the information how the engine found them.

[bob]

Why would you not want the "best move" ordered at the top?

Because that would obscure the fact that the engine might not have found it last, for one. You can always see which had the best score: the score is printed. But the order of the PVs is the only thing that contains the information how the engine found them.

Unfortunately, I like to read top-to-bottom, not bottom-to-top. I suppose that is what you are saying. The most recent PV is on the bottom, and things scroll up as a new one is added? I think that's OK for normal output. But for a "multi-PV" display, to me it would be more readable if ordered in the more natural top-to-bottom, best-to-worst order.

I don't run like that except when having Crafty annotate PGN games, so it doesn't matter and I don't watch it while it is annotating anyway as that is an "offline" type operation for me where I will come back later and look at the output. But in my annotated games, I display best to worst, top-to-bottom if you ask for multiple PVs per game move...

[krazyken]

Why would you not want the "best move" ordered at the top?

Because that would obscure the fact that the engine might not have found it last, for one. You can always see which had the best score: the score is printed. But the order of the PVs is the only thing that contains the information how the engine found them.

The problem with using the score to determine which line is the best is in a case where the pv fails low, then that line is dumped and subsequent lines at a greater depth have a lower score. I don't care too much if it's top to bottom or bottom to top as long as the lines are sorted and I always know which lines are the current best.

[hgm]

The WinBoard engine-output wndow scrolls downward, so normally the last PV would be at the top. With normal seach this is then also the best one, in engines that print a PV every time they change the root move. So the engine should print the multi-PVs low to high score, after the iteration finishes.

Having the multi-PV also appear in the PGN is something that did not occur to me yet. So far no PVs are stored by WB in the PGN at all.

[bob]

Why would you not want the "best move" ordered at the top?

Because that would obscure the fact that the engine might not have found it last, for one. You can always see which had the best score: the score is printed. But the order of the PVs is the only thing that contains the information how the engine found them.

The problem with using the score to determine which line is the best is in a case where the pv fails low, then that line is dumped and subsequent lines at a greater depth have a lower score. I don't care too much if it's top to bottom or bottom to top as long as the lines are sorted and I always know which lines are the current best.

In N-best I would not show fail highs anyway. But if that is the case, I would probably prefer bottom-to-top order, so that the bottom is always the most recent, and the top is the oldest (possibly worst but not necessarily since scores from different depths can't be readily compared).