UCI multipv question

[mar]

So basically I have to wait for k best moves first before sending the PVs to GUI.

You don't have to wait as you can fill with moves from the previous depth (as explained in the previous post). At depth 1, waiting does not matter, but I have created all moves anyway, so I can send PVs of length 1 as shown in the example output.

And what if I have a position where number of legal moves is less than k?

Than the output is the same as with MultiPV n, where n is the number of legal moves.

Thanks

[mar]

Thanks for reply. I also noticed that you use windowing in multipv mode as well. That sounds interesting. I currently don't do any windowing in k-best mode.

I think if you do windowing in MultiPV mode, you should have individual windows for each move. Using just one window for all moves is most likely harmful. The lazy solution is not to do windowing at all in MultiPV mode. My possibly better solution (though I never compared in a tournament with MultiPV enabled) can be found here:

http://talkchess.com/forum/viewtopic.php?t=38404

That's a quite a lot of code. The problem is that I do windowing outside root search (in iterative deepening loop) so implementing that in multipv mode would mean complete rewrite/refactoring of existing code. Anyway multipv showed me that I prune too much, I have a position where most engines find a winning move very fast (11 plies), while my engine needs 20 plies (!!!) in normal mode and 15 plies in 4-best mode. It seems that I prune too much or that I have a serious bug somewhere. It certainly has little to do with eval (9 plies!). I'm sure I generate legal moves. The funny thing that disabling null move, LMR, futility doesn't help... But that's another story...
Thanks for your help Onno!

[hgm]

Don't want to start a protocol flame war, but IMO an elegant protocol is one that has few and simple rules that are straight forward to implement with the minimal quantity of source code.

I don't think the requirement to keep around a large number of old PVs for resending them, and then figuring out which of those exactly have to be resent, and which can be discarded, does very much for minimizing the amount of source code...

How about the Stockfish KBBK (and KBNK) problem mentioned in the other thread, btw? I am still eagerly awaiting a comment from someone from the Stockfish team there.

[Onno Garms]

So basically I have to wait for k best moves first before sending the PVs to GUI.

As said, you don't have to. And it is important not to wait.

If you wait, it's possible that the bestmove gets updated quietly. If you interrupt your search before the first k moves are searched, the engine seemingly suddenly changes its mind on the best move.

In Stockfish, I just found an even worse situation: The pv is sent after all moves are searched. This results in seemingly sudden bestmove changes when the search terminates or gets interrupted, even when MultiPV==1. This is very bad for analysis or if you play a game with Stockfish (say in the engine room of some server) and do the time mangement manually.

Another problem with sending the PV late is that immediately after that the next depth starts. seldepth was just sent with the PV, for starting the next depth no seldepth is sent. This might prevent some GUIs from ever displaying the seldepth. (Shredder GUI with Stockfish displays the seldepth with the PV, but never in the depth field.)

I think that this should be fixed in the next Stockfish release by sending the PV after each bestmove update. Marco, do you agree?

[tano-urayoan]

Hello Mr Garms
Maybe you could check this threads made in openchess forum. The purpose of the people working in it was to make Stockfish more "friendly" for analysis.
http://www.open-chess.org/viewtopic.php?f=5&t=1042
http://www.open-chess.org/viewtopic.php?f=5&t=1234

In fact they design an experimental version that deals with that. Here it is
http://www.open-chess.org/stockfish_pa_gtb_gran2k.7z

And the sources
https://github.com/jeremybernstein/Stockfish_PA_GTB

[mcostalba]

I think that this should be fixed in the next Stockfish release by sending the PV after each bestmove update. Marco, do you agree?

This was the behaviour in the past releases, but raised concerns due to the high frequency of changes in the best move especially at low depths. Currently pv is sent at the end of each iteration, not at the end of the whole search. For small depths this seems the best solution, I agree for high depths, when a single itration takes time, we can do something better.

Regrading the seldepth issue, I was unuware of causing troubles to GUI. What do you think of the below patch ?

Code: Select all

--- a/src/search.cpp
+++ b/src/search.cpp
@@ -561,7 +561,9 @@ namespace {
     while (!StopRequest && ++depth <= PLY_MAX && (!Limits.maxDepth || depth <= Limits.maxDepth&#41;)
     &#123;
         Rml.bestMoveChanges = 0;
-        cout << set960&#40;pos.is_chess960&#40;)) << "info depth " << depth << endl;
+        cout << set960&#40;pos.is_chess960&#40;))
+             << "info depth " << depth
+             << " seldepth 0" << endl;
 
         // Calculate dynamic aspiration window based on previous iterations
         if &#40;MultiPV == 1 && depth >= 5 && abs&#40;bestValues&#91;depth - 1&#93;) < VALUE_KNOWN_WIN&#41;

[Eelco de Groot]

Don't want to start a protocol flame war, but IMO an elegant protocol is one that has few and simple rules that are straight forward to implement with the minimal quantity of source code.

I don't think the requirement to keep around a large number of old PVs for resending them, and then figuring out which of those exactly have to be resent, and which can be discarded, does very much for minimizing the amount of source code...

How about the Stockfish KBBK (and KBNK) problem mentioned in the other thread, btw? I am still eagerly awaiting a comment from someone from the Stockfish team there.

You must have missed Joona answering the day before:

If this is the case, I'd recommend to examine the changes that have been done in the TT scheme. (There are at least some changes related to using the TT in PV.)

Yes, this is most likely the reason. We made a change which made stockfish to remember previous thinking sessions, so it became more useful for analysis, but the change created this bug. We will fix this issue in next relase.

It is likely that that the easier return from VALUE_TYPE_EXACT hits in PV nodes has something to do with it. I believe this was first reported by Frank Quisinsky, in the general forum. But I'm not sure if that was version 2.0. If that was earlier Stockfish it can't be from this change alone. It is not for nothing I think that both Fruit and Glaurung never returned TT hits in PV nodes. Implementation of KBNK in Toga which I did a long time ago did not cause any problems. The narrow aspiration search window is possibly a factor too. Or the change in the TT refresh rules, assuming Frank did not report about version 1.9. I have not tested any of this yet though.

Eelco

[mcostalba]

It is likely that that the easier return from VALUE_TYPE_EXACT hits in PV nodes has something to do with it.

Hi Eelco, yes probably could be something like that, anyhow I really would like to understand exactly how this behaviour can happen, step by step, and I am still not able to figure it out. I hate to fix bugs monkey style, just because it happens to work after the fix.

[hgm]

You must have missed Joona answering the day before:

Actually he answered that an hour after my post (12:52 vs 13:46 on the 15th). Quite possibly in reaction to it. I agree that my talents are awesome, but clairvoyance unfortunately is not amongst them...

[Onno Garms]

Currently pv is sent at the end of each iteration, not at the end of the whole search.

Yes but
1. this is delayed
2. the end of the last iteration is the end of the search, which leads to the problem described above (Stockfish seems to change bestmove at the end of the search)

For small depths this seems the best solution, I agree. [For] high depths, when a single itration takes time, we can do something better.

The UCI specification recommends not to send anything within the first second. This is what I did in Onno.

Your suggestion might be even better. (Some customers complained that Onno sends the first PV so late.) Always send PVs after an iteration. In addition, start sending new PVs after one second. The downside is however, that if the position is easy enough that the search goes to depth 50 immediately, still a lot of PVs are sent quickly.

Regrading the seldepth issue, I was unuware of causing troubles to GUI. What do you think of the below patch ?

Well, em, nothing.

The Shredder GUI has a field where the latest depth is displayed. Before you start an iteration at depth n, you send "depth n", so the GUI displays "n". If you send "depth n seldepth 0", the GUI will display n/0.

The problem is that you send "depth n-1 seldepth m pv <something>" at the end of an iteration and depth n at the beginning of the next iteration, which is just some milliseconds later. So n-1/m might be displayed for some milliseconds (I don't know) but you'lll never see it.

The solution is to send seldepth earlier. I personally send seldepth in the middle of the search, but sending after earch root move together with the pv would still be better than the current situation.