StockFish : best move not found due to MultiPV ?

Discussion of chess software programming and technical issues.

Moderators: hgm, Harvey Williamson, bob

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
User avatar
tissatussa
Posts: 24
Joined: Sat Sep 24, 2016 2:13 am
Location: Netherlands
Full name: Roelof Berkepeis
Contact:

StockFish : best move not found due to MultiPV ?

Post by tissatussa » Fri Jul 06, 2018 10:37 pm

See my 3 screenshots : I use SCID on Linux with the newest StockFish 9 (64 bit) to analyse this position :
3q1rk1/prppnp2/1p4p1/2b1P1N1/5P2/7P/PP5K/R1BQR3 w - - 0 1
BTW. this FEN is from the video "Leela Chess Zero ID 458 plays the Göring Gambit vs Stockfish 5", https://www.youtube.com/watch?v=xFLXxIMQBuM
The best move should be b4(!) but it can take a while before SF finds it ..

In this case I encounter some strange SF behaviour : when starting SCID with this FEN and analyse it with SF 9-64 using MultiPV 8, the engine considers the move b4! at depth 28 for the first time, and at depth 30 b4! is SF first choice. But when starting SCID with this FEN and analyse it with SF 9-64 using MultiPV 6, the engine does not consider the move b4 at all, even at depth 30 .. (lower MultiPV are alike).

Why ?
Attachments
MPV_8_depth_28.png
MultiPV 8 depth 28
MPV_8_depth_28.png (199.23 KiB) Viewed 1663 times
MPV_8_depth_30.png
MultiPV 8 depth 30
MPV_8_depth_30.png (209.21 KiB) Viewed 1663 times
MPV_6_depth_30.png
MultiPV 6 depth 30
MPV_6_depth_30.png (199.37 KiB) Viewed 1663 times
-simple is not always best but best is always simple-

syzygy
Posts: 4415
Joined: Tue Feb 28, 2012 10:56 pm

Re: StockFish : best move not found due to MultiPV ?

Post by syzygy » Sat Jul 07, 2018 12:15 am

It is completely normal that a modern chess engine like Stockfish gives different results on different runs, in particular if search parameters are changed between runs or multiple search threads are used.

It is also not strange that increasing the number of PVs shown results in more solutions being found at a given depth (but for any particular run that can be different, see above).

User avatar
tissatussa
Posts: 24
Joined: Sat Sep 24, 2016 2:13 am
Location: Netherlands
Full name: Roelof Berkepeis
Contact:

Re: StockFish : best move not found due to MultiPV ?

Post by tissatussa » Sat Jul 07, 2018 12:24 am

about "different results on different runs" : i know that's true .. i've tested with threats 2 and every time SF gives a different result set of moves. But that's not my problem, i can accept the (slightly) different results .. the question is : why does SF not include b4! in its result set when MultiPV is less then 8 ? (in this case) .. the move b4! is the best move here, but SF only finds it when MultiPV is high enough !?
-simple is not always best but best is always simple-

Dann Corbit
Posts: 9646
Joined: Wed Mar 08, 2006 7:57 pm
Location: Redmond, WA USA
Contact:

Re: StockFish : best move not found due to MultiPV ?

Post by Dann Corbit » Sat Jul 07, 2018 12:35 am

Since it is the best move, SF should also immediately display it with multi-pv set to 1. I guess that it does not until the search gets deep.

Typically, when you have a position masked by null move difficulties or some similar oddity, it does not get explored deeply unless you force the engine to explore it. Setting multi-pv to a high enough number will accomplish that.

Pruning rules might say something along the lines of, "He tosses a full piece when he makes this move, so reduce the search for this move by 7 plies".

Unfortunately, the game winning move is moved beyond the horizon by the pruning decision.

It sounds like a problem, but really it's not. Because making the engine search what are normally bad moves deeply will make the engine play and analyze poorly. And there are also specialized engines and tweaked versions of engines that have modifications for null move situations and tactical situations that will find the right moves. They are much better at tricky positions like this, but at least 1000 Elo weaker for normal, quiet positions.

Hence, when you analyze a position, it is good to get more than one opinion, and you should also use engines from different families and engines with different goals.

There are engines designed to look for mates like Stockfish Matefinder and Chest.
There are engines tuned for tactical oddities like Houdini Tactical and Sting.
There are engines tuned for game play like Shredder, Stockfish, Komodo, Houdini.
If you really want to understand a particular position, it might take a whole raft of engines to find the right plan.
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.

User avatar
tissatussa
Posts: 24
Joined: Sat Sep 24, 2016 2:13 am
Location: Netherlands
Full name: Roelof Berkepeis
Contact:

Re: StockFish : best move not found due to MultiPV ?

Post by tissatussa » Sat Jul 07, 2018 12:55 am

@Dan Corbit: thanks, this answer gives me insight ! I never read such info about engines .. this way my question seems solved ! (but i would like to get some other comments also)
-simple is not always best but best is always simple-

syzygy
Posts: 4415
Joined: Tue Feb 28, 2012 10:56 pm

Re: StockFish : best move not found due to MultiPV ?

Post by syzygy » Sat Jul 07, 2018 10:49 am

tissatussa wrote:
Sat Jul 07, 2018 12:24 am
about "different results on different runs" : i know that's true .. i've tested with threats 2 and every time SF gives a different result set of moves. But that's not my problem, i can accept the (slightly) different results
I said different, not slightly different.
.. the question is : why does SF not include b4! in its result set when MultiPV is less then 8 ? (in this case) .. the move b4! is the best move here, but SF only finds it when MultiPV is high enough !?
Because of how search engines work nowadays. They pay less attention to moves that don't look so good at first. If you force MultiPV=N, that will force the engine to search the Nth best move as if it were the best move. In some cases that results in the engine realising that what it thought was the Nth best move is really the best move.

Sesse
Posts: 171
Joined: Mon Apr 30, 2018 9:51 pm
Contact:

Re: StockFish : best move not found due to MultiPV ?

Post by Sesse » Sat Jul 07, 2018 11:02 am

It's also meaningless to compare the two on only depth—the interesting question is how long (in seconds) it needs before it starts considering b4 in single-PV mode, versus multi-PV.

There will always exist positions where less pruning is good, of course, but on average, the pruning done by modern engines is super-effective, so running in MultiPV mode all the time would mean a large loss in strength.

User avatar
tissatussa
Posts: 24
Joined: Sat Sep 24, 2016 2:13 am
Location: Netherlands
Full name: Roelof Berkepeis
Contact:

Re: StockFish : best move not found due to MultiPV ?

Post by tissatussa » Sat Jul 07, 2018 11:13 am

Thanks for more good info on this subject. I think my FEN position is good for testing : i want the move b4! to be found at last, regarding processing time .. eg. it may take 10 minutes to reach depth 32, but i don't care to wait.

It's also nice to have SF evaluate quick AND good for most FENs, but for me that's another topic, although i appreciate comments about this.
-simple is not always best but best is always simple-

syzygy
Posts: 4415
Joined: Tue Feb 28, 2012 10:56 pm

Re: StockFish : best move not found due to MultiPV ?

Post by syzygy » Sat Jul 07, 2018 4:14 pm

If one is interested in engine strength, then single positions are not good for testing.

BBauer
Posts: 658
Joined: Wed Mar 08, 2006 7:58 pm

Re: StockFish : best move not found due to MultiPV ?

Post by BBauer » Sun Jul 08, 2018 6:54 pm

What makes you think that 1.b4 is the best move?
Why not 1. Bd2?
Kind regards
Bernhard

Post Reply