New Stockfish contempt

Discussion of anything and everything relating to chess playing software and machines.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Jouni
Posts: 2010
Joined: Wed Mar 08, 2006 7:15 pm

New Stockfish contempt

Post by Jouni » Mon Jan 29, 2018 12:57 pm

It's stunning, that You get +30 ELO with about 10 lines of code! But something must be changed in analysis mode. After 1.e4 SF shows negative evalution e.g. black is better:

Analysis by Stockfish 230118 64 BMI2:

1...e6 2.d4 d5 3.Nd2 c5 4.exd5 exd5 5.Ngf3 Nf6 6.Bb5+ Bd7 7.Bxd7+ Nbxd7 8.0-0 Be7 9.dxc5 Nxc5 10.Nb3 Nce4 11.Nfd4 0-0 12.Nf5 Re8 13.Nxe7+ Rxe7 14.Be3 b6 15.a4 Qc7 16.a5 Rae8 17.axb6 axb6 18.Re1 Nd6 19.Nd4 Nc4
= (-0.11) Depth: 29/43 00:00:44 273mN

This isn't correct behaviour.
Jouni

zullil
Posts: 5670
Joined: Mon Jan 08, 2007 11:31 pm
Location: PA USA
Full name: Louis Zulli

Re: New Stockfish contempt

Post by zullil » Mon Jan 29, 2018 1:17 pm

Jouni wrote:It's stunning, that You get +30 ELO with about 10 lines of code! But something must be changed in analysis mode. After 1.e4 SF shows negative evalution e.g. black is better:

Analysis by Stockfish 230118 64 BMI2:

1...e6 2.d4 d5 3.Nd2 c5 4.exd5 exd5 5.Ngf3 Nf6 6.Bb5+ Bd7 7.Bxd7+ Nbxd7 8.0-0 Be7 9.dxc5 Nxc5 10.Nb3 Nce4 11.Nfd4 0-0 12.Nf5 Re8 13.Nxe7+ Rxe7 14.Be3 b6 15.a4 Qc7 16.a5 Rae8 17.axb6 axb6 18.Re1 Nd6 19.Nd4 Nc4
= (-0.11) Depth: 29/43 00:00:44 273mN

This isn't correct behaviour.
Switch to Cfish: https://github.com/syzygy1/Cfish/commit ... 6f6e0378a9

User avatar
Nordlandia
Posts: 2456
Joined: Fri Sep 25, 2015 7:38 pm
Location: Sortland, Norway

Re: New Stockfish contempt

Post by Nordlandia » Mon Jan 29, 2018 2:52 pm

Set contempt=0 for analysis, case solved :wink:

MikeGL
Posts: 901
Joined: Thu Sep 01, 2011 12:49 pm

Re: New Stockfish contempt

Post by MikeGL » Mon Jan 29, 2018 4:02 pm

Jouni wrote:It's stunning, that You get +30 ELO with about 10 lines of code! But something must be changed in analysis mode. After 1.e4 SF shows negative evalution e.g. black is better:

Analysis by Stockfish 230118 64 BMI2:

1...e6 2.d4 d5 3.Nd2 c5 4.exd5 exd5 5.Ngf3 Nf6 6.Bb5+ Bd7 7.Bxd7+ Nbxd7 8.0-0 Be7 9.dxc5 Nxc5 10.Nb3 Nce4 11.Nfd4 0-0 12.Nf5 Re8 13.Nxe7+ Rxe7 14.Be3 b6 15.a4 Qc7 16.a5 Rae8 17.axb6 axb6 18.Re1 Nd6 19.Nd4 Nc4
= (-0.11) Depth: 29/43 00:00:44 273mN

This isn't correct behaviour.
I knew it! French gives black a slight advantage.

But in a serious note. I guess -0.1 still means equal, but assumes black already equalized his 1 move disadvantage.
I told my wife that a husband is like a fine wine; he gets better with age. The next day, she locked me in the cellar.

jhellis3
Posts: 400
Joined: Fri Aug 16, 2013 10:36 pm

Re: New Stockfish contempt

Post by jhellis3 » Mon Jan 29, 2018 4:17 pm

This isn't correct behaviour.
Yes, it is. Contempt of 20 is being applied to black meaning that Black will play "inferior" moves up to that point. However, to accomplish this the scores are shifted by that amount. So you have -0.11 + 0.20 = 0.09 as the actual eval. But if SF scored it this way, then Black would be happy with a 3-fold, which the whole idea of contempt is to avoid. *Note if contempt were only applied from White's perspective, the eval would be shifted to 0.29 making a 3-fold even more desirable for Black.

User avatar
Eelco de Groot
Posts: 4162
Joined: Sun Mar 12, 2006 1:40 am
Location: Groningen

Re: New Stockfish contempt

Post by Eelco de Groot » Mon Jan 29, 2018 6:33 pm

jhellis3 wrote:
This isn't correct behaviour.
Yes, it is. Contempt of 20 is being applied to black meaning that Black will play "inferior" moves up to that point. However, to accomplish this the scores are shifted by that amount. So you have -0.11 + 0.20 = 0.09 as the actual eval. But if SF scored it this way, then Black would be happy with a 3-fold, which the whole idea of contempt is to avoid. *Note if contempt were only applied from White's perspective, the eval would be shifted to 0.29 making a 3-fold even more desirable for Black.
I think that seems to be accurate but to some this is an offending bug in Stockfish. It may however only need some getting used to :P

In your line of reasoning, the contempt now (here, after 1. e4) helps Black find better moves ('push a little harder' you could say) The scores are not really relevant, only the moves matter. If you now go back to the start position, (having analyzed position after 1. e4), contempt is now for White and White will be pushing a little harder. The score may go up but important is only that Black's moves do not get lost. The back and forth process should smooth things out. Only the moves matter. This is why you do the back and forth in the first place: to find better moves. Now it might be that you can speed this up a little bit, by both sides having contempt on (this would need a code change), but if this were so, then a Stockfish with both sides contempt would also play better against a Stockfish with no contempt.

Although I have not seen anyone actually try this, to me it is a bit implausible. So, in my opinion, it may well be that this (present situation) is actually better than analyzing without contempt even positions scoring around zero. For winning positions I think there is little doubt you don't need to switch contempt off or have double contempt. But this is just a preliminary standpoint. The case becomes stronger because of the really unexpected increase in Elo
Debugging is twice as hard as writing the code in the first
place. Therefore, if you write the code as cleverly as possible, you
are, by definition, not smart enough to debug it.
-- Brian W. Kernighan

jhellis3
Posts: 400
Joined: Fri Aug 16, 2013 10:36 pm

Re: New Stockfish contempt

Post by jhellis3 » Mon Jan 29, 2018 7:36 pm

Ideally, we would de-contempt the score before outputting to UCI. This would allow SF to use contempt but still display the objective score of the position.

That would solve any scoring inconsistency issues with automated analysis (though it seems rather a mountain out of molehill to me).

Separately, we should probably do something like this to resolve any internal TT value issues:

https://github.com/snicolet/Stockfish/c ... ea838b7e95
Last edited by jhellis3 on Mon Jan 29, 2018 7:48 pm, edited 1 time in total.

User avatar
Ovyron
Posts: 2746
Joined: Tue Jul 03, 2007 2:30 am

Re: New Stockfish contempt

Post by Ovyron » Mon Jan 29, 2018 7:45 pm

Nordlandia wrote:Set contempt=0 for analysis, case solved :wink:
No! Don't turn it off!

What Eelco de Groot says is true: what matters is what moves are chosen by the engine, not their scores, and Stockfish outputs better moves with higher Contempt, I have used this since the new Contempt was implemented (note, this Contempt isn't new, it has been out for a while, what is new is that it's the default), and the higher Contempt stands in correspondence time controls.

Is 20 too high? No, actually, you can actually go higher as long as you interact and compare the analysis with other engines. Actually.

This is also true for Komodo and Houdini.

Houdini has a very interesting implementation of Contempt, it seems to detect what side is winning and try to apply Contempt to this side, so you can see really huge swings of evaluations as Houdini goes from liking one side, to liking the other in the next ply.

With Contempt you don't want to analyze both sides of the board, though. If you have analyzed from the white side, what you need to do to have a consistent analysis from the black side is to reverse contempt (say, switch to -20 when you switch sides.) But then you might miss comebacks from this side.

So what you really want to do is have two different Stockfishes loaded in different GUIs, one only analyzes the white side and provides optimistic scores for this, the other only analyzes the black side and does as such for black.

This has been like this for Komodo for a while, and you actually have to create different personalities for Komodo's white side and for Komodo's black side in your GUI.

I'm not sure a checkbox that would apply the 0.20 Centipawns as score shifts so the engine would show consistent scores for white and black would work. First, because, really, white and black should be using differen trees:

White assumes black will play defensively and try to draw, black assumes white will play defensively and try to draw, this produces better moves for both sides. Second, even with a Contempt of 40, Stockfish is showing scores of 0.00 for drawn positions, if all the ideas are refuted by the opponent, this would translate to a score of -0.40 for those equal positions, or a change to a worse move if there's some really -0.39 move that the engine would prefer, wrongly.

Users just need to learn about this concept of using different instances of Stockfish depending on the side of the board they analyze, and to propagate different scores for white and black's viewpoints.

mjlef
Posts: 1427
Joined: Thu Mar 30, 2006 12:08 pm
Contact:

Re: New Stockfish contempt

Post by mjlef » Mon Jan 29, 2018 10:52 pm

It is great that Stockfish has a better form of Contempt now. But switching sides without reversing the Contempt sign is going to fill the hash with contradictory information if it applies contempt to the side to move. Komodo turns off Contempt in analysis mode, unless the "White Contempt" is on. Then it uses Contempt in regular play and analysis, but applies the Contempt number .com white's point of view, switching it as needed for black. This lets you use it in analysis mode without switching analysis side messing up the hash.

syzygy
Posts: 4455
Joined: Tue Feb 28, 2012 10:56 pm

Re: New Stockfish contempt

Post by syzygy » Tue Jan 30, 2018 12:24 am

mjlef wrote:It is great that Stockfish has a better form of Contempt now. But switching sides without reversing the Contempt sign is going to fill the hash with contradictory information if it applies contempt to the side to move. Komodo turns off Contempt in analysis mode, unless the "White Contempt" is on. Then it uses Contempt in regular play and analysis, but applies the Contempt number .com white's point of view, switching it as needed for black. This lets you use it in analysis mode without switching analysis side messing up the hash.
And the hash getting messed up is not the only problem. Contempt switching sign during analysis whenever a move is made on the board doesn't seem very useful to me, either.

Unfortunately many are convinced that this is the best way "to explore active possibilities for both sides" (even though current contempt basically just tries to keep pieces on the board, if need be by closing the position), so it's probably not going to change.

Post Reply