Stockfish no longer has contempt

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Dann Corbit, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Madeleine Birchfield
Posts: 410
Joined: Tue Sep 29, 2020 2:29 pm
Location: Dublin, Ireland
Full name: Madeleine Birchfield

Re: Stockfish no longer has contempt

Post by Madeleine Birchfield » Sat Jun 26, 2021 10:49 pm

syzygy wrote:
Sat Jun 26, 2021 6:05 pm
Madeleine Birchfield wrote:
Tue Jun 22, 2021 8:16 pm
Never understood why they had contempt in the first place, as it interfered with the accuracy of stockfish's evaluation by artificially inflating it by 24 centipawns, especially when it comes in a gui that does not allow the user to change uci options, such as the Lichess Stockfish interface.
Stockfish does not include a GUI, so it is not clear what you are talking about.

If the Lichess developers didn't want contempt, they could have trivially set it to 0 themselves.
https://en.wikipedia.org/wiki/Default_effect
I'm done with this place.

syzygy
Posts: 5048
Joined: Tue Feb 28, 2012 10:56 pm

Re: Stockfish no longer has contempt

Post by syzygy » Sat Jun 26, 2021 11:53 pm

Madeleine Birchfield wrote:
Sat Jun 26, 2021 10:49 pm
syzygy wrote:
Sat Jun 26, 2021 6:05 pm
Madeleine Birchfield wrote:
Tue Jun 22, 2021 8:16 pm
Never understood why they had contempt in the first place, as it interfered with the accuracy of stockfish's evaluation by artificially inflating it by 24 centipawns, especially when it comes in a gui that does not allow the user to change uci options, such as the Lichess Stockfish interface.
Stockfish does not include a GUI, so it is not clear what you are talking about.

If the Lichess developers didn't want contempt, they could have trivially set it to 0 themselves.
https://en.wikipedia.org/wiki/Default_effect
I gave you a full explanation, but you are free not to want to understand.

Michel
Posts: 2248
Joined: Sun Sep 28, 2008 11:50 pm

Re: Stockfish no longer has contempt

Post by Michel » Tue Jun 29, 2021 12:20 pm

syzygy wrote:
Sat Jun 26, 2021 6:05 pm


However, SF's approach to contempt is incompatible with NNUE.
I don't understand why it would be intrinsically incompatible with NNUE. Both HCE and NNUE are evaluation functions. I don't see a reason for them to behave differently with regard to contempt.

Note that I am talking about objective contempt (against weaker engines). Not self play contempt. The fact that that appeared to work with HCE in SF was likely just an accident.
Ideas=science. Simplification=engineering.
Without ideas there is nothing to simplify.

carldaman
Posts: 2129
Joined: Sat Jun 02, 2012 12:13 am

Re: Stockfish no longer has contempt

Post by carldaman » Wed Jun 30, 2021 3:46 am

Doesn't Komodo Dragon, which is now NNUE based, still have contempt working?
If so, they can't be incompatible.

cc2150dx
Posts: 233
Joined: Sat Nov 30, 2013 8:51 am
Location: Canada
Full name: Jason Coombs

Re: Stockfish no longer has contempt

Post by cc2150dx » Wed Jun 30, 2021 5:28 am

yes, Dragons' contempt is working with NNUE.

syzygy
Posts: 5048
Joined: Tue Feb 28, 2012 10:56 pm

Re: Stockfish no longer has contempt

Post by syzygy » Wed Jun 30, 2021 6:09 pm

Michel wrote:
Tue Jun 29, 2021 12:20 pm
syzygy wrote:
Sat Jun 26, 2021 6:05 pm


However, SF's approach to contempt is incompatible with NNUE.
I don't understand why it would be intrinsically incompatible with NNUE. Both HCE and NNUE are evaluation functions. I don't see a reason for them to behave differently with regard to contempt.
SF's eval calculates an evaluation score vector with 2 components (middle game, endgame) and then outputs a weighted average of the two components based on game phase. Contempt skews the middle game component twice as much as the endgame component, which reflects that the advantage of being SF is greater the more pieces there are still on the board. So a 3-rep draw is 0.00 but a completely balanced position with lots of heavy pieces is +something for SF. A completely balanced position with only little material left is close to 0.00, reflecting that SF"s strength advantage is worthless in case of a dead draw.

NNUE does not produce a middle game and an endgame component. So you can't add SF's contempt to it. You can of course still do the old type of content which just changes the draw value in the search, but that has never worked well.

One could think of using two NNs, one for middle game, one for end game, but it would add huge overhead and you can't really train such a thing. The middle game NN would not applied just to middle game positions but it would have to evaluate the position as if it is a middle game position. This hardly makes any sense to begin with, so good luck training it. The reason it can be done for HCE is that it is meaningful to consider the value of an HCE evaluation feature in middle game position and the value of the same feature in endgame positions. With an NN you don't have such isolated features that can be tuned individually.

Perhaps there is a way to add contempt to NNUE, but it will need a new approach.

I guess it should be possible to implement contempt by adding an input feature that indicates which side is being played by SF.

syzygy
Posts: 5048
Joined: Tue Feb 28, 2012 10:56 pm

Re: Stockfish no longer has contempt

Post by syzygy » Wed Jun 30, 2021 6:24 pm

carldaman wrote:
Wed Jun 30, 2021 3:46 am
Doesn't Komodo Dragon, which is now NNUE based, still have contempt working?
If so, they can't be incompatible.
I tend to be precise in my wording: "SF's approach", which was an approach that actually worked.

syzygy
Posts: 5048
Joined: Tue Feb 28, 2012 10:56 pm

Re: Stockfish no longer has contempt

Post by syzygy » Wed Jun 30, 2021 6:44 pm

syzygy wrote:
Wed Jun 30, 2021 6:09 pm
I guess it should be possible to implement contempt by adding an input feature that indicates which side is being played by SF.
But then training data has to come from games of SF vs a weaker engine, and there would be just a single contempt setting.

To get a form of adjustable contempt one would have to add input features encoding the relative strength of the opponent engine. Trying to train it would be terribly messy.

syzygy
Posts: 5048
Joined: Tue Feb 28, 2012 10:56 pm

Re: Stockfish no longer has contempt

Post by syzygy » Thu Jul 01, 2021 12:06 am

syzygy wrote:
Wed Jun 30, 2021 6:09 pm
NNUE does not produce a middle game and an endgame component. So you can't add SF's contempt to it. You can of course still do the old type of content which just changes the draw value in the search, but that has never worked well.
Hmm, perhaps this is not a problem at all. Adding the 2-component HCE and contempt scores and then calculating a weighted average (which is what SF does) gives the same result (up to rounding) as first averaging the two and then adding the weighted averages.

With NNUE there is no averaging of the evaluation scores, but there is probably nothing stopping you from adding a "weighted" contempt score (i.e. a contempt score that tapers off as material disappears from the board).

So it seems I was indeed wrong about the fundamental incompatibility! I just focused too much on SF's very particular (and elegant) implementation, which only requires a single addition and then piggybacks on SF's tapered eval.

I suppose someone tried this already but could not make it work, but I have not followed it too closely.
(Or I am once more overlooking something.)

syzygy
Posts: 5048
Joined: Tue Feb 28, 2012 10:56 pm

Re: Stockfish no longer has contempt

Post by syzygy » Thu Jul 01, 2021 12:26 am

syzygy wrote:
Thu Jul 01, 2021 12:06 am
I suppose someone tried this already but could not make it work, but I have not followed it too closely.
For example:
https://tests.stockfishchess.org/tests/ ... bade2b79d8
It seems there was no clear enough effect.

Post Reply