How important is it? In the Leiden tournament we were playing a significantly weaker opponent in an early round and nearly accepted a draw. The draw score was fixed at the time and was set at zero, a sensible value that assumes you don't know if your opponent is weaker or stronger. We watched helplessly as the result was out of our hands and Komodo seemed to be ok with a draw, feeling it's position was just slightly inferior. As things turned out, Komodo's opponent avoided the draw although there were some repetitions, just not 3-fold. A very close call.
As a result of this we immediately added a contempt factor to the program in order to avoid this situation in the future and also in anticipation of having to play Rybka and other strong programs in the same tournament so we could use a negative contempt (or positive draw score.)
But my question is "How important is contempt?" My feeling at the time was that it would be unlikely to make more than a couple of ELO difference. But I think it's more important than I had imagined. In Komodo the contempt factor is applied to repetition and stalemate, but not other draws.
In my study I am simulating a weaker opponent by using fixed nodes levels and handicapping this "weak" player. "weak" is about 270 ELO weaker and plays a gauntlet against "strong" komodo versions using various contempt factors. An opponent who is 270 ELO weaker still represents a dangerous opponent who will score almost 20% against you. I will repeat this study with opponents who are less different and more different to get a sense of how to compute a contempt factor given and ELO difference. I think that can be computed with a straightforward formula which can be used to convert a chess program evaluation score to a winning percentage (or ELO number) and would have to be calibrated for each program.
The match is still in progress, but the early indications are that this is far more important than I had realized previously.
Has anyone else studied this issue in any serious detail? Here is the current ongoing result of my small study where I'm testing "drawscore" of 0, -10, -20 and -30.
Code: Select all
Rank Name Elo + - games score oppo. draws
1 kse-4290.00-30 3030.5 10.8 10.8 6831 85.7% 2733.1 15.9%
2 kse-4290.00-20 3016.5 10.5 10.5 6831 84.8% 2733.1 17.4%
3 kse-4290.00-10 3013.1 10.4 10.4 6832 84.6% 2733.1 18.1%
4 kse-4290.00-0 3000.0 10.2 10.2 6832 83.7% 2733.1 19.8%
5 weak 2733.1 5.2 5.2 27326 15.3% 3015.0 17.8%