Tuning for rating lists ?

Discussion of chess software programming and technical issues.

Moderators: hgm, Rebel, chrisw

mcostalba
Posts: 2684
Joined: Sat Jun 14, 2008 9:17 pm

Re: Tuning for rating lists ?

Post by mcostalba »

syzygy wrote: Note that not all engines implement "contempt" by tweaking the draw score. For example, H3 seems to implement it by making the evaluation asymmetric (piece value imbalance, king safety imbalance). See here under the heading "Contempt".
King safety is asymmetrical since ages in SF, it is the famous Aggressiveness and Cowardice parameters. Houdini has probably took the idea from here and renamed in contempt (the renaming makes sense to a certain degree).

Instead the idea to value its own pieces higher than the opponent pieces it is original and could make sense.
IWB
Posts: 1539
Joined: Thu Mar 09, 2006 2:02 pm

Re: Tuning for rating lists ?

Post by IWB »

mcostalba wrote: ...If the opponent misses the best reply many times _and_ your eval goes up along the game then you can increase contempt.....just an idea.
If your eval is going up already to increase the contempt is useless as you are already winning. Contempt is helpfull in drawish or slighty worse positions to avoid a draw versus weaker opponents ...

Bye
Ingo
User avatar
Graham Banks
Posts: 41511
Joined: Sun Feb 26, 2006 10:52 am
Location: Auckland, NZ

Re: Tuning for rating lists ?

Post by Graham Banks »

pohl4711 wrote:The first problem are the ratinglists with too much too weak opponents - the strongest engines will always have distorted results in those ratinglists.
That's a pretty bold statement to make.

It would depend a lot of course on the range of opponents that a particular engine has to play. I doubt that there are many ridiculous mismatches.

I use a rough spread of about 200 ELO points for engines outside the top twelve or so.
The top twelve or so would all play many games against each other, as can be seen in the table below:

Image
gbanksnz at gmail.com
User avatar
Graham Banks
Posts: 41511
Joined: Sun Feb 26, 2006 10:52 am
Location: Auckland, NZ

Re: Tuning for rating lists ?

Post by Graham Banks »

mcostalba wrote:There is this magical parameter, called "contempt" that has this interesting property, if enabled lets your engine to become strong with the weakest and weak with the strongest yes, is not a very ethical one :-) Apart from the technical merit of this: I have serious doubts that just tweaking the draw score is enough to enable a more aggressive and risky style of play, but this is another topic.

Here the topic is, assuming that this contempt does the trick, it happens that in the rating lists with many engines weaker than yours, so when your engine is in the top half of the list, this contempt can, more or less, artificially push you up. The side effect is that your engine becomes weaker with the strongest, so if you, for instance, want to participate to a tournament with elimination rounds until the final, this contempt factor perhaps it is wise to disable.

Personally I'd prefer to be strong with the strongest and...merciful :-) with the weakest, IOW I prefer the engine does well in tournaments and in one-to-one direct matches even if this means to give up some points in the rating lists.
I've always been a fan of contempt=0.
The cream should always rise to the top regardless.
gbanksnz at gmail.com
User avatar
hgm
Posts: 27837
Joined: Fri Mar 10, 2006 10:06 am
Location: Amsterdam
Full name: H G Muller

Re: Tuning for rating lists ?

Post by hgm »

Contempt=0 is quite dangerous. Micro-Max has scored draws because of that against opponents that were at least 500 Elo stronger. Problem is that early in the game white always has a slight advantage. And if black then sees an opportunity to repeat, he will take it. So what typically happened is that micro-Max developed its Bishop, the opponent attacked it with its developing Knight, and micro-Max, considering Bishops more valuable than Knights, evaded the attack by moving its Bishop back to where it came from. The opponent then would think "He moves back? Then I move back!" For an easy draw in a slightly inferior position.

Without contempt, you will be vulnerable for such tricks.
User avatar
Don
Posts: 5106
Joined: Tue Apr 29, 2008 4:27 pm

Re: Tuning for rating lists ?

Post by Don »

hgm wrote:Contempt=0 is quite dangerous. Micro-Max has scored draws because of that against opponents that were at least 500 Elo stronger. Problem is that early in the game white always has a slight advantage. And if black then sees an opportunity to repeat, he will take it. So what typically happened is that micro-Max developed its Bishop, the opponent attacked it with its developing Knight, and micro-Max, considering Bishops more valuable than Knights, evaded the attack by moving its Bishop back to where it came from. The opponent then would think "He moves back? Then I move back!" For an easy draw in a slightly inferior position.

Without contempt, you will be vulnerable for such tricks.
Yes, this is very clear. Especially with the black pieces you need contempt to deal with the book which usually stop with black in a slightly inferior position.
Capital punishment would be more effective as a preventive measure if it were administered prior to the crime.
User avatar
sje
Posts: 4675
Joined: Mon Mar 13, 2006 7:43 pm

Re: Tuning for rating lists ?

Post by sje »

Don wrote:
hgm wrote:Contempt=0 is quite dangerous. Micro-Max has scored draws because of that against opponents that were at least 500 Elo stronger. Problem is that early in the game white always has a slight advantage. And if black then sees an opportunity to repeat, he will take it. So what typically happened is that micro-Max developed its Bishop, the opponent attacked it with its developing Knight, and micro-Max, considering Bishops more valuable than Knights, evaded the attack by moving its Bishop back to where it came from. The opponent then would think "He moves back? Then I move back!" For an easy draw in a slightly inferior position.

Without contempt, you will be vulnerable for such tricks.
Yes, this is very clear. Especially with the black pieces you need contempt to deal with the book which usually stop with black in a slightly inferior position.
I am reminded of the Chess 4.x paper of nearly forty years ago which mentions that the program uses a contempt factor to avoid playing for a draw when the book dumps it into a gambit position where the program is down a pawn.

Yet somehow this seems awkward and there needs to be a better way of doing things.
User avatar
Don
Posts: 5106
Joined: Tue Apr 29, 2008 4:27 pm

Re: Tuning for rating lists ?

Post by Don »

sje wrote:
Don wrote:
hgm wrote:Contempt=0 is quite dangerous. Micro-Max has scored draws because of that against opponents that were at least 500 Elo stronger. Problem is that early in the game white always has a slight advantage. And if black then sees an opportunity to repeat, he will take it. So what typically happened is that micro-Max developed its Bishop, the opponent attacked it with its developing Knight, and micro-Max, considering Bishops more valuable than Knights, evaded the attack by moving its Bishop back to where it came from. The opponent then would think "He moves back? Then I move back!" For an easy draw in a slightly inferior position.

Without contempt, you will be vulnerable for such tricks.
Yes, this is very clear. Especially with the black pieces you need contempt to deal with the book which usually stop with black in a slightly inferior position.
I am reminded of the Chess 4.x paper of nearly forty years ago which mentions that the program uses a contempt factor to avoid playing for a draw when the book dumps it into a gambit position where the program is down a pawn.

Yet somehow this seems awkward and there needs to be a better way of doing things.
Agreed. The only reasonable solution to me is that the program is informed of the ELO difference and can do whatever it wants with the information with a provision for having an unknown quantity. This would be used in competitions where the players are known (the tournament would provide the rating estimates in the crosstables) and it could be used for Ipon, CCRL and others where the most up-to-date difference is supplied.

UCI as well as XBOARD already have a provision for reporting this to the programs.

I think we figured out once that contempt of 5 or 6 for every 100 ELO works pretty well.
Capital punishment would be more effective as a preventive measure if it were administered prior to the crime.
User avatar
sje
Posts: 4675
Joined: Mon Mar 13, 2006 7:43 pm

Re: Tuning for rating lists ?

Post by sje »

Don wrote:UCI as well as XBOARD already have a provision for reporting this to the programs.
Other than supplying a vault to a PGN tag, Symbolic ignores rating data from an ICS.

"Play the board, not the man (or machine)."

Does this cost points? Maybe at times, but I don't care. Sometimes it may gain points. I suspect that over time, things are even.
User avatar
Don
Posts: 5106
Joined: Tue Apr 29, 2008 4:27 pm

Re: Tuning for rating lists ?

Post by Don »

sje wrote:
Don wrote:UCI as well as XBOARD already have a provision for reporting this to the programs.
Other than supplying a vault to a PGN tag, Symbolic ignores rating data from an ICS.

"Play the board, not the man (or machine)."

Does this cost points? Maybe at times, but I don't care. Sometimes it may gain points. I suspect that over time, things are even.
It loses ELO whenever your opponent is a lot stronger or a lot weaker if it's not set appropriately. How much? Probably not that much, but I'll be it is as much as 5 or 10 ELO if you consistently play up or down by a lot - but most of the time that is not the case so it's probably not a big deal.
Capital punishment would be more effective as a preventive measure if it were administered prior to the crime.