CCRL live lists with 100 Elo reduction

[Graham Banks]

The latest CCRL Rating Lists and Statistics are available for viewing from the following links:
http://computerchess.org.uk/ccrl/4040.live/ (40/40)
http://www.computerchess.org.uk/ccrl/404.live/ (40/4)
http://www.computerchess.org.uk/ccrl/404FRC/ (FRC 40/4)

Please note that the three lists are often updated separately to each other.
The links given in each update report will give you the currently up to date lists.

The 100 Elo reduction in the main website lists don't show up yet, but will eventually.

[CRoberson]

I made a quick scan of the ccrl forum and didn't find anything pertaining to the reasoning behind the across the board 100 pt reduction.

So, Why?

[Graham Banks]

I made a quick scan of the ccrl forum and didn't find anything pertaining to the reasoning behind the across the board 100 pt reduction.

So, Why?

Our ratings were excessively higher than other rating lists, as many have pointed out at various times. We made the change to come more into line with other lists.

[Adam Hair]

To anyone who reads this:

What would be your reaction if we purposely disconnected the CCRL from any comparison to human ratings?

What if we make the rating for the top engine equal 0 Elo, so that the ratings are such that the rating of each engine directly indicates how many Elo it is behind the leading program?

[mar]

What if we make the rating for the top engine equal 0 Elo, so that the ratings are such that the rating of each engine directly indicates how many Elo it is behind the leading program?

That's what I proposed to Werner when I read about CEGT renormalization! It really doesn't matter. Elo is relative as we all know.
Just imagine you are an engine author and you want to give the users a rough elo estimate (the first question authors get from users when they release their engine, first public version).
People simply like absolute numbers like 2400, 2500 etc. And they tend to compare that to FIDE ratings etc.
Of course the author could state that his engine is n elo stronger/weaker than reference engine X.
Another thing is that you are trying hard to break a certain elo barrier, say 2700. And suddenly all top rating lists renormalize and drop at least 100 elo.
C'mon, more sounds better Just make us programmers happy to have a stable absolute RL. What if someone decides next month to renormalize again? I used to say nn CEGT elo/nn CCRL elo.
Now I can't do that anymore

[Strelkaman]

Nice info Graham.Thanks

[Dan Honeycutt]

To anyone who reads this:

What would be your reaction if we purposely disconnected the CCRL from any comparison to human ratings?

What if we make the rating for the top engine equal 0 Elo, so that the ratings are such that the rating of each engine directly indicates how many Elo it is behind the leading program?

Yuck would be my reaction.

Anybody with any chess experience has a feel for what 1000 or 2000 or 3000 elo means. You go make the numbers negative you'll leave the simpletons such as myself completely perplexed.

Best
Dan H.

[Mincho Georgiev]

To anyone who reads this:

What would be your reaction if we purposely disconnected the CCRL from any comparison to human ratings?

What if we make the rating for the top engine equal 0 Elo, so that the ratings are such that the rating of each engine directly indicates how many Elo it is behind the leading program?

My idea is probably a funny one, but it's what would bring me some comfort with any rating list. There are enough strong FIDE players here that can estimate one single program. If anyone is agreed to do that, use that program for a base.

[CRoberson]

To anyone who reads this:

What would be your reaction if we purposely disconnected the CCRL from any comparison to human ratings?

What if we make the rating for the top engine equal 0 Elo, so that the ratings are such that the rating of each engine directly indicates how many Elo it is behind the leading program?

My thought is to treat them like what they really are -- ranking lists not rating lists. So, drop reporting the ratings; keep them for calculating rankings.

Your solution is reasonable and it does solve the big problem.

The problem is:
Accurate software ratings are impossible. Why? Because, everybody in the world has different hardware. Each processor of my computer is 6x faster than your base machine (AMD 4600). This means that nearly all programs on your list get a rating boost from the speed up. The other big issue is that boost is dynamic: some programs get more than others due to bugs and so forth.

Ares has played several human GM's (since the last version) online and in person. The best they have done is a draw. Of course,
I use my 6x faster hardware. It is quite clear that one number fits all doesn't work.

I see only two directions to fix it.
1) Make the ratings unrelated to humans. They are not well correlated to humans as is.
2) Adjust the time controls to keep up with the best HW not old HW. If the best HW is 6x faster, make the TC's 6x longer or get better HW or do it like SSDF which reports a rating for a HW and SW combination. That is likely best if you can line the ratings up with humans.

Sounds like CCRL conformed for the sake of conformity. Sounds bad. OTOH, y'all are trying!

[Mincho Georgiev]

Use some imagination. Anyone can use their hardware base with an online client.