CCRL update (23rd June 2007)

[Graham Banks]

The June 23rd update of the CCRL Rating Lists and Statistics is now available for viewing at:
http://www.computerchess.org.uk/ccrl/4040/

We now have a "thanks" page. Please note that a few corrections still need to be made and we'd appreciate any errors or omissions being pointed out.
Another new page we've added gives website links.

The links to the various rating lists can be found just beneath the default Best Versions list.
For example there is a 32-bit Single CPU list.

Our standard testing is at 40 moves in 40 minutes repeating while our current blitz testing is at both 40 moves in 4 minutes repeating and 40 moves in 12 minutes repeating, all adjusted to the AMD64 X2 4600+ (2.4GHz).

Currently active testers in our team are:
Graham Banks, Ray Banks, Shaun Brewer, Kirill Kryukov, Dom Leste, Tom Logan, Andreas Schwartmann, Charles Smith, George Speight, Chris Taylor, Chuck Wilson, Gabor Szots and Martin Thoresen.


40/40 Notes

620 games were added to our 40/40 database this week, making a total of 62,689 games.
A big thanks to all testers as usual.

Many engines on our list have few games and in many cases their ratings are likely to fluctuate (markedly for some) until a lot more games are played. Therefore no conclusions should be drawn about their strength yet.
To illustrate this point, when an engine has 200 games played, the error margin is still approximately +/-40 ELO, after 500 games +-25 ELO, after 1000 games +-17 ELO and even after 2000 games there is a +-13 ELO error margin!
This of course highlights the importance of looking at other rating lists that are also available in order to draw comparisons and get a more accurate overall picture.


Multi CPU Engines
Testing of Rybka 2.3.2 64-bit 4CPU has commenced, but with only 138 games so far, little should be read into its current rating.
This can be seen by the fact that Rybka 2.3.2 64-bit 2CPU is currently outperforming it after a similar number of games.
The same situation also exists with the Loop M1-T 64-bit 4CPU and 2CPU versions which we are also in the process of testing.
So the current pecking order amongst 4CPU engines remains Rybka, Zap!Chess Zanzibar, Hiarcs, Naum, Loop, Shredder, Fritz, Junior.


Single CPU Engines
Most input this week has been on Loop 13.6 32-bit gauntlets.
A Fritz 10 gauntlet was completed, so both these engines now have a good number of games.
Current objectives centre on getting more games for Deep Sjeng 2.5 and Strelka 1.0b.
Rybka 2.3.2 32-bit testing is also underway, but few games have been included this week.
The current pecking order amongst single cpu engines is Rybka, Hiarcs/Zap!Chess , Strelka, Fritz, Loop, Shredder.


Amateur News:
Strelka 1.0b continues to impress and is currently second best free engine behind Rybka 1.0. However, this is only based on 167 games, so could change.
Toga II 1.2.1a, Spike 1.2 Turin and Naum 2.1 have been thoroughly tested and remain ahead of the commercial engines Junior 10, Deep Sjeng 2.5, Ktulu 8.0, SmarThink 1.00 and Chess Tiger 2007.
Glaurung 2 epsilon/2 is a nice improvement over previous versions, but needs further games to stabilise its rating.
Other strong amateurs to watch the progress of are Alaric 704, Petir 4.39 and Booot 4.13.1.
We test a very extensive range of amateur engines through our Amateur Championship divisions (32-bit 1CPU) plus other tournaments, all of which can be followed in our public forum.
Our aim is of course to ensure that all engines lower on our lists get at least 200 games.


Blitz Notes

The 40/4 is updated separately to 40/40 with the latest update able to be viewed here:
http://computerchess.org.uk/ccrl/404.live/
There are now 142,000 games in our 40/4 database.


Multi-CPU Engines (both 4CPU and 2CPU)
3102 - Rybka 2.3.2 64-bit 2CPU
3019 - Zap!Chess Zanzibar 64-bit 4CPU
2991 - Hiarcs 11.1 4CPU
2968 - Loop M1-T 64-bit 4CPU
2945 - Deep Shredder 10 64-bit 4CPU
2944 - Deep Fritz 10 4CPU
2941 - Naum 2.1 64-bit 4CPU
2916 - Deep Junior 10 4CPU
2859 - Glaurung 1.2.1 32-bit 4CPU (too few games for Glaurung 2 epsilon/4)
2768 - Deep Sjeng 2.5 2CPU
2754 - Pharaon 3.5.1 2CPU
2732 - Deep Frenzee 3.0 64-bit 2CPU
2723 - Scorpio 1.81 2CPU (Scorpio 1.91 untested)
2713 - Crafty 20.14 2CPU (Crafty 21.5 CPU untested)


Single CPU Engines
3025 - Rybka 2.2 64-bit (Rybka 2.3.2 testing just started)
2914 - Hiarcs 11.1
2909 - Loop 10.32f (too few games for Loop 13.6)
2883 - Fritz 10
2871 - Toga II 1.3x4
2855 - Shredder 10
2850 - Fruit 2.2.1
2850 - Naum 2.1 64-bit
2844 - Zap!Chess Zanzibar 32-bit (64-bit untested)
2841 - Junior 10
2837 - Strelka 1.0b
2834 - Junior 10.1
2827 - Spike 1.2 Turin
2822 - Ktulu 8.0
2820 - Chess Tiger 2007.1
2776 - Deep Sjeng 2.5 1CPU
2761 - Glaurung 2 epsilon/2 32-bit (64-bit untested)
2759 - CM10th Paralyse
2757 - Bright 0.2a
2951 - Scorpio 1.9
2750 - Scorpio 1.91
2733 - SmarThink 1.00 32-bit (64-bit untested)
2720 - Pro Deo 1.2
2718 - CM10th Default
2717 - Alaric 704
2717 - Delfi 5.1
2711 - Slow Chess Blitz WV2.1
2709 - Frenzee 3.0 64-bit
2702 - Gandalf 6
2700 - Pharaon 3.5.1
2699 - Chiron 0.8.7
2695 - WildCat 7
2690 - Movei 0.08.423
2682 - SOS 5.1
2676- Pseudo 0.7c
2672 - Ruffian 1.0.5 (Ruffian 2.1.0 untested)
2668 - Petir 4.39
2666 - Aristarch 4.50


FRC Notes

Ray tests only those engines that can play FRC through the Shredder Classic GUI.
For FRC the best list to look at is the pure list.


Stats/Presentation Notes

The LOS stats to the right hand side of each rating list are "likelihood of superiority" stats. They tell you the likelihood in percentage terms of each engine being superior to the engine directly below them.

All games are available for download through the link given at the top of this post. They can be downloaded by engine or by month.
ELO ratings are now saved in all game databases for those engines that have 150 games or more.

There is now a link at the bottom of the index page showing the engines that we have removed from our lists.

Custom list selections now have the option of including or excluding betas, private engines, settings and others.

A list of games played this week per engine can be found in the update thread in the CCRL public forum, accessible through the link given at the top of this post.

[eriq]

found a typo I think checkout: 2951 - Scorpio 1.9
or it's one tough engine

[Graham Banks]

found a typo I think checkout: 2951 - Scorpio 1.9
or it's one tough engine

It's a typo on my part. Thanks for pointing it out Eriq.

[Tony Thomas]

Toga II 1.2.1a, Spike 1.2 Turin and Naum 2.1 have been thoroughly tested and remain ahead of the commercial engines

Naum 2.1 is commercial.

[Isaksen]

The same could be said about Glaurung. It too is non commercial engine, for now. Don't know if Tord plans releasing it as a commercial one at some time.

[Isaksen]

It certainly is much stronger than most commercial engines.

[Spock]

The same could be said about Glaurung. It too is non commercial engine, for now. Don't know if Tord plans releasing it as a commercial one at some time.

I think Tord has said that it will never be commercial

[Uri Blass]

The same could be said about Glaurung. It too is non commercial engine, for now. Don't know if Tord plans releasing it as a commercial one at some time.

I think Tord has said that it will never be commercial

.
Tord does not plan to make money from glaurung but he did not say that it will never be commercial.

It is the opposite.

Tord allow you to sell Glaurung and even to modify the source if you
give the buyers the source.

Do not forget that not everybody is connected to the internet so not everybody can download glaurung.

people who sell chess program can certainly make glaurung including glaurung's source as part of the package in case that customers like it.

Uri

[Tord Romstad]

I need to write a Glaurung FAQ some day.

Uri is essentially right: I have no intention of ever selling the Glaurung engine. This does not necessarily mean that no version of Glaurung will never be available in commercial form. For instance, I cannot exclude the possibility that some company will some day release some a mass-market chess program (something similar to Chessmaster, perhaps) with Glaurung as the chess engine. It wouldn't be the first time a GPL chess engine is used like this. For instance, the commercial Big Bang Board Games contains GNUChess as its chess engine. I am sure there are other examples.

For purposes like the one described above, it would be possible to use Glaurung even without my explicit permission. But I would also be happy to be actively involved in the project, and to be paid for my work. If some company wanted to use Glaurung as the chess engine in their commercial product, and offered to pay me for continuing to improve my engine and implement the features they wanted, I would seriously consider their offer. But it would be an absolute requirement for me that I would still be allowed to give away the engine itself and its source code for free. I doubt that this requirement would be a problem for a program made for the mass market. Most of the customers wouldn't have any idea what to do with an UCI engine in any case. I am sure that Chessmaster, for instance, would still sell almost exactly the same amount of copies if The King was available as a free, open source UCI engine.

I would also seriously consider the offer if some company offered to pay me for adapting Glaurung to some closed-platform device like a mobile phone or game console. If Apple wrote to me and wanted me to write a chess program for the iPhone, I would probably accept in an instance.

In other words, it is very unlikely, but not entirely impossible that there will some day be a commercial Glaurung (or a commercial product which includes Glaurung as one of its components). But the engine itself and its source code will remain free. I won't accept any commercial offers which force me to close my source code.

Tord

[Graham Banks]

Toga II 1.2.1a, Spike 1.2 Turin and Naum 2.1 have been thoroughly tested and remain ahead of the commercial engines

Naum 2.1 is commercial.

Thanks Tony,

boy - a lot of errors on my part this time round.

I should concentrate better on a Saturday morning!

Regards, Graham.