The Ultimate Computer Chess Challenge !?!?

Discussion of anything and everything relating to chess playing software and machines.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Jouni
Posts: 1979
Joined: Wed Mar 08, 2006 7:15 pm

The Ultimate Computer Chess Challenge !?!?

Post by Jouni » Mon Apr 23, 2007 7:32 pm


User avatar
Mike S.
Posts: 1480
Joined: Thu Mar 09, 2006 4:33 am

Re: The Ultimate Computer Chess Challenge !?!?

Post by Mike S. » Mon Apr 23, 2007 7:51 pm

Although it's supported by FIDE, ICGA, Levy and ChessBase: It's just an exhibition match of two engines, which aren't Top-5 anymore. I think we should not pay too much attention. The various rating lists can provide comparisons for much more than just six games.

It is a shame that the loser wil receive 40,000 $ while stronger engines like Rybka, Hiarcs, Loop, Shredder etc. cannot participate.

There are ICGA members on this message board. What do YOU think? If it's not ok, what are you going to do?

Nothing...
Regards, Mike

Uri Blass
Posts: 8557
Joined: Wed Mar 08, 2006 11:37 pm
Location: Tel-Aviv Israel

Re: The Ultimate Computer Chess Challenge !?!?

Post by Uri Blass » Mon Apr 23, 2007 8:12 pm

Mike S. wrote:Although it's supported by FIDE, ICGA, Levy and ChessBase: It's just an exhibition match of two engines, which aren't Top-5 anymore. I think we should not pay too much attention. The various rating lists can provide comparisons for much more than just six games.

It is a shame that the loser wil receive 40,000 $ while stronger engines like Rybka, Hiarcs, Loop, Shredder etc. cannot participate.

There are ICGA members on this message board. What do YOU think? If it's not ok, what are you going to do?

Nothing...
Rybka is without doubt stronger engine.
I have serious doubts about engines like loop or shredder.

Junior has better ssdf rating than shredder

Uri

User avatar
Mike S.
Posts: 1480
Joined: Thu Mar 09, 2006 4:33 am

Re: The Ultimate Computer Chess Challenge !?!?

Post by Mike S. » Mon Apr 23, 2007 8:21 pm

Uri, please be serious. I guess you know also CCRL and CEGT ratings, not only SSDF.

Junior is a good, interesting and respectable engine! - But it has always been overestimated because of misleading ICGA tournament results.
Regards, Mike

User avatar
George Tsavdaris
Posts: 1575
Joined: Thu Mar 09, 2006 11:35 am

Re: The Ultimate Computer Chess Challenge !?!?

Post by George Tsavdaris » Mon Apr 23, 2007 9:04 pm

Mike S. wrote:Uri, please be serious. I guess you know also CCRL and CEGT ratings, not only SSDF.

Junior is a good, interesting and respectable engine! - But it has always been overestimated because of misleading ICGA tournament results.

It is a shame that the loser wil receive 40,000 $ while stronger engines like Rybka, Hiarcs, Loop, Shredder etc. cannot participate.
SSDF-21/04/2007:

Code: Select all

3 Hiarcs 11.1   2871   51   -51   189   52%  2858
4 Junior 10.1   2867   32   -30   563   72%  2702
5 Junior 10      2851   25   -24   874   70%  2703
8 Shredder 10 2830   25   -24   857   69%  2689

Latest public Hiarcs seems not so stronger than the latest public Junior which seems stronger than the almost latest public Shredder.


The CEGT-40/20 Best versions:
Does not confirm your: "It's just an exhibition match of two engines, which aren't Top-5 anymore" since Deep Fritz 10 is inside top 5.

Enir

Re: The Ultimate Computer Chess Challenge !?!?

Post by Enir » Mon Apr 23, 2007 11:11 pm

Uri Blass wrote:Rybka is without doubt stronger engine.
I would like to see a match Rybka-Hydra. It doesn't have to be organized by FIDE, although it might be considered the absolute world championship of chess.

Enrique

Uri Blass
Posts: 8557
Joined: Wed Mar 08, 2006 11:37 pm
Location: Tel-Aviv Israel

Re: The Ultimate Computer Chess Challenge !?!?

Post by Uri Blass » Tue Apr 24, 2007 5:38 pm

Mike S. wrote:Uri, please be serious. I guess you know also CCRL and CEGT ratings, not only SSDF.

Junior is a good, interesting and respectable engine! - But it has always been overestimated because of misleading ICGA tournament results.
I think that the difference is too small to claim which engine is better
when you consider that the better engine is dependent on the opening line

From CEGT 40/120

6 Deep Shredder 10 x64 2CPU 2858 17 17 925 51.5 % 2847 40.5 %
8 List 11.64 2CPU 2849 20 20 700 47.6 % 2866 40.0 %
12 Deep Junior 10 2CPU 2833 19 19 844 49.0 % 2840 33.8 %

I think that difference of less than 30 elo can be dependent on the opening book that is used and the question if ponder on or ponder off is used.

It is not the fault of CEGT and CCRL but it is impossible to claim without doubt which engine is better when the difference is small regardless of the number of games because engine X can be better in one opening when engine Y can be better in another opening and I see no objective way to decide how many games to play with every opening.

Uri

rdan1987

Re: The Ultimate Computer Chess Challenge !?!?

Post by rdan1987 » Wed Apr 25, 2007 1:49 am

Mike S. wrote:Although it's supported by FIDE, ICGA, Levy and ChessBase: It's just an exhibition match of two engines, which aren't Top-5 anymore.
Since when Deep Fritz 10 isn't in Top 5 anymore???

User avatar
Kirill Kryukov
Posts: 492
Joined: Sun Mar 19, 2006 3:12 am

Re: The Ultimate Computer Chess Challenge !?!?

Post by Kirill Kryukov » Wed Apr 25, 2007 4:28 am

rdan1987 wrote:Since when Deep Fritz 10 isn't in Top 5 anymore???
Since it was surpassed by Rybka, Zap!Chess, Hiarcs, Naum, Deep Shredder and LoopMP. At least in our study: CCRL 40/40. (Not to mention Deep Junior which is 1 point ahead of Deep Fritz right now in our 40/40 list).

Though, in our blitz rating list Deep Fritz is #5 currently.

Best,
Kirill

Alkelele

Re: The Ultimate Computer Chess Challenge !?!?

Post by Alkelele » Wed Apr 25, 2007 10:29 am

Uri Blass wrote:
Mike S. wrote:Uri, please be serious. I guess you know also CCRL and CEGT ratings, not only SSDF.

Junior is a good, interesting and respectable engine! - But it has always been overestimated because of misleading ICGA tournament results.
I think that the difference is too small to claim which engine is better
when you consider that the better engine is dependent on the opening line

From CEGT 40/120

6 Deep Shredder 10 x64 2CPU 2858 17 17 925 51.5 % 2847 40.5 %
8 List 11.64 2CPU 2849 20 20 700 47.6 % 2866 40.0 %
12 Deep Junior 10 2CPU 2833 19 19 844 49.0 % 2840 33.8 %

I think that difference of less than 30 elo can be dependent on the opening book that is used and the question if ponder on or ponder off is used.

It is not the fault of CEGT and CCRL but it is impossible to claim without doubt which engine is better when the difference is small regardless of the number of games because engine X can be better in one opening when engine Y can be better in another opening and I see no objective way to decide how many games to play with every opening.

Uri
Uri, please note that the question "which engine is best" is of a metaphysical nature, and thus it is nonsensical to try and answer it in an absolute sense. All you can really do is making various tests and making various predictions (=creating an Elo list).

When people say engine A is better than engine B, it is thus useful to instead read this as "under these specific conditions, A seems to perform better than B".

Once this is understood, it becomes irrelevant to make observations like "yes, but B may perform better than A under these different conditions, so it is not certain that A is stronger than B". Nobody claimed anything about these different conditions!

Compare to: Which is the most spectacular goal scored at a football world cup? Here it is EASY to see that it is nonsensical to try and answer this in an absolute sense. With a little practice, it becomes equally easy to see that it is pointless to try and answer questions like "which is the stronger engine" in an absolute sense. Specific conditions and metrics always have to be read and understood between the lines.

Post Reply