In my opinion the SSDF-list is the best there is. Ponder on, learning on, opening book on and 40 in 120 is exactly as it should be.
The disadvantage is also very clear. a game 40 in 120 last at least 5 hours.
I test with 3 minutes a game and even then a tournament of 120 games lasts about 10 hours....
In case you missed it: SSDF, shiny new hardware...
Moderators: hgm, Rebel, chrisw
-
- Posts: 39
- Joined: Sat May 21, 2011 12:23 pm
-
- Posts: 4190
- Joined: Wed Nov 25, 2009 1:47 am
Re: In case you missed it: SSDF, shiny new hardware...
40 moves in 120 on strong hardware is just plain rediculous, without special openings, there will be over 90% draw rate between top engines and additionally it is extremely stupid TC once you reach endgame.attakinski wrote:In my opinion the SSDF-list is the best there is. Ponder on, learning on, opening book on and 40 in 120 is exactly as it should be.
The disadvantage is also very clear. a game 40 in 120 last at least 5 hours.
I test with 3 minutes a game and even then a tournament of 120 games lasts about 10 hours....
Moreover, regular books are just too weak already after move 10 for such a TC and strong machine. So you take Brainfish book, maybe tune it a bit, limit it to 10 moves and with the latest SF you can't loose a single game.
-
- Posts: 10121
- Joined: Thu Mar 09, 2006 12:57 am
- Location: van buren,missouri
Re: In case you missed it: SSDF, shiny new hardware...
I agree also. One of the best test site for years.attakinski wrote:Ed and Ray : I agree!
They try not to test clones.
Best,
Gerold.
-
- Posts: 4190
- Joined: Wed Nov 25, 2009 1:47 am
Re: In case you missed it: SSDF, shiny new hardware...
They actually try to test only commercial engines. It's clear that some "sponsoring" money is involved.gerold wrote:They try not to test clones.
It's just pathetic.
-
- Posts: 365
- Joined: Mon May 14, 2007 8:20 pm
- Full name: Boban Stanojević
Re: In case you missed it: SSDF, shiny new hardware...
I haven't posted for years, although I am still reading regularly new threads, and remain very present on the forum. But I felt I had not much to say on many topics -- I regret though not to have thanked the many authors of engines and GUIs who offered us something different and original, useful for training and enjoying chess (Ed, Frank, Steven, HGM, Lucas, Brendan, Norman and many others).
After all these years, I still feel that the SSDF does an excellent job, and that their approach -- testing products, not engines -- herited from years when chess programs were complete software packages, not only engines, with a GUI, an opening book, is the right one. I like also the fact that they test at LTC -- it's probably the most important element for me.
I also do understand why they don't test Houdini, nor every version of engines. That said, in the comments of the chairman, I read that they soon will test the latest version of SF, so it's not that they avoid testing SF. Testing at LTC does severely limit the number of programs/engines they can test, so a choice has to be made. Some things could be done differently, but it's always easy to criticize.
For me, this list has still a lot to offer, especially since it's the only one testing dedicated chess computers, and conserving a testing paradigm I find is usefully complementing the work of other testing groups and individuals.
After all these years, I still feel that the SSDF does an excellent job, and that their approach -- testing products, not engines -- herited from years when chess programs were complete software packages, not only engines, with a GUI, an opening book, is the right one. I like also the fact that they test at LTC -- it's probably the most important element for me.
I also do understand why they don't test Houdini, nor every version of engines. That said, in the comments of the chairman, I read that they soon will test the latest version of SF, so it's not that they avoid testing SF. Testing at LTC does severely limit the number of programs/engines they can test, so a choice has to be made. Some things could be done differently, but it's always easy to criticize.
For me, this list has still a lot to offer, especially since it's the only one testing dedicated chess computers, and conserving a testing paradigm I find is usefully complementing the work of other testing groups and individuals.
-
- Posts: 4190
- Joined: Wed Nov 25, 2009 1:47 am
Re: In case you missed it: SSDF, shiny new hardware...
Code: Select all
1 Komodo 11.01 MP x64 16GB 1800X 3,6 GHz, 3406
Komo9.1 Q6600 25,5-18,5 Stockf6 Q6600 24,5-15,5 DShre13 Q6600 30-10
Komo7.0 Q6600 29,5-10,5 Stockf3 Q6600 34,5-15,5 DRybka4 Q6600 30,5-9,5
DHiar14 Q6600 30,5-9,5 Spike14 Q6600 34-6
Komodo played against 2 versions of itself, 2 prehistoric versions of SF one version of Shredder, Hiaracs, prehistoric Rybka and pathetic Spike. Most of engines it played couldn't even use 6-men. In total 320 games against those "determine" the rating.
In reality the result skewed beyond belief.
-
- Posts: 1494
- Joined: Thu Mar 30, 2006 2:08 pm
Re: In case you missed it: SSDF, shiny new hardware...
"opening books distort a fair competition.". Yes, which is why humans are not allowed to use memory of opening preparation when they play in tournaments. Oh wait. They can use their memory of opening preparation.
It is true that if you knew the opening book used by your opponent you could spend a lot of time and make a counter book to help your winning chances. Hard, but with enough resources it could be done. Komodo has just used a general opening book tuned to lines it understands well, and not against a specific opponent. It is not very practical to have a unique book for each opponent. We do change some lines to prevent repeating games, and Erdo uses his bets judgement to select the first few moves.
I agree it would be unfair to allow one program to use an opening book and another one not. But just as adding an evaluation or search modification to a program can make that program stronger, a well tuned book can help a program play stronger. But I cannot call me adding a new evaluation component to Komodo "unfair" anymore than someone writing a better book.
As long as both parties are allowed to have a book they choose, how is it unfair?
I personally like testing without opening books, but only because I am lazy and am not skilled at making such books. But if you show up to an event like WCCC without a book, your chances will drop a lot.
SSDF has a different set of conditions they use in testing. Variety is good. And I am glad they are back. And I am sure they will test Stockfish 8 or whatever is ready when they get to it. I am happy they are back in action and with some better hardware.
It is true that if you knew the opening book used by your opponent you could spend a lot of time and make a counter book to help your winning chances. Hard, but with enough resources it could be done. Komodo has just used a general opening book tuned to lines it understands well, and not against a specific opponent. It is not very practical to have a unique book for each opponent. We do change some lines to prevent repeating games, and Erdo uses his bets judgement to select the first few moves.
I agree it would be unfair to allow one program to use an opening book and another one not. But just as adding an evaluation or search modification to a program can make that program stronger, a well tuned book can help a program play stronger. But I cannot call me adding a new evaluation component to Komodo "unfair" anymore than someone writing a better book.
As long as both parties are allowed to have a book they choose, how is it unfair?
I personally like testing without opening books, but only because I am lazy and am not skilled at making such books. But if you show up to an event like WCCC without a book, your chances will drop a lot.
SSDF has a different set of conditions they use in testing. Variety is good. And I am glad they are back. And I am sure they will test Stockfish 8 or whatever is ready when they get to it. I am happy they are back in action and with some better hardware.
-
- Posts: 6995
- Joined: Thu Aug 18, 2011 12:04 pm
Re: In case you missed it: SSDF, shiny new hardware...
Nope, just pointing out your (expected) inconsistency.Lyudmil Tsvetkov wrote:are you really that basic-level?Rebel wrote:Are table bases also bad?Lyudmil Tsvetkov wrote:permanent brain = bad;Rebel wrote:We still honor them for what they did in the past, as first one offer a platform for competition long before we heard of Lyudmil TsvetkovLyudmil Tsvetkov wrote:what is SSDF?Dann Corbit wrote:The SSDF is now using AMD Ryzen 7 1800X, which is an 8-core processor at 3.6 GHz. They have opted for 16 GB RAM as standard on this hardware.
I understand in the past, but now?
We still honor them now for willing to play LTC games + permanent brain + opening books + allowing learning software, meaning a rating list where programs are allowed to compete on their strongest settings.
opening books = bad;
learning features = bad;
They are not, some have them, others don't.Lyudmil Tsvetkov wrote:tablebases provide equal conditions for all,
Not at all, an opening book is an essential part of a chess program, in there the programmer decides what suits his program best.Lyudmil Tsvetkov wrote:opening books distort a fair competition.
CCRL | CEGT etc. are all fine, so is the SSDF concept.
-
- Posts: 6995
- Joined: Thu Aug 18, 2011 12:04 pm
Re: In case you missed it: SSDF, shiny new hardware...
The SSDF folks don't buy, As a commercial you have to give them a couple of free versions.Lyudmil Tsvetkov wrote:why not test Houdini?
-
- Posts: 1080
- Joined: Fri Sep 16, 2016 6:55 pm
- Location: USA/Minnesota
- Full name: Leo Anger
Re: In case you missed it: SSDF, shiny new hardware...
Good hardware, bad software. Stockfish 6??Dann Corbit wrote:The SSDF is now using AMD Ryzen 7 1800X, which is an 8-core processor at 3.6 GHz. They have opted for 16 GB RAM as standard on this hardware.
Advanced Micro Devices fan.