CCLS: New tournament (league system) - Introduction

Discussion of computer chess matches and engine tournaments.

Moderators: hgm, Rebel, chrisw

User avatar
CMCanavessi
Posts: 1142
Joined: Thu Dec 28, 2017 4:06 pm
Location: Argentina

CCLS: New tournament (league system) - Introduction

Post by CMCanavessi »

Hi guys, I'm Carlos from Argentina and I've been testing chess engines for a long time, but always in private fashion. This is about to change now and I will share the results with you all. After doing a couple of test for a while I've come up with a league system that looks reasonable (at least to me), with promotions and demotions, longer time controls and fewer engines as the leagues get "better", etc.

CCLS means "Carlos Chess League System", but you can also make a case for "Computer Chess League System" or "Carlos Canavessi League System" or whatever.

Season 1 has already started on December 18 and I'm using it as a real test to see how well or how bad things work out. It is a short season with 3 leagues and a superfinal with the top 3 engines (a total of 66 engines). From Season 2 and beyond there will be 4 leagues + the superfinal (with a total of 122 engines).

The idea is to test as many engines as possible, regardless of their strenght, giving priority to the actively developed ones. Every season (except the shortened 1st one) the engines finishing in the bottom 16 places will be replaced with different ones the following season, but that does not mean they couldn't make a return later on.


The league system is structured as follows:

Entry league (skipped in season 1): 64 engines, bottom 16 are replaced with new engines the following season, top 8 are promoted to the Junior league. Time control is 1 minutes + 1 second and matches will be played in 2 x round robin format changing colors every round. A total of 4032 matches will be played, and the whole run will take aprox. 13 days to complete (estimation).

Junior league: 32 engines, bottom 8 are demoted to the Entry league, top 4 are promoted to the Super league. Time control is 2 minutes + 1 second and matches will be played in 4 x round robin format changing colors every round. A total of 1984 matches will be played, and the whole run will take aprox. 9 days to complete (took 9,5 days for season 1, already finished).

Super league: 24 engines, bottom 4 are demoted, top 2 are promoted to the Elite league. Time control is 3 minutes + 2 seconds and matches will be played in 6 x round robin format changing colors every round. A total of 1656 matches will be played, and the whole run will take aprox. 13 days to complete (more than 60% played so far and it's right there with the estimation).

Elite league: 16 engines, bottom 2 are demoted to the Super league, top 3 play the Superfinal. Time control is 5 minutes + 5 seconds and matches will be played in 8 x round robin format changing colors every round. A total of 960 matches will be played, and the whole run will take aprox. 15 days to complete (estimation).

Superfinal: 3 engines finishing on top of the Elite league. Time control will be 15 minutes + 15 seconds and matches will be played in 20 x round robin format changing colors every round. A total of 60 matches will be played, and the whole run will take aprox. 7 days to complete (estimation).


General rules:

Only 1 thread for each engine
Ponder ON
Opening book is Libra8 from the Numpty engine author (click here for book info), though that may change in future seasons
4GB Hash or biggest size the engine can handle if less than 4GB
Using Arena GUI on Windows 10, Ryzen 1700 processor with 32 GB total Ram


I'm still deciding what to do with the commercial engines. I have Shredder 13, Fritz 16 and Hiarcs 14 but no Komodo or Houdini; so I might have to use Houdini 1.5a and Komodo 9.02 which are free. After writing this post here I'm gonna email Mark and Robert to see if they are willing to provide the latest public releases of their engines so I can test them. If not, free versions for all, which I know it's gonna be disappointing.

Also, I'll contact Ginkgo author and see if he wants to participate from Season 2 (will have to start from the entry league, but it will be easy to promote fast to the elite league).

I don't want to test "dev" versions of engines, as that would complicate things for me. Only public releases at least for now.


In the future I would love to broadcast the matches for everyone to watch, but I'm still testing different options on how to do that. I use Arena and that limits things a lot.



So with that said, I'm gonna go ahead and make another thread with the results that I have so far (Season 1 Junior league 100% complete, Super league still going).

Ideas and suggestions always welcome. :idea:
User avatar
Graham Banks
Posts: 41412
Joined: Sun Feb 26, 2006 10:52 am
Location: Auckland, NZ

Re: CCLS: New tournament (league system) - Introduction

Post by Graham Banks »

All the best with your tournaments. I'll follow your results. :)
gbanksnz at gmail.com
User avatar
CMCanavessi
Posts: 1142
Joined: Thu Dec 28, 2017 4:06 pm
Location: Argentina

Re: CCLS: New tournament (league system) - Introduction

Post by CMCanavessi »

Thanx Graham, we'll see where this leads to. Still working on getting broadcast to work.
User avatar
CMCanavessi
Posts: 1142
Joined: Thu Dec 28, 2017 4:06 pm
Location: Argentina

Re: CCLS: New tournament (league system) - Introduction

Post by CMCanavessi »

Ginkgo 2.02 presence confirmed. It will start in Season 1 already in the Elite league, and "Critter" will be taken out (it's not been updated in ages, even if it's still super strong) and placed in the entry list in season 2, so it'll probably climb fast anyways. :D
User avatar
CMCanavessi
Posts: 1142
Joined: Thu Dec 28, 2017 4:06 pm
Location: Argentina

Re: CCLS: New tournament (league system) - Introduction

Post by CMCanavessi »

Houdini 6.03 presence confirmed. Now Stockfish 8 has some serious competition for the top spot.
User avatar
Nordlandia
Posts: 2821
Joined: Fri Sep 25, 2015 9:38 pm
Location: Sortland, Norway

Re: CCLS: New tournament (league system) - Introduction

Post by Nordlandia »

CMCanavessi wrote:Houdini 6.03 presence confirmed. Now Stockfish 8 has some serious competition for the top spot.
Carlos: Stockfish 8 was released 13 months ago. Nobody test SF8 anymore.
User avatar
CMCanavessi
Posts: 1142
Joined: Thu Dec 28, 2017 4:06 pm
Location: Argentina

Re: CCLS: New tournament (league system) - Introduction

Post by CMCanavessi »

Nordlandia wrote:
CMCanavessi wrote:Houdini 6.03 presence confirmed. Now Stockfish 8 has some serious competition for the top spot.
Carlos: Stockfish 8 was released 13 months ago. Nobody test SF8 anymore.
I know, but it's the latest official release. It's not my (or our) fault that the Stockfish team doesn't want to release 9 yet (for whatever reason). I don't like to test dev versions because that generates a mess. In stockfish case for example, there's almost a version every week, and even every day if you compile it yourself. So my policy is to only test official public releases (unless unavoidable).
Adam Hair
Posts: 3226
Joined: Wed May 06, 2009 10:31 pm
Location: Fuquay-Varina, North Carolina

Re: CCLS: New tournament (league system) - Introduction

Post by Adam Hair »

I noticed that the Argentinian engine Gaviota is missing from the Junior league. Are you a Boca Juniors fan? :D
User avatar
CMCanavessi
Posts: 1142
Joined: Thu Dec 28, 2017 4:06 pm
Location: Argentina

Re: CCLS: New tournament (league system) - Introduction

Post by CMCanavessi »

No!! http://clubaindependiente.com/


Gaviota 1.0 is already on the list for Season 2 in the Entry League (which will probably win or finish 2nd behind Critter). I hope Miguel will update it sometime as its already super old (but still strong). I asked him about 1.01 at the old TCEC chat and he said he would release it, but it hasn't happened yet.
Adam Hair
Posts: 3226
Joined: Wed May 06, 2009 10:31 pm
Location: Fuquay-Varina, North Carolina

Re: CCLS: New tournament (league system) - Introduction

Post by Adam Hair »

Miguel is a huge River Plate fan.

Yeah, it has been 3 1/2 years or so since the last update. I have been waiting for Miguel to find the time and the interest to start working on Gaviota again. Maybe one day it will happen. The only reason why I brought this up is that you included several engines that have been inactive much longer period of time than Gaviota.