My take on the whole "End of an era" thing.
Posted: Wed May 01, 2019 9:28 am
I think the end of an era is more than about Lc0 beating Stockfish in the TCEC Superfinal and that it's pretty much here right now.
I'm not a programmer, I don't know how to code. I'm a casual chess player, chess fan, and computer chess "follower". I guess.
These are my observations having follwed the devleopment of certain engines on GitHub and other websites, and having wathed TCEC avidly since Season 10 which started in September 2017 shortly after the release of Houdini 6.
My observation that there are, at most, only 5 "CPU only" engines that are in anything resembling serious development at the moment. This is looking at engines of TCEC Division 3 strength or above. They are
Stockfish
Komodo (mainly KMCTS)
Fire
Andscacs
Rofchade
Stockfish, we all know about.
Komodo is making great strides with KMCTS.
I still class Fire as being in development because after version 7.1 was released in May 2018, we've seen development versions from January and February 2019 playing at TCEC and CCC which show that it's still being worked on. I don't know how much stronger Fire 8 will be than these development versions, or when it will be released. I shouldn't think it will be more than 20 Elo, which will mean that the gap between Fire and Komodo/Houdini is still pretty big.
It's Andscacs 0.95 was released on 17 December 2018. There hasn't been an official release since, but development version 0.95105 has played at CCC and 0.95123 has played a TCEC, so we have proof that development, however slow, is still continuing to some extent.
Rofchade 2.1 was released on 26 April 2019. It's very strong for such a young engine. I don't know how far it can go, but it's going well at the moment.
Ethereal has now ceased development having got to version 11.38, with 11.25 being the last official release.
Laser hasn't had a patch since 24 March 2019 and OpenBench has fallen silent showing that there is nothing new in the works for Laser.
Xiphos hasn't had a patch since 30 March 2019, which for Xiphos, if you've followed its development to any extent on GitHub, you'll know is a long time to go without a patch.
Every other chess engine list below is basically dead. The list is shown in the order that the engines will start TCEC Season 16. Premier Division is still subject to change but probably won't change that much.
Season 16
Premier Division
1. Stockfish
2. Leela
3. Komodo
4. AllieStein
5. Houdini (Nothing since the release on 6.03 on 20 November 2017)
6. KMCTS
7. TBC
8. TBC
Division 1
1. Fire (development versions used at TCEC and CCC show that version 8 is probably arriving at some point later on in 2019)
2. Ethereal (dead)
3. Xiphos (likely dead because it's been 4 weeks without a single patch and the version submitted for the upcoming TCEC Cup 3 is the "version" 0.5.3 that's been available for those 4 weeks from 30 March 2019)
4. Laser (likely dead, nothing new showing on OpenBench)
5. Andscacs (very slow development, barely hanging on)
6. Fizbo (Dead. Same version used at TCEC for 5 consecutive seasons)
7. TBC
8. TBC
Division 2
1. Jonny (Dead - same version 8.1 has been used in TCEC for 6 consecutive seasons)
2. Chiron (Dead. Version 4 was released on 10 January 2017 and we've seen nothing more for well over 2 years. It now looks shaky in TCEC Division 2. At one time development versions were challenging Fire in the Premier Division).
3. Ginkgo (Porbably dead. Development seems to have stalled completely)
4. ChessBrainVB (Dead. Same version has now been used for 2 consecutive TCEC seasons)
5. Booot (8 July 2018 release and nothing new since. Dropped to 32 threads as soon as it hit TCEC and has never recovered).
6. Rofchade (All good for now. Hopefully it can catch Ethereal and maybe Fire).
7. TBC
8. TBC
Division 3
1. Fritz (Dead. Version 16 was released on 12 November 2017 and has been used at TCEC for 5 consecutive seasons)
2. Nirvana ( Dead. Nothing since the release of version 2.4 on 11 August 2017)
3. Arasan (Still plodding away on version 21.3, perhaps this one should make the cut?)
4. Texel (Dead Version 1.07 was released on 30 September 2017. Nothing since. TCEC and CCC seem to be stuck on 1.08a13 for many seasons/tournaments. Texel has fallen from Division 1)
5. Vajolet2 (recently upgraded from 2.6 to 2.7 but still below Division 2 strength).
6. Gull (dead since 2014)
7. TBC
8. TBC
So there we have it. Traditional chess engines. Very few of them appear to be improving at a rate that is going to keep pace with NN development. I appreciate that there may be many Division 4 strength engines being actively worked on, but unless they can get to at least Division 1 within 2 TCEC seasons, it doesn't really change the landscape. It seems that no traditional engine is capable of getting above Fire's level of playing strength.
Fire is now being outplayed by Leela, Leelenstein, AllieStein and Antifish. They have been around for a lot less time and have already surpassed got to the strength of Fire that used to be the engine that sat alone between the old "Big 3" and the rest of the field.
Whilst it's sad that the majority of TCEC is basically a graveyard, there are, apparently 4 or 5 new NN engines taking part in Season 16.
How long before the old favourites are no longer strong enough to compete.
I'm not a programmer, I don't know how to code. I'm a casual chess player, chess fan, and computer chess "follower". I guess.
These are my observations having follwed the devleopment of certain engines on GitHub and other websites, and having wathed TCEC avidly since Season 10 which started in September 2017 shortly after the release of Houdini 6.
My observation that there are, at most, only 5 "CPU only" engines that are in anything resembling serious development at the moment. This is looking at engines of TCEC Division 3 strength or above. They are
Stockfish
Komodo (mainly KMCTS)
Fire
Andscacs
Rofchade
Stockfish, we all know about.
Komodo is making great strides with KMCTS.
I still class Fire as being in development because after version 7.1 was released in May 2018, we've seen development versions from January and February 2019 playing at TCEC and CCC which show that it's still being worked on. I don't know how much stronger Fire 8 will be than these development versions, or when it will be released. I shouldn't think it will be more than 20 Elo, which will mean that the gap between Fire and Komodo/Houdini is still pretty big.
It's Andscacs 0.95 was released on 17 December 2018. There hasn't been an official release since, but development version 0.95105 has played at CCC and 0.95123 has played a TCEC, so we have proof that development, however slow, is still continuing to some extent.
Rofchade 2.1 was released on 26 April 2019. It's very strong for such a young engine. I don't know how far it can go, but it's going well at the moment.
Ethereal has now ceased development having got to version 11.38, with 11.25 being the last official release.
Laser hasn't had a patch since 24 March 2019 and OpenBench has fallen silent showing that there is nothing new in the works for Laser.
Xiphos hasn't had a patch since 30 March 2019, which for Xiphos, if you've followed its development to any extent on GitHub, you'll know is a long time to go without a patch.
Every other chess engine list below is basically dead. The list is shown in the order that the engines will start TCEC Season 16. Premier Division is still subject to change but probably won't change that much.
Season 16
Premier Division
1. Stockfish
2. Leela
3. Komodo
4. AllieStein
5. Houdini (Nothing since the release on 6.03 on 20 November 2017)
6. KMCTS
7. TBC
8. TBC
Division 1
1. Fire (development versions used at TCEC and CCC show that version 8 is probably arriving at some point later on in 2019)
2. Ethereal (dead)
3. Xiphos (likely dead because it's been 4 weeks without a single patch and the version submitted for the upcoming TCEC Cup 3 is the "version" 0.5.3 that's been available for those 4 weeks from 30 March 2019)
4. Laser (likely dead, nothing new showing on OpenBench)
5. Andscacs (very slow development, barely hanging on)
6. Fizbo (Dead. Same version used at TCEC for 5 consecutive seasons)
7. TBC
8. TBC
Division 2
1. Jonny (Dead - same version 8.1 has been used in TCEC for 6 consecutive seasons)
2. Chiron (Dead. Version 4 was released on 10 January 2017 and we've seen nothing more for well over 2 years. It now looks shaky in TCEC Division 2. At one time development versions were challenging Fire in the Premier Division).
3. Ginkgo (Porbably dead. Development seems to have stalled completely)
4. ChessBrainVB (Dead. Same version has now been used for 2 consecutive TCEC seasons)
5. Booot (8 July 2018 release and nothing new since. Dropped to 32 threads as soon as it hit TCEC and has never recovered).
6. Rofchade (All good for now. Hopefully it can catch Ethereal and maybe Fire).
7. TBC
8. TBC
Division 3
1. Fritz (Dead. Version 16 was released on 12 November 2017 and has been used at TCEC for 5 consecutive seasons)
2. Nirvana ( Dead. Nothing since the release of version 2.4 on 11 August 2017)
3. Arasan (Still plodding away on version 21.3, perhaps this one should make the cut?)
4. Texel (Dead Version 1.07 was released on 30 September 2017. Nothing since. TCEC and CCC seem to be stuck on 1.08a13 for many seasons/tournaments. Texel has fallen from Division 1)
5. Vajolet2 (recently upgraded from 2.6 to 2.7 but still below Division 2 strength).
6. Gull (dead since 2014)
7. TBC
8. TBC
So there we have it. Traditional chess engines. Very few of them appear to be improving at a rate that is going to keep pace with NN development. I appreciate that there may be many Division 4 strength engines being actively worked on, but unless they can get to at least Division 1 within 2 TCEC seasons, it doesn't really change the landscape. It seems that no traditional engine is capable of getting above Fire's level of playing strength.
Fire is now being outplayed by Leela, Leelenstein, AllieStein and Antifish. They have been around for a lot less time and have already surpassed got to the strength of Fire that used to be the engine that sat alone between the old "Big 3" and the rest of the field.
Whilst it's sad that the majority of TCEC is basically a graveyard, there are, apparently 4 or 5 new NN engines taking part in Season 16.
How long before the old favourites are no longer strong enough to compete.