+100 elo breakthrough in new rewritten Lco engine ( Ceres)

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Dann Corbit, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
mwyoung
Posts: 2727
Joined: Wed May 12, 2010 8:00 pm

Re: +100 elo breakthrough in new rewritten Lco engine ( Ceres)

Post by mwyoung » Sun Jan 03, 2021 10:30 pm

mwyoung wrote:
Sun Jan 03, 2021 9:02 pm
Some observations of Ceres in the current version being tested.

1. Ceres is +100 Elo. False
2. Ceres is 2.3x faster. False.
3. And Lc0 is out searching Ceres by many ply with the same size net.

Current score Lc0 vs Ceres. +2 =28 -0 +23 Elo for Lc0

I will switch Lc0 to the same exact NN after this test.
End of match.

Code: Select all

DESKTOP-CORSAIR, Blitz 3.0min+2.0sec  0

                           
1   Lc0 v0.26.3 (1)   +17  +2/=38/-0 52.50%   21.0/40
2   Ceres             -17  +0/=38/-2 47.50%   19.0/40

"The worst thing that can happen to a forum is a running wild attacking moderator(HGM) who is not corrected by the community." - Ed Schröder
But my words like silent raindrops fell. And echoed in the wells of silence.

mwyoung
Posts: 2727
Joined: Wed May 12, 2010 8:00 pm

Re: +100 elo breakthrough in new rewritten Lco engine ( Ceres)

Post by mwyoung » Sun Jan 03, 2021 10:44 pm

Live Ceres (703810) vs Lc0 0.26.3 (703810) (TC=3m+2s)

Same NN for each engine. 703810

Hardware 2950x, RTX 2080TI

The top ten 7 man tablebases.

Opening book to 6 moves

Live Stream: https://www.youtube.com/watch?v=61of1koXhdY
"The worst thing that can happen to a forum is a running wild attacking moderator(HGM) who is not corrected by the community." - Ed Schröder
But my words like silent raindrops fell. And echoed in the wells of silence.

cc2150dx
Posts: 233
Joined: Sat Nov 30, 2013 8:51 am
Location: Canada
Full name: Jason Coombs

Re: +100 elo breakthrough in new rewritten Lco engine ( Ceres)

Post by cc2150dx » Sun Jan 03, 2021 11:07 pm

Odd exceptions:




User avatar
Laskos
Posts: 10949
Joined: Wed Jul 26, 2006 8:21 pm
Full name: Kai Laskos

Re: +100 elo breakthrough in new rewritten Lco engine ( Ceres)

Post by Laskos » Sun Jan 03, 2021 11:10 pm

Albert Silver wrote:
Sat Jan 02, 2021 2:56 pm
MMarco wrote:
Sat Jan 02, 2021 1:44 pm
Albert Silver wrote:
Sat Jan 02, 2021 12:02 am
How does that compare with Lc0 v26?
I did a quick bench on my 3080 of CERES and LC0-26.3-cuda. Default settings for CERES, defaults and NNcache=10M, smart-pruning-factor=0 for Lc0. The measured "speedup" (if we can say so, we are comparing nps from two different engines): 30x384 = 15%, 20x256 = 28%, 10x128 = 68%.
Startposition, go movetime 60000

30x384
*****************
LC0-66740
info depth 12 seldepth 42 time 59800 nodes 1383904 score cp 10 nps 25720
CERES-66740
info depth 14 seldepth 36 time 60006 nodes 1773283 score cp 8 nps 29552

20x256
******************
LC0-LS-15
info depth 23 seldepth 68 time 59800 nodes 4135219 score cp 15 nps 77013
CERES-LS-15
info depth 25 seldepth 53 time 56979 nodes 5618322 score cp 11 nps 98603

10x128
*******************
LC0-703810
info depth 14 seldepth 50 time 59800 nodes 8140461 score cp 12 nps 151242
CERES-703810
info depth 17 seldepth 51 time 59991 nodes 15210428 score cp 11 nps 253544
Ok, thanks. This is very interesting and actually seems to match what I suspected is happening. While there may be a normal speedup, if modest, over Lc0 v26, its biggest strength (for play, not for development) is removing the scaling shackles. Meaning smaller nets will gain the most from powerful cards, and above all will suffer less from the scaling issues on multi-gpu setups. Time will tell how accurate this read is, but it is what I am gathering from the results.
It seems the CPU bottleneck is not as pronounced as with Lc0. Ceres can probably use better the smaller nets with big hardware. With Lc0 and big hardware, larger nets were preferred because of the CPU bottleneck. Ceres seems to be using variable batchsizes and unlimited cache. Its NPS seems to grow starting from being slower than Lc0 at very short times to significantly faster at longer thinking times. Maybe Ceres scales better to longer TC if NPS are some indication? I checked at 30s + 0.3s in 400 games on RTX 2070 with 66970 large net, Ceres seems marginally stronger (10-20 Elo points) than Lc0 v26.3. Had harder time running it with EPD test suites, indications seem to be that it is stronger tactically, but I was unable to check.

mwyoung
Posts: 2727
Joined: Wed May 12, 2010 8:00 pm

Re: +100 elo breakthrough in new rewritten Lco engine ( Ceres)

Post by mwyoung » Sun Jan 03, 2021 11:18 pm

cc2150dx wrote:
Sun Jan 03, 2021 11:07 pm
Odd exceptions:



If you have your GPU overclocked then do not overclock your GPU. Ceres did not like any overclock. And I got the same exception. Running stock speed fixed the issue.
"The worst thing that can happen to a forum is a running wild attacking moderator(HGM) who is not corrected by the community." - Ed Schröder
But my words like silent raindrops fell. And echoed in the wells of silence.

cc2150dx
Posts: 233
Joined: Sat Nov 30, 2013 8:51 am
Location: Canada
Full name: Jason Coombs

Re: +100 elo breakthrough in new rewritten Lco engine ( Ceres)

Post by cc2150dx » Mon Jan 04, 2021 1:01 am

mwyoung wrote:
Sun Jan 03, 2021 11:18 pm
cc2150dx wrote:
Sun Jan 03, 2021 11:07 pm
Odd exceptions:



If you have your GPU overclocked then do not overclock your GPU. Ceres did not like any overclock. And I got the same exception. Running stock speed fixed the issue.
No, just a base EVGA RTX 2060. I don't even know how to overclock even if I wanted to :oops:

mwyoung
Posts: 2727
Joined: Wed May 12, 2010 8:00 pm

Re: +100 elo breakthrough in new rewritten Lco engine ( Ceres)

Post by mwyoung » Mon Jan 04, 2021 1:16 am

cc2150dx wrote:
Mon Jan 04, 2021 1:01 am
mwyoung wrote:
Sun Jan 03, 2021 11:18 pm
cc2150dx wrote:
Sun Jan 03, 2021 11:07 pm
Odd exceptions:



If you have your GPU overclocked then do not overclock your GPU. Ceres did not like any overclock. And I got the same exception. Running stock speed fixed the issue.
No, just a base EVGA RTX 2060. I don't even know how to overclock even if I wanted to :oops:
We will fix that today. Never leave performance on the table. When it is free!

Memory should be easily OC to over 8000 Mhz give or take. GPU is good with around 100 Mhz increase. You will know if it crashes in your games. No harm will happen, and just lower the clock.

Memory OC is good and can be left on at all times when you find the OC stability. It will not crash Lc0 or Cares, and other games.

Just download MSI afterburner. This will work on all RTX cards.

Make sure you use the maximum power limit allowed, and temp limit allowed.

https://www.msi.com/Landing/afterburner
"The worst thing that can happen to a forum is a running wild attacking moderator(HGM) who is not corrected by the community." - Ed Schröder
But my words like silent raindrops fell. And echoed in the wells of silence.

mwyoung
Posts: 2727
Joined: Wed May 12, 2010 8:00 pm

Re: +100 elo breakthrough in new rewritten Lco engine ( Ceres)

Post by mwyoung » Mon Jan 04, 2021 2:30 am

mwyoung wrote:
Sun Jan 03, 2021 10:44 pm
Live Ceres (703810) vs Lc0 0.26.3 (703810) (TC=3m+2s)

Same NN for each engine. 703810

Hardware 2950x, RTX 2080TI

The top ten 7 man tablebases.

Opening book to 6 moves

Live Stream: https://www.youtube.com/watch?v=61of1koXhdY
Ceres takes the lead +2 =24 -0 +27 Elo.
"The worst thing that can happen to a forum is a running wild attacking moderator(HGM) who is not corrected by the community." - Ed Schröder
But my words like silent raindrops fell. And echoed in the wells of silence.

mwyoung
Posts: 2727
Joined: Wed May 12, 2010 8:00 pm

Re: +100 elo breakthrough in new rewritten Lco engine ( Ceres)

Post by mwyoung » Mon Jan 04, 2021 3:21 am

mwyoung wrote:
Mon Jan 04, 2021 2:30 am
mwyoung wrote:
Sun Jan 03, 2021 10:44 pm
Live Ceres (703810) vs Lc0 0.26.3 (703810) (TC=3m+2s)

Same NN for each engine. 703810

Hardware 2950x, RTX 2080TI

The top ten 7 man tablebases.

Opening book to 6 moves

Live Stream: https://www.youtube.com/watch?v=61of1koXhdY
Ceres takes the lead +2 =24 -0 +27 Elo.
End of this match.

Code: Select all

DESKTOP-CORSAIR, Blitz 3.0min+2.0sec  0

                           
1   Ceres             +22  +2/=30/-0 53.12%   17.0/32
2   Lc0 v0.26.3 (2)   -22  +0/=30/-2 46.88%   15.0/32

"The worst thing that can happen to a forum is a running wild attacking moderator(HGM) who is not corrected by the community." - Ed Schröder
But my words like silent raindrops fell. And echoed in the wells of silence.

User avatar
AdminX
Posts: 5858
Joined: Mon Mar 13, 2006 1:34 pm
Location: Acworth, GA
Contact:

Re: +100 elo breakthrough in new rewritten Lco engine ( Ceres)

Post by AdminX » Mon Jan 04, 2021 11:47 am

Ceres wrote: Ceres is a newly released chess engine which has a re-implementation of MCTS search over the lc0 backends. It shows very strong search performance characteristics which will be interesting to learn from. The specific details of some of those performance characteristics would likely be difficult to copy in to lc0, without also requiring a full rewrite. So there has been some talk of possibly migrating to Ceres as a a replacement for the lc0 engine in general. This post isn’t about that though. Instead I wanted a place to capture the ‘details’ of difference as I read through the Ceres codebase.
Read More: https://www.themissingdocs.net/?p=874

Also see the latest LC0 Blog: https://lczero.org/blog/2021/01/announcing-ceres/
"Good decisions come from experience, and experience comes from bad decisions."
__________________________________________________________________
Ted Summers

Post Reply