I read the following opinion in the lc0's forum:
It's not only a question of whether openings should be used, but also in what way. I can't be sure, but given the almost equal result, I suspect that Leela would be able to win if:
a) Openings were used to provide variety, but for less plies, and not as sharp, or
b) No opening lines were forced but stockfish could use an opening book
I do not believe that b is correct but I may be wrong and I wonder if it is possible to test stockfish using BookX.bin against lc0 with no book to see what are the results(other conditions should be the same as the last lc0-stockfish match from TCEC when lc0 won with no books).
I know that in the past BookX.bin helped stockfish against lc0 but I believe lc0 improved from that time and it was also at faster time control than the last TCEC match
when I guess a book can help more at faster time control.
can stockfish with book beat leela without book?
Moderators: hgm, Rebel, chrisw
-
- Posts: 10282
- Joined: Thu Mar 09, 2006 12:37 am
- Location: Tel-Aviv Israel
-
- Posts: 1470
- Joined: Mon Apr 23, 2018 7:54 am
Re: can stockfish with book beat leela without book?
If in b) you give SF the biggest book possible, Lc0 won't win a single game.
Lc0 maybe's got stronger since Kai's book tests, but won't the effects still be the same qualitatively?
Lc0 maybe's got stronger since Kai's book tests, but won't the effects still be the same qualitatively?
-
- Posts: 10948
- Joined: Wed Jul 26, 2006 10:21 pm
- Full name: Kai Laskos
Re: can stockfish with book beat leela without book?
a) is probably true, b) is debatable. Maybe using latest BraniFish is even better than BookX for SF. As of now, T30 is marginally the strongest at 60s + 1s time control, not sure at 240s + 4s, as T40 of today is very close to T30 at 60s + 1s TC, but T40 seems to scale better. They just dropped LR some 2 days ago, I expect further gains soon, maybe I will perform in 1-3 days a test of T40 latest net against the latest BrainFish at 60s + 1s (100 games maybe). BookX gave some 80 Elo points boost to SF against Lc0 (no book) compared to 3-4 mover regular GM openings (used by both), probably Cerebellum will fare even better, but as you said, the advantage of the book diminishes with time control.Uri Blass wrote: ↑Wed Mar 06, 2019 9:22 am I read the following opinion in the lc0's forum:
It's not only a question of whether openings should be used, but also in what way. I can't be sure, but given the almost equal result, I suspect that Leela would be able to win if:
a) Openings were used to provide variety, but for less plies, and not as sharp, or
b) No opening lines were forced but stockfish could use an opening book
I do not believe that b is correct but I may be wrong and I wonder if it is possible to test stockfish using BookX.bin against lc0 with no book to see what are the results(other conditions should be the same as the last lc0-stockfish match from TCEC when lc0 won with no books).
I know that in the past BookX.bin helped stockfish against lc0 but I believe lc0 improved from that time and it was also at faster time control than the last TCEC match
when I guess a book can help more at faster time control.
Also, your entire question is dependent on hardware GPU/CPU balance, my balance is somewhat skewed towards GPU. And as stated, the answer depends on time control used too.
-
- Posts: 10282
- Joined: Thu Mar 09, 2006 12:37 am
- Location: Tel-Aviv Israel
Re: can stockfish with book beat leela without book?
Biggest book possible does not make sense because it is going to include many bad lines(for example 1.f3 e5 2.g4 for white)
I doubt if there is a combination of a book and an engine that you can be sure that LC0 won't win a single game against the combination.
-
- Posts: 708
- Joined: Mon Jan 16, 2012 6:34 am
Re: can stockfish with book beat leela without book?
For TCEC hardware, I still put my bet on Lco that Lco without book is still better than SF + brainfish (or whatever book you can get on earth) in adjusted allocated time for book use.
What I mean is that , if you give book to one side, it should not be time cheat to one side. In spcc testing, I noticed that Brainfish has approx >+ 80/+100 elo above SF in 1 min bullet. In fact brainfish got not only better opening line but also about 30% time cheat which is non-sense.
If you would like to use brainfish book for stockfish, (use whatever depth the book have, may be depth 40- 60) but just start your time control only when both engines are out of book.
What I mean is that , if you give book to one side, it should not be time cheat to one side. In spcc testing, I noticed that Brainfish has approx >+ 80/+100 elo above SF in 1 min bullet. In fact brainfish got not only better opening line but also about 30% time cheat which is non-sense.
If you would like to use brainfish book for stockfish, (use whatever depth the book have, may be depth 40- 60) but just start your time control only when both engines are out of book.
-
- Posts: 1470
- Joined: Mon Apr 23, 2018 7:54 am
Re: can stockfish with book beat leela without book?
It won't happen, of course, but if a top GM allowed SF to borrow his opening book, it's very unlikely Lc0 would win a game. When A0 played SF8 with a book (not the latest SF, not the best book), it was almost all draws.Uri Blass wrote: ↑Wed Mar 06, 2019 3:43 pmBiggest book possible does not make sense because it is going to include many bad lines(for example 1.f3 e5 2.g4 for white)
I doubt if there is a combination of a book and an engine that you can be sure that LC0 won't win a single game against the combination.
-
- Posts: 1080
- Joined: Fri Sep 16, 2016 6:55 pm
- Location: USA/Minnesota
- Full name: Leo Anger
Re: can stockfish with book beat leela without book?
What is BookX.bin ?
Advanced Micro Devices fan.
-
- Posts: 10948
- Joined: Wed Jul 26, 2006 10:21 pm
- Full name: Kai Laskos
Re: can stockfish with book beat leela without book?
Yes, at 4min + 4s time control, Lc0 T41367 beats soundly latest SF_dev + BookX.bin in Cutechess-Cli UI. Unfortunately, the variety of games was not great, I am not sure why, polyglot book choices were constricted in Cutechess and Lc0 itself played almost deterministically. The result in 40 games was:Uri Blass wrote: ↑Wed Mar 06, 2019 9:22 am I read the following opinion in the lc0's forum:
It's not only a question of whether openings should be used, but also in what way. I can't be sure, but given the almost equal result, I suspect that Leela would be able to win if:
a) Openings were used to provide variety, but for less plies, and not as sharp, or
b) No opening lines were forced but stockfish could use an opening book
I do not believe that b is correct but I may be wrong and I wonder if it is possible to test stockfish using BookX.bin against lc0 with no book to see what are the results(other conditions should be the same as the last lc0-stockfish match from TCEC when lc0 won with no books).
I know that in the past BookX.bin helped stockfish against lc0 but I believe lc0 improved from that time and it was also at faster time control than the last TCEC match
when I guess a book can help more at faster time control.
Score of lc0_41367 vs SF_dev_BookX: 7 - 2 - 31 [0.563] 40
Elo difference: 43.66 +/- 50.34
Finished match
LOS=95%
Also, it seems that above 2min + 2s time control on strong RTX GPU, T40 as of today is stronger than the best T30 nets (and scales better to longer time control). T40 seems to have a bright future at TCEC and CCC, and SF won't probably win finals anymore in those conditions, aside from very weird opening positions.
-
- Posts: 3283
- Joined: Wed Mar 08, 2006 8:15 pm
Re: can stockfish with book beat leela without book?
In CCC 6 Stockfish is dominating clearly against four Leelas including latest T40. Any excuses ?
Jouni
-
- Posts: 12540
- Joined: Wed Mar 08, 2006 8:57 pm
- Location: Redmond, WA USA
Re: can stockfish with book beat leela without book?
A good book is worth about 100 Elo
https://sites.google.com/site/computers ... book-cs-23
SCCT XXIII's Champion: Attacker - Congrats to Adrian Costinescu!
Difference of 111 Elo (give or take a little).
Now, SF recently won TCEC so it is likely a little stronger on equal ground.
Hence with a book, given a long tournament, winning is likely.
But consider all the unstated variables...
LC0 Elo seems to grow faster than SF Elo.
Shall we give LC0 four of these 2080 TI cards?
https://www.segmentblue.com/best-graphi ... tx-2080-ti
Or how about a whole rack of these:
https://www.nextplatform.com/2018/05/10 ... processor/
But maybe we give Stockfish the "128 cpu's x32 threads Cluster System" that gets 4,801,341,606 NPS using 4096 threads described here:
http://www.ipmanchess.yolasite.com/amd- ... -bench.php
Of course the real question we are all asking ourselves is, "What does the future hold for chess engines?"
We can wonder what direction things will take. Will a hybrid strategy be best?
But of course, nobody knows the answer to any of these questions.
Still, it's always fun to discuss how many angels can dance on the head of a pin.
I think it's 13.
https://sites.google.com/site/computers ... book-cs-23
SCCT XXIII's Champion: Attacker - Congrats to Adrian Costinescu!
Code: Select all
# PLAYER : RATING POINTS GAMES (%)
1 Attacker Pro 3 : 3562.9 866.5 1460 59.3%
...
92 No Book : 3451.3 281.0 640 43.9%
Now, SF recently won TCEC so it is likely a little stronger on equal ground.
Hence with a book, given a long tournament, winning is likely.
But consider all the unstated variables...
LC0 Elo seems to grow faster than SF Elo.
Shall we give LC0 four of these 2080 TI cards?
https://www.segmentblue.com/best-graphi ... tx-2080-ti
Or how about a whole rack of these:
https://www.nextplatform.com/2018/05/10 ... processor/
But maybe we give Stockfish the "128 cpu's x32 threads Cluster System" that gets 4,801,341,606 NPS using 4096 threads described here:
http://www.ipmanchess.yolasite.com/amd- ... -bench.php
Of course the real question we are all asking ourselves is, "What does the future hold for chess engines?"
We can wonder what direction things will take. Will a hybrid strategy be best?
But of course, nobody knows the answer to any of these questions.
Still, it's always fun to discuss how many angels can dance on the head of a pin.
I think it's 13.
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.