StockfishASM_150916NB-64 => TestNB.bin
StockfishASM_150916-64 => GS_medium.bin
StockfishASM_150916AB-64 => Noomen2016.ctg via Aquarium bookadapter
Book details:
----------------
GS_medium.bin = general book, a bit handtuned, max depth 30 plies, 3.8MB
TestNB.bin = just a general 4 plies book for variation/randomization for the 'NB' engine (avoid dupes and only few lines), 9KB
Noomen2016.ctg = set to max 80 plies, tuned ex-commercial book, 1.05GB! (one caveat is that some settings cannot be set in the book adapter,
e.g. min games, but at least it seems to play only marked tournament moves)
Time control 20s+0.25s, 1000 games, 2 cores, Ponder Off, 256MB (per core it seems? Taskmanager says so), Syzygy 5 men, WB 4.80b, adjudication at move 120, manual result correction (BTW no time losses)
# PLAYER : RATING ERROR POINTS PLAYED (%)
1 StockfishASM_150916AB-64 : 3247.81 9.02 259.0 411 63.0
2 StockfishASM_150916NB-64 : 3152.19 9.02 152.0 411 37.0
White advantage = 52.87 +/- 9.37
Draw rate (equal opponents) = 76.32 % +/- 2.37
Head to head statistics:
1) StockfishASM_150916AB-64 3247.81 : 411 (+125,=268,-18), 63.0 %
vs. : games ( +, =, -), (%) : Diff, SD, CFS (%)
StockfishASM_150916NB-64 : 411 ( 125, 268, 18), 63.0 : +95.61, 9.21, 100.0
-17 for NB SFAsm against GS_medium
-100(ca.) against Noomen2016
More detailed data in the next days.
I finished my testing of Stockfish with BookX against Stockfish with a generic 2-move book for randomization of the openings. My conclusion would be: the importance of the opening book diminishes with longer time control, but it's not clear whether it's due to generally higher draw rate, Wins/Losses ratio remains pretty constant.
Book advantage:
20s + 0.2s --> 111 ELO points 400 games
360s + 3.6s --> 59 ELO points 200 games
3600s + 36s --> 35 ELO points 60 games
The last 3600s + 36s match finished +6 -0 =54, no-book Stockfish didn't manage a single win in 60 games.
For curiosity, I had a book competition, Brainfish book blows away the strongest of Polyglot books:
I have a question.
Up to what Move Number is it safe to use even a strong Opening Book in a LTC game ?
I heard most of the Books around have been tuned for STC games.
I'd like to avoid the unpleasant surprise of coming out of a supposedly strong Book and finding myself -1 or -2 down in a LTC game (while my opponent had come out of Book many moves back ).
Laskos wrote: My conclusion would be: the importance of the opening book diminishes with longer time control, but it's not clear whether it's due to
I should have thought that the reason was obvious.
In LTC game, the Engine has more time to figure out the best move, which may be as good if not better than the Book move.
While in an STC or Blitz game, the Engine has barely a few seconds at the most to find the best move, so a good Opening Book is of more value.
Only logical I should think.
Hence my question above.
"Now a month after the release and much feedback, I have a better understanding of it.
With a time of 1 minute for the whole game, Cerebellum is of little value, because Stockfish often is not able to continue correctly after the last move out of the book, which is in many cases calculated with a time of about 1 minute on a fast quadcore. And with that time, the book can easily be outplayed by a statistical book simply by playing many 1 minute games, and collecting statistical data about the errors Stockfish is making st that time.
Around 5 minutes is the break even point with statistical books, and above that, for example at 12+1, is where Cerebellum really begins to shine, that's what the results showed so far."
"at 12+1, is where Cerebellum really begins to shine, that's what the results showed so far."
The Engine Masters say otherwise.
What do they say? I had a mediocre result too at 1 hour + increment time control with Cerebellum book against a generic 2-mover, +3 -0 =37, no better than MyFrineds book, but it might have been a statistical fluke.
"at 12+1, is where Cerebellum really begins to shine, that's what the results showed so far."
The Engine Masters say otherwise.
That's not so obvious as it my seem. For example I myself (Gilgamesch) playing with Brainfish and Cerebellum ended 4th place on a quadcore, that should be ok.
The only other person which I know for sure that he uses Brainfish + Cerebellum was Longin on 15th place, with a 4 or 6 core I think.
Some people are using only Brainfish but not Cerebellum, because it is a bit faster than most Stockfish compiles or because it has Numa support. Or they are using Cerebellum with an additional Gui book.
The players on the last two places used Brainfish, but as far as I can see from checking their games, with an additional Gui book, so that is something different.
Hi Thomas
When are you going to release the Commercial version ?
Right now every Tom, Dick and Harry is using your excellent Cerebellum book, which you make with so much effort, for free....gives me no advantage.
People who are prepared to pay money, should get an advantage over those who want only free gifts !
Thomas Zipproth wrote:Some people are using only Brainfish but not Cerebellum, because it is a bit faster than most Stockfish compiles or because it has Numa support. Or they are using Cerebellum with an additional Gui book.
The players on the last two places used Brainfish, but as far as I can see from checking their games, with an additional Gui book, so that is something different.
I assumed they were using the whole package, using a subpar book instead of Cerebellum never crossed my mind. More data is needed if actually just two players were using it correctly. For the moment, i retract my statement.
