mwyoung wrote: ↑Mon Jun 03, 2019 2:34 am
I want to know what each engine is able to bring with their best configuration.
The best configuration includes a Tournament Book, say, you hire Jeroen Noomen to produce a private book for Leela that goes into positions that the engine plays best, and that will be great against whatever opponent Leela is facing, then you will see her true strength. He did it for Rebel and Rybka when they were the best, so why not Leela?
The problem with TCEC, the CCRL, and other testing groups is that they're testing with generic books, and in those, 99% of the openings played are irrelevant.
Assume for a moment that 1.d4 is actually bad and Leela struggles with it or that the Sicilian is actually bad and Leela struggles with it, yet it's being forced to play into those openings and Stockfish reigns supreme, but a Tournament book would show that Leela can kick ass with just 1.e4 and 1...e5 against 1.e4, and that all the results of the generic openings she wouldn't play are irrelevant.
This is all made up, but you should get the point that minimal books or extensive opening books are not the answer.
You don't force Magnus Carlsen to forget all the chess theory he knows at move 4 and think by himself from there. You also don't force Magnus Carlsen and his opponents to play from a generic opening position they may not have much experience in. You let them play with a "Tournament Book" that is on their head thanks to their preparation. Why aren't engines treated the same?
All the testing of Stockfish and Leela has been nothing but charades, back in my day (...2007, yeah, I'm getting old) Zappa Mexico proved to be the most superior engine, over Rybka, under these conditions, proving that all the generic testing (where Rybka was the best) was a farce.