jp wrote: ↑Sun Mar 24, 2019 10:11 am
Ovyron wrote: ↑Sun Mar 24, 2019 10:03 am
Oh, I've been testing this privately since 2007. Not only that, whenever a new engine pops up, I don't care about its ELO at all, I use it to analyze my Correspondence games against the strongest opposition I can find, and the same analysis methods that can beat some opponent easily, can easily lose against them if one uses very weak engines in comparison with what they're using.
The correlation between an engine's ELO and the true quality of its move choices is extraordinarily high.
Very interesting. Have you tested Lc like that yet?
I tried. Terrible results. Okay, so I couldn't even run it in my GPU, so this is Leela running on a 9 year old CPU. The "Time/Quality Ratio" forced me to stick to Depth 8. Depth 9 is just unbearable (specially when 99% of the time Leela sticks to its D8 choice and you waste whole minutes that just accumulate.)
This is the most useless thing I've used, even more useless than the SMIRF chess engine (which, I bring up, because, years ago, I used the SMIRF chess engine with success to analyze my human games, it provided great ideas easy to understand and gave me some insights on the moves that were good in the kind of positions I used to play, back when I was rated 1100; now that I'm rated 1700, lichess automatic system is good enough for me. Well, I tried using SMIRF for my correspondence chess analysis and its move choices were terrible, the most terrible I had seen since then. It was providing worse move choices than Zillions! And Zillions is an engine that can play any game that you program into it, so at corr chess, SMIRF was surprisingly bad.)
I've gone and used engines that are at the bottom of the CCRL list, and even TSCP was better than Leela. Do you know why? Because, at least, those engines will not show some 2.00 winning score for moves in positions, unless the 2 pawn advantage is real, or at least they think it's real. At least for the common positions that you will find in normal games (all engines will give huge scores to dead drawn positions now and then, but that's rare.)
Leela? Oh no, Leela will go and say "oh! ooooh! Look at this move! It's totally winning! awesome! go play it!", and you can go very deep and wide examining it, while the opposing engine (say, Stockfish) thinks it's a blunder, and Leela will remain stuck on stupid with really high scores, until finally you play a move that busts its variation, and Leela will agree it's a blunder (with huge swing in score.)
Back on the root, Leela still says "okay, that move was bad... but wait! See this one! It's gonna win you the game!", but what does Stockfish say? "Blunder!" And guess who is right...
Examining variations with Leela is almost like tossing a coin to check what move to analyze, and, hey, I can examine random moves by myself. It reminds me of the crazy personalities I was creating for Pro Deo back in the day, it'd show some PV where it was giving away a rook and a knight just to maximize the number of checks it was giving, but if you forces the moves, it'd realize it had ran out of checks and come back to the reality that it was losing.
Leela is just tactically blind.
No doubt there could be some critical position where Leela is the only one that suggest the best move or something, but most positions are quiet (several moves are playable, but all of them are ok, and you need a plan), or easy (Stockfish on MultiPV will quickly show there's a best move clearly better than the rest). If I had a detector of critical positions I'd use it before anything else every time, but I don't, so exploring losing blunders with Leela (because she thinks they're winning) is a losing proposition.
Sorry if I sound angry, I'm not, I'm passionate, because I was hoping to get more from Leela. Maybe I'd need some high end graphics card to enjoy the benefits. Good luck to the project, and I'm looking forward to be defeated by a corr chess opponent that uses Leela while I don't, because it'll be very clear I'd be defeated by a "science fiction move" (as kingscrusher calls them), but I'm not even sure those exist at 50 day/game with 50 day increment every 10 moves time control, where I'm defeated because my opponents are understanding better than me what is going on the chessboard, and no amount of engines or hardware can change that.
Great spirits have always encountered violent opposition from mediocre minds.
