TalkChess.com

jp wrote: ↑Mon Mar 04, 2019 11:52 am

Gary Internet wrote: ↑Mon Mar 04, 2019 8:07 am All they did was change the time control from 7200+15 to 720+3 and remove the opening book.

What does "remove the opening book" mean? 100 games from the initial start position with no book for SF?
There are huge problems with doing that.

I totally agree.

Leo wrote: ↑Mon Mar 04, 2019 6:58 pm

jp wrote: ↑Mon Mar 04, 2019 11:52 am

Gary Internet wrote: ↑Mon Mar 04, 2019 8:07 am All they did was change the time control from 7200+15 to 720+3 and remove the opening book.

What does "remove the opening book" mean? 100 games from the initial start position with no book for SF?
There are huge problems with doing that.

I totally agree.

Problems for who? TCEC ran a 100 game match between the same versions of Lc0 and Stockfish that played in the superfinal. They used the same hardware but reduced the time control from 7200+15 to 720+3. Yes, every game started from the standard initial start position. From what I saw, and I watched a fair bit of the match, it ran very smoothly.

Gary Internet wrote: ↑Tue Mar 05, 2019 7:09 am

Leo wrote: ↑Mon Mar 04, 2019 6:58 pm

jp wrote: ↑Mon Mar 04, 2019 11:52 am What does "remove the opening book" mean? 100 games from the initial start position with no book for SF?
There are huge problems with doing that.

I totally agree.

Problems for who?

Problems getting chess results that mean anything. It's totally flawed.

I didn't mean TCEC would e.g. have problems with computer crashes, etc. I'm sure we can all run meaningless things smoothly.

Gary Internet wrote: ↑Tue Mar 05, 2019 7:09 am

Leo wrote: ↑Mon Mar 04, 2019 6:58 pm

jp wrote: ↑Mon Mar 04, 2019 11:52 am

Gary Internet wrote: ↑Mon Mar 04, 2019 8:07 am All they did was change the time control from 7200+15 to 720+3 and remove the opening book.

What does "remove the opening book" mean? 100 games from the initial start position with no book for SF?
There are huge problems with doing that.

I totally agree.

Problems for who? TCEC ran a 100 game match between the same versions of Lc0 and Stockfish that played in the superfinal. They used the same hardware but reduced the time control from 7200+15 to 720+3. Yes, every game started from the standard initial start position. From what I saw, and I watched a fair bit of the match, it ran very smoothly.

It was discussed on topic of DeemMind paper here too. 100 games from the same position, moreover from the sole starting position Lc0 is trained with, is a very bad practice. The games have large coorrelation as outcomes go due to the sole starting chosen position, and favor Lc0. Even playing from 2moves_v1.pgn, on which SF is trained on, is not that bad, as while maybe favoring SF, they are at least diverse. I usually test on 3-4 mover pgn files of GM games, a thing favoring Lc0 a bit compared to 2moves_v1.pgn openings, but in my view are fair, nobody serious will play most of these 2-mover often silly positions of SF testing framework.

TCEC is just an entertainment venue for chess engine lovers. It probably will not give you the most accurate indication of the strongest chess engine as the opening positions are designed to be unbalanced to make as many decisive games as possible. One of the main reasoning behind opening book choices is to see what year the books caused the percentage of the most decisive results. I think for Lc0 it would be more advantageous if equal opening positions were chosen ... and thus you could say that by choosing certain opening positions, you could dictate who wins TCEC. Obviously that was not the case in TCEC 14 and I am sure no one had a clue which opening lines would favor which engine. I think SF deserved to win as if Lc0 had won with so many glaring weaknesses that even 2000 ELO engines don't have ... well that would have been scandalous. In the end the more mature engine won out (just barely) and what is clear is that once Lc0 fixes a few of those weaknesses, it will easily beat any engine out there in any opening position choice. Another fact that works in Lc0 favor is that the hardware for AI is just getting started ... I expect a dramatic improvement in AI hardware which obviously would tilt the balance even more so in Lc0 favor.

M ANSARI wrote: ↑Tue Mar 05, 2019 8:40 am I think SF deserved to win as if Lc0 had won with so many glaring weaknesses that even 2000 ELO engines don't have ... well that would have been scandalous. In the end the more mature engine won out (just barely) and what is clear is that once Lc0 fixes a few of those weaknesses, it will easily beat any engine out there in any opening position choice.

There are ideas how to fix Lc0's endgame weakness, but we don't know if they'll work and they are non-zero ideas. If they (TCEC, etc.) give Lc0 TBs, obviously there's no reason not to give other engines opening books.

As you say, it's about entertainment for CCC & TCEC, and entertainment means handicapping to make it all close.

I would like to see a match that allows both engines to use any book they want. Like a freestyle match. I think SF would do well if its given a powerful book.

Theoretically, a powerful book will beat any other engine without a powerful book. I think a powerful book takes away from an engine playing chess as it depends completely on the book maker's ability. Better would be to have an opening and both sides have to play colors reversed.

Guenther wrote: ↑Mon Mar 04, 2019 12:46 pm
I have a script, which creates nice column style traditional pgn output from the TCEC format.
The file is attached. Note that I was too lazy to work around the unusual 'timeleft', thus only score/depth is included.

Can you please share the script ? It looks great. The pgn is so easily readable.