Stockfish Beats LC0 to Win 2019 Chess.com Tourney

[Laskos]

I also think that in TCEC the +10 rule should be changed as +10 for Lc0 does not mean the same as +10 for Komodo or SF or Houdini and on many occasions Lc0 will not find the win even if it is +10 ahead.

Yes, I saw the danger with adjudication and Lc0, so all the matches I've played so far have been with adjudication turned off completely. Makes for long tortuous games sometimes that is for sure. I'll find a compromise perhaps.

Yes, this is a big problem with TCEC. A compromise is to have draw adjudication rules but play out wins to checkmate or a TB win.

I think this is a problem with Leela, not TCEC. The fact that her wpct -> cp formula results in such absurd numbers which don't fit the rather rough values that other engines output are the issue here, not the +10 for 10 rule. If you are -10 for 10 ply you should be hopelessly lost and if not then there is an issue (eg #fortress) with your eval.

The rule used to be -6 for a couple of moves iirc. That worked completely fine and nobody really complained with the very rare exception when the engines both clearly saw the win, but your average chess joe did not. I don't think I myself ever encountered a winning result under those rules that I had to really disagree with. Now can you imagine Leela under those rules?

Imo there is no justification whatsoever to not adjust the formula. You can complain about CP not meaning anything, you don't have to like CP, you don't have to use CP when you run the program at home, but it is a standard that people are used to and you would like to be on a similar scale to all the other engines that added this output.

Well, who came there with arctangent formula? While almost all other engines have some kind of logistic if CP measure is roughly kept? Were they not aware of that? These have very different tails (high evals regions). I wrote several times about this issue here, but they don't seem to read often CCC (or they don't care).

[Modern Times]

No complaints from me, just recognising that Lc0 is different and thus you have to take that into account in certain situations like adjudication.

[jorose]

No complaints from me, just recognising that Lc0 is different and thus you have to take that into account in certain situations like adjudication.

When an engine has a faulty UCI implementation you don't adjust the rules of the event to accommodate it, the engine with the faulty implementation just has to live with the consequences. If the faulty implementation ends up benefitting the engine in some way, then you have to take a look at the rules of the event.

I also think that in TCEC the +10 rule should be changed as +10 for Lc0 does not mean the same as +10 for Komodo or SF or Houdini and on many occasions Lc0 will not find the win even if it is +10 ahead.

Yes, I saw the danger with adjudication and Lc0, so all the matches I've played so far have been with adjudication turned off completely. Makes for long tortuous games sometimes that is for sure. I'll find a compromise perhaps.

Yes, this is a big problem with TCEC. A compromise is to have draw adjudication rules but play out wins to checkmate or a TB win.

I think this is a problem with Leela, not TCEC. The fact that her wpct -> cp formula results in such absurd numbers which don't fit the rather rough values that other engines output are the issue here, not the +10 for 10 rule. If you are -10 for 10 ply you should be hopelessly lost and if not then there is an issue (eg #fortress) with your eval.

The rule used to be -6 for a couple of moves iirc. That worked completely fine and nobody really complained with the very rare exception when the engines both clearly saw the win, but your average chess joe did not. I don't think I myself ever encountered a winning result under those rules that I had to really disagree with. Now can you imagine Leela under those rules?

Imo there is no justification whatsoever to not adjust the formula. You can complain about CP not meaning anything, you don't have to like CP, you don't have to use CP when you run the program at home, but it is a standard that people are used to and you would like to be on a similar scale to all the other engines that added this output.

Well, who came there with arctangent formula? While almost all other engines have some kind of logistic if CP measure is roughly kept? Were they not aware of that? These have very different tails (high evals regions). I wrote several times about this issue here, but they don't seem to read often CCC (or they don't care).

I just took a look at the recent commits and it seems there was a commit addressing the output. https://github.com/LeelaChessZero/lc0/pull/860

In my opinion this is much better. If my understanding is correct than the centipawn evals in the low win percent range have been increased, but the evals in the higher win percent range have been significantly reduced. I am rather ambivalent in the case of the former as I never really had an issue with the evals it output there and the tails look an order of magnitude better, even if still on the high side relative to SF.

[Leo]

Nice win for Stockfish.

[whereagles]

didn't Kai once made a flowchart of komodo eval into wins? that should make it possible to transform an eval into win percentage and its reverse

[ChiefPushesWood]

For those interested, the new client for Leela (released a couple of weeks ago but not in time for TCEC) has already adjusted her WP/CP formulas to be more inline with SF and Kom. They also worked (lightly) on making her more direct in going for wins, etc. Still more work to do in this area but the new eval display is fine now.

CPW

[jp]

didn't Kai once made a flowchart of komodo eval into wins? that should make it possible to transform an eval into win percentage and its reverse

That might apply just to Komodo.

[Nordlandia]

cutechess gui and cli support two sided adjudication. So insane evals don't matter much in the late endgame.

[Modern Times]

cutechess gui and cli support two sided adjudication. So insane evals don't matter much in the late endgame.

Yes - I'm going to switch to that. Currently my Lc0 games are played under Shredder GUI with adjudication off totally.

[jp]

cutechess gui and cli support two sided adjudication. So insane evals don't matter much in the late endgame.

Yes - I'm going to switch to that. Currently my Lc0 games are played under Shredder GUI with adjudication off totally.

Two-sided adjudication won't help when the evals are sane but Lc0 just doesn't have the technique to convert the win, which for older NNs was very often (I don't know about the latest ones).