But they are about 14 times as much as they should be.
According to who?
I think it is fairly common for Lc0 to report scores in the balllpark of 10 pawns for positions where it cannot demonstrate a win against itself.
Perhaps for old nets produced before the temperature drop and TB re-scoring. And there may be some issues still due to stubborn admins protecting their fragile egos, but I don't think I would characterize the issue as fairly common. But that is just me....
jhellis3 wrote: ↑Sat Feb 16, 2019 6:15 pm
Criticizing LC0 for having a superior eval (more quickly recognizing known wins) seems a bit silly, especially when K itself reports scores of +250 for TB wins where it may only be a pawn up....
Or if you want an objective measure, have Komodo play white in 100 games (different openings) against LC0 a knight down.
What makes you believe Lc0 has a superior eval? Its evals have always been too optimistic.
jhellis3 wrote: ↑Sat Feb 16, 2019 6:15 pm
Criticizing LC0 for having a superior eval (more quickly recognizing known wins) seems a bit silly, especially when K itself reports scores of +250 for TB wins where it may only be a pawn up....
Or if you want an objective measure, have Komodo play white in 100 games (different openings) against LC0 a knight down.
What makes you believe Lc0 has a superior eval? Its evals have always been too optimistic.
A superior eval is identified by how properly it orders a set of positions by their desirability, not by how well the underlying scores of that function line up with some preconceived notion of value.
Of course not. I believe that the win/draw threshold on a fairly full board is somewhere around 0.7 eval by Komodo. Lc0 gives knight odds as about 1.3% win exp., which is probably a realistic figure for ultra-fast engine games but way too high for realistic time controls. The point is that Lc0 is NOT giving a 0 win exp or nearly so; it seems to give scores of about 0.3% when the position is obviously resignable even against a weak player. The fact that knight odds is winning for Black is irrelevant if Lc0 does not score it as such. Anyway for me it just means that Lc0 should only be used in win% display mode until the conversion formula is corrected.
Can someone tell me at what time control or node counts the win percentages are in general likely to be most accurate in Lc0?
jhellis3 wrote: ↑Sat Feb 16, 2019 6:15 pm
Criticizing LC0 for having a superior eval (more quickly recognizing known wins) seems a bit silly...
What makes you believe Lc0 has a superior eval? Its evals have always been too optimistic.
A superior eval is identified by how properly it orders a set of positions by their desirability, not by how well the underlying scores of that function line up with some preconceived notion of value.
But that hasn't been identified, has it?
What do you think is the preconceived notion of value? The knight value Larry mentioned?
I'll try to explain it a different way. Suppose we decided in the next Komodo release to multiply all the reported evals by ten. No moves would be chosen differently, the relative scores (as ratios) of all moves would be the same, but the relative scores in terms of numerical difference would be ten times as large. Users would be very upset with us, and rightly so. By definition, a loss of one pawn in a "typical" position should swing the reported score by 100 centipawns, otherwise they are not centipawns. You can argue about whether this should be a full board position, an endgame, or some average over all positions, but you can't call it a centipawn if you don't accept that a loss or win of a pawn should (in some normal situation) show up as a 100 centipawn score change.
I am becoming a fan of Lc0 myself, I already much prefer it over Stockfish in most situations prior to the endgame. I think it is very good at evaluating whether one position is better than another, and at estimating the winning percentage in middlegame positions. My only complaint here is that it should not pretend to be giving a centipawn evaluation unless it is generally true that winning a pawn will increase the score by about 100 centipawns. This is simple to correct and won't affect play, but it's not much of a concern for me now that I just use the win percentage display.
The UCI standard requires scores to be either reported as centiPawns or distance to mate. When LC0 counts an advantage of 4 Pawns (which would presumably enough to fully compensate the Knight, if not over-compensate it) as +4800, it is obviously non-compliant. EGT-derived scores are basically distance-to-mate score. If Komodo reports those as cP this would also be non-compliant.
Note that no matter how grave the sins of Komodo might be, it has absolutely no bearing on what would be acceptable from LC0.
