Knight equals 48 pawns?
Moderators: hgm, Rebel, chrisw
-
- Posts: 5960
- Joined: Sun Jan 10, 2010 6:15 am
- Location: Maryland USA
Knight equals 48 pawns?
I downloaded the latest Lc0 about a week ago on my laptop (with GPU 1060), and looked at its analysis (infinite mode) on lots of positions. Typically, in the opening evals are roughly double what they should be; in general a clean pawn up shows as about two pawns. However, the initial position with knight odds (b1 off) shows a fairly stable evaluation of around -48 pawns!!! Even if we divide by two (since the evals are double what they should be in normal positions), we get one knight = 24 pawns!! Moreover changing the knight to the g1 knight or to a bishop only made the score even more lopsided. I think that mathematically this is due to using the arctangent in the formula, but why would Lc0 choose to display such obvious nonsense, and is there any version that displays sensible evals? It shouldn't affect the play, just the display.
Komodo rules!
-
- Posts: 1797
- Joined: Thu Sep 18, 2008 10:24 pm
Re: Knight equals 48 pawns?
It's definitely not showing probability odds, like Alpha Zero?
-
- Posts: 5960
- Joined: Sun Jan 10, 2010 6:15 am
- Location: Maryland USA
Re: Knight equals 48 pawns?
Nothing wrong with converting win probs to scores in pawn units, we do the same in Komodo MCTS. But we don't output nonsense scores like this. Looks like no one bothered to check whether the conversion formula made any sense.
Komodo rules!
-
- Posts: 10948
- Joined: Wed Jul 26, 2006 10:21 pm
- Full name: Kai Laskos
Re: Knight equals 48 pawns?
Sure arctan use and the calibration. What was with logistic a 4.00 eval meaning some very roughly exp(-eval) or here exp(-4) performance, now is 1/eval or 1/48 performance (very roughly). The tails of logistic are exponentially diminishing, of arctan inversely linearly (hyperbolic) diminishing. Maybe it's better to use with Leela the performance % display. I am not even sure when this arctan display became popular, earlier most engines showed some "real pawns", and the eval obeyed pretty well the logistic in this case (some engines like Texel even tuned the eval to logistic).lkaufman wrote: ↑Wed Feb 13, 2019 7:23 pm I downloaded the latest Lc0 about a week ago on my laptop (with GPU 1060), and looked at its analysis (infinite mode) on lots of positions. Typically, in the opening evals are roughly double what they should be; in general a clean pawn up shows as about two pawns. However, the initial position with knight odds (b1 off) shows a fairly stable evaluation of around -48 pawns!!! Even if we divide by two (since the evals are double what they should be in normal positions), we get one knight = 24 pawns!! Moreover changing the knight to the g1 knight or to a bishop only made the score even more lopsided. I think that mathematically this is due to using the arctangent in the formula, but why would Lc0 choose to display such obvious nonsense, and is there any version that displays sensible evals? It shouldn't affect the play, just the display.
-
- Posts: 126
- Joined: Thu Jun 05, 2014 5:29 am
- Location: Malaysia
Re: Knight equals 48 pawns?
Someone already opened an issue and proposed a better conversion formula on Lc0 Github a few days ago.
https://github.com/LeelaChessZero/lc0/issues/728
https://github.com/LeelaChessZero/lc0/issues/728
-
- Posts: 1797
- Joined: Thu Sep 18, 2008 10:24 pm
Re: Knight equals 48 pawns?
Test 50 looks like it might change this
https://medium.com/@veedrac/leela-chess ... 5896becfac
-
- Posts: 1627
- Joined: Thu Mar 09, 2006 12:35 pm
Re: Knight equals 48 pawns?
Leela's evals should not be considered with normal thinking about pieces values and connections between Pawns-Knights etc but only as probabilities of winning/drawing.lkaufman wrote: ↑Wed Feb 13, 2019 7:23 pm I downloaded the latest Lc0 about a week ago on my laptop (with GPU 1060), and looked at its analysis (infinite mode) on lots of positions. Typically, in the opening evals are roughly double what they should be; in general a clean pawn up shows as about two pawns. However, the initial position with knight odds (b1 off) shows a fairly stable evaluation of around -48 pawns!!! Even if we divide by two (since the evals are double what they should be in normal positions), we get one knight = 24 pawns!! Moreover changing the knight to the g1 knight or to a bishop only made the score even more lopsided. I think that mathematically this is due to using the arctangent in the formula, but why would Lc0 choose to display such obvious nonsense, and is there any version that displays sensible evals? It shouldn't affect the play, just the display.
Leela's evals are just a somewhat arbitrary function that maps her probabilities for draw/win to centiPawns and it's(assuming the weaker side has 0 probability to win):
http://pasted.co/1a396b09
After his son's birth they've asked him:
"Is it a boy or girl?"
YES! He replied.....
"Is it a boy or girl?"
YES! He replied.....
-
- Posts: 89
- Joined: Sat Sep 13, 2014 4:12 pm
- Location: Zagreb, Croatia
- Full name: Branko Radovanović
Re: Knight equals 48 pawns?
Compare this to Houdini (http://www.cruxis.com/chess/manual/inde ... istory.htm):George Tsavdaris wrote: ↑Thu Feb 14, 2019 11:59 amLeela's evals should not be considered with normal thinking about pieces values and connections between Pawns-Knights etc but only as probabilities of winning/drawing.lkaufman wrote: ↑Wed Feb 13, 2019 7:23 pm I downloaded the latest Lc0 about a week ago on my laptop (with GPU 1060), and looked at its analysis (infinite mode) on lots of positions. Typically, in the opening evals are roughly double what they should be; in general a clean pawn up shows as about two pawns. However, the initial position with knight odds (b1 off) shows a fairly stable evaluation of around -48 pawns!!! Even if we divide by two (since the evals are double what they should be in normal positions), we get one knight = 24 pawns!! Moreover changing the knight to the g1 knight or to a bishop only made the score even more lopsided. I think that mathematically this is due to using the arctangent in the formula, but why would Lc0 choose to display such obvious nonsense, and is there any version that displays sensible evals? It shouldn't affect the play, just the display.
Leela's evals are just a somewhat arbitrary function that maps her probabilities for draw/win to centiPawns and it's(assuming the weaker side has 0 probability to win):
http://pasted.co/1a396b09
So, 75% chance of winning corresponds to +1 eval for Houdini and +6.7 eval for Leela, if I'm interpreting this correctly. That would indeed be highly distorted.The evaluations have again been calibrated to correlate directly with the win expectancy in the position. A +1.00 pawn advantage gives a 75% chance of winning the game against an equal opponent at blitz time control. At +1.50 the engine will win 90% of the time, and at +2.50 about 99% of the time. To win nearly 50% of the time, you need an advantage of about +0.60 pawn.
-
- Posts: 5960
- Joined: Sun Jan 10, 2010 6:15 am
- Location: Maryland USA
Re: Knight equals 48 pawns?
It's even worse than this I think. The 75% figure for winning when up one pawn is not 75% score exp., but actual wins. If most of the other 25% are draws (as would surely be the case) the actual scoring percentage expectancy when a pawn up is about 85%, not 75%. The conversion formula should be adjusted accordingly. I think the figures are pretty similar for Komodo, although it varies a bit from version to version, and the time limit makes some difference too.
Komodo rules!
-
- Posts: 546
- Joined: Sat Aug 17, 2013 12:36 am
Re: Knight equals 48 pawns?
So you disagree that the side starting the game a knight down is likely objectively lost?