First move advantage

Discussion of anything and everything relating to chess playing software and machines.

Moderator: Ras

Lyudmil Tsvetkov
Posts: 6052
Joined: Tue Jun 12, 2012 12:41 pm

First move advantage

Post by Lyudmil Tsvetkov »

Looking at the games of the match between Houdini and Komodo, run by Clemens Keck (http://www.clemens-keck.de/ReplayZone/1010.htm ), I notice that the engines have somewhat different evaluation of the general starting position (or openings very close to the starting position, probably deviating only slightly from perfect play): barring some exceptions, like Houdini estimating very well white's chances in the Caro-Kann and the Petrov (some +35 centipawns), Houdini sticks to an evaluation of some 9-12 centipawns in favour of white, while Komodo thinks white's advantage is somewhere around 22-25 centipawns.
I remember also that Rybka exhibited generally some 25 centipawns in white's favour.

Me, personally, I think that objectively white's advantage should be estimated at some 14-15 centipawns (do not ask me how I am trying to guess this, it is purely intuitive, although a while ago I experimented some erratic calculations). So, if this is anywhere near about the truth (most probably very very far from it), Houdini in general should be a bit closer to such an estimate.

What is your opinion on the matter? Maybe you have some relevant data, estimates, etc. I think this is very important for a top program to have a consistent evaluation and might even have a bearing on other stages of the game.

Best regards, Lyudmil

PS. I hope this posting is not to wild to answer :)
User avatar
Eelco de Groot
Posts: 4743
Joined: Sun Mar 12, 2006 2:40 am
Full name:   Eelco de Groot

Re: First move advantage

Post by Eelco de Groot »

Well you see, for Larry as a grandmaster and senior champion, Larry does a lot of his opening preparation and study also with Komodo, he has stated since his Rybka times that he finds this correlation between human evaluations and computer evaluations very important, he and Don work hard at it in Komodo.

Stockfish on the other hand, to name another program, is developed with very little of this kind of input, and we don't have grandmasters on the team, as far as I know. Endusers have to really get used to its evaluations being a bit different from at least Rybka, Houdini and Komodo I think. But for the program itself, it only becomes a problem if its (mis)evaluations would make it give up any long term advantages. The material calculations in the short term, apart from horizon effects are never a problem, compared to most human play I mean. It is true that some positional characteristics are really very hard to judge for most programs, and maybe the commercials are working very hard on that. But as long as the program would scale all its positional evalterms the same way (in relation to each other), at least roughly, it does not really matter if it seems in general overestimating or underestimating things, as long as it does not give up good long term traits for worse ones. So for all evaluations staying under a pawn, giving up a pawn too soon because of misevaluations is not an extreme risk. And some of the other long term traits, well the program simply does not understand those anyway 8-).

And for the people who have studied Fruit it is not a secret: some of the remaining misevaluations will disappear, as by magic, as the game progresses, solving some of the problems literally automatically. That is because since the time of Fruit, but I think Fruit was not really the first to do this, all the eval terms have both a middlegame and endgame component. And based on some, maybe crude, but effective, rules depending mostly on the material situation, every term is then scaled between those two components as the game inevitably progresses towards less material, towards the endgames. An opening book helps with the very early stages. Of course it remains extremely important to tune everything for maximum effectiveness and minimum blunders/horizon effects, but that already does not have much to do anymore with how the grandmaster looks at a position, how all the things he has learned are balanced out on the board. The total sum of the terms is about the same but the grandmaster adds different terms to come to his conclusion than the computerprogram. In the end the silicon grandmasters are still mostly counting beans in the eyes of human topgrandmastters I think.

Hope this gives some programming perspective to your question!
Regards, Eelco
Debugging is twice as hard as writing the code in the first
place. Therefore, if you write the code as cleverly as possible, you
are, by definition, not smart enough to debug it.
-- Brian W. Kernighan
lkaufman
Posts: 6304
Joined: Sun Jan 10, 2010 6:15 am
Location: Maryland USA
Full name: Larry Kaufman

Re: First move advantage

Post by lkaufman »

Lyudmil Tsvetkov wrote:Looking at the games of the match between Houdini and Komodo, run by Clemens Keck (http://www.clemens-keck.de/ReplayZone/1010.htm ), I notice that the engines have somewhat different evaluation of the general starting position (or openings very close to the starting position, probably deviating only slightly from perfect play): barring some exceptions, like Houdini estimating very well white's chances in the Caro-Kann and the Petrov (some +35 centipawns), Houdini sticks to an evaluation of some 9-12 centipawns in favour of white, while Komodo thinks white's advantage is somewhere around 22-25 centipawns.
I remember also that Rybka exhibited generally some 25 centipawns in white's favour.

Me, personally, I think that objectively white's advantage should be estimated at some 14-15 centipawns (do not ask me how I am trying to guess this, it is purely intuitive, although a while ago I experimented some erratic calculations). So, if this is anywhere near about the truth (most probably very very far from it), Houdini in general should be a bit closer to such an estimate.

What is your opinion on the matter? Maybe you have some relevant data, estimates, etc. I think this is very important for a top program to have a consistent evaluation and might even have a bearing on other stages of the game.

Best regards, Lyudmil

PS. I hope this posting is not to wild to answer :)

I can answer your question rather precisely. I have two Aquarium IDeA projects going, one using Houdini 3, the other Komodo. Both have very deeply analyzed all the openings (to around moves 15 to 20 usually) of all games played at 2700 Elo average and above in recent times, plus many others. The resultant minimaxed tree gives the opinion of each program as to the true advantage of the White pieces as well as of every opening. Both Komodo and Houdini are showing +.15 for the initial position (there is some rounding done by Aquarium, this isn't an exact figure), so basically you are right on the money. Currently Komodo gives both 1.d4 and 1.e4 as +.15 and 1.Nf3 as +.11, while Houdini gives 1.d4 and 1.Nf3 as +.15 and 1.e4 as +.11. Against 1.e4 Komodo gives French as best and the other "big three" (1...e5, Sic., Caro) as equal second. Houdini gives French and Sicilian as equal best, 1...e5 as third, but doesn't like the Caro. Against 1.d4 they both rate Slav and Gruenfeld as the two best defenses (K prefers Slav slightly, H prefers Gruenfeld slightly.
So basically, there is remarkably little difference between their evaluations when done this way.
Lyudmil Tsvetkov
Posts: 6052
Joined: Tue Jun 12, 2012 12:41 pm

Re: First move advantage

Post by Lyudmil Tsvetkov »

lkaufman wrote:
Lyudmil Tsvetkov wrote:Looking at the games of the match between Houdini and Komodo, run by Clemens Keck (http://www.clemens-keck.de/ReplayZone/1010.htm ), I notice that the engines have somewhat different evaluation of the general starting position (or openings very close to the starting position, probably deviating only slightly from perfect play): barring some exceptions, like Houdini estimating very well white's chances in the Caro-Kann and the Petrov (some +35 centipawns), Houdini sticks to an evaluation of some 9-12 centipawns in favour of white, while Komodo thinks white's advantage is somewhere around 22-25 centipawns.
I remember also that Rybka exhibited generally some 25 centipawns in white's favour.

Me, personally, I think that objectively white's advantage should be estimated at some 14-15 centipawns (do not ask me how I am trying to guess this, it is purely intuitive, although a while ago I experimented some erratic calculations). So, if this is anywhere near about the truth (most probably very very far from it), Houdini in general should be a bit closer to such an estimate.

What is your opinion on the matter? Maybe you have some relevant data, estimates, etc. I think this is very important for a top program to have a consistent evaluation and might even have a bearing on other stages of the game.

Best regards, Lyudmil

PS. I hope this posting is not to wild to answer :)

I can answer your question rather precisely. I have two Aquarium IDeA projects going, one using Houdini 3, the other Komodo. Both have very deeply analyzed all the openings (to around moves 15 to 20 usually) of all games played at 2700 Elo average and above in recent times, plus many others. The resultant minimaxed tree gives the opinion of each program as to the true advantage of the White pieces as well as of every opening. Both Komodo and Houdini are showing +.15 for the initial position (there is some rounding done by Aquarium, this isn't an exact figure), so basically you are right on the money. Currently Komodo gives both 1.d4 and 1.e4 as +.15 and 1.Nf3 as +.11, while Houdini gives 1.d4 and 1.Nf3 as +.15 and 1.e4 as +.11. Against 1.e4 Komodo gives French as best and the other "big three" (1...e5, Sic., Caro) as equal second. Houdini gives French and Sicilian as equal best, 1...e5 as third, but doesn't like the Caro. Against 1.d4 they both rate Slav and Gruenfeld as the two best defenses (K prefers Slav slightly, H prefers Gruenfeld slightly.
So basically, there is remarkably little difference between their evaluations when done this way.
Hi Larry.
Thanks for answering, and congrats on Komdo btw.
I still have problems with making compatible the results of your Aquarium tests and the evals of both top engines I see on Clemens Keck's site.

For Komodo specifically I see:
+0.23 after 1.e4 c5 2.Nf3 Nc6 3.d4 cxd4
+0.23 after 1.e4 c6 2.d4 d5 3.e5
+0.19 after 1.d4 d5
+0.22 after 1.d4 Nf6 2.c4
+0.26 after 1.e4 e5 2.Nf3 Nf6 3.d4
+0.21 after 1.e4 e5 2.Nf3 Nc6 3.Bb5 a6
+0.27 after 1.d4 Nf6 2.c4,
etc.

And the engine basically sticks to those values for quite some time (meaning fairly good consistency).

For me, this seems quite different from the 0.15 values you have stated.

I think you understand very well the implications of couple of centipawns' difference that could gradually transpose to a more significant one.

Best, Lyudmil
lkaufman
Posts: 6304
Joined: Sun Jan 10, 2010 6:15 am
Location: Maryland USA
Full name: Larry Kaufman

Re: First move advantage

Post by lkaufman »

Lyudmil Tsvetkov wrote:
lkaufman wrote:
Lyudmil Tsvetkov wrote:Looking at the games of the match between Houdini and Komodo, run by Clemens Keck (http://www.clemens-keck.de/ReplayZone/1010.htm ), I notice that the engines have somewhat different evaluation of the general starting position (or openings very close to the starting position, probably deviating only slightly from perfect play): barring some exceptions, like Houdini estimating very well white's chances in the Caro-Kann and the Petrov (some +35 centipawns), Houdini sticks to an evaluation of some 9-12 centipawns in favour of white, while Komodo thinks white's advantage is somewhere around 22-25 centipawns.
I remember also that Rybka exhibited generally some 25 centipawns in white's favour.

Me, personally, I think that objectively white's advantage should be estimated at some 14-15 centipawns (do not ask me how I am trying to guess this, it is purely intuitive, although a while ago I experimented some erratic calculations). So, if this is anywhere near about the truth (most probably very very far from it), Houdini in general should be a bit closer to such an estimate.

What is your opinion on the matter? Maybe you have some relevant data, estimates, etc. I think this is very important for a top program to have a consistent evaluation and might even have a bearing on other stages of the game.

Best regards, Lyudmil

PS. I hope this posting is not to wild to answer :)

I can answer your question rather precisely. I have two Aquarium IDeA projects going, one using Houdini 3, the other Komodo. Both have very deeply analyzed all the openings (to around moves 15 to 20 usually) of all games played at 2700 Elo average and above in recent times, plus many others. The resultant minimaxed tree gives the opinion of each program as to the true advantage of the White pieces as well as of every opening. Both Komodo and Houdini are showing +.15 for the initial position (there is some rounding done by Aquarium, this isn't an exact figure), so basically you are right on the money. Currently Komodo gives both 1.d4 and 1.e4 as +.15 and 1.Nf3 as +.11, while Houdini gives 1.d4 and 1.Nf3 as +.15 and 1.e4 as +.11. Against 1.e4 Komodo gives French as best and the other "big three" (1...e5, Sic., Caro) as equal second. Houdini gives French and Sicilian as equal best, 1...e5 as third, but doesn't like the Caro. Against 1.d4 they both rate Slav and Gruenfeld as the two best defenses (K prefers Slav slightly, H prefers Gruenfeld slightly.
So basically, there is remarkably little difference between their evaluations when done this way.
Hi Larry.
Thanks for answering, and congrats on Komdo btw.
I still have problems with making compatible the results of your Aquarium tests and the evals of both top engines I see on Clemens Keck's site.

For Komodo specifically I see:
+0.23 after 1.e4 c5 2.Nf3 Nc6 3.d4 cxd4
+0.23 after 1.e4 c6 2.d4 d5 3.e5
+0.19 after 1.d4 d5
+0.22 after 1.d4 Nf6 2.c4
+0.26 after 1.e4 e5 2.Nf3 Nf6 3.d4
+0.21 after 1.e4 e5 2.Nf3 Nc6 3.Bb5 a6
+0.27 after 1.d4 Nf6 2.c4,
etc.

And the engine basically sticks to those values for quite some time (meaning fairly good consistency).

For me, this seems quite different from the 0.15 values you have stated.

I think you understand very well the implications of couple of centipawns' difference that could gradually transpose to a more significant one.

Best, Lyudmil
Well, there is no mystery here. The scores you report are probably from a 25 to 30 ply search. My scores are from a similar (or deeper) search starting on average around ply 30! The deeper you search, the closer the score for a sound defense gets to the theoretical zero score for a draw.

Best,
Larry
Lyudmil Tsvetkov
Posts: 6052
Joined: Tue Jun 12, 2012 12:41 pm

Re: First move advantage

Post by Lyudmil Tsvetkov »

Eelco de Groot wrote:Well you see, for Larry as a grandmaster and senior champion, Larry does a lot of his opening preparation and study also with Komodo, he has stated since his Rybka times that he finds this correlation between human evaluations and computer evaluations very important, he and Don work hard at it in Komodo.

Stockfish on the other hand, to name another program, is developed with very little of this kind of input, and we don't have grandmasters on the team, as far as I know. Endusers have to really get used to its evaluations being a bit different from at least Rybka, Houdini and Komodo I think. But for the program itself, it only becomes a problem if its (mis)evaluations would make it give up any long term advantages. The material calculations in the short term, apart from horizon effects are never a problem, compared to most human play I mean. It is true that some positional characteristics are really very hard to judge for most programs, and maybe the commercials are working very hard on that. But as long as the program would scale all its positional evalterms the same way (in relation to each other), at least roughly, it does not really matter if it seems in general overestimating or underestimating things, as long as it does not give up good long term traits for worse ones. So for all evaluations staying under a pawn, giving up a pawn too soon because of misevaluations is not an extreme risk. And some of the other long term traits, well the program simply does not understand those anyway 8-).

And for the people who have studied Fruit it is not a secret: some of the remaining misevaluations will disappear, as by magic, as the game progresses, solving some of the problems literally automatically. That is because since the time of Fruit, but I think Fruit was not really the first to do this, all the eval terms have both a middlegame and endgame component. And based on some, maybe crude, but effective, rules depending mostly on the material situation, every term is then scaled between those two components as the game inevitably progresses towards less material, towards the endgames. An opening book helps with the very early stages. Of course it remains extremely important to tune everything for maximum effectiveness and minimum blunders/horizon effects, but that already does not have much to do anymore with how the grandmaster looks at a position, how all the things he has learned are balanced out on the board. The total sum of the terms is about the same but the grandmaster adds different terms to come to his conclusion than the computerprogram. In the end the silicon grandmasters are still mostly counting beans in the eyes of human topgrandmastters I think.

Hope this gives some programming perspective to your question!
Regards, Eelco
Hi Eelco.
Thanks for answering.
The many games with the corresponding evaluations of a variety of engines I have studied have quickly led me to the conclusion, that the more consistent an engine's evaluation is throughout different plies, the stronger it would play overall. I have very very rarely indeed observed the case of an engine with the less consistent evals faring better than its opponent. And for an engine to have a consistent score, you can not rely on a full pawn weights. In order to be consistent, you have to rely on taking into account very small assets and liabilities.

I observed that once with Rybka, and it is even truer now.

And, of course, you should not forget that a couple of centipawns advantage might easily and legitimately increase even with none of the sides making observable mistakes with the passage of time (obviously because the number of variations for the defending side keeping the score would gradually decrease with the disappearance of material).
Just my couple of cents.
User avatar
Ajedrecista
Posts: 2267
Joined: Wed Jul 13, 2011 9:04 pm
Location: Madrid, Spain.

Re: First move advantage.

Post by Ajedrecista »

Hello Larry:
lkaufman wrote:
Lyudmil Tsvetkov wrote:Looking at the games of the match between Houdini and Komodo, run by Clemens Keck (http://www.clemens-keck.de/ReplayZone/1010.htm), I notice that the engines have somewhat different evaluation of the general starting position (or openings very close to the starting position, probably deviating only slightly from perfect play): barring some exceptions, like Houdini estimating very well white's chances in the Caro-Kann and the Petrov (some +35 centipawns), Houdini sticks to an evaluation of some 9-12 centipawns in favour of white, while Komodo thinks white's advantage is somewhere around 22-25 centipawns.
I remember also that Rybka exhibited generally some 25 centipawns in white's favour.

Me, personally, I think that objectively white's advantage should be estimated at some 14-15 centipawns (do not ask me how I am trying to guess this, it is purely intuitive, although a while ago I experimented some erratic calculations). So, if this is anywhere near about the truth (most probably very very far from it), Houdini in general should be a bit closer to such an estimate.

What is your opinion on the matter? Maybe you have some relevant data, estimates, etc. I think this is very important for a top program to have a consistent evaluation and might even have a bearing on other stages of the game.

Best regards, Lyudmil

PS. I hope this posting is not to wild to answer :)

I can answer your question rather precisely. I have two Aquarium IDeA projects going, one using Houdini 3, the other Komodo. Both have very deeply analyzed all the openings (to around moves 15 to 20 usually) of all games played at 2700 Elo average and above in recent times, plus many others. The resultant minimaxed tree gives the opinion of each program as to the true advantage of the White pieces as well as of every opening. Both Komodo and Houdini are showing +.15 for the initial position (there is some rounding done by Aquarium, this isn't an exact figure), so basically you are right on the money. Currently Komodo gives both 1.d4 and 1.e4 as +.15 and 1.Nf3 as +.11, while Houdini gives 1.d4 and 1.Nf3 as +.15 and 1.e4 as +.11. Against 1.e4 Komodo gives French as best and the other "big three" (1...e5, Sic., Caro) as equal second. Houdini gives French and Sicilian as equal best, 1...e5 as third, but doesn't like the Caro. Against 1.d4 they both rate Slav and Gruenfeld as the two best defenses (K prefers Slav slightly, H prefers Gruenfeld slightly.
So basically, there is remarkably little difference between their evaluations when done this way.
Could you please share more results you have, please? I am curious about this:
lkaufman wrote:Houdini gives French and Sicilian as equal best, 1...e5 as third, but doesn't like the Caro.
As a Caro-Kann player (a very mediocre player by the way), I refuse to think that Caro-Kann is bad for black.

I am also a d4 player as white, so I would like to know the deep impressions of these engines for other defences (QGD, QGA, KID, QID, Nimzo-Indian, Catalan, etc.). Thanks in advance.

Congratulations for the new Komodo! It has done a good job at IPON. :)

Regards from Spain.

Ajedrecista.
lkaufman
Posts: 6304
Joined: Sun Jan 10, 2010 6:15 am
Location: Maryland USA
Full name: Larry Kaufman

Re: First move advantage.

Post by lkaufman »

Ajedrecista wrote:Hello Larry:
lkaufman wrote:
Lyudmil Tsvetkov wrote:Looking at the games of the match between Houdini and Komodo, run by Clemens Keck (http://www.clemens-keck.de/ReplayZone/1010.htm), I notice that the engines have somewhat different evaluation of the general starting position (or openings very close to the starting position, probably deviating only slightly from perfect play): barring some exceptions, like Houdini estimating very well white's chances in the Caro-Kann and the Petrov (some +35 centipawns), Houdini sticks to an evaluation of some 9-12 centipawns in favour of white, while Komodo thinks white's advantage is somewhere around 22-25 centipawns.
I remember also that Rybka exhibited generally some 25 centipawns in white's favour.

Me, personally, I think that objectively white's advantage should be estimated at some 14-15 centipawns (do not ask me how I am trying to guess this, it is purely intuitive, although a while ago I experimented some erratic calculations). So, if this is anywhere near about the truth (most probably very very far from it), Houdini in general should be a bit closer to such an estimate.

What is your opinion on the matter? Maybe you have some relevant data, estimates, etc. I think this is very important for a top program to have a consistent evaluation and might even have a bearing on other stages of the game.

Best regards, Lyudmil

PS. I hope this posting is not to wild to answer :)

I can answer your question rather precisely. I have two Aquarium IDeA projects going, one using Houdini 3, the other Komodo. Both have very deeply analyzed all the openings (to around moves 15 to 20 usually) of all games played at 2700 Elo average and above in recent times, plus many others. The resultant minimaxed tree gives the opinion of each program as to the true advantage of the White pieces as well as of every opening. Both Komodo and Houdini are showing +.15 for the initial position (there is some rounding done by Aquarium, this isn't an exact figure), so basically you are right on the money. Currently Komodo gives both 1.d4 and 1.e4 as +.15 and 1.Nf3 as +.11, while Houdini gives 1.d4 and 1.Nf3 as +.15 and 1.e4 as +.11. Against 1.e4 Komodo gives French as best and the other "big three" (1...e5, Sic., Caro) as equal second. Houdini gives French and Sicilian as equal best, 1...e5 as third, but doesn't like the Caro. Against 1.d4 they both rate Slav and Gruenfeld as the two best defenses (K prefers Slav slightly, H prefers Gruenfeld slightly.
So basically, there is remarkably little difference between their evaluations when done this way.
Could you please share more results you have, please? I am curious about this:
lkaufman wrote:Houdini gives French and Sicilian as equal best, 1...e5 as third, but doesn't like the Caro.
As a Caro-Kann player (a very mediocre player by the way), I refuse to think that Caro-Kann is bad for black.

I am also a d4 player as white, so I would like to know the deep impressions of these engines for other defences (QGD, QGA, KID, QID, Nimzo-Indian, Catalan, etc.). Thanks in advance.

Congratulations for the new Komodo! It has done a good job at IPON. :)

Regards from Spain.

Ajedrecista.
In the case of Caro, I think Komodo gets it right and Houidini gets it wrong. I know why, but you will understand that I'm not inclined to give free advice to Houdini.
I don't have the exact numbers handy, but in general terms, both engines agree pretty well. Nimzo is fine for Black, and the Catalan also okay, but if White avoids by 3.Nf3 all the options are a bit worse than Slav, Gruenfeld, or Nimzo. QID and QGD are about equal and a bit worse than the previously named ones. QGA a bit worse yet. KI in general gets the worst scores, especially if Black avoids the sideline 7...exd4 in the Classical. The MarDel Plata gets huge scores for White, but for practical play OTB this is rather misleading, as White must defend perfectly.

Larry
User avatar
Mike S.
Posts: 1480
Joined: Thu Mar 09, 2006 5:33 am

Re: First move advantage

Post by Mike S. »

The first move advantage as defined by Hans Berliner, is - very obviosly - 0.5 plies.

That's pretty clear and I do not understand any requirement of discussion about this particular fact.
Regards, Mike
lkaufman
Posts: 6304
Joined: Sun Jan 10, 2010 6:15 am
Location: Maryland USA
Full name: Larry Kaufman

Re: First move advantage

Post by lkaufman »

Mike S. wrote:The first move advantage as defined by Hans Berliner, is - very obviosly - 0.5 plies.

That's pretty clear and I do not understand any requirement of discussion about this particular fact.
That is obviously true but not helpful by itself. If three tempi equal a pawn (as many books say), then first move advantage is 1/6 of a pawn. Since in the opening a pawn value is usually defined as less than one average pawn, if we say 0.9 pawns then 1/6 of that is .15, the exact figure we agreed on in this thread!