Castling long

[lucasart]

Castling is not only one way to reach a position that you could otherwise reach with artificial castling in many moves, but the eval staying the same; that might be the case, if you do not give bonuses for an existing option of castling. An existing option of castling would mean the possibility to go to a safe square. An unexisting option in the middlegame (when a rook or the king has already moved, or temporarily inexistant, like your excellent h6 knight from the game prevents the black king from castling short) would quite often already completely change the position characteristics.

Actually there is some castling right dependant stuff in the eval. A rook blocked by an uncastled king (even if the uncastled king is back on E7/E8) is penalized, especially if the file is not semi open (for a rook lift). So the penalty is really for the rook not being active and hard to activate, but it's specifically castling rights dependant.

Concerning square agnosticity of king safety in Discocheck, do not you have penalties for king in the center in the middlegame, squares on the first rank better than squares on the second, etc.? That might make some difference, although the attacking potential and the distance from the king remain the same.

Yes, there is a PST (piece on square bonus/malus table) for opening and endgame (using linear interpolation between the two based on remaining material). The PST(Opening) penalizes based on rank and file: optimal files are B and G (equal) and optimal rank is obviously 1 (for white), penalty increases quickly with rank, so king in the center is penalized. But it's agnositc to castling side. To give you a better idea:

Code: Select all

PST(Opening,King)=f(rank)+g(file), where g(B)=g(G), and f(rank)=constant*rank.

The King safety is a different feature from the PST. It only start to give penalties if there are enemy attacks around the king. Typiclly form of king safety is something quadratic like:

Code: Select all

nb_attacks * sum(weights of attacks)

There is in the end of my code something that adds the distance to safety

Code: Select all

nb_attacks := nb_attacks + min(distance to B1/B8, distance to G1/G8)

Concerning the vulnerability of the king on the queen side, what makes it more vulnerable than a king on the king side, apart from some possible tempo losses?

I don't know in general. Every position is different. But often a deep search prefers OO to OOO, and it seems easier and more effective to attack a long castled king than a short castled one. But yes, OOO is fine in some games, just not that many.

Good game. I think up until move 16 Discocheck is better, maybe winning.
That is why you should not blame long castling for the loss. 17. Qd2 is an obvious mistake. 17. Qe3 would do nice. Do you consider x-ray attacks in Discocheck? That might solve some of Disco's problems in the current game. Disco probably misses that the c6 black rook attacks c2 on an x-ray, and that actually wins the game. But I acknowledge that Fruit combination was exquisite, probably difficult to see.

Yes, Qd2 was a mistake. It was a blitz game in 5min+3sec. But the combination that Fruit managed to pull, only in a few seconds of search, is quite impressive, DiscoCheck completely missed it, and it took a while to see the score of DiscoCheck agree with the Fruit score.

Perhaps after Qe3 DiscoCheck was still fine, but still not as good as it was a few moves earlier IMO. At least OOO puts DiscoCheck on the defensive, in a difficult position where it is easy to blunder, and losing a tempo can be fatal.

Fruit's tactics are still impressive, especially for an engine from 2005!

[lucasart]

Majority of games
come from books or positions where already done castling.

Search Statistics
appears unreliable.
Unless the effect has been searched.

I composed a suite with the king in the center
to test my engine, I almost always make castling,
I have to re-prepare it lost "castling rights".

My statistics are based on a 5 move opening book where about 9% of white side has castling and about 14% of the black openings have castling.

I just made a 3 move (6 ply) opening book that has no castling - I'll look to see what kind of statistics Komodo produces without 10% of the castling moves being forced on it. This gives me 5,174 openings.

Don

Your highly varied and shallow books are of great interest to me (for self-testing). Currently I'm using some 49 thousand FEN positions, that I extracted from Adam Hair's 8move GM PGN file, posted on the Stockfish developpement forum.

How do you create them ? Do you use your Multi-PV to make Komodo create the book ex-nihilo ? If so, that makes me want to implement multi-PV to generate my own search-based books with any depth/branching factor that I choose.

[Don]

If you allow me some remarks.
Castling is not only one way to reach a position that you could otherwise reach with artificial castling in many moves, but the eval staying the same; that might be the case, if you do not give bonuses for an existing option of castling. An existing option of castling would mean the possibility to go to a safe square. An unexisting option in the middlegame (when a rook or the king has already moved, or temporarily inexistant, like your excellent h6 knight from the game prevents the black king from castling short) would quite often already completely change the position characteristics.

Concerning square agnosticity of king safety in Discocheck, do not you have penalties for king in the center in the middlegame, squares on the first rank better than squares on the second, etc.? That might make some difference, although the attacking potential and the distance from the king remain the same.

Concerning the vulnerability of the king on the queen side, what makes it more vulnerable than a king on the king side, apart from some possible tempo losses?

The queen side of the board is better supported for attacking as a general rule, probably because the queen is on that side of the board at the beginning of the game to better support an attack. If both side castling on opposite wings the classic strategy is for both sides to push their pawns and the game becomes a race, the first to get there wins. This strategy makes the queen come alive too.

This has nothing to do with extra imagination any more than any other well known chess concepts, the strategy is very old and very well known. Castle opposite side and push your pawns at the king.

Concerning Don, that is the most difficult one, I personally do not know who is right; I simply noticed that Alekhine and Kasparov (2 of the best chess players according to me, you might add also Fischer and maybe Karpov) would play long castling much more often than the average. That certainly rings a bell. Theoretically, there must be an optimal occurence rate for long castling, maybe 7% (my crazy intuitive estimation, obviously very wrong), maybe the 5% Komodo exhibits, maybe the 3% Houdini exhibits, why not the 15% of Kasparov or the whopping 30% Alekhine figure. There is a wide room for discussion, evidence here. Or maybe it is just a matter of style, as Don says, I do not know. But the problem is there and might provide some clues to something, even if meaningless from a certain point of view.

Don't get me wrong - I don't know the answer either but the strongest players in the world are computers and they seem to castle short much more than all humans, not just the players you mention. However that may only be because program authors are concerned about castling and perhaps focus on king side castling heuristics.

After creating a 3 move book I see that Komodo is castling queen side a little more often, but the statistical noise is too high to be sure. I'm getting 3.79 % now but only 84 instances of castling long. It makes sense that it should increase slightly since some of the king side castling was forced on the program.

If I get the time, I will dip into the source code and made a couple of adjustments that will make Komodo want to castle queen side more often and see what impact it has on the stats.

I am a bit sceptical about those 3200 elo figures. That is a rating of a computer consisting of many cores, you could increase the cores endlessly; why not, in such a case, give Alekhine a thinking time x times bigger (x being the number of cores) for comparison purposes? I think this would make sense. And in this case you would see that the difference in rating is not all that significant (although computers might still be better).

Many people don't know this, but humans are "scalable" too and in fact it is very much like computers - but even more so. It's contrary to intuition because most humans don't realize they play much better with more time, probably because they are always facing opponents who do too.

This was very clear in the old days of computer chess when computers were still very weak at tournament time controls but were starting to equal humans in 5 minute chess. There was a huge disparity here - and as the thinking time got longer humans did better and better. Please note that doubling the thinking time was an ENOURMOUS benefit to computers, and yet STILL humans did better - proving that humans follow a similar time vs ELO curve.

Assuming there is no help from computers humans play correspondence chess at extremely high levels. If there was a reasonable way to compress a correspondence players thinking time to 1 minute and match him against a world champion calibre player, ordinary masters would easily win. Of course this is an impossible experiment to perform.

Correspondence play probably adds just a few hundred ELO because you don't really get 24 hours, you have to sleep and do other things but I think most really good correspondence players take on average 30 minutes or more per move. I say on average because I doubt they spend a lot of time on EVERY move. I played some correspondence games 30 years ago and probably spent an hour per move on average. I assert that humans are much more scalable than computers - they zero in and focus on what is relevant far better than computers and make the time count.

What I know for certain, is that Alekhine is not a guy to ridicule, especially when it comes to opening theory.

Alekhine was chump

Best, Lyudmil

PS. I hope I did not say something completely unjustified, that will release the outrage of a leading programmer

Do you mean a programmer like me? I'm actually finding this humorous that anyone should be afraid of me. I don't take myself that seriously and neither should you.

[Don]

Majority of games
come from books or positions where already done castling.

Search Statistics
appears unreliable.
Unless the effect has been searched.

I composed a suite with the king in the center
to test my engine, I almost always make castling,
I have to re-prepare it lost "castling rights".

My statistics are based on a 5 move opening book where about 9% of white side has castling and about 14% of the black openings have castling.

I just made a 3 move (6 ply) opening book that has no castling - I'll look to see what kind of statistics Komodo produces without 10% of the castling moves being forced on it. This gives me 5,174 openings.

Don

My observation was in a general way,
say, CCRL games, ect.

If you keep on own tests, about 5 moves, are true.

But even so agree,
or agree with you,

Lyudmid tend to express their opinions, as certain..

Apparently, so do I. In this forum I don't feel pressure to be overly formal or precise and probably others feel the same way about their own statements. So when you express and opinion it can come across as being more than it is. It's usually not a dogmatic statement but just an assertion that is tossed out there to chew on.

It seems that simply do a "long castling", and is a sign of superior
and as he says "LucasArt" Alekhine, Alekhine, those were different times, Alekhine could give pieze advantage and win the game.

[Lyudmil Tsvetkov]

Hi Don again.
Here just a small further message just to keep alive the discussion (this is a discussion board, is not it?). Please, do not take it as any kind of infringement upon your status, or other programmers' status, war on engines, etc. It is just to keep the discussion going (btw., thank you very much for your statistics, I appreciated them enormously, and they are quite an useful amount of information).

I have no idea what you are talking about here - I think you are misunderstanding me completely.

Here are the castle long statistics, I probably don't have enough Stockfish games to make this accurate though and I'm putting them from most to least order:

Stockfish 3.71 percent
Komodo 3.60 percent
Houdini 2.89 percent

Note that Stockfish will castle long much more often than Houdini and this is a stylistic difference. The similarity tester works so well because each program has its own style of play and preferences and biases.

Thanks, Don,
very interesting statistics, you went to awful pains to do that.

It seems that Komodo and Stockfish do not have to do muc more than lower their long casling rate in order to catch Houdini.

If you have the time to answer, do not you think that, by perfect play, the rate of castling long/short should be a fixed number (although difficult to ascertain)?

Also, how reliable is your statistics in relation to the strength of moves reproduced on the board? If short and very short time control games have been used predominantly, that would mean that the engine would have some 5 seconds to take a complicated decision, while Alekhine some 5 minutes - a ratio of 1 to 60! I think that things might be further complicated by the fact that the decision which way to castle (almost tantamount to where to cover your king) is probably one of the most complicated ones. You need time for that, to evaluate exactly pawn storm opportunities. Could it be true that more complicated decisions in general are avoided in short time control games?

Thanks again for the statistics.

[Don]

Hi Don again.
Here just a small further message just to keep alive the discussion (this is a discussion board, is not it?). Please, do not take it as any kind of infringement upon your status, or other programmers' status, war on engines, etc. It is just to keep the discussion going (btw., thank you very much for your statistics, I appreciated them enormously, and they are quite an useful amount of information).

I have no idea what you are talking about here - I think you are misunderstanding me completely.

Here are the castle long statistics, I probably don't have enough Stockfish games to make this accurate though and I'm putting them from most to least order:

Stockfish 3.71 percent
Komodo 3.60 percent
Houdini 2.89 percent

Note that Stockfish will castle long much more often than Houdini and this is a stylistic difference. The similarity tester works so well because each program has its own style of play and preferences and biases.

Thanks, Don,
very interesting statistics, you went to awful pains to do that.

It seems that Komodo and Stockfish do not have to do muc more than lower their long casling rate in order to catch Houdini.

If you have the time to answer, do not you think that, by perfect play, the rate of castling long/short should be a fixed number (although difficult to ascertain)?

Also, how reliable is your statistics in relation to the strength of moves reproduced on the board? If short and very short time control games have been used predominantly, that would mean that the engine would have some 5 seconds to take a complicated decision, while Alekhine some 5 minutes - a ratio of 1 to 60! I think that things might be further complicated by the fact that the decision which way to castle (almost tantamount to where to cover your king) is probably one of the most complicated ones. You need time for that, to evaluate exactly pawn storm opportunities. Could it be true that more complicated decisions in general are avoided in short time control games?

Thanks again for the statistics.

My experiences with the similarity tester and move matching shows that the style of a program hardly changes with level - even though ELO does change enormously with the level. If you really get familiar enough with a program to understand how it plays it will be same at 5 minute chess and 5 hour chess. I believe that castling is more stylistic than anything else.

I don't think it's easy to say that you should castle a certain way a certain percentage of the time to be optimal as that depends greatly on the openings that are played.

I did get a surprise with the 3 move book - Komodo is castling long 4.41 percent of the time now - a pretty big increase. I'm not playing 5 minute chess, I'm playing 5 SECOND chess. Game in 3 seconds + 0.03 fischer increment! Average depth of search is only about 11 ply. I will run some much longer time control games to see if the statistics change - my guess is that they won't but I could be wrong.

[Lyudmil Tsvetkov]

There is always the possibility that Alekhine and Kasparov castled long more frequently than others not because they thought it was objectively best, but because the resulting positions were more unbalanced (more likely to lead to a decisive result for one side or the other).

-Sam

By the level of Alekhine and Kasparov I would doubt that very much. How often do you remember Kasparov playing an objectively inferior move just to bluff the opponent? I think their chess grandmaster's ethics would even almost completely exclude the possibility of playing inferior moves. They might have tried to experiment sometimes, but that would only happen extremely rare (5% of the games maybe), and only when the chosen option would not compromise the position beyond a point where this would already constitute a danger for the end result.

Lyudmil

[Don]

There is always the possibility that Alekhine and Kasparov castled long more frequently than others not because they thought it was objectively best, but because the resulting positions were more unbalanced (more likely to lead to a decisive result for one side or the other).

-Sam

By the level of Alekhine and Kasparov I would doubt that very much. How often do you remember Kasparov playing an objectively inferior move just to bluff the opponent? I think their chess grandmaster's ethics would even almost completely exclude the possibility of playing inferior moves. They might have tried to experiment sometimes, but that would only happen extremely rare (5% of the games maybe), and only when the chosen option would not compromise the position beyond a point where this would already constitute a danger for the end result.

Lyudmil

There are degrees. Perhaps they would not necessarily play inferior moves but instead play moves that give more risk vs reward. A LOT of GM's are very conservative, keeping a draw in hand and all that. But there are players who were willing to take a risk to get a reward.

[velmarin]

By the level of Alekhine and Kasparov I would doubt that very much. How often do you remember Kasparov playing an objectively inferior move just to bluff the opponent?
Lyudmil

Kasparov, something I remember, pretty, is my idol.

Alekhine, I remember less. I think there's something written, but were other times....
The charm of chess, now everything is immediate.


If I remember Karpov playing the Caro kann, and "castling long" often with black .

Alekhine, Capablanca, Murphy, those were different times .....
is made a move in tourney and everything see worldwide ...
Today came the sound of wind from Mars..(curiosity)

[Lyudmil Tsvetkov]

If you allow me some remarks.
Castling is not only one way to reach a position that you could otherwise reach with artificial castling in many moves, but the eval staying the same; that might be the case, if you do not give bonuses for an existing option of castling. An existing option of castling would mean the possibility to go to a safe square. An unexisting option in the middlegame (when a rook or the king has already moved, or temporarily inexistant, like your excellent h6 knight from the game prevents the black king from castling short) would quite often already completely change the position characteristics.

Concerning square agnosticity of king safety in Discocheck, do not you have penalties for king in the center in the middlegame, squares on the first rank better than squares on the second, etc.? That might make some difference, although the attacking potential and the distance from the king remain the same.

Concerning the vulnerability of the king on the queen side, what makes it more vulnerable than a king on the king side, apart from some possible tempo losses?

The queen side of the board is better supported for attacking as a general rule, probably because the queen is on that side of the board at the beginning of the game to better support an attack. If both side castling on opposite wings the classic strategy is for both sides to push their pawns and the game becomes a race, the first to get there wins. This strategy makes the queen come alive too.

This has nothing to do with extra imagination any more than any other well known chess concepts, the strategy is very old and very well known. Castle opposite side and push your pawns at the king.

Concerning Don, that is the most difficult one, I personally do not know who is right; I simply noticed that Alekhine and Kasparov (2 of the best chess players according to me, you might add also Fischer and maybe Karpov) would play long castling much more often than the average. That certainly rings a bell. Theoretically, there must be an optimal occurence rate for long castling, maybe 7% (my crazy intuitive estimation, obviously very wrong), maybe the 5% Komodo exhibits, maybe the 3% Houdini exhibits, why not the 15% of Kasparov or the whopping 30% Alekhine figure. There is a wide room for discussion, evidence here. Or maybe it is just a matter of style, as Don says, I do not know. But the problem is there and might provide some clues to something, even if meaningless from a certain point of view.

Don't get me wrong - I don't know the answer either but the strongest players in the world are computers and they seem to castle short much more than all humans, not just the players you mention. However that may only be because program authors are concerned about castling and perhaps focus on king side castling heuristics.

After creating a 3 move book I see that Komodo is castling queen side a little more often, but the statistical noise is too high to be sure. I'm getting 3.79 % now but only 84 instances of castling long. It makes sense that it should increase slightly since some of the king side castling was forced on the program.

If I get the time, I will dip into the source code and made a couple of adjustments that will make Komodo want to castle queen side more often and see what impact it has on the stats.

I am a bit sceptical about those 3200 elo figures. That is a rating of a computer consisting of many cores, you could increase the cores endlessly; why not, in such a case, give Alekhine a thinking time x times bigger (x being the number of cores) for comparison purposes? I think this would make sense. And in this case you would see that the difference in rating is not all that significant (although computers might still be better).

Many people don't know this, but humans are "scalable" too and in fact it is very much like computers - but even more so. It's contrary to intuition because most humans don't realize they play much better with more time, probably because they are always facing opponents who do too.

This was very clear in the old days of computer chess when computers were still very weak at tournament time controls but were starting to equal humans in 5 minute chess. There was a huge disparity here - and as the thinking time got longer humans did better and better. Please note that doubling the thinking time was an ENOURMOUS benefit to computers, and yet STILL humans did better - proving that humans follow a similar time vs ELO curve.

Assuming there is no help from computers humans play correspondence chess at extremely high levels. If there was a reasonable way to compress a correspondence players thinking time to 1 minute and match him against a world champion calibre player, ordinary masters would easily win. Of course this is an impossible experiment to perform.

Correspondence play probably adds just a few hundred ELO because you don't really get 24 hours, you have to sleep and do other things but I think most really good correspondence players take on average 30 minutes or more per move. I say on average because I doubt they spend a lot of time on EVERY move. I played some correspondence games 30 years ago and probably spent an hour per move on average. I assert that humans are much more scalable than computers - they zero in and focus on what is relevant far better than computers and make the time count.

What I know for certain, is that Alekhine is not a guy to ridicule, especially when it comes to opening theory.

Alekhine was chump

Best, Lyudmil

PS. I hope I did not say something completely unjustified, that will release the outrage of a leading programmer

Do you mean a programmer like me? I'm actually finding this humorous that anyone should be afraid of me. I don't take myself that seriously and neither should you.

Hi Don.
Thanks again for the effort.

Could you repeat the experiment with doubling the time control? I guess Komodo will castle long even more frequently, not to mention trebling it.

What is a 'chump'?

When I was talking of a 'leading programmer', actually I was referring to Lucas

Best, Lyudmil