Endgame fortress handling

[Lyudmil Tsvetkov]

Sorry for posting a second time, I just forgot to say that tbs, even 7-men, will not solve the problem at all.

6-men fortresses are far rarer than 7-men fortresses, as is to be expected, but those in turn are much much rarer than 8-men fortresses, for which we absolutely do not have tbs, and so forth.

So 5-men tbs might resolve 2% of all eg fortresses, 6-men another 5%, 7-men another 10-15%, but what about the remaining 80-85%?

[zullil]

Sorry for posting a second time, I just forgot to say that tbs, even 7-men, will not solve the problem at all.

6-men fortresses are far rarer than 7-men fortresses, as is to be expected, but those in turn are much much rarer than 8-men fortresses, for which we absolutely do not have tbs, and so forth.

So 5-men tbs might resolve 2% of all eg fortresses, 6-men another 5%, 7-men another 10-15%, but what about the remaining 80-85%?

OK, now I'm 3/3 wrong.

[Lyudmil Tsvetkov]

Sorry for posting a second time, I just forgot to say that tbs, even 7-men, will not solve the problem at all.

6-men fortresses are far rarer than 7-men fortresses, as is to be expected, but those in turn are much much rarer than 8-men fortresses, for which we absolutely do not have tbs, and so forth.

So 5-men tbs might resolve 2% of all eg fortresses, 6-men another 5%, 7-men another 10-15%, but what about the remaining 80-85%?

OK, now I'm 3/3 wrong.

I hate when you are wrong.

[cdani]

You can add a huge amount of elo, if your engine was able to recognize fortress positions early into the search and prune those lines to the benefit of other more promising lines.

Sure. For the moment I was not able to find a way. I will try some more generic shuffle-detection code. Maybe someone will succeed.

[zullil]

Sorry for posting a second time, I just forgot to say that tbs, even 7-men, will not solve the problem at all.

6-men fortresses are far rarer than 7-men fortresses, as is to be expected, but those in turn are much much rarer than 8-men fortresses, for which we absolutely do not have tbs, and so forth.

So 5-men tbs might resolve 2% of all eg fortresses, 6-men another 5%, 7-men another 10-15%, but what about the remaining 80-85%?

OK, now I'm 3/3 wrong.

I hate when you are wrong.

My training as a research mathematician has taught me that:

• a) many of my conjectures/ideas/dreams will be wrong
  b) being wrong is OK---especially if you see why and then move forward

[Lyudmil Tsvetkov]

You can add a huge amount of elo, if your engine was able to recognize fortress positions early into the search and prune those lines to the benefit of other more promising lines.

Sure. For the moment I was not able to find a way. I will try some more generic shuffle-detection code. Maybe someone will succeed.

What about this:

- you try with detecting shuffling with smaller depth, just 10 plies
- say, you are at the root of a certain line, you trace the development of the line in terms of score, you observe that at ply 5 you reach a eg fortress score - 300cps edge, so you follow this line another 5 moves/10 plies, to see if the score changes substantially. If the score changes substantially for those 10 plies, then this is a normal position and you proceed as usual; if the score does not change substantially during those 10 plies, say changes less than 20-30cps, 50cps, you should tune this margin, then you presume the engine is shuffling and this might well be an eg fortress

- in case you detect 300cps score for the first time deeper into the tree, say at ply 20, and the whole line is searched only to ply say 22, you extent this particular line for another 8 plies until it reaches 10 plies distance from the ply at which the 300cps score is detected for the first time; then you compare the scores and again draw your conclusions

[Lyudmil Tsvetkov]

Or, if extending that many plies is impossible, you might want to extend just a couple of plies after you reach a 300cps score in the eg.
You use the fact that large eg scores tend to increase very fast, with tens of cps or even more at every move, so if the score stays almost the same, decreases, etc., this is very suspicious, and quite likely you have stumbled upon an eg fortress.

Small scores increase very slowly, if at all, but large scores normally increase rapidly.
Extremely big scores, like more than 3 full pawns, should sky-rocket in the immediate future.

So say, you encounter 300cps eg score, you check the line 4 plies later, or 6 plies later, to see the score changes, if the line is not that long, you extend with the necessary number of plies, and then check the score again.

If score does not change significantly or decreases, you cut that line as a draw, if you have other positive lines.

[hgm]

I am fully certain search can do the job much better instead, but you need a lot of testing to tune the respective parameters, but, first of all, an engine whose eval does not bounce back and forth like mad, but is more or less stable. Unfortunately, even among the top, such engines are rare.

So you are wrong. As Uri's example, and the engine analysis of it convincingly showed. Scores will continue to increase steadily with search depth even in dead drawn fortress positions, in Uri's case from +4 to +7 in some 35 ply, while the initial position is already a dead draw, and could have been trivially recognized as such. (Only Rook Pawns and a Bishop, the King on the promotion square where your Bishop cannot get at it.)

And it still seemed not to have sunk in with you that in alpha-beta searches large scores typically do not increase at all, in non-PV branches of the tree (i.e. 99.99999% of the branches...), but on the contrary decrease, from +3, to +0.01 (compared to the PV score).

Finally that the number of lines added to a code would determine the slowdown is also a misconception. It doesn't matter how many lines you add. Just how many of them are typically executed, and how often. Adding 50000 lines of code does not have to cause any measurable slowdown, if only a small fraction of them would be executed, and then even in rare situations where there really is a fortress. And whether executing additional code is a slowdown or a speedup is determined by what the alternative behavior is. If that would be a 20-ply search on the position that could otherwise have been pruned without further search becasue a fortress was detected, even executing all 50000 lines would be a speedup of a factor 100.

[Lyudmil Tsvetkov]

I am fully certain search can do the job much better instead, but you need a lot of testing to tune the respective parameters, but, first of all, an engine whose eval does not bounce back and forth like mad, but is more or less stable. Unfortunately, even among the top, such engines are rare.

So you are wrong. As Uri's example, and the engine analysis of it convincingly showed. Scores will continue to increase steadily with search depth even in dead drawn fortress positions, in Uri's case from +4 to +7 in some 35 ply, while the initial position is already a dead draw, and could have been trivially recognized as such. (Only Rook Pawns and a Bishop, the King on the promotion square where your Bishop cannot get at it.)

And it still seemed not to have sunk in with you that in alpha-beta searches large scores typically do not increase at all, in non-PV branches of the tree (i.e. 99.99999% of the branches...), but on the contrary decrease, from +3, to +0.01 (compared to the PV score).

Finally that the number of lines added to a code would determine the slowdown is also a misconception. It doesn't matter how many lines you add. Just how many of them are typically executed, and how often. Adding 50000 lines of code does not have to cause any measurable slowdown, if only a small fraction of them would be executed, and then even in rare situations where there really is a fortress. And whether executing additional code is a slowdown or a speedup is determined by what the alternative behavior is. If that would be a 20-ply search on the position that could otherwise have been pruned without further search becasue a fortress was detected, even executing all 50000 lines would be a speedup of a factor 100.

All 3 things you stated are completely wrong.

- in all fortress positions scores do not increase at all, or just marginally, to reflect changing piece psqt and mobility, as practically nothing else changes

- large scores tend to increase very rapidly always, unless the position is a fortress (maybe though they decrease in the case of FairyMax, I am not certain ):shock:

- for the first time I hear adding many lines of code with heavy specific rules only speeds up the engine; I hope Marco does not see this

[hgm]

All 3 things you stated are completely wrong.

- in all fortress positions scores do not increase at all, or just marginally, to reflect changing piece psqt and mobility, as practically nothing else changes

Already refuted by Uri's example: the score clearly rose from +4 to +6 in the presence of the fortress, and would have risen further with more depth.

- large scores tend to increase very rapidly always, unless the position is a fortress (maybe though they decrease in the case of FairyMax, I am not certain ):shock:

You really don't know anything about engines, do you? All lines always tend in score towards the PV, because the engine would not make any effort whatsoever to increase it, or even keep the advantage, if it is already better than the PV.

- for the first time I hear adding many lines of code with heavy specific rules only speeds up the engine; I hope Marco does not see this

We already concluded that your knowledge of programming, and Chess engines in particular, is dismal. Could you even manage to install and run one? And why do you hope Marco would overlook things? Are you unfriendly to the Stockfish development?