Endgame fortress handling

[Lyudmil Tsvetkov]

Black simply plays a5-a4-a3-a2, and the bishop should take the pawn in 3 moves time. Otherwise, if the bishop blocks the pawn on a2, then, when the white king goes to a3 to capture the pawn, the black king will capture both of white's pawns.

But it is white's turn, and I would play 1. Bc6. Then not much of your a5-a4-a3-a2 plan would remain, if you would not want me to make progress by gaining the black Pawn immediately. And then I would advance the first white Pawn under cover of my King to h6, whith my King on g6, so that black has to keep his King on g8-h8 to prevent me promoting. After that I would move the Bishop via a5 to c2 (so it covers h7 as soon as I move my King). Up to that point moving up the black Pawn loses the Pawn without compensation, as my own Pawns are still protected by the King.

Then I move Kf6, and at that point luring the Bishop away with a5 still doesn't get you the white Pawn back in compensation, as after you attack it with Kh7 I can still protect it with Kg5. And one move later I step into the Pawns 'quadrate' with Ke5, so the King will now eat the Pawn, and my own Pawn will move to h7 under protection of the Bishop, so you will never get it. And you cannot go to the rear Pawn because you cannot step behind the front one without allowing it to promote...

But I guess you missed Uri's point. which is that you can always supplement an iron-clad fortress with some indefensible superfluous material, to allow the opponent some progress by putting up a fight for gaining that (and eventually succeeding), and that would delay the recognition of the fortress by your heuristic even more.

If you are able to play the white bishop from a5 to c2, I am not surprised you do not see white should capture the black pawn in 3 moves time, as on your suggested Bc6, black again plays simply a4, and then a3.

[hgm]

Well, so I made a typo (a5 instead of a4). Doesn't alter the fact that Uri was 100% correct showing this position as an example where white is going to make progress by gaining a black Pawn and advancing its own Pawns to 6th and 7th rank, and that your imagined defense against this utterly sucks...

To see how engines perceive the progress, this is what Fairy-Max says:

Code: Select all

dep	score	nodes	time	(not shown:  tbhits	knps	seldep)
 26	+5.61 	257.7M	3:55.28	g2c6 h8g7 h1g2 g7f6 h3h4 f6g6 g2g3 g6f5 g3f3 f5f6 f3g4 f6g6 h4h5 g6h6 c6b5 h6h7 g4g5 h7g7 h5h6
 25	+5.60 	110.9M	1:38.50	g2c6 h8g7 h1g2 g7h6 h3h4 h6g6 g2g3 g6f5 g3f3 f5g6 f3g4 g6h7 h4h5 h7h6
 24	+5.49 	69.7M  	1:01.32	g2c6 h8g7 h1g2 g7f6 g2g3 f6g5 c6e8 g5f5 h3h4 f5e5 e8a4 e5f5 g3f3 f5f6 f3f4 f6g6 f4g4
 23	+4.95 	46.6M  	0:40.93	g2c6 h8g7 h1g2 g7g6 h3h4 g6f6 g2h3 f6f5 h3g3 f5f6 g3g4 f6g6 c6a4 g6h6 a4e8 h6h7 g4g5 h7h8 g5f4 h8h7 f4g4
 22	+5.01 	27.3M  	0:23.36	g2c6 h8g7 h1g2 g7h6 h3h4 h6g6 g2f3 g6h5 f3g3 h5g6 g3g4 g6h6 c6e8 h6g7 h4h5 g7h6 g4h4 h6h7 h4g5
 21	+5.05 	15.0M  	0:12.51	g2c6 h8g7 h1g2 g7f6 g2f3 f6g5 f3g3 g5h5 h3h4 h5h6 g3g4
 20	+4.45 	9.67M  	0:08.00	g2c6 h8g7 h1g2 g7f6 h3h4 f6e5 g2g3 e5f5 g3f3 f5e5 f3g4 e5e6 h4h5 e6f6
 19	+4.45 	6.31M  	0:05.17	g2c6 h8g7 h1g2 g7f6 h3h4 f6e6 g2g3 e6f5 g3f3 f5f6 f3g4 f6g6 h4h5 g6f6 g4f4 f6f7 f4g4
 18	+4.42 	3.00M  	0:02.39	g2c6 h8g7 h1g2 g7g6 h3h4 g6h6 g2h3 h6h5 h3g3
 17	+4.42 	2.02M  	0:01.61	g2c6 h8g7 h1g2 g7g6 h3h4 g6h6 g2h3 h6h5 h3g3
 16	+3.88 	1.34M  	0:01.06	g2c6
 16	+3.87 	1.27M  	0:01.00	g2c6 h8g7 h1g2 g7g6 h3h4 g6h5 g2g3 h5h6 g3g4 h6g6 h4h5 g6h6 h2h4 h6h7 c6e4 h7g7
 15	+3.91 	824888	0:00.64	g2c6 h8g7 h1g2 g7f6 h3h4 f6g6 g2g3 g6h5 h2h3 h5g6
 14	+3.82 	506191	0:00.39	g2c6 h8g7 h1g2 g7f6 h3h4 f6g6 g2g3 g6f6 g3f4 f6g6 f4g4 g6g7 g4f4 g7g6
 13	+3.83 	346476	0:00.26	g2c6 h8g7 h1g2 g7f6 h3h4 f6g6 g2g3 g6f5 g3f3
 12	+3.67 	174709	0:00.14	g2c6 h8g7 h1g2 g7f6 g2f3 f6g6 f3f4 g6f6 f4f3
 11	+3.69 	90650  	0:00.07	g2c6 h8g7 h1g2 g7f6 g2f3 f6f5 h3h4 f5f6 h2h3 f6e5 c6e4
 10	+3.84 	46958  	0:00.04	g2c6 h8g7 h1g2 g7f6 g2f3 f6f5 c6e4 f5e5 h3h4 a5a4
  9	+3.69 	24296  	0:00.01	g2c6 h8g7 h1g2 g7f6 g2f3 f6f5 c6e4 f5e5 h3h4
  8	+3.83 	12082  	0:00.01	g2c6 h8g7 h1g2 g7f6 g2f3 f6e5 c6e4 a5a4
  7	+3.84 	6131    	0:00.00	g2c6 h8g7 h1g2 g7f6 g2f3 f6e5 c6e4
  6	+3.70 	4527    	0:00.00	g2c6 h8g7 h1g2 g7f6 g2f3 f6e5
  6	+3.34 	2901    	0:00.00	h3h4 a5a4 g2d5 a4a3 h1g2 h8g7
  5	+3.68 	2074    	0:00.00	h3h4 a5a4 h4h5 h8g7 g2e4
  5	+3.63 	1567    	0:00.00	g2c6 h8g7 h1g2 g7f6 g2f3
  4	+3.56 	1081    	0:00.00	g2c6 h8g7 h1g2 g7f6
  4	+3.41 	967      	0:00.00	g2d5 a5a4 h1g2 a4a3
  4	+3.26 	818      	0:00.00	g2e4 a5a4 h1g2 a4a3
  4	+3.10 	569      	0:00.00	g2f1 a5a4 h1g2 a4a3
  4	+3.01 	407      	0:00.00	g2b7 a5a4 b7e4 a4a3
  3	+3.83 	218      	0:00.00	g2b7 h8g7 b7e4
  3	+3.75 	152      	0:00.00	g2f3 h8g7 h1g2
  3	+3.72 	113      	0:00.00	g2e4 h8g7 h1g2
  2	+3.73 	20        	0:00.00	g2e4 h8g7
  1	+3.95 	11        	0:00.00	g2e4

You see, the score rises steadily, and will eventually reach 7, if you let it think long enough.

[hgm]

Two more iterations, still rising...

Code: Select all

dep	score	nodes	time	(not shown:  tbhits	knps	seldep)
 28	+5.90 	1.43G	21:42.17	g2c6 h8h7 h1g2 h7h6 g2g3 h6g5 h3h4 g5f5 g3f3 f5f6 f3g4 f6g6 h4h5 g6h6 h2h4 h6h7 g4g5 h7h8 g5g6 h8g8 c6b5 a5a4 b5a4 g8f8 h5h6 f8g8
 27	+5.86 	665.0M	10:04.87	g2c6 h8g7 h1g1 g7h7 g1g2 h7g6 h3h4 g6f6 g2f3 f6f5 f3g3 f5e5 g3g4 e5f6 h4h5 f6f7 g4g5 f7g7

It finally grabbed the Pawn at 28 ply. But there still is about 150 cP to gain by moving one Pawn from 4th rank to 7th rank, in the final position of that PV. (A Pawn on 7th rank is worth about 250cP to Fairy-Max.) Only after that (36 ply?) it would flatten out. And 20 ply later your algorithm would notice that.

[Lyudmil Tsvetkov]

Two more iterations, still rising...

Code: Select all

dep	score	nodes	time	(not shown:  tbhits	knps	seldep)
 28	+5.90 	1.43G	21:42.17	g2c6 h8h7 h1g2 h7h6 g2g3 h6g5 h3h4 g5f5 g3f3 f5f6 f3g4 f6g6 h4h5 g6h6 h2h4 h6h7 g4g5 h7h8 g5g6 h8g8 c6b5 a5a4 b5a4 g8f8 h5h6 f8g8
 27	+5.86 	665.0M	10:04.87	g2c6 h8g7 h1g1 g7h7 g1g2 h7g6 h3h4 g6f6 g2f3 f6f5 f3g3 f5e5 g3g4 e5f6 h4h5 f6f7 g4g5 f7g7

It finally grabbed the Pawn at 28 ply. But there still is about 150 cP to gain by moving one Pawn from 4th rank to 7th rank, in the final position of that PV. (A Pawn on 7th rank is worth about 250cP to Fairy-Max.) Only after that (36 ply?) it would flatten out. And 20 ply later your algorithm would notice that.

What kind of engine is that, FairyMax?

Stronger or weaker than SF?

As said, after Bc6, black simply plays a4.

[hgm]

Well, than white plays Bxa4, and the first 100cP of progress is already in the pocket...

And at the same depth Fairy-Max would of course be much stronger than Stockfish. It prunes much less. In fact, in this position, it doesn't prune at all. No null move, no LMR. Just a fixed-depth alpha-beta. You cannot get them any stronger than that, at fixed depth.

[hgm]

I take your silence as that we all agree this unholy theory of yours has now permanent been referred to the dust bin?

[zullil]

Interesting why, if the eval approach should really work with fortresses, why more than half a century of very active computer chess development, the only fortress position even the top engines manage to assess correctly is the very simple wrong bishop fortress? Is not this just a bit too negligeable a progress, if the eval approach really could have worked?

I see no reason why pattern-recognition code couldn't be added to evaluation to better evaluate fortress positions. As to why it hasn't been added, several thoughts come to mind immediately:

1) Fortress detection would rarely be of value in practical play, and such code would add almost nothing to ELO.

2) Most chess programmers would find such coding very boring, probably like a lot of evaluation coding. Lots of pattern-matching and case-checking.

3) Endgame tables will detect many of these positions already.

But maybe I'm wrong about one or more of these.

[Lyudmil Tsvetkov]

I take your silence as that we all agree this unholy theory of yours has now permanent been referred to the dust bin?

Until someone reports he has gained 50 elo with it.

[Lyudmil Tsvetkov]

Interesting why, if the eval approach should really work with fortresses, why more than half a century of very active computer chess development, the only fortress position even the top engines manage to assess correctly is the very simple wrong bishop fortress? Is not this just a bit too negligeable a progress, if the eval approach really could have worked?

I see no reason why pattern-recognition code couldn't be added to evaluation to better evaluate fortress positions. As to why it hasn't been added, several thoughts come to mind immediately:

1) Fortress detection would rarely be of value in practical play, and such code would add almost nothing to ELO.

2) Most chess programmers would find such coding very boring, probably like a lot of evaluation coding. Lots of pattern-matching and case-checking.

3) Endgame tables will detect many of these positions already.

But maybe I'm wrong about one or more of these.

You can add a huge amount of elo, if your engine was able to recognize fortress positions early into the search and prune those lines to the benefit of other more promising lines. 20-30-50 elo, or even more as eg fortress positions abound.

This has nothing to do with boredom - if they were able to do it, they would have done it. It is simply that, they did try many fortress ideas in the past, and the only one that worked was the very simple wrong bishop rule. They were able to do this, and only partially, only because it was very simple indeed, and without many exceptions.

As said, you need to add 50 000 lines of code or so to properly define all eg fortresses, and who would do that without slowing their engine?
Adding just 500 lines of code will not gain you any measurable elo, quite probably will even lose a certain amount of elo.

OK, I can define very sensible eval rules for a range of fortress positions: rook vs queen, minor piece vs rook, positions with all pawns blocked, etc., but such a definition would still miss a very large amount of all available fortresses, so no big benefit here, only adding code that is far from perfect.

I am fully certain search can do the job much better instead, but you need a lot of testing to tune the respective parameters, but, first of all, an engine whose eval does not bounce back and forth like mad, but is more or less stable. Unfortunately, even among the top, such engines are rare.

So you need a lot of tuning here.

[zullil]

Interesting why, if the eval approach should really work with fortresses, why more than half a century of very active computer chess development, the only fortress position even the top engines manage to assess correctly is the very simple wrong bishop fortress? Is not this just a bit too negligeable a progress, if the eval approach really could have worked?

I see no reason why pattern-recognition code couldn't be added to evaluation to better evaluate fortress positions. As to why it hasn't been added, several thoughts come to mind immediately:

1) Fortress detection would rarely be of value in practical play, and such code would add almost nothing to ELO.

2) Most chess programmers would find such coding very boring, probably like a lot of evaluation coding. Lots of pattern-matching and case-checking.

3) Endgame tables will detect many of these positions already.

But maybe I'm wrong about one or more of these.

You can add a huge amount of elo, if your engine was able to recognize fortress positions early into the search and prune those lines to the benefit of other more promising lines. 20-30-50 elo, or even more as eg fortress positions abound.

This has nothing to do with boredom - if they were able to do it, they would have done it. It is simply that, they did try many fortress ideas in the past, and the only one that worked was the very simple wrong bishop rule. They were able to do this, and only partially, only because it was very simple indeed, and without many exceptions.

As said, you need to add 50 000 lines of code or so to properly define all eg fortresses, and who would do that without slowing their engine?
Adding just 500 lines of code will not gain you any measurable elo, quite probably will even lose a certain amount of elo.

OK, I can define very sensible eval rules for a range of fortress positions: rook vs queen, minor piece vs rook, positions with all pawns blocked, etc., but such a definition would still miss a very large amount of all available fortresses, so no big benefit here, only adding code that is far from perfect.

I am fully certain search can do the job much better instead, but you need a lot of testing to tune the respective parameters, but, first of all, an engine whose eval does not bounce back and forth like mad, but is more or less stable. Unfortunately, even among the top, such engines are rare.

So you need a lot of tuning here.

Well, I guess I'm at least 2/3 wrong then.