STS (v3.0) - Knight Outposts/Centralization/Repositioning

[Dann Corbit]

I have suggested the following revision to the record:
[d]rqr3k1/1p2bppp/3pn3/p3p1Pn/P3P3/1PNBBP2/1P1Q3P/2KR3R b - - bm Nd4; c0 "Nd4=10, Nef4=9, Nhf4=9"; id "STS: Knight Outposts/Repositioning/Centralization.100";

Both Nef4 and Nhf4 are favored by some engines after a long search (but not by Naum or Rybka).

[bob]

This is 3 hour Rybka output. Other engines churning now

Code: Select all

Analysis from C:\tmp\question.epd   
6/4/2009 5:17:08 PM Level: 10800 Seconds
Analyzing engine: Rybka 3

1) Ne6d4; pv Ne6d4 Bd3c 
    Searching move: Ne6-d4, Ne6-d4, Bd3-c2
    Best move (Rybka 3): Ne6-d4
    identical moves! Found in: 00:02
      2	00:00	         443	28.352	+0.53	Ne6d4
      3	00:00	       1.132	72.448	+0.31	Ne6d4
      4	00:00	       1.444	92.416	+0.22	Ne6d4
      5	00:00	       2.644	84.608	+0.12	Ne6d4 Bd3c2
      6	00:00	       7.162	116.410	+0.08	Ne6d4 Bd3c2 Nd4xc2 Qd2xc2 Nh5f4
      6	00:00	       8.932	115.776	+0.10	Nh5f4 h2h4 d6d5
      7	00:00	      19.933	130.008	+0.02	Nh5f4 h2h4 b7b6 Bd3c4 Qb8b7
      7	00:00	      22.903	124.748	+0.04	Ne6d4 Bd3c2 Nd4xc2
      8	00:00	      31.155	127.610	 0.00	Ne6d4 Bd3c2 Nd4xc2 Qd2xc2 Ra8a6 Kc1b1 Ra6c6 Qc2d2
      8	00:00	      38.556	126.138	+0.01	Nh5f4 h2h4 d6d5 Be3xf4 Ne6xf4 e4xd5 Be7b4 Kc1b1
      9	00:00	      65.126	129.493	-0.07	Nh5f4 h2h4 Nf4xd3+ Qd2xd3 Rc8c6 Kc1b1 Qb8e8 Nc3d5 Be7d8
      9	00:00	      78.311	128.304	-0.05	Ne6d4 Bd3c2 Nd4xc2 Qd2xc2 Rc8c6 Kc1b1 Qb8e8 Qc2d2
     10	00:01	     114.232	127.007	-0.12	Ne6d4 Bd3c2 Nd4xc2 Qd2xc2 Rc8c6 Kc1b1
     10	00:01	     164.215	129.850	-0.11	Nh5f4 h2h4 Ne6c5 Bd3c2 Nf4e6 Kc1b1 Rc8c6 Nc3d5 Qb8d8
     11	00:01	     221.320	124.113	-0.15	Nh5f4 h2h4 Ne6c5 Bd3c2 Nc5e6 Kc1b1 b7b6 Rd1c1 Ra8a7 Nc3d5
     11	00:02	     278.671	126.995	-0.11	Ne6d4 Bd3c2 Nd4xc2
     12	00:02	     318.404	126.619	-0.08	Ne6d4 Bd3c2 Nd4xc2 Qd2xc2 Rc8c6 Kc1b1 Qb8c8 Qc2f2 Qc8h3 Nc3d5 Be7d8 Rh1g1
     13	00:04	     463.479	120.702	-0.10	Ne6d4 Bd3c2 Nd4xc2 Qd2xc2 Rc8c6 Kc1b1 Qb8c8 Qc2f2 Qc8h3 Nc3d5 Be7d8 Rh1g1 Ra8c8
     14	00:08	     934.809	115.330	-0.16	Ne6d4 Bd3c2 Nd4xc2 Qd2xc2 Rc8c6 Kc1b1 Qb8c8 Qc2f2 Qc8e6 Nc3d5 Ra8c8 Qf2h4 g7g6 Qh4g4 Be7d8 Rd1c1
     15	00:12	   1.418.731	113.286	-0.16	Ne6d4 Bd3c2 Nd4xc2 Qd2xc2 Rc8c6 Kc1b1 Qb8c8 Qc2f2 Qc8e6 Nc3d5 Ra8c8 Qf2h4 g7g6 Qh4g4 Be7d8 Rd1c1
     16	00:23	   2.629.125	115.511	-0.15	Ne6d4 Bd3c2 Nd4xc2 Qd2xc2 Rc8c6 Kc1b1 Qb8c8 Qc2f2 Qc8e6 Nc3d5 Be7d8 Qf2h4 g7g6 Qh4f2 Ra8c8 Rd1c1 Qe6d7
     17	00:43	   4.873.045	117.425	-0.19	Ne6d4 Bd3c2 Nd4xc2 Qd2xc2 Rc8c6 Kc1b1 Qb8c8 Qc2f2 Qc8e6 Nc3d5 Be7d8 Qf2g2 Ra8c8 Qg2g4 g7g6 Rd1d2 f7f5
     18	01:44	  12.059.702	118.557	-0.22	Ne6d4 Bd3c2 Nd4xc2 Qd2xc2 Rc8c6 Kc1b1 Qb8c8 h2h4 Qc8e6 Rd1d5 b7b6 Rh1d1 Ra8c8 Qc2f2 f7f6 Nc3b5 Qe6f7
     19	02:31	  17.360.643	117.516	-0.22	Ne6d4 Bd3c2 Nd4xc2 Qd2xc2 Rc8c6 Kc1b1 Qb8c8 h2h4 Qc8e6 Rd1d5 b7b6 Rh1d1 Ra8c8 Qc2f2 f7f6 Nc3b5 Qe6f7
     20	06:59	  49.364.289	126.047	-0.29	Ne6d4 Bd3c2 Nd4xc2 Qd2xc2 Rc8c6 Kc1b1 f7f6 g5xf6 Nh5xf6 Qc2d3 Qb8e8 Qd3b5 Be7f8 Rh1f1 Qe8f7 Kb1a2 Rc6c8 Rd1c1 g7g6 Rf1d1 Rc8c6 Rc1c2
     21	17:56	 140.434.366	133.562	-0.42	Ne6d4 Kc1b1 Nd4xb3 Qd2g2 Nh5f4 Be3xf4 Rc8xc3 b2xc3 e5xf4 Bd3b5 Qb8c7 Qg2a2 Nb3c5 h2h4 Ra8c8 Qa2c4 Qc7d8 e4e5 Nc5e6 Qc4d5 d6xe5 Qd5xd8+ Rc8xd8 Rd1xd8+ Be7xd8 Kb1c2 Ne6c5 Rh1d1
     22	49:59	 410.550.987	140.148	-0.43	Ne6d4 Kc1b1 Nd4xb3 Qd2g2 Nh5f4 Be3xf4 Rc8xc3 b2xc3 e5xf4 Bd3b5 Qb8c7 Qg2a2 Nb3c5 h2h4 Ra8c8 Qa2c4 Qc7d8 e4e5 Nc5e6 Qc4d5 d6xe5 Qd5xd8+ Rc8xd8 Rd1xd8+ Be7xd8 Kb1c2 Ne6c5 Rh1d1
     23	1:35:24	 782.530.785	139.984	-0.43	Ne6d4 Kc1b1 Nd4xb3 Qd2g2 Nh5f4 Be3xf4 Rc8xc3 b2xc3 e5xf4 Bd3b5 Qb8c7 Qg2a2 Nb3c5 h2h4 Ra8c8 Qa2c4 Qc7d8 e4e5 Nc5e6 Qc4d5 d6xe5 Qd5xd8+ Rc8xd8 Rd1xd8+ Be7xd8 Kb1c2 Ne6c5 Rh1d1
     24	2:27:36	1.234.938.892	142.779	-0.42	Ne6d4 Kc1b1 Nd4xb3 Qd2g2 Nh5f4 Be3xf4 Rc8xc3 b2xc3 e5xf4 Bd3b5 Qb8c7 Qg2a2 Nb3c5 h2h4 Ra8c8 Qa2c4 Be7f8 Rh1h2 Qc7d8 Qc4d5 Qd8e7 Rh2b2 g7g6 Rb2d2
   6/4/2009 8:17:15 PM, Time for this analysis: 03:00:00, Rated time: 00:02

1 of 1 matching moves
6/4/2009 8:17:16 PM, Total time: 3:00:07 AM
Rated time: 00:02 = 2 Seconds

To me that looks bad. Score is steadily _dropping_....

These test sets are not tactical problems that will show a win. They are intended to be strategic moves where the best choice has long term implications. Several of the problems will have small negative scores after a very long think.

Right, but steadily dropping suggest more than just a positional issue is going on. There may well be some deeper tactics that are affecting the backed up score as well.

[bob]

This is 3 hour Rybka output. Other engines churning now

Code: Select all

Analysis from C:\tmp\question.epd   
6/4/2009 5:17:08 PM Level: 10800 Seconds
Analyzing engine: Rybka 3

1) Ne6d4; pv Ne6d4 Bd3c 
    Searching move: Ne6-d4, Ne6-d4, Bd3-c2
    Best move (Rybka 3): Ne6-d4
    identical moves! Found in: 00:02
      2	00:00	         443	28.352	+0.53	Ne6d4
      3	00:00	       1.132	72.448	+0.31	Ne6d4
      4	00:00	       1.444	92.416	+0.22	Ne6d4
      5	00:00	       2.644	84.608	+0.12	Ne6d4 Bd3c2
      6	00:00	       7.162	116.410	+0.08	Ne6d4 Bd3c2 Nd4xc2 Qd2xc2 Nh5f4
      6	00:00	       8.932	115.776	+0.10	Nh5f4 h2h4 d6d5
      7	00:00	      19.933	130.008	+0.02	Nh5f4 h2h4 b7b6 Bd3c4 Qb8b7
      7	00:00	      22.903	124.748	+0.04	Ne6d4 Bd3c2 Nd4xc2
      8	00:00	      31.155	127.610	 0.00	Ne6d4 Bd3c2 Nd4xc2 Qd2xc2 Ra8a6 Kc1b1 Ra6c6 Qc2d2
      8	00:00	      38.556	126.138	+0.01	Nh5f4 h2h4 d6d5 Be3xf4 Ne6xf4 e4xd5 Be7b4 Kc1b1
      9	00:00	      65.126	129.493	-0.07	Nh5f4 h2h4 Nf4xd3+ Qd2xd3 Rc8c6 Kc1b1 Qb8e8 Nc3d5 Be7d8
      9	00:00	      78.311	128.304	-0.05	Ne6d4 Bd3c2 Nd4xc2 Qd2xc2 Rc8c6 Kc1b1 Qb8e8 Qc2d2
     10	00:01	     114.232	127.007	-0.12	Ne6d4 Bd3c2 Nd4xc2 Qd2xc2 Rc8c6 Kc1b1
     10	00:01	     164.215	129.850	-0.11	Nh5f4 h2h4 Ne6c5 Bd3c2 Nf4e6 Kc1b1 Rc8c6 Nc3d5 Qb8d8
     11	00:01	     221.320	124.113	-0.15	Nh5f4 h2h4 Ne6c5 Bd3c2 Nc5e6 Kc1b1 b7b6 Rd1c1 Ra8a7 Nc3d5
     11	00:02	     278.671	126.995	-0.11	Ne6d4 Bd3c2 Nd4xc2
     12	00:02	     318.404	126.619	-0.08	Ne6d4 Bd3c2 Nd4xc2 Qd2xc2 Rc8c6 Kc1b1 Qb8c8 Qc2f2 Qc8h3 Nc3d5 Be7d8 Rh1g1
     13	00:04	     463.479	120.702	-0.10	Ne6d4 Bd3c2 Nd4xc2 Qd2xc2 Rc8c6 Kc1b1 Qb8c8 Qc2f2 Qc8h3 Nc3d5 Be7d8 Rh1g1 Ra8c8
     14	00:08	     934.809	115.330	-0.16	Ne6d4 Bd3c2 Nd4xc2 Qd2xc2 Rc8c6 Kc1b1 Qb8c8 Qc2f2 Qc8e6 Nc3d5 Ra8c8 Qf2h4 g7g6 Qh4g4 Be7d8 Rd1c1
     15	00:12	   1.418.731	113.286	-0.16	Ne6d4 Bd3c2 Nd4xc2 Qd2xc2 Rc8c6 Kc1b1 Qb8c8 Qc2f2 Qc8e6 Nc3d5 Ra8c8 Qf2h4 g7g6 Qh4g4 Be7d8 Rd1c1
     16	00:23	   2.629.125	115.511	-0.15	Ne6d4 Bd3c2 Nd4xc2 Qd2xc2 Rc8c6 Kc1b1 Qb8c8 Qc2f2 Qc8e6 Nc3d5 Be7d8 Qf2h4 g7g6 Qh4f2 Ra8c8 Rd1c1 Qe6d7
     17	00:43	   4.873.045	117.425	-0.19	Ne6d4 Bd3c2 Nd4xc2 Qd2xc2 Rc8c6 Kc1b1 Qb8c8 Qc2f2 Qc8e6 Nc3d5 Be7d8 Qf2g2 Ra8c8 Qg2g4 g7g6 Rd1d2 f7f5
     18	01:44	  12.059.702	118.557	-0.22	Ne6d4 Bd3c2 Nd4xc2 Qd2xc2 Rc8c6 Kc1b1 Qb8c8 h2h4 Qc8e6 Rd1d5 b7b6 Rh1d1 Ra8c8 Qc2f2 f7f6 Nc3b5 Qe6f7
     19	02:31	  17.360.643	117.516	-0.22	Ne6d4 Bd3c2 Nd4xc2 Qd2xc2 Rc8c6 Kc1b1 Qb8c8 h2h4 Qc8e6 Rd1d5 b7b6 Rh1d1 Ra8c8 Qc2f2 f7f6 Nc3b5 Qe6f7
     20	06:59	  49.364.289	126.047	-0.29	Ne6d4 Bd3c2 Nd4xc2 Qd2xc2 Rc8c6 Kc1b1 f7f6 g5xf6 Nh5xf6 Qc2d3 Qb8e8 Qd3b5 Be7f8 Rh1f1 Qe8f7 Kb1a2 Rc6c8 Rd1c1 g7g6 Rf1d1 Rc8c6 Rc1c2
     21	17:56	 140.434.366	133.562	-0.42	Ne6d4 Kc1b1 Nd4xb3 Qd2g2 Nh5f4 Be3xf4 Rc8xc3 b2xc3 e5xf4 Bd3b5 Qb8c7 Qg2a2 Nb3c5 h2h4 Ra8c8 Qa2c4 Qc7d8 e4e5 Nc5e6 Qc4d5 d6xe5 Qd5xd8+ Rc8xd8 Rd1xd8+ Be7xd8 Kb1c2 Ne6c5 Rh1d1
     22	49:59	 410.550.987	140.148	-0.43	Ne6d4 Kc1b1 Nd4xb3 Qd2g2 Nh5f4 Be3xf4 Rc8xc3 b2xc3 e5xf4 Bd3b5 Qb8c7 Qg2a2 Nb3c5 h2h4 Ra8c8 Qa2c4 Qc7d8 e4e5 Nc5e6 Qc4d5 d6xe5 Qd5xd8+ Rc8xd8 Rd1xd8+ Be7xd8 Kb1c2 Ne6c5 Rh1d1
     23	1:35:24	 782.530.785	139.984	-0.43	Ne6d4 Kc1b1 Nd4xb3 Qd2g2 Nh5f4 Be3xf4 Rc8xc3 b2xc3 e5xf4 Bd3b5 Qb8c7 Qg2a2 Nb3c5 h2h4 Ra8c8 Qa2c4 Qc7d8 e4e5 Nc5e6 Qc4d5 d6xe5 Qd5xd8+ Rc8xd8 Rd1xd8+ Be7xd8 Kb1c2 Ne6c5 Rh1d1
     24	2:27:36	1.234.938.892	142.779	-0.42	Ne6d4 Kc1b1 Nd4xb3 Qd2g2 Nh5f4 Be3xf4 Rc8xc3 b2xc3 e5xf4 Bd3b5 Qb8c7 Qg2a2 Nb3c5 h2h4 Ra8c8 Qa2c4 Be7f8 Rh1h2 Qc7d8 Qc4d5 Qd8e7 Rh2b2 g7g6 Rb2d2
   6/4/2009 8:17:15 PM, Time for this analysis: 03:00:00, Rated time: 00:02

1 of 1 matching moves
6/4/2009 8:17:16 PM, Total time: 3:00:07 AM
Rated time: 00:02 = 2 Seconds

To me that looks bad. Score is steadily _dropping_....

These test sets are not tactical problems that will show a win. They are intended to be strategic moves where the best choice has long term implications. Several of the problems will have small negative scores after a very long think.

Right, but steadily dropping suggests more than just a positional issue is going on. There may well be some deeper tactics that are affecting the backed up score as well.

[swami]

The positions are taken from games where both players were formidable and the chosen move was found in a game that was won by the player making the move. So the moves definitely have winning chances. I do not know what game Swami took the move from. I have only one game with that position, the key move was chosen in that game, and the player making that move won the game.

Yes, I always take positions from the 'winning' side. I never took positions from the 'losing' side. You're probably right that this game was from Hiarcs vs Zappa. Since I've more than once mentioned that I have databases of free games from CCRL,CEGT, SSDF, Correspondence games, and GM games.

This position seems a rare exception, in that winning side that played Nd4, despite being slightly minus and a transformation from plus to slight minus, actually won against very strong opposition.

[swami]

The positions are taken from games where both players were formidable and the chosen move was found in a game that was won by the player making the move. So the moves definitely have winning chances. I do not know what game Swami took the move from. I have only one game with that position, the key move was chosen in that game, and the player making that move won the game.

Yes, I always take positions from the 'winning' side. I never took positions from the 'losing' side. You're probably right that this game was from Hiarcs. Since I've more than once mentioned that I have databases of games from CCRL, Correspondence games, and GM games.

This position seems a rare exception, in that winning side that played Nd4, despite being slightly minus and a transformation from plus to slight minus, actually won against very strong opposition.

That said, I have no problem if this particular problem has to be replaced with something else that brings positive scoring for the winning side. After all that is what bugfixing is for.

[swami]

I have suggested the following revision to the record:
[d]rqr3k1/1p2bppp/3pn3/p3p1Pn/P3P3/1PNBBP2/1P1Q3P/2KR3R b - - bm Nd4; c0 "Nd4=10, Nef4=9, Nhf4=9"; id "STS: Knight Outposts/Repositioning/Centralization.100";

Both Nef4 and Nhf4 are favored by some engines after a long search (but not by Naum or Rybka).

Ok, the file in the download section is now updated with:

Code: Select all

rqr3k1/1p2bppp/3pn3/p3p1Pn/P3P3/1PNBBP2/1P1Q3P/2KR3R b - - bm Nd4; c0 "Nd4=10, Nef4=9, Nhf4=9"; id "STS: Knight Outposts/Repositioning/Centralization.100";

[swami]

1. piece/square table to centralize knights.

2. outpost knights, with a bonus for no enemy pawn that can attack the knight and another bonus if no enemy knight or bishop can attack the knight.

3. Mobility with mobility to central squares being worth more than mobility to edge squares.

Interesting. Makes me wonder if all programmers of >2600 engines have good understanding of advanced rules in chess strategies that's required of a IM or other strong chess players? Did you study a lot of strategical books in chess, Bob, to be able to determine some sort of importances for certain concepts? If so, that's impressive.

Maybe you heard of these rules over years of experience discussing chess with IM's, and that they gave you some sort of rules on Do's/Don'ts in strategies, and you just implemented them promtly as you go along?

perhaps its all too simple and common knowledge for all programmers, and that I'm just not expecting much and holding on to my belief that they are all secret and difficult to obtain knowledge?

Am I right in thinking that there's too much principles and values to implement on a strong playing chess program, so that it wouldn't make any positional errors?

Do you have the complete details of whatever inputs related to values, bonuses, principles and strategies, you've so far given to Crafty? I'd be interested in such a documentation.

For your last question first, no, other than what is in the comments. I have tried to include all the "normal" ideas into the evaluation. pawn structure, piece placement, etc. But there is a long way to go, since the knowledge/speed tradeoff is ever-present.

I think our mobility is done "right", although it is not as good as what we had in the last versions of Cray Blitz that were lost. I do not believe that mobility, just for the sake of mobility is very good. we currently weight squares based on centrality (for knights). In Cray Blitz we weighted squares by computing the weights as we did the eval, so that we had an idea of which squares were weak, which pawns were weak, etc, and mobility factored in those weights that could be different for every square (cute vector merge instruction let us get away with this for zero cost). In any case, knights are not magic in what we do, attract them toward the center, evaluate outposts as I mentioned previously, and then factor in mobility. The knight's position is factored into king safety, but not in the code that evaluates knights, this is done in the EvaluateKing() code.

As far as "where did the ideas come from?" I've read most books that others read over the years, and probably some that have been missed. "Point count chess" appeared to be a perfect one for computer chess, but although the ideas were good, it was unusable as an evaluation model. "My System" was my favorite for improving my own play and many of his ideas therefore influenced Crafty's evaluation. Other ideas came from prodding by Roman Dzhindi (candidate pawn evaluation, as one example).

If you liked "My System", I'm sure Battle Royale 1924 and Alexei Bartanishkov's Principles of chess Strategies are worth checking out. They've list of rules outlined for several conditions.

You'd find high quality stuffs from those books that you can implement them to Crafty.

[swami]

Test - 10 seconds per Position.
Hardware: Core 2 Quad Q6600 2 GM RAM

Grade A: (80% +)

Code: Select all

Rybka

Grade B: (70%+)

Code: Select all

Pro Deo - 76
Pharaon - 73
TwistedLogic - 73
Fruit - 72
Spike - 72
The King TrailBlazer - 71
Bright - 71
ETChess - 71
Glaurung - 70

Grade C: (60% +)

Code: Select all

Baron - 68
NOW - 68
Scorpio - 68
Cerebro - 68
Alfil - 67
Knight Dreamer - 67
Bug Chess - 67
Movei - 67
Chiron - 66
Pepito - 66
Hamsters - 66
Lambchop - 66
Slowchess - 65
Delfi - 64
WildCat - 64
Arasan - 64
Green Light chess - 62
Gromit - 62
Alaric - 61
Delphil -61
Abrok - 60
Horizon - 60
Muse - 60 
Gaia - 60

Grade D: (50% +)

Code: Select all

Learning Lemming 59
Natwarlal - 59
Aristarch - 58
Little Thought - 58
Knightx - 58
Rotor - 58
Asterisk - 58
Der Bringer - 58 
Little Goliath Revival - 58
Flux - 57
Phalanx - 56
Trace - 56
Dorky - 55
Nejmet - 55
Yace Paderborn - 54
Eve Ann - 54
Djinn - 50

Grade E (40% +) (Many more engines yet to be tested)

Code: Select all

Beowulf - 44
Lodocase - 43