King Indian like and usefulness of engines

[BeyondCritics]

Those are good but Kasparov did not always win with the classical king side clamp/attack.

Here is a good recent correspondence game that shows Black winning:

[pgn]
[Event "MT-Borzenko (RUS)"]
[Site "ICCF"]
[Date "2015.09.01"]
[Round "?"]
[White "Shapiro, Vladislav Borisovic"]
[Black "Nekhaev, Andrey Ivanovich"]
[Result "0-1"]
[ECO "E99"]
[WhiteElo "2358"]
[BlackElo "2525"]
[PlyCount "118"]
[EventDate "2015.??.??"]

1. d4 Nf6 2. c4 g6 3. Nc3 Bg7 4. e4 d6 5. Nf3 O-O 6. Be2 e5 7. O-O Nc6 8. d5
Ne7 9. Ne1 Nd7 10. f3 f5 11. g4 a5 12. Nd3 b6 13. Kh1 Nc5 14. Rg1 f4 15. Bd2
Bd7 16. b3 g5 17. Nb5 Qb8 18. Bc3 Ng6 19. Nxc5 bxc5 20. Rb1 Kf7 21. Qd2 Qb6 22.
Qc2 Rh8 23. Bf1 Qb8 24. a3 Nh4 25. Qd1 h5 26. Bh3 Qc8 27. Rb2 Bf6 28. Rf2 Be7
29. Rc2 Rh6 30. Qf1 Ng6 31. b4 axb4 32. axb4 cxb4 33. Bxb4 Nh4 34. Rb2 Ra4 35.
Qd3 hxg4 36. Bxg4 Bxg4 37. fxg4 Qd7 38. Nc3 Ra8 39. h3 Ng6 40. Rh2 Kg7 41. Nb1
Nf8 42. Nd2 Qa4 43. Rb1 Nd7 44. Kg2 Nf6 45. Rhh1 Qd7 46. Qf3 c6 47. Bc3 c5 48.
Kf2 Rah8 49. Ra1 Qc8 50. Kg2 Rh4 51. Rag1 Qg8 52. Nb3 Qh7 53. Be1 Nxe4 54. Bxh4
gxh4 55. Qd3 Rf8 56. Ra1 f3+ 57. Kh2 Ng3 58. Qxh7+ Kxh7 59. Ra7 Re8 0-1
[/pgn]

Lets do a cursory analysis of this game. In this position
[d]r1bq1rk1/1ppnn1bp/3p2p1/p2Ppp2/2P1P1P1/2N2P2/PP2B2P/R1BQNRK1 w - - 0 12
the textbook defence plan i believe, is to prepare h4 (e.g Ng2, Be3) and then simply stay there. After h5, white plays g5 and after g5 white play h5, thus avoiding opening of lines near his king.
In this game however white choose an alternative, risky plan, but around move 25 or so it became tangible, that his strategy failed completely. Improvements for white should be there, after move 11 and way before move 25.

So far so good, but could you now extract reliable training data from such a game manually? I have never really attempted that, but i think it is rather difficult, since in closed positions there are typically many variations possible.
My idea is, that if you want to generate test data manually, you should consider using commented master games, since there is already a lot of work done for you.

[corres]

[quote="PK"]
Sometimes I think that data structures commonly used in chess programs are just a bit too simplistic to allow for applying (or perhaps learning) a new knowledge. Chess programs achieved tremendous success with them, but they come from the days when both memory and processor speed were rare goods, and this heritage is clearly visible until today.
[/quote]
Lot of years ago I thought that this issue can be resolved only using one dimension matrix. There are many positions with equivalent position value (= value of matrix) but different mobility value, different king safety value, different material value, different free pawn value, etc. These differences can be handled with matrix only.

[BeyondCritics]

A few days ago I was talking to GM Daniel Alsina and told me something that is well known about the inability of the engines to understand some structures in some opening or defenses, like the typical white pawns on f3, e4, d5, and black d6, e5, f4 in the King indian.
[...]
When the plan requires a lot of quiet moves and creation of good coordination for only later crashing trhough the position, the current engines are not able to do this as they are optimized for tactics, which of course reign in most of the positions but not all.

[...]

This was certainly true without any discussion whatsoever, say 10 years ago. But when i today switch on stockfish to analyze some KID position, i can only admire the depth and brilliancy of the stockfish analysis, which i get within seconds. In seconds!

Maybe there are still exceptions. Certainly a grand master has a broader scope of experience than an amateur. So if someone believes he can do better, he should preferably post examples, to back up his claim.

Closed positions can be extremly difficult. So it is expected that even 3000+ engines gets it wrong here and there.
But if some concept is "not understood" consistently, that should be extremly interesting for testers to hear about.

[corres]

[quote="PK"]
I have a gut feeling that serious data mining coupled with increasing the number of piece/square tables (maybe using different sets for different central pawns structures and different king positions) might result in something as spectacular as AlphaGo neural networks. Knowledge needs a structure in which it can reside, and current chess engines don't seem to have that structure.
[/quote]
Good old engine Fritz 3 and probable the next Fritz-s had such a king-pawn table. In the case of Fritz 3 this table could be set by the user . A well tuned table enhanced endgame knowledge of Fritz some tens Elo mainly in the area of pawn endings and pawn+light figure endings.

[corres]

[quote="cdani"]
Maybe this can be aborded by tuning some parameters specifically for this type of structures. Or instead of tuning parameters, something can be done adding a helper function like the ones we tend to do for the endgames.
[/quote]
The main issue is that the parameters what are good for closed positions but these are bad for open positions. I thought that it would need such a chess engine what consist of a main engine and more sub-engines. The main engine would analyze the position globally and would distribute the task to the engine possessed special knowledge. Nowadays there are PC-s with many cores more and more so it is the time to utilize the opportunity of them.

[corres]

[quote="PK"]
There is a lot of human positional knowledge that has not been encoded in the modern chess engines yet.
[/quote]
Every single logic decrease the speed of engine. For this reason you must choice between two possibility:
Engine 1 - You make an engine with lot of logic for being very clever and because of this to be very slow,
or
Engine 2 - You make a fast but silly fool engine with some logic.
Old tests prove that the power of an engine is mainly determined by the speed of that engine.
So if you want to make a World Champion engine you have more chance with Engine 2 than Engine 1.
I think that developers of Stockfish, Houdini and Komodo have a success in balancing between speed and knowledge.

[Lyudmil Tsvetkov]

Those are good but Kasparov did not always win with the classical king side clamp/attack.

Here is a good recent correspondence game that shows Black winning:

[pgn]
[Event "MT-Borzenko (RUS)"]
[Site "ICCF"]
[Date "2015.09.01"]
[Round "?"]
[White "Shapiro, Vladislav Borisovic"]
[Black "Nekhaev, Andrey Ivanovich"]
[Result "0-1"]
[ECO "E99"]
[WhiteElo "2358"]
[BlackElo "2525"]
[PlyCount "118"]
[EventDate "2015.??.??"]

1. d4 Nf6 2. c4 g6 3. Nc3 Bg7 4. e4 d6 5. Nf3 O-O 6. Be2 e5 7. O-O Nc6 8. d5
Ne7 9. Ne1 Nd7 10. f3 f5 11. g4 a5 12. Nd3 b6 13. Kh1 Nc5 14. Rg1 f4 15. Bd2
Bd7 16. b3 g5 17. Nb5 Qb8 18. Bc3 Ng6 19. Nxc5 bxc5 20. Rb1 Kf7 21. Qd2 Qb6 22.
Qc2 Rh8 23. Bf1 Qb8 24. a3 Nh4 25. Qd1 h5 26. Bh3 Qc8 27. Rb2 Bf6 28. Rf2 Be7
29. Rc2 Rh6 30. Qf1 Ng6 31. b4 axb4 32. axb4 cxb4 33. Bxb4 Nh4 34. Rb2 Ra4 35.
Qd3 hxg4 36. Bxg4 Bxg4 37. fxg4 Qd7 38. Nc3 Ra8 39. h3 Ng6 40. Rh2 Kg7 41. Nb1
Nf8 42. Nd2 Qa4 43. Rb1 Nd7 44. Kg2 Nf6 45. Rhh1 Qd7 46. Qf3 c6 47. Bc3 c5 48.
Kf2 Rah8 49. Ra1 Qc8 50. Kg2 Rh4 51. Rag1 Qg8 52. Nb3 Qh7 53. Be1 Nxe4 54. Bxh4
gxh4 55. Qd3 Rf8 56. Ra1 f3+ 57. Kh2 Ng3 58. Qxh7+ Kxh7 59. Ra7 Re8 0-1
[/pgn]

Lets do a cursory analysis of this game. In this position
[d]r1bq1rk1/1ppnn1bp/3p2p1/p2Ppp2/2P1P1P1/2N2P2/PP2B2P/R1BQNRK1 w - - 0 12
the textbook defence plan i believe, is to prepare h4 (e.g Ng2, Be3) and then simply stay there. After h5, white plays g5 and after g5 white play h5, thus avoiding opening of lines near his king.
In this game however white choose an alternative, risky plan, but around move 25 or so it became tangible, that his strategy failed completely. Improvements for white should be there, after move 11 and way before move 25.

So far so good, but could you now extract reliable training data from such a game manually? I have never really attempted that, but i think it is rather difficult, since in closed positions there are typically many variations possible.
My idea is, that if you want to generate test data manually, you should consider using commented master games, since there is already a lot of work done for you.

Both players simply made too many mistakes.

6.Be2, as well as any other move apart from h3, like Bg5, Be3, etc., gives black full equality.
when black pushes f5-f4, black already has an advantage, white should have captured on f5.
white could have held the draw by playing h2-h3 instead of Bh3, placing the white king on f2 and rook on h-file, with blocked position.

As said, no matter how strange this might seem to someone, 6.h3, followed by g4 and castling long, quite probably simply wins the game for white. I can see no refutation, if someone sees, please tell me.

If that were true, than mainline KID is busted, though in the majority of lines, other than 6.h3, for example 6.Bg5, 6.Be3, 6.Be2, black gets full equality or even better, if longer chain is created.

In that way of thinking, 1.d4 Nf6?!, 1.d4 c5!

[Lyudmil Tsvetkov]

old tests prove that the power of an engine is mainly determined by the speed of that engine.

at least try to follow latest top engine developments, before commenting.

Even from SF7 to SF 8 (migth check this on their abrok site, if not in code), SF added quite a few new eval terms, while just a couple were partially simplified. Similarly, SF search only grew more sophisticated, adding new search stages and conditions, fewer search routines were simplified. Overall, top engines' search speed only tends to decrease to the benefit of more knowledge added.

So, there is a single road ahead: Shannon's type A strategy.

[Rebel]

A few days ago I was talking to GM Daniel Alsina and told me something that is well known about the inability of the engines to understand some structures in some opening or defenses, like the typical white pawns on f3, e4, d5, and black d6, e5, f4 in the King indian.

The KID is a blast from the AEGON past. If you scroll through its games you see many victories from average players against the comps because they did not understand the building up of the black pieces on the king side. It was the way to beat the comps. As soon as folks got the chance to close the position with f5-f4 then the comp was toasted provided the human did not blunder in the final touch.

[Ras]

you see many victories from average players against the comps because they did not understand the building up of the black pieces on the king side.

The CT800 has an asymmetric evaluation penalising positions with a blocked centre for the computer side only. This is a simple kind of anti human mode. The idea is to work around the shortcomings by keeping the human from even reaching these positions.

Of course, the opening book must also be in line with this approach for avoiding to throw the computer into these positions right from the book.