Narrative AI analysis of a chess game

[whelanh]

I was intrigued by a blog post that used AI to produce a long analysis of their chess game. Their analysis broke the game into phases and commented on key points, learning items etc. in each section. The article is: https://mattplayschess.com/narrative-ai-game-analysis/

I have created an attempt to operationalize this approach at https://github.com/whelanh/chessGameReport My version does give you the option to just generate the prompt instead of automatically submitting it to Gemini (use the --prompt-only argument).

Sample output is shown in the MyAnalysis.txt file in the repo. My interpretation of it is that most of it makes sense, but AI is prone to inventing things and/or repeating generalities. In another game I used it on, it told me Black's pawns were doubled, but they were not

This is just a first attempt. If anyone is interested, I would welcome ideas for improving it.

[Ferdy]

My attempt in integrating llm in chess annotation with gpt3.

The idea is to collect features such as material value, mobility value, king threats value, etc for both sides and a final evaluation of the position. Send those info to llm and let it do the talking. Get back the response and save it as annotation.

If you have more features to send, the better. If we need llm to comment on passers, then scan the board for passers and tell llm that there is a white passer in a4 square for example, this way llm can accurately tell that there is a passer in a4 square. This is typical for weaknesses like doubled pawns, isolated pawns, hanging pieces. Can also be tactical motifs like forks - which square a knight can jump to fork opponent's non-knight pieces and what is the square of this knight, etc. To make llm comment on material quality, we need to calculate the material balance how many white rooks, etc.

An interesting challenge is to interpret variation. Given engine variation, walk the moves and get the positions. Get the features and values of those positions, send them to llm and prompt that this is the sequence of moves and positions with features and values also send the final evaluation of the pv.

I think llm now can generate more interesting comments with some clever prompts. Chess fine-tuned llm is also interesting to try.

Have fun with your explorations.

[whelanh]

My attempt in integrating llm in chess annotation with gpt3.

The idea is to collect features such as material value, mobility value, king threats value, etc for both sides and a final evaluation of the position. Send those info to llm and let it do the talking. Get back the response and save it as annotation.

If you have more features to send, the better. If we need llm to comment on passers, then scan the board for passers and tell llm that there is a white passer in a4 square for example, this way llm can accurately tell that there is a passer in a4 square. This is typical for weaknesses like doubled pawns, isolated pawns, hanging pieces. Can also be tactical motifs like forks - which square a knight can jump to fork opponent's non-knight pieces and what is the square of this knight, etc. To make llm comment on material quality, we need to calculate the material balance how many white rooks, etc.

An interesting challenge is to interpret variation. Given engine variation, walk the moves and get the positions. Get the features and values of those positions, send them to llm and prompt that this is the sequence of moves and positions with features and values also send the final evaluation of the pv.

I think llm now can generate more interesting comments with some clever prompts. Chess fine-tuned llm is also interesting to try.

Have fun with your explorations.

Thanks very much for sharing and for the good suggestions.