GeminiChess, an LLM built engine

[smatovic]

Out of curiosity, did you program already a chess engine by yourself? Or is chess programming a "foreign territory" to you?

--
Srdja

[glav]

Out of curiosity, did you program already a chess engine by yourself? Or is chess programming a "foreign territory" to you?

--
Srdja

I am passionate about chess and programming, but I never programmed an engine. I have done some unrelated work in C, though. These days I am mostly using R and python, but I found out that chatgpt can often do the work for me, especially the basic stuff.

[glav]

I forgot to add the link to the new version:

[Werewolf]

I did exactly this with ChatGPT 5 Pro - also in bitboard, written in C.

Nice. Would you have an executable of your program since I was unable to compile it?

Did debugging take you ages by any chance?

Not really. Although the model submitted several wrong answers (it had allucinations, produced non-compilable code or code that crashed or didn't pass the tests, a couple of times produced even header files with the '.hh' (!) extension, etc.), the driver script was merciless in rejecting these proposals and asking for new ones. It was not too stressful: once started the run, I never had any direct interaction with the LLM, though I was closely watching.

Can you PM me with your email? I'll send you the file directly. However, it is not 100% bug free yet.

[glav]

Apparently, I wasn't the first one to use Gemini for building a chess playing program. Heiko Hotz had already developed one 5 months ago and, moreover, it was also more complete than mine, because it also had a GUI. Nice also the idea to let the program to explain his reasoning. All in all, a nice read. Srdja might wish to update his list.


https://github.com/heiko-hotz/gemini-ch ... t/thoughts

[smatovic]

Apparently, I wasn't the first one to use Gemini for building a chess playing program. Heiko Hotz had already developed one 5 months ago and, moreover, it was also more complete than mine, because it also had a GUI. Nice also the idea to let the program to explain his reasoning. All in all, a nice read. Srdja might wish to update his list.

https://medium.com/google-cloud/build-a ... fc46af7329
https://github.com/heiko-hotz/gemini-ch ... t/thoughts

Thx for posting, but I see no chess engine in the given repository. As far as I got it, you play chess against LLMs with that app.

--
Srdja

[glav]

Apparently, I wasn't the first one to use Gemini for building a chess playing program. Heiko Hotz had already developed one 5 months ago and, moreover, it was also more complete than mine, because it also had a GUI. Nice also the idea to let the program to explain his reasoning. All in all, a nice read. Srdja might wish to update his list.

https://medium.com/google-cloud/build-a ... fc46af7329
https://github.com/heiko-hotz/gemini-ch ... t/thoughts

Thx for posting, but I see no chess engine in the given repository. As far as I got it, you play chess against LLMs with that app.

--
Srdja

Oops! You are right. Too bad: I was already dreaming of a tournament among LLM-built engines.

[jdart]

My experience so far with AIs for coding (with Claude Code and Github Copilot) is that they are pretty good at implementing things, or restructuring, but quite poor at debugging. Claude seems to be better than Copilot: the latter generated code with some pretty elementary C++ coding errors. I haven't tried generating a complete program. One of my learnings from decades in computer chess is that testing is very important. It is amazing how broken a program can be and still play legal moves. Only a good set of tests can demonstrate that everything is functioning properly. I have both unit tests for program components and also of course use game match testing.

[Werewolf]

My experience so far with AIs for coding (with Claude Code and Github Copilot) is that they are pretty good at implementing things, or restructuring, but quite poor at debugging. Claude seems to be better than Copilot: the latter generated code with some pretty elementary C++ coding errors. I haven't tried generating a complete program. One of my learnings from decades in computer chess is that testing is very important. It is amazing how broken a program can be and still play legal moves. Only a good set of tests can demonstrate that everything is functioning properly. I have both unit tests for program components and also of course use game match testing.

ChatGPT 5 Pro is the best here. One trick you can try is playing them off against each other for de-bugging since they seem blind to their own mistakes but critical of other's code.