Using GAN to play chess

[EvgeniyZh]

Authors of Style Transfer Generative Adversarial Networks: Learning to Play Chess Differently proposed to use Style Transfer techniques to teach NN to play chess.

What do you think?

[Leo]

Authors of Style Transfer Generative Adversarial Networks: Learning to Play Chess Differently proposed to use Style Transfer techniques to teach NN to play chess.

What do you think?

Interesting paper.

[Tony P.]

I've just downloaded the paper and haven't figured out its formulas yet...

Evgeniy, could you please give us a brief explanation of what 'style transfer' means?

Do I take it right that it means picking up a 'model' human player and setting up a loss function (to use in NN training) in a way that penalizes the NN for playing 'unlike' the model? E.g. if one wants to train the NN to play in the Chigorin style, then one may add a term to the loss function corresponding to an exchange of a knight for a bishop, so that the AI become less likely to give a knight up, as Chigorin preferred knights to bishops.

If so, it looks like the primary application of style transfer is training an AI tailored to score well against a particular human. E.g. one may want to train an NN to play like Carlsen and then train an 'anti-Carlsen' NN that prefers to create positions where the model of Magnus is most likely to play an inaccurate move, which can then be used by a world championship contender in off-the-board opening preparation to help him defeat Magnus in the title match.

If I understand the concept correctly, then I'll surely be using style transfer in my future AIs, and actually, I had thought about this before I learned from you that NNs using this method already exist.

[Tony P.]

E.g. if one wants to train the NN to play in the Chigorin style, then one may add a term to the loss function corresponding to an exchange of a knight for a bishop, so that the AI become less likely to give a knight up, as Chigorin preferred knights to bishops.

No, my impression was wrong. The GAN and STGAN algorithms don't add hand-coded terms corresponding to specific features of a player's style, they only subtract a generic 'discriminator term' that tells only that the move chosen by the generator NN is different from the one chosen by the discriminator NN trained on master games but doesn't distinguish between the reasons why the move is different. So the learning occurs in a rather model-free fashion.

Clearly, the application of GAN to chess is in its infancy. In particular, I don't like the naive 768-dimensional board representation by piece occupancy bitmaps used by the authors. The more elaborate board representation used in Matthew Lai's engine Giraffe is more promising as it's smoother (similar positions are closer in the feature space), which speeds up the convergence.

[Tony P.]

Curiously, David Pfau and Oriol Vinyals, who're working for DeepMind, have pointed out the close connection between GANs and actor-critic reinforcement learning methods, and the latter have been shown to be efficient at outperforming humans at video games.