Using GAN to play chess

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Harvey Williamson, bob

Post Reply
EvgeniyZh
Posts: 42
Joined: Fri Sep 19, 2014 2:54 pm
Location: Israel
Contact:

Using GAN to play chess

Post by EvgeniyZh » Thu Feb 23, 2017 12:32 pm

Authors of Style Transfer Generative Adversarial Networks: Learning to Play Chess Differently proposed to use Style Transfer techniques to teach NN to play chess.

What do you think?

Leo
Posts: 427
Joined: Fri Sep 16, 2016 4:55 pm
Location: Saint Paul, Minnesota USA

Re: Using GAN to play chess

Post by Leo » Thu Feb 23, 2017 4:46 pm

EvgeniyZh wrote:Authors of Style Transfer Generative Adversarial Networks: Learning to Play Chess Differently proposed to use Style Transfer techniques to teach NN to play chess.

What do you think?
Interesting paper.
Brainfish fan. AMD fan.

Tony P.
Posts: 81
Joined: Sun Jan 22, 2017 7:30 pm
Location: Russia

Re: Using GAN to play chess

Post by Tony P. » Thu Feb 23, 2017 4:59 pm

I've just downloaded the paper and haven't figured out its formulas yet...

Evgeniy, could you please give us a brief explanation of what 'style transfer' means?

Do I take it right that it means picking up a 'model' human player and setting up a loss function (to use in NN training) in a way that penalizes the NN for playing 'unlike' the model? E.g. if one wants to train the NN to play in the Chigorin style, then one may add a term to the loss function corresponding to an exchange of a knight for a bishop, so that the AI become less likely to give a knight up, as Chigorin preferred knights to bishops.

If so, it looks like the primary application of style transfer is training an AI tailored to score well against a particular human. E.g. one may want to train an NN to play like Carlsen and then train an 'anti-Carlsen' NN that prefers to create positions where the model of Magnus is most likely to play an inaccurate move, which can then be used by a world championship contender in off-the-board opening preparation to help him defeat Magnus in the title match.

If I understand the concept correctly, then I'll surely be using style transfer in my future AIs, and actually, I had thought about this before I learned from you that NNs using this method already exist.

Tony P.
Posts: 81
Joined: Sun Jan 22, 2017 7:30 pm
Location: Russia

Re: Using GAN to play chess

Post by Tony P. » Thu Feb 23, 2017 6:24 pm

Tony P. wrote:E.g. if one wants to train the NN to play in the Chigorin style, then one may add a term to the loss function corresponding to an exchange of a knight for a bishop, so that the AI become less likely to give a knight up, as Chigorin preferred knights to bishops.
No, my impression was wrong. The GAN and STGAN algorithms don't add hand-coded terms corresponding to specific features of a player's style, they only subtract a generic 'discriminator term' that tells only that the move chosen by the generator NN is different from the one chosen by the discriminator NN trained on master games but doesn't distinguish between the reasons why the move is different. So the learning occurs in a rather model-free fashion.

Clearly, the application of GAN to chess is in its infancy. In particular, I don't like the naive 768-dimensional board representation by piece occupancy bitmaps used by the authors. The more elaborate board representation used in Matthew Lai's engine Giraffe is more promising as it's smoother (similar positions are closer in the feature space), which speeds up the convergence.

Tony P.
Posts: 81
Joined: Sun Jan 22, 2017 7:30 pm
Location: Russia

Re: Using GAN to play chess

Post by Tony P. » Thu Feb 23, 2017 7:47 pm

Curiously, David Pfau and Oriol Vinyals, who're working for DeepMind, have pointed out the close connection between GANs and actor-critic reinforcement learning methods, and the latter have been shown to be efficient at outperforming humans at video games.

Post Reply