Teaching Deep Convolutional Neural Networks to Play Go

[Michel]

Any comments from people that are knowledgeable about computer go?

http://arxiv.org/abs/1412.3409

[AlvaroBegue]

That was circulated in the computer-go mailing list (computer-go@computer-go.org) a couple of days ago. Most of us are excited about it and several of us had been thinking about similar things for a while. Aja Huang ( https://chessprogramming.wikispaces.com ... hieh+Huang ) says he will soon publish even better results.

The most exciting thing would be a clever hybrid of Monte Carlo techniques and Neural Network techniques, because they have very different strengths. Some people pointed out that a technique CrazyStone used in 2007 is roughly equivalent to using a shallow NN within the MC simulations and in creating priors for the tree-search part of MCTS. This type of deep NN could be a better method for the latter use.

You can follow the thread in the mailing list's archive here: http://computer-go.org/pipermail/comput ... 07010.html (I am always confused by the navigation in those threads.)

[Daniel Shawul]

Is deep NN more classification achieved by using more layers? Many Go engines may already have pretty good pattern recognition used in the selection phase of MCTS, but not something that is used for game play by its own, as is the case here and in GnuGo. I would be surprized if this new method beats Remi's elo-based pattern recognition for the purpose of improving MCTS. How much this new NN helps current state-of-art Go engines depends on how much their move selection/ordering routines sucked before. In any case, I guess that even a 1% increase in move ordering quality would be beneficial.

My 0.2 cents.

PS: Be an applied Mathematician so you don't have to envy engineers

[AlvaroBegue]

Is deep NN more classification achieved by using more layers? Many Go engines may already have pretty good pattern recognition used in the selection phase of MCTS, but not something that is used for game play by its own, as is the case here and in GnuGo. I would be surprized if this new method beats Remi's elo-based pattern recognition for the purpose of improving MCTS. How much this new NN helps current state-of-art Go engines depends on how much their move selection/ordering routines sucked before. In any case, I guess that even a 1% increase in move ordering quality would be beneficial.

Computer vision has seen a recent shift. The traditional techniques (say, 5 years ago) consisted of hand-crafted feature detectors followed by some simple machine-learning technique. Convolutional NNs have revolutionized the field, by extending the machine learning to also decide what the feature detectors should be.

Rémi's Elo-based method of combining patterns is simply SoftMax by another name (if I remember correctly; it's been years since I read the paper), which is a very reasonable choice for a machine-learning mechanism to combine features, and it's also what the last layer of a CNN would probably be. Extending the machine learning to include the definition of the features might allow for the discovery of non-obvious patterns. Particularly there is a concept of "good shape" in go that a CNN might be able to capture better than previous techniques.

[jdart]

One of the issues with NNs is that generally they are not optimized for runtime efficiency. So you can get an accurate model but not one it's feasible to evaluate very, very frequently.

[zamar]

Any comments from people that are knowledgeable about computer go?

http://arxiv.org/abs/1412.3409

Not that I'm too knowledgeable, but the result was very surprising for me. I couldn't image that a neural network could play so strong go without any search, that's remarkable.

However there might be some overoptimism in the air. NNs are likely to be able to give some boost for MCTS based searches, but they are not going to help with the misevaluation of life & death or semeais during the playout phase (which is one of the key blockers)

[Michel]

Follow up with even better results

http://arxiv.org/abs/1412.6564

97% against GnuGo without any search.

[Vinvin]

Follow up with even better results

http://arxiv.org/abs/1412.6564

97% against GnuGo without any search.

Great !
How many times does it take for a move ?