Teaching Deep Convolutional Neural Networks to Play Go

Discussion of chess software programming and technical issues.

Moderators: hgm, Harvey Williamson, bob

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Post Reply
Michel
Posts: 2000
Joined: Sun Sep 28, 2008 11:50 pm

Teaching Deep Convolutional Neural Networks to Play Go

Post by Michel » Tue Dec 16, 2014 7:27 pm

Any comments from people that are knowledgeable about computer go?

http://arxiv.org/abs/1412.3409
Ideas=science. Simplification=engineering.
Without ideas there is nothing to simplify.

AlvaroBegue
Posts: 912
Joined: Tue Mar 09, 2010 2:46 pm
Location: New York
Full name: Álvaro Begué (RuyDos)

Re: Teaching Deep Convolutional Neural Networks to Play Go

Post by AlvaroBegue » Tue Dec 16, 2014 7:40 pm

That was circulated in the computer-go mailing list (computer-go@computer-go.org) a couple of days ago. Most of us are excited about it and several of us had been thinking about similar things for a while. Aja Huang ( https://chessprogramming.wikispaces.com ... hieh+Huang ) says he will soon publish even better results.

The most exciting thing would be a clever hybrid of Monte Carlo techniques and Neural Network techniques, because they have very different strengths. Some people pointed out that a technique CrazyStone used in 2007 is roughly equivalent to using a shallow NN within the MC simulations and in creating priors for the tree-search part of MCTS. This type of deep NN could be a better method for the latter use.

You can follow the thread in the mailing list's archive here: http://computer-go.org/pipermail/comput ... 07010.html (I am always confused by the navigation in those threads.)

Daniel Shawul
Posts: 3561
Joined: Tue Mar 14, 2006 10:34 am
Location: Ethiopia
Contact:

Re: Teaching Deep Convolutional Neural Networks to Play Go

Post by Daniel Shawul » Tue Dec 16, 2014 11:31 pm

Is deep NN more classification achieved by using more layers? Many Go engines may already have pretty good pattern recognition used in the selection phase of MCTS, but not something that is used for game play by its own, as is the case here and in GnuGo. I would be surprized if this new method beats Remi's elo-based pattern recognition for the purpose of improving MCTS. How much this new NN helps current state-of-art Go engines depends on how much their move selection/ordering routines sucked before. In any case, I guess that even a 1% increase in move ordering quality would be beneficial.

My 0.2 cents.

PS: Be an applied Mathematician so you don't have to envy engineers :)

AlvaroBegue
Posts: 912
Joined: Tue Mar 09, 2010 2:46 pm
Location: New York
Full name: Álvaro Begué (RuyDos)

Re: Teaching Deep Convolutional Neural Networks to Play Go

Post by AlvaroBegue » Wed Dec 17, 2014 12:44 am

Daniel Shawul wrote:Is deep NN more classification achieved by using more layers? Many Go engines may already have pretty good pattern recognition used in the selection phase of MCTS, but not something that is used for game play by its own, as is the case here and in GnuGo. I would be surprized if this new method beats Remi's elo-based pattern recognition for the purpose of improving MCTS. How much this new NN helps current state-of-art Go engines depends on how much their move selection/ordering routines sucked before. In any case, I guess that even a 1% increase in move ordering quality would be beneficial.
Computer vision has seen a recent shift. The traditional techniques (say, 5 years ago) consisted of hand-crafted feature detectors followed by some simple machine-learning technique. Convolutional NNs have revolutionized the field, by extending the machine learning to also decide what the feature detectors should be.

Rémi's Elo-based method of combining patterns is simply SoftMax by another name (if I remember correctly; it's been years since I read the paper), which is a very reasonable choice for a machine-learning mechanism to combine features, and it's also what the last layer of a CNN would probably be. Extending the machine learning to include the definition of the features might allow for the discovery of non-obvious patterns. Particularly there is a concept of "good shape" in go that a CNN might be able to capture better than previous techniques.

jdart
Posts: 3663
Joined: Fri Mar 10, 2006 4:23 am
Location: http://www.arasanchess.org

Re: Teaching Deep Convolutional Neural Networks to Play Go

Post by jdart » Mon Dec 22, 2014 12:46 am

One of the issues with NNs is that generally they are not optimized for runtime efficiency. So you can get an accurate model but not one it's feasible to evaluate very, very frequently.

zamar
Posts: 613
Joined: Sun Jan 18, 2009 6:03 am

Re: Teaching Deep Convolutional Neural Networks to Play Go

Post by zamar » Mon Dec 29, 2014 10:41 pm

Michel wrote:Any comments from people that are knowledgeable about computer go?

http://arxiv.org/abs/1412.3409
Not that I'm too knowledgeable, but the result was very surprising for me. I couldn't image that a neural network could play so strong go without any search, that's remarkable.

However there might be some overoptimism in the air. NNs are likely to be able to give some boost for MCTS based searches, but they are not going to help with the misevaluation of life & death or semeais during the playout phase (which is one of the key blockers)
Joona Kiiski

Michel
Posts: 2000
Joined: Sun Sep 28, 2008 11:50 pm

Re: Teaching Deep Convolutional Neural Networks to Play Go

Post by Michel » Wed Dec 31, 2014 11:38 am

Follow up with even better results

http://arxiv.org/abs/1412.6564

97% against GnuGo without any search.
Ideas=science. Simplification=engineering.
Without ideas there is nothing to simplify.

Vinvin
Posts: 4174
Joined: Thu Mar 09, 2006 8:40 am
Full name: Vincent Lejeune

Re: Teaching Deep Convolutional Neural Networks to Play Go

Post by Vinvin » Wed Dec 31, 2014 12:42 pm

Michel wrote:Follow up with even better results

http://arxiv.org/abs/1412.6564

97% against GnuGo without any search.
Great !
How many times does it take for a move ?

Post Reply