Move Evaluation in Go Using Deep Convolutional Neural Networks
arXiv (Cornell University)Published 20 December 2014Open access
Chris J. Maddison, Aja Huang, Ilya Sutskever, David Silver
Citations92
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
Abstract
The game of Go is more challenging than other board games, due to the difficulty of constructing a position or move evaluation function. In this paper we investigate whether deep convolutional networks can be used to directly represent and learn this knowledge. We train a large 12-layer convolutional neural network by supervised learning from a database of human professional games. The network correctly predicts the expert move in 55% of positions, equalling the accuracy of a 6 dan human player. When the trained convolutional network was used directly to play games of Go, without any search, it beat the traditional search program GnuGo in 97% of games, and matched the performance of a state-of-the-art Monte-Carlo tree search that simulates a million positions per move.
Keywords
PsychologyComputer ScienceEconomics, Econometrics and Finance
Communications of the ACMImageNet classification with deep convolutional neural networks
75,709 Citations2017Alex Krizhevsky, Ilya Sutskever +1 more
A large, deep convolutional neural network was trained to classify the 1.2 million high-resolution images in the ImageNet LSVRC-2010 contest into the 1000 different classes and employed a recently developed regularization method called "dropout" that proved to be very effective.
arXiv (Cornell University)Very Deep Convolutional Networks for Large-Scale Image Recognition
75,540 Citations2014Karen Simonyan, Andrew Zisserman
This work investigates the effect of the convolutional network depth on its accuracy in the large-scale image recognition setting using an architecture with very small convolution filters, which shows that a significant improvement on the prior-art configurations can be achieved by pushing the depth to 16-19 weight layers.
Going deeper with convolutions
46,825 Citations2015Christian Szegedy, Wei Liu +7 more
A deep convolutional neural network architecture codenamed Inception is proposed that achieves the new state of the art for classification and detection in the ImageNet Large-Scale Visual Recognition Challenge 2014 (ILSVRC14).
Large Scale Distributed Deep Networks
2,916 Citations2012Jay B. Dean, Greg S. Corrado +10 more
This paper considers the problem of training a deep network with billions of parameters using tens of thousands of CPU cores and develops two algorithms for large-scale distributed training, Downpour SGD and Sandblaster L-BFGS, which increase the scale and speed of deep network training.
Lecture notes in computer scienceBandit Based Monte-Carlo Planning
2,802 Citations2006Levente Kocsis, Csaba Szepesvári
A new algorithm is introduced, UCT, that applies bandit ideas to guide Monte-Carlo planning and is shown to be consistent and finite sample bounds are derived on the estimation error due to sampling.
Lecture notes in computer scienceEfficient Selectivity and Backup Operators in Monte-Carlo Tree Search
1,223 Citations2007Rémi Coulom
A new framework to combine tree search with Monte-Carlo evaluation, that does not separate between a min-max phase and a Monte- carlo phase is presented, that provides finegrained control of the tree growth, at the level of individual simulations, and allows efficient selectivity.
New Mathematics and Natural ComputationPROGRESSIVE STRATEGIES FOR MONTE-CARLO TREE SEARCH
436 Citations2008Guillaume Chaslot, Mark H. M. Winands +3 more
Two progressive strategies for MCTS are introduced, called progressive bias and progressive unpruning, which enable the use of relatively time-expensive heuristic knowledge without speed reduction.
Artificial IntelligenceMonte-Carlo tree search and rapid action value estimation in computer Go
370 Citations2011Sylvain Gelly, David Silver
The Monte-Carlo revolution in computer Go is surveyed, the key ideas that led to the success of MoGo and subsequent Go programs are outlined, and for the first time a comprehensive description, in theory and in practice, of this extended framework for Monte- Carlo tree search is provided.
IEEE Transactions on Computational Intelligence and AI in GamesFuego—An Open-Source Framework for Board Games and Go Engine Based on Monte Carlo Tree Search
176 Citations2010Markus Enzenberger, Martin Müller +2 more
An overview of the development and current state of the FUEGO project is given, which describes the reusable components of the software framework and specific algorithms used in the Go engine.
Edinburgh Research Explorer (University of Edinburgh)Training Deep Convolutional Neural Networks to Play Go
125 Citations2015Christopher Clark, Amos Storkey
The convolutional neural networks trained in this work can consistently defeat the well known Go program GNU Go and win some games against state of the art Go playing program Fuego while using a fraction of the play time.
MPG.PuRe (Max Planck Society)Temporal Difference Learning of Position Evaluation in the Game of Go
120 Citations1993Nicol N. Schraudolph, Peter Dayan +1 more
This work demonstrates a viable alternative by training networks to evaluate Go positions via temporal difference (TD) learning, based on network architectures that reflect the spatial organization of both input and reinforcement signals on the Go board, and training protocols that provide exposure to competent (though unlabelled) play.
Lecture notes in computer sciencePACHI: State of the Art Open Source Go Program
64 Citations2012P Baudis, Jean-loup Gailly
A state of the art implementation of the Monte Carlo Tree Search algorithm for the game of Go and three notable original improvements: an adaptive time control algorithm, dynamic komi, and the usage of the criticality statistic are described.
Lecture notes in computer scienceMonte-Carlo Simulation Balancing in Practice
51 Citations2011Shih-Chieh Huang, Rémi Coulom +1 more
The effectiveness of simulation balancing is demonstrated in a more realistic setting and a state-of-the-art program, Erica, learned an improved playout policy on the 9×9 board, without requiring any external expert to provide position evaluations.
Lecture notes in computer scienceMimicking Go Experts with Convolutional Neural Networks
37 Citations2008Ilya Sutskever, Vinod Nair
The main result is that the ensemble learns to predict 36.9% of the moves made in test expert Go games, improving upon the state of the art, and that the best single convolutional neural network of the ensemble achieves 34% accuracy.
arXiv (Cornell University)Teaching Deep Convolutional Neural Networks to Play Go
13 Citations2014Christopher Clark, Amos Storkey
Convolutional neural networks trained to play Go are able to consistently defeat the well known Go program GNU Go, indicating it is state of the art among programs that do not use Monte Carlo Tree Search.
Lecture notes in computer scienceInvestigating the Limits of Monte-Carlo Tree Search Methods in Computer Go
13 Citations2014Shih-Chieh Huang, Martin Müller
This chapter discusses Monte-Carlo Tree Search methods for Go, and some of the challenges faced in dealing with several simultaneous fights, including the two safe groups problem, and dealing with coexistence in seki.