alphago原理简述.pdfVIP

Mastering the Game of Go With Deep Neural Networks and Tree Search Nabiha Asghar 27th May 2016 AlphaGo by Google DeepMind 鈥 Go: ancient Chinese board game. Simple rules, but far more complicated than Chess 鈥 Oct 鈥?5: defeated Fan Hui (2-dan European Go champion) 5 鈥 0 (news delayed till January 2016 to coincide with the publication in Nature) 鈥 Mar 鈥?6: defeated Lee Se-dol (9-dan South Korean Go player) 4 鈥 1 鈥 鈥淟ast night was very gloomy鈥 Many people drank alcohol鈥滫 South Korean newspaper after Lee鈥檚 first defeat Before AlphaGo The strongest previous Go programs were all based on Monte Carlo Tree Search (MCTS) 鈥 Crazy Stone 鈥 2006 鈥 Mogo 鈥 2007 鈥 Fuego 鈥 2010 鈥 Pachi 鈥 2012 Game Tree Game Tree Game Tree 鈥 Optimal value of a node = best possible value the node鈥檚 player can guarantee for himself 鈥 Optimal value function: 肀?肀涰睖肀戫睊) 鈫 肀滍睗肀№睎肀氿€亷肀 肀ｐ€亷肀欗雹肀 Monte Carlo Simulations Q: How do we estimate the value of a node? Monte Carlo Simulations Q: How do we estimate the value of a node? Idea: 鈥 Run several simulations from that node by sampling actions from a policy distribution 饊亷 ~肀?饊亷|肀? 肀 鈥 Average the rewards from the simulations to obtain a Monte Carlo value estimate of the node Monte Carlo Tree Search (MCTS) Combine Monte Carlo simulations with game tree search 1. Selection: Select the action leading to the node with highest value in the tree 2. Evaluation/Rollout: When a leaf is encountered in the tree, use a stochastic policy to select actions for both players, till the game terminates 1 2 3. Backup/Update: Update the statistics (# of visits, # of win