Games are to AI researchers what fruit flies are to biology. A new AI has mastered many classic video games by combining two types of machine learning:
The first, called deep learning, uses a brain-inspired architecture in which connections between layers of simulated neurons are strengthened on the basis of experience. Deep-learning systems can then draw complex information from reams of unstructured data (see Nature 505, 146–148; 2014). Google, of Mountain View, California, uses such algorithms to automatically classify photographs and aims to use them for machine translation.
The second is reinforcement learning, a decision-making system inspired by the neurotransmitter dopamine reward system in the animal brain. Using only the screen’s pixels and game score as input, the algorithm learnedby trial and error which actions — such as go left, go right or fire — to take at any given time to bring the greatest rewards. After spending several hours on each game, it mastered a range of arcade classics, including car racing, boxing and Space Invaders.
Only games with a simple and timely relationship between actions and score were amenable to reinforcement learning.