Two Types of Machine Learning

Tuesday, March 3rd, 2015

Games are to AI researchers what fruit flies are to biology. A new AI has mastered many classic video games by combining two types of machine learning:

The first, called deep learning, uses a brain-inspired architecture in which connections between layers of simulated neurons are strengthened on the basis of experience. Deep-learning systems can then draw complex information from reams of unstructured data (see Nature 505, 146–148; 2014). Google, of Mountain View, California, uses such algorithms to automatically classify photographs and aims to use them for machine translation.

The second is reinforcement learning, a decision-making system inspired by the neuro­transmitter dopamine reward system in the animal brain. Using only the screen’s pixels and game score as input, the algorithm learnedby trial and error which actions — such as go left, go right or fire — to take at any given time to bring the greatest rewards. After spending several hours on each game, it mastered a range of arcade classics, including car racing, boxing and Space Invaders.

Only games with a simple and timely relationship between actions and score were amenable to reinforcement learning.

Designing The Best Board Game On The Planet

Wednesday, January 21st, 2015

Twilight Struggle is the best board game on the planet — at least according to BoardGameGeek:

Twilight Struggle traces its roots to the early 2000s and a board gaming club at George Washington University. That’s where Gupta and co-designer Jason Matthews met. Not GW students themselves, they were friends with some, and would go to the school to play and also to bemoan the increasing complexity of historical games — a genre especially dear to them. The rulebooks were overlong, the game mechanics baroque.

Simplification, to Gupta and Matthews, was the name of their design philosophy. Rather than overwhelm players with a fat rulebook at the start, the designers spread the information required throughout the gameplay, on cards. A typical Twilight Struggle card reads, “Truman Doctrine: Remove all USSR Influence from a single uncontrolled country in Europe.” The Twilight Struggle rulebook is a relatively slender 24 pages.

They originally intended to do a game about the Spanish Civil War but realized they’d been scooped by a guy in Spain. “We’re probably not going to do a better job than he is,” Gupta joked. They eventually settled on the Cold War. Most games on the topic had focused on when the Cold War got hot. But thermonuclear war is depressing. Gupta and Matthews instead designed a game about the geopolitics, rather than a hypothetical military conflict.

Matthews, of Alexandria, Virginia, is an American history expert and was the legislative director for Sen. Mary Landrieu of Louisiana. Gupta, a history buff, was doing policy work at a think tank, then was in school for computer science, before dropping out after he landed his first job in the video-game industry. The two would discuss key aspects of the Cold War — the domino theory, the arms race, the space race — and these would make their way into the game.

But publishers balked. “The Cold War? Why would anyone want to play a game about the Cold War?” Gupta recalled being asked.

Salvation came in the form of the company GMT Games, and its Project 500 — a kind of Kickstarter before Kickstarter was cool. Interested gamers would pledge money, and GMT would print the game if enough capital was raised. Even then, it took a grinding 18 months for Twilight Struggle to generate enough pledges to warrant a printing.

That first printing sold out in 20 minutes. It has gone on to amass 17,781 ratings on BoardGameGeek, as I write, with an average rating of 8.33.


Twilight Struggle is emblematic of a sea change from older, magisterial games with titles like Rise and Decline of the Third Reich, War in Europe and The Civil War. (The Civil War’s listed playing time is 1,200 minutes.) The redistribution of game information from massive rulebooks onto game cards was a revolution that can be traced to Mark Herman’s We the People, a game about the American Revolution, and Paths of Glory, a World War I game by Ted Raicer.

“What that meant was the game was a lot easier to learn,” Gupta said. “That started a renaissance in historical gaming.”

The Packers of Catan

Saturday, January 17th, 2015

Apparently the Green Bay Packers are obsessed with Settlers of Catan.

I suppose they’ll move on to The Cones of Dunshire:

The Checkered Game of Life

Friday, December 26th, 2014

The Checkered Game of Life, released in 1860 by Milton Bradley, evolved into the very different modern-day game of Life:

Instead of becoming hair stylists or police officers and poking plastic peg-shaped children into candy-colored SUVs, players of the original game landed on spaces marked with “virtues” and “vices.”

Spaces like “honesty” and “truth” sprung you forward; spaces like “gambling” and “disgrace” slowed your progress.

“I think religion really affected how games were made,” Keren-Detar says. “In Europe, there were more excuses to play, whereas in the US, we didn’t have an established aristocracy. So the idea of playing was very negative, or thought of as lazy and idle and sinful. These games had morphed themselves into being entertaining, but also educational, so you wouldn’t get in trouble for playing a game on a Sunday if it’s based on how to become a better Christian.”

Checkered Game of Life

The pre-Civil War game Mansion of Happiness was even more righteous, Keren-Detar says. Some of its illustrated squares showed characters suffering for their sins with consequences like whipping posts or pillories.

Neither Mansion of Happiness nor The Checkered Game of Life featured dice, probably because dice were still strongly associated with gambling and sin. Checkered Game of Life used a spinning number wheel instead, a feature that survives to this day.

The shift in the narrative of Life over the centuries, Keren-Detar says, suggests a shift in American values.

“The narrative wasn’t dying and going to heaven—it was trying to go to college and be productive and get money,” Keren-Detar says of the version of Life we’re most familiar with. “A lot of games that came out around that time changed from being religious to being industrious.”

Denigration of the Great Revolution

Thursday, November 20th, 2014

The latest Assassin’s Creed video game takes place in Paris during the French Revolution, and French leftists are appalled that the heroes of the People are depicted as bloodthirsty savages:

The former leftist French presidential candidate, Jean-Luc Mélenchon, called it “propaganda against the people, the people who are [portrayed as] barbarians, bloodthirsty savages,” while the “cretin” that is Marie-Antoinette and the “treacherous” Louis XVI are portrayed as noble victims. “The denigration of the great Revolution is a dirty job to instill more self-loathing and déclinisme in the French,” he told Le Figaro (link in French). The secretary general of the Left Front, Alexis Corbière, said on his blog (link in French):

To all those who will buy Assassin’s Creed: Unity, I wish them a good time, but I also tell them that the pleasure of playing does not stop you from thinking. Play, yes, but do not let yourself be manipulated by those who make propaganda.

Ubisoft, the maker of the Assassin’s Creed series of video games, which has been going since 2007 and has sold more than 70 million copies, is in fact French. One of the makers of the game replied (link in French) that Assassin’s Creed: Unity is a “consumer video game, not a history lesson” but did say that his team hired a historian and specialists on the Terror and other aspects of the Revolution. Le Monde lays out seven errors in the game here (in French).

Greek Hold ‘Em

Wednesday, November 19th, 2014

Existential Comics turns its philosophical eye toward poker — Greek Hold ‘Em:

Existential Comics Greek Hold Em 2

Creator of “Choose Your Own Adventure” Dies

Tuesday, November 18th, 2014

R.A. Montgomery, co-author and publisher of the “Choose Your Own Adventure” books, has passed away.

Choose Your Own Adventure 001f Cave of Time

His partner, Edward Packard, was considered the better writer, but Montgomery had his strengths:

Montgomery, on the other hand, often eschewed internal consistency in favor of big ideas, and his books have their own bizarre charm. While Packard was writing the standard sword-and-sorcery story The Forbidden Castle about dragons, knights, and princesses, Montgomery unleashed the berserk House of Danger which involved super-intelligent monkeys plotting to destabilize the world economy via counterfeiting, psychic detectives, Civil War ghosts, alien abduction, holograms, age regression, cannibalism, secret environmental conspiracies, and one ending that has the reader turned into Genghis Khan.

Dogs Playing Dungeons & Dragons

Friday, November 14th, 2014

Dogs Playing Poker is fine, but Dogs Playing Dungeons & Dragons is better:

Dogs Playing D&D

The Philosophy of the Science of Poker

Friday, October 10th, 2014

Existential Comics illustrates the philosophy of the science of poker:

Existential Comics Philosophy of Science of Poker

(Read the whole thing.)

Sports are Broken

Thursday, October 2nd, 2014

Most sports were invented years ago, Scott Adams(Dilbert) notes, and much has changed since then:

Equipment technology has improved. We have far more knowledge of health risks. Our attention spans have shrunk, and our options for leisure activities have increased. If you were to invent the rules of sports today, from a blank slate, you would do a lot of things differently.

For example, when tennis was invented, serving was just a way to start the rally. One player bunted the ball into the service box and it was on.

Fast-forward to 2014.

Now the pros are 6’8″, their rackets and strings are made from exotic materials, and they are trained to serve at 140 miles per hour. As you might imagine, that creates a lot of double-faults and aces. Both are boring.

To fix tennis, eliminate the serve. That is already happening where I live. A group of folks in my town already play without the serve. Under the no-serve rules either player can start the rally and the point is live on the third hit. You play to 21, win by two, so no more funky tennis scoring with the 15-30-40 ridiculousness. This version of tennis is about twice as fun as playing serve-and-miss while wishing you were getting some exercise.

In 2014 we know a lot about the dangers of concussions. Football wouldn’t be allowed as a youth sport if it were invented today. Soccer players wouldn’t be allowed to head the ball for the same reason. So let’s get rid of football entirely, at least for kids, and make it a penalty to head a soccer ball.

Speaking of soccer, if we invented that game today the goals would be 50% wider to create more scoring and there would be TV timeouts built into the game design so the major networks could more easily monetize with commercials. And the off-side rule has to go; that is just boring. And while we are at it, let’s put up a glass wall around the field so the ball stays in play.

Baseball could be interesting if it were slow-pitch and any ball hit out of the park were ruled an out. I might add another player to the outfield, but the idea is to have lots of hits and lots of defense. In the age of smartphones, no one has the patience to watch nine guys standing around in the grass wondering when something might happen.

Volleyball has one of the most ridiculous rules in sports. The players need to rotate positions after every point. The well-coached teams do a quick, synchronized rotation as soon as the serve is hit to get into the positions they prefer instead of the positions the game rules require. Let’s just lose the player rotation rule.

Golf also needs to be fixed. The main problem is that 18 holes is far too much time commitment and 9 holes seem too little. I hear that 12-hole courses are being built for exactly that reason. That makes sense in 2014.

Another thing that golf needs to lose is the annoying foursome behind you that makes you feel rushed and guilty. I don’t know how to fix that in an economically way, but it sure would improve the game if someone did.

World Of PeaceCraft

Friday, August 22nd, 2014

“You’ve fought terrorists in Call of Duty and alien hordes in Gears of War. Well, now get ready for the opposite of that“:

(From Last Week Tonight with John Oliver.)

Creating History Is Hard

Thursday, August 14th, 2014

I still haven’t played Dwarf Fortress, but its world-building process intrigues me:

Adams has, over the years, developed an intricate process to simulate eons of complex geologic time, a way of stacking fractal layers and blending them with algorithms to give life to each world.

The first layer plots the annual rainfall of each map location. Then a separate fractal simulates the deposition of mineral elements throughout the underground strata, giving the land itself a kind of texture. A temperature fractal is generated and rough biomes emerge as contiguous tiles on the map that contain a subset of closely related flora and fauna.

The order here is important, because in the next step — drainage — Dwarf Fortress begins to simulate the complex forces of erosion. Only after the biomes have been created can the rivers run, slashing deep valleys as they flow toward unnamed oceans. When they finally meet the sea a salinity algorithm kicks in to define the areas for swampy river deltas, alluvial islands and mangrove swamps.


In his research for the game Adams learned that in the real world when warm, wet air travels up the side of a mountain it loses moisture. Rain precipitates out creating areas like rain forests and snow capped peaks. On the other side of the mountain deserts form in areas that are called “rain shadows.”


After the map has been locked into place, the game assigns a kind of energy to each region, ranging from good to evil on a scale of one to 20. It then uses the positive and negative energy of each area to generate place names — The Ocean of Muting sit along the edge of The Jungles of Mire near the Ivory Hills — and on and on creating hundreds of uniquely named regions.

But these are just the names as translated for the player. Adams says that each area of the map has been named by one of four cultures. Human, dwarven, elvish and goblin languages are actually programmed into the game.

Can Video Games Make You Smarter?

Friday, July 11th, 2014

Can video games make you smarter? Yeah, sort of:

Magnus Carlsen’s Route to Chess

Wednesday, July 2nd, 2014

World Chess Champion Magnus Carlsen’s route to chess took longer than his subsequent progress might suggest:

Henrik, 52, a keen chess player himself, remembers introducing the game to Magnus and his older sister, Ellen, now 25, when his son was turning 5. But after a month or two, Henrik says, “I gave up, basically, in the sense that we continued to play chess occasionally, but I didn’t have any ambitions.” He knew that legendary players such as Capablanca and Kasparov had understood the game — he clicks his fingers — “just like that.” Magnus and his sister, he says, “learned the rules quickly, and they could capture a piece, but to get two or more pieces working together, which is what chess is about, this spatial vision took a long time.”

At the time, Henrik reconciled himself to the fact that chess would simply be an enjoyable family pastime. “I felt, OK, they’re definitely not geniuses, but it doesn’t matter. Because, I mean, we loved our children. Chess was something we could do together, just a hobby, like playing cards or anything else.” In the meantime, there were signs that Magnus had the aptitude and the determination to perform impressive mental feats. Sigrun, 51, recalls her son sitting for hours with puzzles or making advanced Lego models, patiently working his way through pages and pages of instructions meant for children a decade older. “He had the ability to sit for a very long time, even when he was small,” she recalls.

This quality has contributed in no small measure to his success; chess commentators draw attention to his ability to wear down opponents, to wait patiently for them to make the tiniest mistake. Magnus himself maintains that he is an aggressive player but that audacity isn’t always what’s called for. “When you play against the best people in the world, they see through your plans, and you cannot win with a swashbuckling attack all the time,” he says. “You just need to take what’s there.”

His parents are eager to point out that he wasn’t an obviously faster learner than his sisters (he also has two younger siblings, Ingrid, 20, and Signe, 17) but that he kept on going, focusing his attention on a specific subject, such as car brands, until he knew it inside out. When I ask Magnus about his childhood proficiency, he replies simply: “I didn’t particularly know if I was good at it or not; I just tried to do it.”

Then came a turning point. Just before Magnus turned 8, says Henrik, “Ellen suddenly understood enough to make it interesting for me to play with her.” Magnus would sit to watch them and, a little later, join in. Henrik’s dilemma was that if he adopted poor strategy, his children wouldn’t learn anything, but he also didn’t want them to become discouraged. So he began to play with limited resources — just his king and a pawn — slowly adding pieces as they learned the game. Magnus’s interest started to grow, although Henrik maintains that “he just wanted to beat his sister.” He had a competitive streak even as a small child? “Yes, absolutely,” Sigrun says, “he still has that.” More competitive than his sisters? “Absolutely.” She laughs and gestures to her husband. “It’s not from me, it’s from him!”

Soon he was entering and very quickly winning tournaments. At home, during dinner, he began sitting apart from the family so he could study his chessboard while eating. “He was in the same room,” remembers Sigrun, “so we could speak to him if we wanted to; he could hear what we were talking about if he wanted to join.” Despite their unorthodox meals, they were, and remain, a close family.

The Mystery of Go

Wednesday, June 11th, 2014

In 1965 New Scientist published I.J. Good’s The Mystery of Go:

Go, the Japanese national pastime, was recently described by Ralph Fox, a Princeton professor of mathematics, as the most interesting game in the world. At any rate many expert chess players, including Emanuel Lasker, who was World Chess Champion for 28 years, have held that Go is more interesting than chess, and it is not easy to think of any third game that is a serious rival. Another, unrelated chess master, Edward Lasker, believes that Go will replace chess as the leading intellectual game of the Occident just as it has reigned supreme in the Orient for some four thousand years.

In Japan the game is known as I-go or Go, in China as Wei-k’i or Wei-Chi, in Korea as Patok. It is played by a high proportion of educated people in Japan, including many Geisha girls, and ability at Go is relevant to promotion in many firms.


The rules are basically so simple that perhaps a game very much like Go is played in many extra-terrestrial places, even within our own galaxy.


A weakness in chess is that there is too much knowledge about the openings, and it is constantly increasing. It is not good for a game if its mastery requires much rote learning. In Go, although play in the corners of the board is somewhat stereotyped among masters, one can become a competent Go player by occidental standards with very little knowledge of these so-called Joseki.

In chess, the parrots can be defeated by playing Randomised Chess, wherein the pieces on the back lines are permuted at random. Similarly, Randomised Go could be defined in terms of a random deletion of some of the vertices near the corners of the board. The corners could even be abolished by playing on a cylinder or an anchor ring: this could be done, without using a magnetic set, by identification of opposite sides of the ordinary board, either one pair of sides or both pairs. Another form of Go is that with more than two players, all against all, with one colour for each player. Since it is possible for players to form coalitions, this form of Go bears some resemblance to power politics and is liable to create an emotional scene.

But ordinary Go is fascinating enough for people with an IQ between 110 and 190 — in fact, too fascinating. Sometimes the game becomes an addiction, like smoking, food, drink, television, chess and women. It seems to appeal especially to scientists and mathematicians, because of the emergence of the Gestalt — a unity with a significant pattern — out of a collection of discrete entities and axioms.

Go on a computer? — In order to programme a computer to play a reasonable game of Go — rather than merely a legal game — it is necessary to formalise the principles of good strategy, or to design a learning programme. The prlnciples are more qualitative and mysterious than in chess, and depend more on judgment. So I think it will be even more difficult to programme a computer to play a reasonable game of Go than of chess.

The experienced player will often be unable to explain convincingly to a beginner why one move is better than another. A move might be regarded as good because it looks influential, or combines attack and defence, or preserves the initiative, or because if we had not played at that vertex the opponent would have done so; or it might be regarded as bad because it was too bold or too timid, or too close to the enemy or too far away. If these and other qualitative judgments. could be expressed in precise quantitative terms, then good strategy could be programmed for a computer; but hardly any progress has been made in this direction.

Indeed, hardly any progress was made in computer Go for decades after Good’s article came out, until Rémi Coulom’s Crazy Stone combined a tree search with Monte Carlo methods:

He christened the new algorithm Monte Carlo Tree Search, or MCTS, and in January of 2006, Crazy Stone won its first tournament. After he published his findings, other programmers quickly integrated MCTS into their Go programs, and for the next two years, Coulom vied for dominance with another French program, Mogo, that ran a refined version of the algorithm.

Although Crazy Stone ended up winning the UEC Cup in 2007 and 2008, Mogo’s team used man-machine matches to win the publicity war. Coulom felt the lack of attention acutely. When neither the public nor his university gave him the recognition he deserved, he lost motivation and stopped working on Go for nearly two years.

Coulom might have given up forever had it not been for a 2010 email from Ikeda Osamu, the CEO of Unbalance, a Japanese computer game company. Ikeda wanted to know if he’d be willing to license Crazy Stone. Unbalance controlled about a third of the million-dollar global market in computer Go, but Zen’s commercial version had begun to increase its market share. Ikeda needed Coulom to give his company’s software a boost.

The first commercial version of Crazy Stone hit the market in spring of 2011. In March of 2013, Coulom’s creation returned to the UEC Cup, beating Zen in the finals and — given a four-stone head-start — winning the first Densei-sen against Japanese professional Yoshio “The Computer” Ishida. The victories were huge for Coulom, both emotionally and financially. You can see their significance in the gift shop of the Japan Go Association, where a newspaper clipping, taped to the wall behind display copies of Crazy Stone, shows the pro grimly succumbing to Coulom’s creation.