Secret of Googlenomics

Tuesday, May 26th, 2009

Steven Levy explains the Secret of Googlenomics:

But as the business grew, Kamangar and Veach decided to price the slots on the side of the page by means of an auction. Not an eBay-style auction that unfolds over days or minutes as bids are raised or abandoned, but a huge marketplace of virtual auctions in which sealed bids are submitted in advance and winners are determined algorithmically in fractions of a second. Google hoped that millions of small and medium companies would take part in the market, so it was essential that the process be self-service. Advertisers bid on search terms, or keywords, but instead of bidding on the price per impression, they were bidding on a price they were willing to pay each time a user clicked on the ad. (The bid would be accompanied by a budget of how many clicks the advertiser was willing to pay for.) The new system was called AdWords Select, while the ads at the top of the page, with prices still set by humans, was renamed AdWords Premium.

One key innovation was that all the sidebar slots on the results page were sold off in a single auction. (Compare that to an early pioneer of auction-driven search ads, Overture, which held a separate auction for each slot.) The problem with an all-at-once auction, however, was that advertisers might be inclined to lowball their bids to avoid the sucker’s trap of paying a huge amount more than the guy just below them on the page. So the Googlers decided that the winner of each auction would pay the amount (plus a penny) of the bid from the advertiser with the next-highest offer. (If Joe bids $10, Alice bids $9, and Sue bids $6, Joe gets the top slot and pays $9.01. Alice gets the next slot for $6.01, and so on.) Since competitors didn’t have to worry about costly overbidding errors, the paradoxical result was that it encouraged higher bids.

“Eric Veach did the math independently,” Kamangar says. “We found out along the way that second-price auctions had existed in other forms in the past and were used at one time in Treasury auctions.” (Another crucial innovation had to do with ad quality, but more on that later.)

Google’s homemade solution to its ad problem impressed even Paul Milgrom, the Stanford economist who is to auction theory what Letitia Baldridge is to etiquette. “I’ve begun to realize that Google somehow stumbled on a level of simplification in ad auctions that was not included before,” he says. And applying a variation on second-price auctions wasn’t just a theoretical advance. “Google immediately started getting higher prices for advertising than Overture was getting.”

This is the secret sauce though:

In fact, there’s a key component that few users know about and even sophisticated advertisers don’t fully understand. The bids themselves are only a part of what ultimately determines the auction winners. The other major determinant is something called the quality score. This metric strives to ensure that the ads Google shows on its results page are true, high-caliber matches for what users are querying. If they aren’t, the whole system suffers and Google makes less money.

Google determines quality scores by calculating multiple factors, including the relevance of the ad to the specific keyword or keywords, the quality of the landing page the ad is linked to, and, above all, the percentage of times users actually click on a given ad when it appears on a results page. (Other factors, Google won’t even discuss.) There’s also a penalty invoked when the ad quality is too low — in such cases, the company slaps a minimum bid on the advertiser. Google explains that this practice — reviled by many companies affected by it — protects users from being exposed to irrelevant or annoying ads that would sour people on sponsored links in general. Several lawsuits have been filed by would-be advertisers who claim that they are victims of an arbitrary process by a quasi monopoly.

You can argue about fairness, but arbitrary it ain’t. To figure out the quality score, Google needs to estimate in advance how many users will click on an ad. That’s very tricky, especially since we’re talking about billions of auctions. But since the ad model depends on predicting clickthroughs as perfectly as possible, the company must quantify and analyze every twist and turn of the data. Susan Wojcicki, who oversees Google’s advertising, refers to it as “the physics of clicks.”

Google has gone a bit auction-crazy:

The company used auctions to place ads on other Web sites (that program was dubbed AdSense). “But the really gutsy move,” Varian says, “was using it in the IPO.” In 2004, Google used a variation of a Dutch auction for its IPO; Brin and Page loved that the process leveled the playing field between small investors and powerful brokerage houses. And in 2008, the company couldn’t resist participating in the FCC’s auction to reallocate portions of the radio spectrum.

Google even uses auctions for internal operations, like allocating servers among its various business units. Since moving a product’s storage and computation to a new data center is disruptive, engineers often put it off. “I suggested we run an auction similar to what the airlines do when they oversell a flight. They keep offering bigger vouchers until enough customers give up their seats,” Varian says. “In our case, we offer more machines in exchange for moving to new servers. One group might do it for 50 new ones, another for 100, and another won’t move unless we give them 300. So we give them to the lowest bidder—they get their extra capacity, and we get computation shifted to the new data center.”

I love the story of how Google’s chief economist decided to go into economics in the first place:

It’s a satisfying development for Varian, a guy whose career as an economist was inspired by a sci-fi novel he read in junior high. “In Isaac Asimov’s first Foundation trilogy, there was a character who basically constructed mathematical models of society, and I thought this was a really exciting idea. When I went to college, I looked around for that subject. It turned out to be economics.”

Leave a Reply