Exclusive: Inside Google Spanner, the Largest Single Database on Earth

Each morning, when Andrew Fikes sat down at his desk inside Google headquarters in Mountain View, California, he turned on the “VC” link to New York.
VC is Google shorthand for video conference. Looking up at the screen on his desk, Fikes could see Wilson Hsieh sitting inside a Google office in Manhattan, and Hsieh could see him. They also ran VC links to a Google office in Kirkland, Washington, near Seattle. Their engineering team spanned three offices in three different parts of the country, but everyone could still chat and brainstorm and troubleshoot without a moment’s delay, and this is how Google built Spanner.
“You walk into our cubes, and we’ve got VC on — all the time,” says Fikes, who joined Google in 2001 and now ranks among the company’s distinguished software engineers. “We’ve been doing this for years. It lowers all the barriers to communication that you typically have.”

‘As a distributed-systems developer, you’re taught from — I want to say childhood — not to trust time. What we did is find a way that we could trust time — and understand what it meant to trust time.’

— Andrew Fikes

The arrangement is only appropriate. Much like the engineering team that created it, Spanner is something that stretches across the globe while behaving as if it’s all in one place Unveiled the fall after years of hints and rumors, it’s the first worldwide database worthy of the name — a database designed to seamlessly operate across hundreds of data centers and millions of machines and trillions of rows of information.
Spanner is a creation so large, some have trouble wrapping their heads around it. But the end result is easily explained: With Spanner, Google can offer a web service to a worldwide audience, but still ensure that something happening on the service in one part of the world doesn’t contradict what’s happening in another.
Google’s new-age database is already part of the company’s online ad system — the system that makes its millions — and it could signal where the rest of the web is going. Google caused a stir when it published a research paper detailing Spanner in mid-September, and the buzz was palpable among the hard-core computer systems engineers when Wilson Hsieh presented the paper at a conference in Hollywood, California, a few weeks later.
“It’s definitely interesting,” says Raghu Murthy, one of the chief engineers working on the massive software platform that underpins Facebook — though he adds that Facebook has yet to explore the possibility of actually building something similar.

Google’s web operation is significantly more complex than most, and it’s forced to build custom software that’s well beyond the scope of most online outfits. But as the web grows, its creations so often trickle down to the rest of the world.
Before Spanner was revealed, many didn’t even think it was possible. Yes, we had “NoSQL” databases capable of storing information across multiple data centers, but they couldn’t do so while keeping that information “consistent” — meaning that someone looking at the data on one side of the world sees the same thing as someone on the other side. The assumption was that consistency was barred by the inherent delays that come when sending information between data centers.
But in building a database that was both global and consistent, Google’s Spanner engineers did something completely unexpected. They have a history of doing the unexpected. The team includes not only Fikes and Hsieh, who oversaw the development of BigTable, Google’s seminal NoSQL database, but also legendary Googlers Jeff Dean and Sanjay Ghemawat and a long list of other engineers who worked on such groundbreaking data-center platforms as Megastore and Dremel.
This time around, they found a new way of keeping time.
“As a distributed systems developer, you’re taught from — I want to say childhood — not to trust time,” says Fikes. “What we did is find a way that we could trust time — and understand what it meant to trust time.”

Time Is of the Essence

On the net, time is of the essence. Yes, in running a massive web service, you need things to happen quickly. But you also need a means of accurately keeping track of time across the many machines that underpin your service. You have to synchronize the many processes running on each server, and you have to synchronize the servers themselves, so that they too can work in tandem. And that’s easier said than done.
Typically, data-center operators keep their servers in sync using what’s called the Network Time Protocol, or NTP. This is essentially an online service that connects machines to the official atomic clocks that keep time for organizations across the world. But because it takes time to move information across a network, this method is never completely accurate, and sometimes, it breaks altogether. In July, several big-name web operations experienced problems — including Reddit, Gawker, and Mozilla — because their software was'nt prepared to handle a “leap second” that was added to the world’s atomic clocks.

‘We wanted something that we were confident in. It’s a time reference that’s owned by Google.’

— Andrew Fikes

But with Spanner, Google discarded the NTP in favor of its own time-keeping mechanism. It’s called the TrueTime API. “We wanted something that we were confident in,” Fikes says. “It’s a time reference that’s owned by Google.”
Rather than rely on outside clocks, Google equips its Spannerized data centers with its own atomic clocks and GPS (global positioning system) receivers, not unlike the one in your iPhone. Tapping into a network of satellites orbiting the Earth, a GPS receiver can pinpoint your location, but it can also tell time.
These time-keeping devices connect to a certain number of master servers, and the master servers shuttle time readings to other machines running across the Google network. Basically, each machine on the network runs a daemon — a background software process — that is constantly checking with masters in the same data center and in other Google data centers, trying to reach a consensus on what time it is. In this way, machines across the Google network can come pretty close to running a common clock.

Written by Arslan Asif

We are Creative Blogger Theme Wavers which provides user friendly, effective and easy to use themes. Each support has free and providing HD support screen casting.