It’s very rare that we get a peek inside the core algorithms and programs that Google uses to manage its worldwide data needs. So when we do, it’s often fascinating and cryptic at the same time.
I have recently been reading the papers that Google have released about various components of their system, particularly BigTable. I also discovered recently this presentation [PDF] given by Jeff Dean, a Google Fellow, on Google’s infrastructure, systems, and processing statistics that I found extremely interesting for a number of reasons.
The first reason I found it interesting is just some of the numbers they are using casually; I thought 300 million realtime Retweet buttons a day at work was impressive – it’s nothing!
The other reason I found it interesting was this is the first mention I’ve heard of Google Spanner, a product (not a wrench) that it is rumoured they are using internally already. Here’s why it’s interesing.