For the start of this week I would like to come back to a talk delivered at Usenix (The Advanced Computing Systems Association) Conference in 2012 by Wilson Hsieh about Google’s distributed cloud database product Spanner.
This is still a timely talk, even if 5 years in the computing world is a small eternity, for the increasingly importance of cloud based database systems for the business IT departments does not show signs of abating. Quite the contrary, the trend may see acceleration in coming years, as software automation and artificial intelligence (AI) systems take further hold of enterprise database systems.
The research paper supporting the talk is well worth a thorough readership. In it we can check the details of Spanner’s research implementations, and its ability to perform distributed computing of complex datasets, perform replication of integrated concurrency of data, and control, correctness and fault-tolerant issues. The consistency of global operations, atomic schemas and MapReduce executions are features this Google product excels at:
As a globally distributed database, Spanner provides several interesting features. First, the replication configurations for data can be dynamically controlled at a fine grain by applications. Applications can specify constraints to control which datacenters contain which data, how far data is from its users (to control read latency), how far replicas are from each other (to control write latency), and how many replicas are maintained (to control durability, availability, and read performance). Data can also be dynamically and transparently moved between datacenters by the system to balance resource usage across datacenters. Second, Spanner has two features that are difficult to implement in a distributed database: it provides externally consistent  reads and writes, and globally consistent reads across the database at a timestamp. These features enable Spanner to support consistent backups, consistent MapReduce executions , and atomic schema updates, all at global scale, and even in the presence of ongoing transactions.
Interestingly theses features are achieved within an extremely complex distributed global network of datacenters, where non-linearities of all kinds would be expected. Spanner overcomes these challenges achieving linearizability by a clever serialization order of external consistent timestamps, being the first system with this characteristic at a global scale:
These features are enabled by the fact that Spanner assigns globally meaningful commit timestamps to transactions, even though transactions may be distributed. The timestamps reflect serialization order. In addition, the serialization order satisfies external consistency (or equivalently, linearizability ): if a transaction T1 commits before another transaction T2 starts, then T1’s commit timestamp is smaller than T2’s. Spanner is the first system to provide such guarantees at global scale.
The systems have proved especially efficient within social networks datacenters operations, where concurrency of data is paramount. The future, or even the present given the 5 year span of this paper and talk, will certainly incorporate better automated database technology. Low-latency and low synchronization costs in complex global datacenter networks are features that Spanner may have an undisputable comparative advantage over other approaches and offerings. From the conclusion in the paper:
To summarize, Spanner combines and extends on ideas from two research communities: from the database community, a familiar, easy-to-use, semi-relational interface, transactions, and an SQL-based query language; from the systems community, scalability, automatic sharding, fault tolerance, consistent replication, external consistency, and wide-area distribution. Since Spanner’s inception, we have taken more than 5 years to iterate to the current design and implementation. Part of this long iteration phase was due to a slow realization that Spanner should do more than tackle the problem of a globally replicated namespace, and should also focus on database features that Bigtable was missing. One aspect of our design stands out: the linchpin of Spanner’s feature set is TrueTime. We have shown that reifying clock uncertainty in the time API makes it possible to build distributed systems with much stronger time semantics. In addition, as the underlying system enforces tighter bounds on clock uncertainty, the overhead of the stronger semantics decreases. As a community, we should no longer depend on loosely synchronized clocks and weak time APIs in designing distributed algorithms.
One other important application of these technologies is within modern financial services industries such as banking, insurance companies and pension/asset management firms. The massive complex datasets of a global reach is a well suited use case for Spanner. The Information Age will be wise to check further developments within these technologies and its also relevant related issues.
featured image: Spanner osdi2012