Talk at June 29, 2010 Social Developer Summit in San Francisco.
With the rising popularity of non-relational databases like Cassandra, CouchDB, MongoDB, and Amazon SimpleDB, are relational databases doomed? Companies like Facebook and other large product development teams are using Hadoop or simply sharding indefinitely, what's the best strategy for storing information on the social web? When does it make the most sense to use each type of database and what are some innovative solutions being used by the biggest social web companies today?
Database sharding is a method of horizontal partitioning in a database or search engine. Each individual partition is referred to as a shard or database shard. Horizontal partitioning is a design principle whereby rows of a database table are held separately, rather than splitting by columns (as for normalization). Each partition forms part of a shard, which may in turn be located on a separate database server or physical location. The advantage is the number of rows in each table is reduced (this reduces index size, thus improves search performance).