Video and slides synchronized, mp3 and slide download available at http://bit.ly/YpQCCJ.
Sean Cribbs compares ACID with BASE, explaining the virtues and tradeoffs of eventually consistent systems and what developers should know in order to feel comfortable working with EC systems. Filmed at qconsf.com.
Sean Cribbs is a Software Engineer at Basho Technologies, where he works on Riak, the fault-tolerant, highly-scalable distributed database. Prior to Basho, Sean was a freelance developer and consultant who also managed the development of the open-source Radiant web publishing system. He briefly studied Music Theory after receiving degrees in Computer Science and Music from University of Tulsa.
2. Watch the video with slide
synchronization on InfoQ.com!
http://www.infoq.com/presentations
/Embrace-Eventual-Consistency
InfoQ.com: News & Community Site
• 750,000 unique visitors/month
• Published in 4 languages (English, Chinese, Japanese and Brazilian
Portuguese)
• Post content from our QCon conferences
• News 15-20 / week
• Articles 3-4 / week
• Presentations (videos) 12-15 / week
• Interviews 2-3 / week
• Books 1 / month
3. Presented at QCon San Francisco
www.qconsf.com
Purpose of QCon
- to empower software development by facilitating the spread of
knowledge and innovation
Strategy
- practitioner-driven conference designed for YOU: influencers of
change and innovation in your teams
- speakers and topics driving the evolution and innovation
- connecting and catalyzing the influencers and innovators
Highlights
- attended by more than 12,000 delegates since 2007
- held in 9 cities worldwide
11. Why BASE / EC?
• “Omniscience” is expensive and slow.
12. Why BASE / EC?
• “Omniscience” is expensive and slow.
• Availability is often correlated to revenue.
13. Why BASE / EC?
• “Omniscience” is expensive and slow.
• Availability is often correlated to revenue.
• Failures happen all the time.
14. Why BASE / EC?
• “Omniscience” is expensive and slow.
• Availability is often correlated to revenue.
• Failures happen all the time.
“Any sufficiently large system is in a
constant state of partial failure.”
Justin Sheehy, Basho CTO
15. Why BASE / EC?
• “Omniscience” is expensive and slow.
• Availability is often correlated to revenue.
• Failures happen all the time.
• You’re probably doing it already.
22. Liveness
• “Good things • starvation
eventually freedom
happen”
• termination
• Always in
future • guaranteed
service
23. ACID vs. BASE
• Strong consistency • Weak consistency
• Isolation • Availability first
• Focus on “commit” • Best effort
• Nested transactions • Approximate answer
• Conservative • Aggressive
(pessimistic) (optimistic)
Fox, Gribble, Chawathe, Brewer, Gauthier -
Cluster-Based Scalable Network Services (SOSP97)
24. Peter Bailis
Eventual
consistency is
not safe
“...it’s easy to satisfy liveness
without being useful... If all
replicas return the value 42 in
response to every request, the
system is eventually
consistent.”
http://www.bailis.org/blog/safety-and-liveness-eventual-consistency-is-not-safe/
28. Domain Name Service
• Federated, hierarchical database
• How qconsf.com becomes 77.66.16.106
• Layered system with caching
29. Domain Name Service
• Federated, hierarchical database
• How qconsf.com becomes 77.66.16.106
• Layered system with caching
Diagrams from Wikimedia
30. Domain Name Service
• Federated, hierarchical database
• How qconsf.com becomes 77.66.16.106
• Layered system with caching
Diagrams from Wikimedia
31. DNS Liveness
• Convergence - caches eventually expire
• Consensus-free - local authority over subtree
updates
• Responsiveness - intermediaries can cache
results and reply quicker
• Resilience - authority servers can be replicated/
load-balanced
33. BitTorrent
• Peer-to-peer
cooperative large-file
transfer
• Dynamic membership
and block discovery
through the “tracker”
node
• Epidemic effect
http://computer.howstuffworks.com/bittorrent2.htm
34. BitTorrent Liveness
• Convergence - all peers that remain connected
eventually become seeds
• Resilience - loss of one peer doesn’t impede
progress
• Responsiveness - closer, faster peers tend to be
preferred
36. The Web
• Sparsely-connected graph of hypertext
documents identified by URIs
• Rich caching semantics: expiration, validation,
control
• Fluid evolution through uniform interface
• Layered system (federated)
37. Web: Liveness
• Consensus-free - local documents can be
changed, moved, removed without coordination
• Convergence - caching semantics prevent
unbounded staleness, redirection
• Responsiveness - many parties can proxy, cache
• Resilience - failure of one server doesn’t stop
the system
39. Dynamo
1
• Key-value store: distributed,
replicated, partitioned
2
• Client requests can go to any
node
• Low-latency at high percentiles 3
• Many clones: Riak, Cassandra,
Voldemort