SlideShare uma empresa Scribd logo
1 de 45
Fear No More:
Embrace Eventual
  Consistency
     Sean Cribbs
     @seancribbs
Watch the video with slide
                         synchronization on InfoQ.com!
                      http://www.infoq.com/presentations
                         /Embrace-Eventual-Consistency

       InfoQ.com: News & Community Site
• 750,000 unique visitors/month
• Published in 4 languages (English, Chinese, Japanese and Brazilian
  Portuguese)
• Post content from our QCon conferences
• News 15-20 / week
• Articles 3-4 / week
• Presentations (videos) 12-15 / week
• Interviews 2-3 / week
• Books 1 / month
Presented at QCon San Francisco
                          www.qconsf.com
Purpose of QCon
- to empower software development by facilitating the spread of
knowledge and innovation
Strategy
 - practitioner-driven conference designed for YOU: influencers of
change and innovation in your teams
- speakers and topics driving the evolution and innovation
- connecting and catalyzing the influencers and innovators
Highlights
- attended by more than 12,000 delegates since 2007
- held in 9 cities worldwide
Distributed Systems Experts
Fear No More: Embrace Eventual Consistency
FEAR



       Photo from Wikimedia
ACID vs. BASE
ACID vs. BASE
  • Strong consistency    • Weak consistency
  • Isolation             • Availability first
  • Focus on “commit”     • Best effort
  • Nested transactions   • Approximate answer
  • Conservative          • Aggressive
    (pessimistic)           (optimistic)

   Fox, Gribble, Chawathe, Brewer, Gauthier -
Cluster-Based Scalable Network Services (SOSP97)
ACID           vs.       BASE
  “Inconsistency is       “Being unavailable
the worst thing that       is the worst thing
   could happen.”        that could happen.”
Why BASE / EC?
Why BASE / EC?

• “Omniscience” is expensive and slow.
Why BASE / EC?

• “Omniscience” is expensive and slow.
• Availability is often correlated to revenue.
Why BASE / EC?

• “Omniscience” is expensive and slow.
• Availability is often correlated to revenue.
• Failures happen all the time.
Why BASE / EC?

• “Omniscience” is expensive and slow.
• Availability is often correlated to revenue.
• Failures happen all the time.
 “Any sufficiently large system is in a
   constant state of partial failure.”
      Justin Sheehy, Basho CTO
Why BASE / EC?

• “Omniscience” is expensive and slow.
• Availability is often correlated to revenue.
• Failures happen all the time.
• You’re probably doing it already.
Safety &
Liveness
Leslie Lamport
1977
Safety
Safety
• “Bad things
 don’t happen”

• Point-in-time
 identifiable
Safety
• “Bad things      • mutual
 don’t happen”       exclusion

• Point-in-time    • partial
 identifiable        correctness

                   • first-come,
                     first-serve
Liveness
Liveness
• “Good things
 eventually
 happen”

• Always in
 future
Liveness
• “Good things     • starvation
 eventually         freedom
 happen”
                   • termination
• Always in
 future            • guaranteed
                    service
ACID vs. BASE
  • Strong consistency    • Weak consistency
  • Isolation             • Availability first
  • Focus on “commit”     • Best effort
  • Nested transactions   • Approximate answer
  • Conservative          • Aggressive
    (pessimistic)           (optimistic)

   Fox, Gribble, Chawathe, Brewer, Gauthier -
Cluster-Based Scalable Network Services (SOSP97)
Peter Bailis
 Eventual
 consistency is
 not safe
 “...it’s easy to satisfy liveness
 without being useful... If all
 replicas return the value 42 in
 response to every request, the
 system is eventually
 consistent.”

http://www.bailis.org/blog/safety-and-liveness-eventual-consistency-is-not-safe/
Liveness of BASE

• Convergence - “eventual delivery”
• Responsiveness - “eventual service”
• Resilience - “eventual recovery”
• Consensus-free - “eventual progress”
Safety of BASE


• Durability - “accepted writes are not lost”
• Integrity - “data is not corrupted”
• Authenticity - “data is not forged”
Real BASE Systems



            Photo from Wikimedia
Domain Name Service
• Federated, hierarchical database
  • How qconsf.com becomes 77.66.16.106
• Layered system with caching
Domain Name Service
• Federated, hierarchical database
  • How qconsf.com becomes 77.66.16.106
• Layered system with caching




                                 Diagrams from Wikimedia
Domain Name Service
• Federated, hierarchical database
  • How qconsf.com becomes 77.66.16.106
• Layered system with caching




                                 Diagrams from Wikimedia
DNS Liveness

• Convergence - caches eventually expire
• Consensus-free - local authority over subtree
  updates
• Responsiveness - intermediaries can cache
  results and reply quicker
• Resilience - authority servers can be replicated/
  load-balanced
DNS Safety



• Authenticity - forgery prevented by DNSSEC
BitTorrent
• Peer-to-peer
  cooperative large-file
  transfer
• Dynamic membership
  and block discovery
  through the “tracker”
  node
• Epidemic effect
                      http://computer.howstuffworks.com/bittorrent2.htm
BitTorrent Liveness

• Convergence - all peers that remain connected
  eventually become seeds
• Resilience - loss of one peer doesn’t impede
  progress
• Responsiveness - closer, faster peers tend to be
  preferred
BitTorrent Safety



• Integrity - each block is checksummed to
  prevent corruption
The Web

• Sparsely-connected graph of hypertext
  documents identified by URIs
• Rich caching semantics: expiration, validation,
  control
• Fluid evolution through uniform interface
• Layered system (federated)
Web: Liveness

• Consensus-free - local documents can be
  changed, moved, removed without coordination
• Convergence - caching semantics prevent
  unbounded staleness, redirection
• Responsiveness - many parties can proxy, cache
• Resilience - failure of one server doesn’t stop
  the system
Web: Safety


• Privacy & Authenticity - HTTPS/SSL/TLS
• Integrity - POST responses don’t pollute caches
Dynamo
                                    1
• Key-value store: distributed,
  replicated, partitioned
                                        2
• Client requests can go to any
  node
• Low-latency at high percentiles       3
• Many clones: Riak, Cassandra,
  Voldemort
Dynamo: Liveness

• Convergence - read-repair, hash-tree
  exchanges, vector-clocks
• Resilience - hinted-handoff, sloppy quorums
• Responsiveness - replication
• Consensus-free - loose coordination, concurrent
  updates
Dynamo: Safety


• Authenticity - won’t serve data you didn’t store
• Durability - confirmed writes are not lost
ACID vs. BASE
CONFLICT

           Photo by Associated Press
SPECTRUM

           Photo by Associated Press
hu g s!



Embrace
Eventual
Consistency

Mais conteúdo relacionado

Mais de C4Media

Streaming a Million Likes/Second: Real-Time Interactions on Live Video
Streaming a Million Likes/Second: Real-Time Interactions on Live VideoStreaming a Million Likes/Second: Real-Time Interactions on Live Video
Streaming a Million Likes/Second: Real-Time Interactions on Live VideoC4Media
 
Next Generation Client APIs in Envoy Mobile
Next Generation Client APIs in Envoy MobileNext Generation Client APIs in Envoy Mobile
Next Generation Client APIs in Envoy MobileC4Media
 
Software Teams and Teamwork Trends Report Q1 2020
Software Teams and Teamwork Trends Report Q1 2020Software Teams and Teamwork Trends Report Q1 2020
Software Teams and Teamwork Trends Report Q1 2020C4Media
 
Understand the Trade-offs Using Compilers for Java Applications
Understand the Trade-offs Using Compilers for Java ApplicationsUnderstand the Trade-offs Using Compilers for Java Applications
Understand the Trade-offs Using Compilers for Java ApplicationsC4Media
 
Kafka Needs No Keeper
Kafka Needs No KeeperKafka Needs No Keeper
Kafka Needs No KeeperC4Media
 
High Performing Teams Act Like Owners
High Performing Teams Act Like OwnersHigh Performing Teams Act Like Owners
High Performing Teams Act Like OwnersC4Media
 
Does Java Need Inline Types? What Project Valhalla Can Bring to Java
Does Java Need Inline Types? What Project Valhalla Can Bring to JavaDoes Java Need Inline Types? What Project Valhalla Can Bring to Java
Does Java Need Inline Types? What Project Valhalla Can Bring to JavaC4Media
 
Service Meshes- The Ultimate Guide
Service Meshes- The Ultimate GuideService Meshes- The Ultimate Guide
Service Meshes- The Ultimate GuideC4Media
 
Shifting Left with Cloud Native CI/CD
Shifting Left with Cloud Native CI/CDShifting Left with Cloud Native CI/CD
Shifting Left with Cloud Native CI/CDC4Media
 
CI/CD for Machine Learning
CI/CD for Machine LearningCI/CD for Machine Learning
CI/CD for Machine LearningC4Media
 
Fault Tolerance at Speed
Fault Tolerance at SpeedFault Tolerance at Speed
Fault Tolerance at SpeedC4Media
 
Architectures That Scale Deep - Regaining Control in Deep Systems
Architectures That Scale Deep - Regaining Control in Deep SystemsArchitectures That Scale Deep - Regaining Control in Deep Systems
Architectures That Scale Deep - Regaining Control in Deep SystemsC4Media
 
ML in the Browser: Interactive Experiences with Tensorflow.js
ML in the Browser: Interactive Experiences with Tensorflow.jsML in the Browser: Interactive Experiences with Tensorflow.js
ML in the Browser: Interactive Experiences with Tensorflow.jsC4Media
 
Build Your Own WebAssembly Compiler
Build Your Own WebAssembly CompilerBuild Your Own WebAssembly Compiler
Build Your Own WebAssembly CompilerC4Media
 
User & Device Identity for Microservices @ Netflix Scale
User & Device Identity for Microservices @ Netflix ScaleUser & Device Identity for Microservices @ Netflix Scale
User & Device Identity for Microservices @ Netflix ScaleC4Media
 
Scaling Patterns for Netflix's Edge
Scaling Patterns for Netflix's EdgeScaling Patterns for Netflix's Edge
Scaling Patterns for Netflix's EdgeC4Media
 
Make Your Electron App Feel at Home Everywhere
Make Your Electron App Feel at Home EverywhereMake Your Electron App Feel at Home Everywhere
Make Your Electron App Feel at Home EverywhereC4Media
 
The Talk You've Been Await-ing For
The Talk You've Been Await-ing ForThe Talk You've Been Await-ing For
The Talk You've Been Await-ing ForC4Media
 
Future of Data Engineering
Future of Data EngineeringFuture of Data Engineering
Future of Data EngineeringC4Media
 
Automated Testing for Terraform, Docker, Packer, Kubernetes, and More
Automated Testing for Terraform, Docker, Packer, Kubernetes, and MoreAutomated Testing for Terraform, Docker, Packer, Kubernetes, and More
Automated Testing for Terraform, Docker, Packer, Kubernetes, and MoreC4Media
 

Mais de C4Media (20)

Streaming a Million Likes/Second: Real-Time Interactions on Live Video
Streaming a Million Likes/Second: Real-Time Interactions on Live VideoStreaming a Million Likes/Second: Real-Time Interactions on Live Video
Streaming a Million Likes/Second: Real-Time Interactions on Live Video
 
Next Generation Client APIs in Envoy Mobile
Next Generation Client APIs in Envoy MobileNext Generation Client APIs in Envoy Mobile
Next Generation Client APIs in Envoy Mobile
 
Software Teams and Teamwork Trends Report Q1 2020
Software Teams and Teamwork Trends Report Q1 2020Software Teams and Teamwork Trends Report Q1 2020
Software Teams and Teamwork Trends Report Q1 2020
 
Understand the Trade-offs Using Compilers for Java Applications
Understand the Trade-offs Using Compilers for Java ApplicationsUnderstand the Trade-offs Using Compilers for Java Applications
Understand the Trade-offs Using Compilers for Java Applications
 
Kafka Needs No Keeper
Kafka Needs No KeeperKafka Needs No Keeper
Kafka Needs No Keeper
 
High Performing Teams Act Like Owners
High Performing Teams Act Like OwnersHigh Performing Teams Act Like Owners
High Performing Teams Act Like Owners
 
Does Java Need Inline Types? What Project Valhalla Can Bring to Java
Does Java Need Inline Types? What Project Valhalla Can Bring to JavaDoes Java Need Inline Types? What Project Valhalla Can Bring to Java
Does Java Need Inline Types? What Project Valhalla Can Bring to Java
 
Service Meshes- The Ultimate Guide
Service Meshes- The Ultimate GuideService Meshes- The Ultimate Guide
Service Meshes- The Ultimate Guide
 
Shifting Left with Cloud Native CI/CD
Shifting Left with Cloud Native CI/CDShifting Left with Cloud Native CI/CD
Shifting Left with Cloud Native CI/CD
 
CI/CD for Machine Learning
CI/CD for Machine LearningCI/CD for Machine Learning
CI/CD for Machine Learning
 
Fault Tolerance at Speed
Fault Tolerance at SpeedFault Tolerance at Speed
Fault Tolerance at Speed
 
Architectures That Scale Deep - Regaining Control in Deep Systems
Architectures That Scale Deep - Regaining Control in Deep SystemsArchitectures That Scale Deep - Regaining Control in Deep Systems
Architectures That Scale Deep - Regaining Control in Deep Systems
 
ML in the Browser: Interactive Experiences with Tensorflow.js
ML in the Browser: Interactive Experiences with Tensorflow.jsML in the Browser: Interactive Experiences with Tensorflow.js
ML in the Browser: Interactive Experiences with Tensorflow.js
 
Build Your Own WebAssembly Compiler
Build Your Own WebAssembly CompilerBuild Your Own WebAssembly Compiler
Build Your Own WebAssembly Compiler
 
User & Device Identity for Microservices @ Netflix Scale
User & Device Identity for Microservices @ Netflix ScaleUser & Device Identity for Microservices @ Netflix Scale
User & Device Identity for Microservices @ Netflix Scale
 
Scaling Patterns for Netflix's Edge
Scaling Patterns for Netflix's EdgeScaling Patterns for Netflix's Edge
Scaling Patterns for Netflix's Edge
 
Make Your Electron App Feel at Home Everywhere
Make Your Electron App Feel at Home EverywhereMake Your Electron App Feel at Home Everywhere
Make Your Electron App Feel at Home Everywhere
 
The Talk You've Been Await-ing For
The Talk You've Been Await-ing ForThe Talk You've Been Await-ing For
The Talk You've Been Await-ing For
 
Future of Data Engineering
Future of Data EngineeringFuture of Data Engineering
Future of Data Engineering
 
Automated Testing for Terraform, Docker, Packer, Kubernetes, and More
Automated Testing for Terraform, Docker, Packer, Kubernetes, and MoreAutomated Testing for Terraform, Docker, Packer, Kubernetes, and More
Automated Testing for Terraform, Docker, Packer, Kubernetes, and More
 

Último

Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Adtran
 
UiPath Clipboard AI: "A TIME Magazine Best Invention of 2023 Unveiled"
UiPath Clipboard AI: "A TIME Magazine Best Invention of 2023 Unveiled"UiPath Clipboard AI: "A TIME Magazine Best Invention of 2023 Unveiled"
UiPath Clipboard AI: "A TIME Magazine Best Invention of 2023 Unveiled"DianaGray10
 
9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding TeamAdam Moalla
 
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationUsing IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationIES VE
 
Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024D Cloud Solutions
 
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAAnypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAshyamraj55
 
Machine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfMachine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfAijun Zhang
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostMatt Ray
 
UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6DianaGray10
 
All in AI: LLM Landscape & RAG in 2024 with Mark Ryan (Google) & Jerry Liu (L...
All in AI: LLM Landscape & RAG in 2024 with Mark Ryan (Google) & Jerry Liu (L...All in AI: LLM Landscape & RAG in 2024 with Mark Ryan (Google) & Jerry Liu (L...
All in AI: LLM Landscape & RAG in 2024 with Mark Ryan (Google) & Jerry Liu (L...Daniel Zivkovic
 
COMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a WebsiteCOMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a Websitedgelyza
 
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1DianaGray10
 
Cybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxCybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxGDSC PJATK
 
Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Commit University
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsIgniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsSafe Software
 
Nanopower In Semiconductor Industry.pdf
Nanopower  In Semiconductor Industry.pdfNanopower  In Semiconductor Industry.pdf
Nanopower In Semiconductor Industry.pdfPedro Manuel
 
Comparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioComparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioChristian Posta
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesMd Hossain Ali
 
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfJamie (Taka) Wang
 
Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )Brian Pichman
 

Último (20)

Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™
 
UiPath Clipboard AI: "A TIME Magazine Best Invention of 2023 Unveiled"
UiPath Clipboard AI: "A TIME Magazine Best Invention of 2023 Unveiled"UiPath Clipboard AI: "A TIME Magazine Best Invention of 2023 Unveiled"
UiPath Clipboard AI: "A TIME Magazine Best Invention of 2023 Unveiled"
 
9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team
 
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationUsing IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
 
Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024
 
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAAnypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
 
Machine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfMachine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdf
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
 
UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6
 
All in AI: LLM Landscape & RAG in 2024 with Mark Ryan (Google) & Jerry Liu (L...
All in AI: LLM Landscape & RAG in 2024 with Mark Ryan (Google) & Jerry Liu (L...All in AI: LLM Landscape & RAG in 2024 with Mark Ryan (Google) & Jerry Liu (L...
All in AI: LLM Landscape & RAG in 2024 with Mark Ryan (Google) & Jerry Liu (L...
 
COMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a WebsiteCOMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a Website
 
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
 
Cybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxCybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptx
 
Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsIgniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
 
Nanopower In Semiconductor Industry.pdf
Nanopower  In Semiconductor Industry.pdfNanopower  In Semiconductor Industry.pdf
Nanopower In Semiconductor Industry.pdf
 
Comparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioComparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and Istio
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
 
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
 
Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )
 

Fear No More: Embrace Eventual Consistency

  • 1. Fear No More: Embrace Eventual Consistency Sean Cribbs @seancribbs
  • 2. Watch the video with slide synchronization on InfoQ.com! http://www.infoq.com/presentations /Embrace-Eventual-Consistency InfoQ.com: News & Community Site • 750,000 unique visitors/month • Published in 4 languages (English, Chinese, Japanese and Brazilian Portuguese) • Post content from our QCon conferences • News 15-20 / week • Articles 3-4 / week • Presentations (videos) 12-15 / week • Interviews 2-3 / week • Books 1 / month
  • 3. Presented at QCon San Francisco www.qconsf.com Purpose of QCon - to empower software development by facilitating the spread of knowledge and innovation Strategy - practitioner-driven conference designed for YOU: influencers of change and innovation in your teams - speakers and topics driving the evolution and innovation - connecting and catalyzing the influencers and innovators Highlights - attended by more than 12,000 delegates since 2007 - held in 9 cities worldwide
  • 6. FEAR Photo from Wikimedia
  • 8. ACID vs. BASE • Strong consistency • Weak consistency • Isolation • Availability first • Focus on “commit” • Best effort • Nested transactions • Approximate answer • Conservative • Aggressive (pessimistic) (optimistic) Fox, Gribble, Chawathe, Brewer, Gauthier - Cluster-Based Scalable Network Services (SOSP97)
  • 9. ACID vs. BASE “Inconsistency is “Being unavailable the worst thing that is the worst thing could happen.” that could happen.”
  • 10. Why BASE / EC?
  • 11. Why BASE / EC? • “Omniscience” is expensive and slow.
  • 12. Why BASE / EC? • “Omniscience” is expensive and slow. • Availability is often correlated to revenue.
  • 13. Why BASE / EC? • “Omniscience” is expensive and slow. • Availability is often correlated to revenue. • Failures happen all the time.
  • 14. Why BASE / EC? • “Omniscience” is expensive and slow. • Availability is often correlated to revenue. • Failures happen all the time. “Any sufficiently large system is in a constant state of partial failure.” Justin Sheehy, Basho CTO
  • 15. Why BASE / EC? • “Omniscience” is expensive and slow. • Availability is often correlated to revenue. • Failures happen all the time. • You’re probably doing it already.
  • 18. Safety • “Bad things don’t happen” • Point-in-time identifiable
  • 19. Safety • “Bad things • mutual don’t happen” exclusion • Point-in-time • partial identifiable correctness • first-come, first-serve
  • 21. Liveness • “Good things eventually happen” • Always in future
  • 22. Liveness • “Good things • starvation eventually freedom happen” • termination • Always in future • guaranteed service
  • 23. ACID vs. BASE • Strong consistency • Weak consistency • Isolation • Availability first • Focus on “commit” • Best effort • Nested transactions • Approximate answer • Conservative • Aggressive (pessimistic) (optimistic) Fox, Gribble, Chawathe, Brewer, Gauthier - Cluster-Based Scalable Network Services (SOSP97)
  • 24. Peter Bailis Eventual consistency is not safe “...it’s easy to satisfy liveness without being useful... If all replicas return the value 42 in response to every request, the system is eventually consistent.” http://www.bailis.org/blog/safety-and-liveness-eventual-consistency-is-not-safe/
  • 25. Liveness of BASE • Convergence - “eventual delivery” • Responsiveness - “eventual service” • Resilience - “eventual recovery” • Consensus-free - “eventual progress”
  • 26. Safety of BASE • Durability - “accepted writes are not lost” • Integrity - “data is not corrupted” • Authenticity - “data is not forged”
  • 27. Real BASE Systems Photo from Wikimedia
  • 28. Domain Name Service • Federated, hierarchical database • How qconsf.com becomes 77.66.16.106 • Layered system with caching
  • 29. Domain Name Service • Federated, hierarchical database • How qconsf.com becomes 77.66.16.106 • Layered system with caching Diagrams from Wikimedia
  • 30. Domain Name Service • Federated, hierarchical database • How qconsf.com becomes 77.66.16.106 • Layered system with caching Diagrams from Wikimedia
  • 31. DNS Liveness • Convergence - caches eventually expire • Consensus-free - local authority over subtree updates • Responsiveness - intermediaries can cache results and reply quicker • Resilience - authority servers can be replicated/ load-balanced
  • 32. DNS Safety • Authenticity - forgery prevented by DNSSEC
  • 33. BitTorrent • Peer-to-peer cooperative large-file transfer • Dynamic membership and block discovery through the “tracker” node • Epidemic effect http://computer.howstuffworks.com/bittorrent2.htm
  • 34. BitTorrent Liveness • Convergence - all peers that remain connected eventually become seeds • Resilience - loss of one peer doesn’t impede progress • Responsiveness - closer, faster peers tend to be preferred
  • 35. BitTorrent Safety • Integrity - each block is checksummed to prevent corruption
  • 36. The Web • Sparsely-connected graph of hypertext documents identified by URIs • Rich caching semantics: expiration, validation, control • Fluid evolution through uniform interface • Layered system (federated)
  • 37. Web: Liveness • Consensus-free - local documents can be changed, moved, removed without coordination • Convergence - caching semantics prevent unbounded staleness, redirection • Responsiveness - many parties can proxy, cache • Resilience - failure of one server doesn’t stop the system
  • 38. Web: Safety • Privacy & Authenticity - HTTPS/SSL/TLS • Integrity - POST responses don’t pollute caches
  • 39. Dynamo 1 • Key-value store: distributed, replicated, partitioned 2 • Client requests can go to any node • Low-latency at high percentiles 3 • Many clones: Riak, Cassandra, Voldemort
  • 40. Dynamo: Liveness • Convergence - read-repair, hash-tree exchanges, vector-clocks • Resilience - hinted-handoff, sloppy quorums • Responsiveness - replication • Consensus-free - loose coordination, concurrent updates
  • 41. Dynamo: Safety • Authenticity - won’t serve data you didn’t store • Durability - confirmed writes are not lost
  • 43. CONFLICT Photo by Associated Press
  • 44. SPECTRUM Photo by Associated Press