Artificial intelligence and machine learning are currently all the rage. Every organisation is trying to jump on this bandwagon and cash in on their data reserves. At ThoughtWorks, we’d agree that this tech has huge potential — but as with all things, realising value depends on understanding how best to use it.
6. Build it they will come!
● Seen primarily as an infrastructure problem
● Pinning down uses cases & value stream is hard
● Analysis paralysis & overengineering
@kiran_p
11. Focus on initiatives which
align with business outcomes.
Structure teams around
business capabilities.
Product
Thinking
Self service platform for
storage, catalogue,
computation, access rights
and pipelines etc.
Autonomous teams with clear
bounded context building and
running products
independently.
Platform
Thinking
Domain Driven
Design
The Data Mesh Paradigm
@kiran_p
13. Project vs Product
Project Mode Product Mode
START Solution (often) defined at outset.
Problem identified at outset.
Solution developed iteratively and tested.
STOP Team moves on when solution delivered. Team moves on when problem verifiably fixed.
FOCUS Features delivered in a given time & budget.
Progress made on key business goals
(measured by metrics).
HAS FIXED SCOPE? Usually. Almost never.
@lucyfedia
14. Product teams have two jobs and two customers
● Deliver business capabilities
- External User
● Expose their domain’s data for others to consume
- (often) Internal User
@lucyfedia
25. Reduce fraud by
5% per year
Identify
fraudulent
claims
Reduce vehicle damage
claims by 2% per year
Increase conversion
rate by 2%
Predict Weather
Patterns
Upselling
Insurance
Products
The Use-Cases
@lucyfedia
29. Not a technology problem
Becoming data-driven
is usually an
organisational problem
Work with cross functional
product teams and real use-
cases to deliver business
value.
Build by autonomous cross
functional teams using data
platforms instead of
centralized data lake
Domain data
is a product
Distributed
Data Mesh
Key Takeaways
@kiran_p & @lucyfedia
30. Kiran Prakash
@kiran_p
Thank you
Lucy Chambers
@lucyfedia
How to Move Beyond a
Monolithic Data Lake to
a Distributed Data Mesh
martinfowler.com/articles/
data-monolith-to-mesh.html
The Curse of the
Data Lake Monster
thoughtworks.com/insights/
blog/curse-data-lake-monster