SlideShare uma empresa Scribd logo
1 de 31
Baixar para ler offline
Continuous Testing
Digital Transformation Requires Continuous Testing
Ingo Philipp
Businesses must
continuously exploit
digital technologies to both
create new sources
of customer value and
increase operational
agility in service of
customers.
© Forrester Research
Digital
Disruption
Across industries, companies face the
challenge of software-led transformation
Plan Code Version Build Provision Test Deploy Monitor
© Melissa Perri
The product is a solution.
If the problem isn't solved,
the product doesn't work.
The goal is to
accelerate this
value exchange
Identify
Needs
Receive
Money
Build
Solution
Your Company Your Customer
Requirements Design Implementation Testing Acceptance Deployment
Inception Elaboration Construction Transition
Development Operations
DevOps
Agile Horizon
Digital Singularity
Incremental
Rational Unified Process
4
Agile
Scrum, Kanban
2
Continuous
DevOps
1
6+
Big Bang
Waterfall
Development Response
Flexible Framework
Rigid Rules
Following a Plan
Contract Negotiation
Copious Documentation
Processes & Tools
Individuals & Interactions
Working Software
Customer Collaboration
Responding to Change
It’s not the strongest
that survive, nor the most
intelligent, but the one most
responsive to change
Charles Darwin
Because software is not written, it is rewritten
80%
Manual Testing
20%
Test Automation
The reliance on
manual testing is the
top technical challenge
in app development
World Quality Report, 2015/16
Test automation
requires developers
UI LoadAPI
Present
Traditional Testing
Future
Continuous Testing
+85%
Test Automation
Exploratory Tests
Automated UI Tests
API Tests
Load Tests
80%
Manual Testing
20%
Test Automation
Present
Traditional Testing
Orchestrated Service Virtualization
100%
0%
Sprint
Start
Sprint
End
API
UI
Start testing early,
shift left test automation Degree of
Completion
Time
Efficiency
Gain
+4x
Creation
+6x
Maintenance
+20x
Execution
*voke, market snapshot report on service virtualization - 2012
On average, organizations require access to 33systems for development or testing.
18Average # of systems with
unrestricted access
96% of testers have
restricted test lab access
Testing
Present
Enterprise system landscapes are alike disease gene networks.
Welcome to the tester’s hell
Automation doesn't make testing
easy, it makes testing possible
Lesson Learned
© Wolfgang Platz
A
B
Visiting London
~100 Billion Billion Billion Billion Billion Billion Billion Combinations
= 50
Testing harder isn’t the
answer, testing smarter is!
Lesson Learned
© Wolfgang Platz
Test Case
Passing
Test Case
Passing
Test Case
@iamdevloper
© BBC
Testing is exactly like
washing a pig. Because
it’s messy. It has no rules. No
clear beginning, middle, or end.
It’s kind of a pain in the ass,
and when you’re done you’re
not sure if the pig is really clean
or even why you were washing
a pig in the first place.
© Luke Sullivan © BBC
Testing is exactly like
washing a pig. Because
it’s messy. It has no rules. No
clear beginning, middle, or end.
It’s kind of a pain in the ass,
and when you’re done you’re
not sure if the pig is really clean
or even why you were washing
a pig in the first place.
© Luke Sullivan
That’s what we can know
Testing is exactly like
washing a pig. Because
it’s messy. It has no rules. No
clear beginning, middle, or end.
It’s kind of a pain in the ass,
and when you’re done you’re
not sure if the pig is really clean
or even why you were washing
a pig in the first place.
© Luke Sullivan
That’s what we can know
That’s what we know
That’s
what we
check
Problem
Knowledge Gap
© Richard Bradshaw
© Cem Kaner
© Richard Bradshaw
The purpose of testing is to
close the knowledge gap
The goal is information,
not gratuitous automation
Testing is and always will be
a search for information
That’s what we can know
That’s what we know
That’s
what we
check
Problem
Knowledge Gap
That’s what we can know
That’s what we know
That’s
what we
check
Problem
Knowledge Gap
Checking
« Requires Processing »
© Rich Rogers
Verify through
Instructions
Pay attention to
Deviations
Create
Test Cases
Follow
Procedure
Examine
Requirements
Factory
Process
» Here You Monitor Known Risks
That’s what we can know
That’s what we know
That’s
what we
check
Problem
Knowledge Gap
Confirmatory Testing
« Requires Processing »
© Rich Rogers
Verify through
Instructions
Pay attention to
Deviations
Create
Test Cases
Follow
Procedure
Examine
Requirements
Factory
Process
MechanicalProcess
Confirmatory Testing
« Requires Processing »
© Rich Rogers
Verify through
Instructions
Pay attention to
Deviations
Create
Test Cases
Follow
Procedure
Examine
Requirements
Factory
Process
Exploratory Testing
« Requires Thinking »
Investigate through
Experiments
Pay attention to
Oracles
Create
Test Ideas
Follow
Clues
Examine
Risks
Adaptive
Investigation
» Here You Analyze Potential Risks
CognitiveProcess
MechanicalProcess » Here You Monitor Known Risks
Checking
Efficient Confirmatory Testing
Exploring
Effective Exploratory Testing
Testing
Thorough Testing
+
=
Agile
Testing Equation
© Elisabeth Hendrickson
If this doesn’t make an impact, then
its absence doesn’t make a difference
Our Proof
© Trey Smith
100%
Manual
Testing
0%
Automated
Testing
Testing
Present
There’s a better
way to do it. Find it!
Project Charter
11
Manual
Testers
4755
Manual
Test Cases
?
Unknown
Coverage
10
Weeks
Execution
Risk Coverage Optimization Test Data Management UI/API Test Automation Continuous Integration & Distributed Execution
48
Hours
Execution
Single Agent
8
Hours
Execution
Multiple Agents
11
Manual
Testers
4755
Manual
Test Cases
?
Unknown
Coverage
10
Weeks
Execution
89%
Business Risk
Coverage
1193
Manual
Test Cases
37%
Business Risk
5
Weeks
Execution
2.5
Weeks
Execution
92%
Automated
Test Cases
53%
Business Risk
Coverage
Smoke Testing
34
Minutes
Execution
Smoke Testing
Test Case Count 7%
75% Redundancy 50% Effort Test Data
64% API Tests
8
Hours
Execution
Multiple Agents
72%
Critical Defect Reduction
Production Defects
89%
Business Risk
Coverage
1193
Manual
Test Cases
5
Weeks
Execution
2.5
Weeks
Execution
92%
Automated
Test Cases
34
Minutes
Execution
Smoke Testing
53%
Business Risk
Coverage
Smoke Testing
Test Case Count 7%
Test Case Design & Exploratory Testing
64% API Tests
Risk Coverage Optimization Test Data Management UI/API Test Automation Continuous Integration & Distributed Execution
The future doesn’t just happen,
it gets happened, so make it happen
Conclusion
© Paul Tagliablue
Questions
The show is over. It’s your turn.
?

Mais conteúdo relacionado

Mais procurados

DevOps - A Gentle Introduction
DevOps - A Gentle IntroductionDevOps - A Gentle Introduction
DevOps - A Gentle IntroductionGanesh Samarthyam
 
DevOps Kaizen: Practical Steps to Start & Sustain a Transformation
DevOps Kaizen: Practical Steps to Start & Sustain a TransformationDevOps Kaizen: Practical Steps to Start & Sustain a Transformation
DevOps Kaizen: Practical Steps to Start & Sustain a Transformationdev2ops
 
What is DevOps
What is DevOpsWhat is DevOps
What is DevOpsspajus
 
JUG Bonn June 2021 - The DevOps disaster
JUG Bonn June 2021 - The DevOps disasterJUG Bonn June 2021 - The DevOps disaster
JUG Bonn June 2021 - The DevOps disasterBert Jan Schrijver
 
Behind the Book: Gene Kim's Top Takeaways from Researching and Writing 'The D...
Behind the Book: Gene Kim's Top Takeaways from Researching and Writing 'The D...Behind the Book: Gene Kim's Top Takeaways from Researching and Writing 'The D...
Behind the Book: Gene Kim's Top Takeaways from Researching and Writing 'The D...XebiaLabs
 
Continuous Delivery in a Legacy Shop—One Step at a Time
Continuous Delivery in a Legacy Shop—One Step at a TimeContinuous Delivery in a Legacy Shop—One Step at a Time
Continuous Delivery in a Legacy Shop—One Step at a TimeTechWell
 
DevOps: A Value Proposition
DevOps: A Value PropositionDevOps: A Value Proposition
DevOps: A Value PropositionNicole Forsgren
 
Introduction to devops - update 2017
Introduction to devops - update 2017Introduction to devops - update 2017
Introduction to devops - update 2017gjdevos
 
Quality Jam 2016 Product Roadmap
Quality Jam 2016 Product RoadmapQuality Jam 2016 Product Roadmap
Quality Jam 2016 Product RoadmapQASymphony
 
Understanding DevOps in simpler way with Continuous Delivery
Understanding DevOps in simpler way with Continuous DeliveryUnderstanding DevOps in simpler way with Continuous Delivery
Understanding DevOps in simpler way with Continuous DeliverySwapnil Jain
 
The Anti-Transformation transformation @DevOps Summit Amsterdam
The Anti-Transformation transformation @DevOps Summit AmsterdamThe Anti-Transformation transformation @DevOps Summit Amsterdam
The Anti-Transformation transformation @DevOps Summit AmsterdamMirco Hering
 
Jan de Vries - How to convince your boss that it is DevOps that he wants
Jan de Vries - How to convince your boss that it is DevOps that he wantsJan de Vries - How to convince your boss that it is DevOps that he wants
Jan de Vries - How to convince your boss that it is DevOps that he wantsAgile Lietuva
 
Introduction to DevOps
Introduction to DevOpsIntroduction to DevOps
Introduction to DevOpsJoão Miranda
 
DevOps vs Agile | DevOps Tutorial For Beginners | DevOps Training | Edureka
DevOps vs Agile | DevOps Tutorial For Beginners | DevOps Training | EdurekaDevOps vs Agile | DevOps Tutorial For Beginners | DevOps Training | Edureka
DevOps vs Agile | DevOps Tutorial For Beginners | DevOps Training | EdurekaEdureka!
 
The Four Keys - Measuring DevOps Success
The Four Keys - Measuring DevOps SuccessThe Four Keys - Measuring DevOps Success
The Four Keys - Measuring DevOps SuccessDina Graves Portman
 
DevOps MythBusters
DevOps MythBustersDevOps MythBusters
DevOps MythBustersXebiaLabs
 

Mais procurados (20)

DevOps - A Gentle Introduction
DevOps - A Gentle IntroductionDevOps - A Gentle Introduction
DevOps - A Gentle Introduction
 
DevOps Kaizen: Practical Steps to Start & Sustain a Transformation
DevOps Kaizen: Practical Steps to Start & Sustain a TransformationDevOps Kaizen: Practical Steps to Start & Sustain a Transformation
DevOps Kaizen: Practical Steps to Start & Sustain a Transformation
 
DevOps introduction
DevOps introductionDevOps introduction
DevOps introduction
 
What is DevOps
What is DevOpsWhat is DevOps
What is DevOps
 
JUG Bonn June 2021 - The DevOps disaster
JUG Bonn June 2021 - The DevOps disasterJUG Bonn June 2021 - The DevOps disaster
JUG Bonn June 2021 - The DevOps disaster
 
Behind the Book: Gene Kim's Top Takeaways from Researching and Writing 'The D...
Behind the Book: Gene Kim's Top Takeaways from Researching and Writing 'The D...Behind the Book: Gene Kim's Top Takeaways from Researching and Writing 'The D...
Behind the Book: Gene Kim's Top Takeaways from Researching and Writing 'The D...
 
Dev ops
Dev opsDev ops
Dev ops
 
Continuous Delivery in a Legacy Shop—One Step at a Time
Continuous Delivery in a Legacy Shop—One Step at a TimeContinuous Delivery in a Legacy Shop—One Step at a Time
Continuous Delivery in a Legacy Shop—One Step at a Time
 
DevOps: A Value Proposition
DevOps: A Value PropositionDevOps: A Value Proposition
DevOps: A Value Proposition
 
Introduction to devops - update 2017
Introduction to devops - update 2017Introduction to devops - update 2017
Introduction to devops - update 2017
 
Why Even DevOp?
Why Even DevOp?Why Even DevOp?
Why Even DevOp?
 
Devops ppt
Devops pptDevops ppt
Devops ppt
 
Quality Jam 2016 Product Roadmap
Quality Jam 2016 Product RoadmapQuality Jam 2016 Product Roadmap
Quality Jam 2016 Product Roadmap
 
Understanding DevOps in simpler way with Continuous Delivery
Understanding DevOps in simpler way with Continuous DeliveryUnderstanding DevOps in simpler way with Continuous Delivery
Understanding DevOps in simpler way with Continuous Delivery
 
The Anti-Transformation transformation @DevOps Summit Amsterdam
The Anti-Transformation transformation @DevOps Summit AmsterdamThe Anti-Transformation transformation @DevOps Summit Amsterdam
The Anti-Transformation transformation @DevOps Summit Amsterdam
 
Jan de Vries - How to convince your boss that it is DevOps that he wants
Jan de Vries - How to convince your boss that it is DevOps that he wantsJan de Vries - How to convince your boss that it is DevOps that he wants
Jan de Vries - How to convince your boss that it is DevOps that he wants
 
Introduction to DevOps
Introduction to DevOpsIntroduction to DevOps
Introduction to DevOps
 
DevOps vs Agile | DevOps Tutorial For Beginners | DevOps Training | Edureka
DevOps vs Agile | DevOps Tutorial For Beginners | DevOps Training | EdurekaDevOps vs Agile | DevOps Tutorial For Beginners | DevOps Training | Edureka
DevOps vs Agile | DevOps Tutorial For Beginners | DevOps Training | Edureka
 
The Four Keys - Measuring DevOps Success
The Four Keys - Measuring DevOps SuccessThe Four Keys - Measuring DevOps Success
The Four Keys - Measuring DevOps Success
 
DevOps MythBusters
DevOps MythBustersDevOps MythBusters
DevOps MythBusters
 

Semelhante a Continuous Testing: Preparing for DevOps

Eric Ries - The Lean Startup - RailsConf 2011
Eric Ries - The Lean Startup - RailsConf 2011Eric Ries - The Lean Startup - RailsConf 2011
Eric Ries - The Lean Startup - RailsConf 2011Eric Ries
 
Why unvalidated assumption is the enemy of good product
Why unvalidated assumption is the enemy of good productWhy unvalidated assumption is the enemy of good product
Why unvalidated assumption is the enemy of good productSeb Agertoft
 
2010 10 15 the lean startup at tech_hub london
2010 10 15 the lean startup at tech_hub london2010 10 15 the lean startup at tech_hub london
2010 10 15 the lean startup at tech_hub londonEric Ries
 
How do we fix testing
How do we fix testingHow do we fix testing
How do we fix testingPeter Varhol
 
A Rapid Introduction to Rapid Software Testing
A Rapid Introduction to Rapid Software TestingA Rapid Introduction to Rapid Software Testing
A Rapid Introduction to Rapid Software TestingTechWell
 
2011 10 12 eric ries lean startup web 2.0 expo ny keynote
2011 10 12 eric ries lean startup web 2.0 expo ny keynote2011 10 12 eric ries lean startup web 2.0 expo ny keynote
2011 10 12 eric ries lean startup web 2.0 expo ny keynoteEric Ries
 
A Rapid Introduction to Rapid Software Testing
A Rapid Introduction to Rapid Software TestingA Rapid Introduction to Rapid Software Testing
A Rapid Introduction to Rapid Software TestingTechWell
 
A Rapid Introduction to Rapid Software Testing
A Rapid Introduction to Rapid Software TestingA Rapid Introduction to Rapid Software Testing
A Rapid Introduction to Rapid Software TestingTechWell
 
2010 10 28 the lean startup at ucsd
2010 10 28 the lean startup at ucsd2010 10 28 the lean startup at ucsd
2010 10 28 the lean startup at ucsdEric Ries
 
2011 03 11 eric ries - the lean startup sxsw
2011 03 11 eric ries - the lean startup sxsw2011 03 11 eric ries - the lean startup sxsw
2011 03 11 eric ries - the lean startup sxswEric Ries
 
Eric Ries - The Lean Startup - Google Tech Talk
Eric Ries - The Lean Startup - Google Tech TalkEric Ries - The Lean Startup - Google Tech Talk
Eric Ries - The Lean Startup - Google Tech TalkEric Ries
 
Eric Ries - The lean startup
Eric Ries - The lean startupEric Ries - The lean startup
Eric Ries - The lean startupmomentummi
 
Robert and Anne Sabourin: Gauging Software Health
Robert and Anne Sabourin: Gauging Software HealthRobert and Anne Sabourin: Gauging Software Health
Robert and Anne Sabourin: Gauging Software HealthAnna Royzman
 
User Stories Suck by David Hawks at North Dallas Product Owners Meetup
User Stories Suck by David Hawks at North Dallas Product Owners MeetupUser Stories Suck by David Hawks at North Dallas Product Owners Meetup
User Stories Suck by David Hawks at North Dallas Product Owners MeetupAgile Velocity
 
Understanding Root Cause Analysis
Understanding Root Cause AnalysisUnderstanding Root Cause Analysis
Understanding Root Cause Analysisleanadvisors
 
'Stakeholder Engagement Shortcuts': Ilan Goldstein @ Colombo Agile Conference...
'Stakeholder Engagement Shortcuts': Ilan Goldstein @ Colombo Agile Conference...'Stakeholder Engagement Shortcuts': Ilan Goldstein @ Colombo Agile Conference...
'Stakeholder Engagement Shortcuts': Ilan Goldstein @ Colombo Agile Conference...ColomboCampsCommunity
 
eric ries - the lean startup dogpatch LUXr nyc
eric ries - the lean startup dogpatch LUXr nyceric ries - the lean startup dogpatch LUXr nyc
eric ries - the lean startup dogpatch LUXr nycEric Ries
 
Lean Software Development: Validated Learning
Lean Software Development: Validated LearningLean Software Development: Validated Learning
Lean Software Development: Validated LearningÒscar Vilaplana
 

Semelhante a Continuous Testing: Preparing for DevOps (20)

Eric Ries - The Lean Startup - RailsConf 2011
Eric Ries - The Lean Startup - RailsConf 2011Eric Ries - The Lean Startup - RailsConf 2011
Eric Ries - The Lean Startup - RailsConf 2011
 
Why unvalidated assumption is the enemy of good product
Why unvalidated assumption is the enemy of good productWhy unvalidated assumption is the enemy of good product
Why unvalidated assumption is the enemy of good product
 
2010 10 15 the lean startup at tech_hub london
2010 10 15 the lean startup at tech_hub london2010 10 15 the lean startup at tech_hub london
2010 10 15 the lean startup at tech_hub london
 
How do we fix testing
How do we fix testingHow do we fix testing
How do we fix testing
 
A Rapid Introduction to Rapid Software Testing
A Rapid Introduction to Rapid Software TestingA Rapid Introduction to Rapid Software Testing
A Rapid Introduction to Rapid Software Testing
 
2011 10 12 eric ries lean startup web 2.0 expo ny keynote
2011 10 12 eric ries lean startup web 2.0 expo ny keynote2011 10 12 eric ries lean startup web 2.0 expo ny keynote
2011 10 12 eric ries lean startup web 2.0 expo ny keynote
 
A Rapid Introduction to Rapid Software Testing
A Rapid Introduction to Rapid Software TestingA Rapid Introduction to Rapid Software Testing
A Rapid Introduction to Rapid Software Testing
 
A Rapid Introduction to Rapid Software Testing
A Rapid Introduction to Rapid Software TestingA Rapid Introduction to Rapid Software Testing
A Rapid Introduction to Rapid Software Testing
 
2010 10 28 the lean startup at ucsd
2010 10 28 the lean startup at ucsd2010 10 28 the lean startup at ucsd
2010 10 28 the lean startup at ucsd
 
2011 03 11 eric ries - the lean startup sxsw
2011 03 11 eric ries - the lean startup sxsw2011 03 11 eric ries - the lean startup sxsw
2011 03 11 eric ries - the lean startup sxsw
 
Eric Ries - The Lean Startup - Google Tech Talk
Eric Ries - The Lean Startup - Google Tech TalkEric Ries - The Lean Startup - Google Tech Talk
Eric Ries - The Lean Startup - Google Tech Talk
 
Eric Ries - The lean startup
Eric Ries - The lean startupEric Ries - The lean startup
Eric Ries - The lean startup
 
Robert and Anne Sabourin: Gauging Software Health
Robert and Anne Sabourin: Gauging Software HealthRobert and Anne Sabourin: Gauging Software Health
Robert and Anne Sabourin: Gauging Software Health
 
User Stories Suck by David Hawks at North Dallas Product Owners Meetup
User Stories Suck by David Hawks at North Dallas Product Owners MeetupUser Stories Suck by David Hawks at North Dallas Product Owners Meetup
User Stories Suck by David Hawks at North Dallas Product Owners Meetup
 
Understanding Root Cause Analysis
Understanding Root Cause AnalysisUnderstanding Root Cause Analysis
Understanding Root Cause Analysis
 
'Stakeholder Engagement Shortcuts': Ilan Goldstein @ Colombo Agile Conference...
'Stakeholder Engagement Shortcuts': Ilan Goldstein @ Colombo Agile Conference...'Stakeholder Engagement Shortcuts': Ilan Goldstein @ Colombo Agile Conference...
'Stakeholder Engagement Shortcuts': Ilan Goldstein @ Colombo Agile Conference...
 
Martijn Beijk & Charles Goodall
Martijn Beijk & Charles GoodallMartijn Beijk & Charles Goodall
Martijn Beijk & Charles Goodall
 
eric ries - the lean startup dogpatch LUXr nyc
eric ries - the lean startup dogpatch LUXr nyceric ries - the lean startup dogpatch LUXr nyc
eric ries - the lean startup dogpatch LUXr nyc
 
Lean startup
Lean startup Lean startup
Lean startup
 
Lean Software Development: Validated Learning
Lean Software Development: Validated LearningLean Software Development: Validated Learning
Lean Software Development: Validated Learning
 

Mais de STePINForum

Am I a Social Monster?
Am I a Social Monster?Am I a Social Monster?
Am I a Social Monster?STePINForum
 
Code to Release using Artificial Intelligence and Machine Learning
Code to Release using Artificial Intelligence and Machine LearningCode to Release using Artificial Intelligence and Machine Learning
Code to Release using Artificial Intelligence and Machine LearningSTePINForum
 
Predictive Analytics based Regression Test Optimization
Predictive Analytics based Regression Test OptimizationPredictive Analytics based Regression Test Optimization
Predictive Analytics based Regression Test OptimizationSTePINForum
 
Blockchain and AI - Match made in Heaven
Blockchain and AI - Match made in HeavenBlockchain and AI - Match made in Heaven
Blockchain and AI - Match made in HeavenSTePINForum
 
Machine Learning & Artificial Intelligence - Machine Controlled Data Dispensa...
Machine Learning & Artificial Intelligence - Machine Controlled Data Dispensa...Machine Learning & Artificial Intelligence - Machine Controlled Data Dispensa...
Machine Learning & Artificial Intelligence - Machine Controlled Data Dispensa...STePINForum
 
Self Healing - Bringing Intelligence into Automation
Self Healing - Bringing Intelligence into AutomationSelf Healing - Bringing Intelligence into Automation
Self Healing - Bringing Intelligence into AutomationSTePINForum
 
Quality engineering in a world with AI and IoT
Quality engineering in a world with AI and IoTQuality engineering in a world with AI and IoT
Quality engineering in a world with AI and IoTSTePINForum
 
How accurate are the Wearable fitness tracker showing 10000 steps in a day: A...
How accurate are the Wearable fitness tracker showing 10000 steps in a day: A...How accurate are the Wearable fitness tracker showing 10000 steps in a day: A...
How accurate are the Wearable fitness tracker showing 10000 steps in a day: A...STePINForum
 
Performance Testing Internet of Things
Performance Testing Internet of ThingsPerformance Testing Internet of Things
Performance Testing Internet of ThingsSTePINForum
 
Blockchain and AI: Powering the future
Blockchain and AI: Powering the futureBlockchain and AI: Powering the future
Blockchain and AI: Powering the futureSTePINForum
 
Docker–Grid (A On demand and Scalable dockerized selenium grid architecture)
Docker–Grid (A On demand and Scalable dockerized selenium grid architecture)Docker–Grid (A On demand and Scalable dockerized selenium grid architecture)
Docker–Grid (A On demand and Scalable dockerized selenium grid architecture)STePINForum
 
AI driven classification framework for advanced Test Automation
AI driven classification framework for advanced Test AutomationAI driven classification framework for advanced Test Automation
AI driven classification framework for advanced Test AutomationSTePINForum
 
Jugaadu Automation - The Real Desi Test Automation Story
Jugaadu Automation - The Real Desi Test Automation StoryJugaadu Automation - The Real Desi Test Automation Story
Jugaadu Automation - The Real Desi Test Automation StorySTePINForum
 

Mais de STePINForum (13)

Am I a Social Monster?
Am I a Social Monster?Am I a Social Monster?
Am I a Social Monster?
 
Code to Release using Artificial Intelligence and Machine Learning
Code to Release using Artificial Intelligence and Machine LearningCode to Release using Artificial Intelligence and Machine Learning
Code to Release using Artificial Intelligence and Machine Learning
 
Predictive Analytics based Regression Test Optimization
Predictive Analytics based Regression Test OptimizationPredictive Analytics based Regression Test Optimization
Predictive Analytics based Regression Test Optimization
 
Blockchain and AI - Match made in Heaven
Blockchain and AI - Match made in HeavenBlockchain and AI - Match made in Heaven
Blockchain and AI - Match made in Heaven
 
Machine Learning & Artificial Intelligence - Machine Controlled Data Dispensa...
Machine Learning & Artificial Intelligence - Machine Controlled Data Dispensa...Machine Learning & Artificial Intelligence - Machine Controlled Data Dispensa...
Machine Learning & Artificial Intelligence - Machine Controlled Data Dispensa...
 
Self Healing - Bringing Intelligence into Automation
Self Healing - Bringing Intelligence into AutomationSelf Healing - Bringing Intelligence into Automation
Self Healing - Bringing Intelligence into Automation
 
Quality engineering in a world with AI and IoT
Quality engineering in a world with AI and IoTQuality engineering in a world with AI and IoT
Quality engineering in a world with AI and IoT
 
How accurate are the Wearable fitness tracker showing 10000 steps in a day: A...
How accurate are the Wearable fitness tracker showing 10000 steps in a day: A...How accurate are the Wearable fitness tracker showing 10000 steps in a day: A...
How accurate are the Wearable fitness tracker showing 10000 steps in a day: A...
 
Performance Testing Internet of Things
Performance Testing Internet of ThingsPerformance Testing Internet of Things
Performance Testing Internet of Things
 
Blockchain and AI: Powering the future
Blockchain and AI: Powering the futureBlockchain and AI: Powering the future
Blockchain and AI: Powering the future
 
Docker–Grid (A On demand and Scalable dockerized selenium grid architecture)
Docker–Grid (A On demand and Scalable dockerized selenium grid architecture)Docker–Grid (A On demand and Scalable dockerized selenium grid architecture)
Docker–Grid (A On demand and Scalable dockerized selenium grid architecture)
 
AI driven classification framework for advanced Test Automation
AI driven classification framework for advanced Test AutomationAI driven classification framework for advanced Test Automation
AI driven classification framework for advanced Test Automation
 
Jugaadu Automation - The Real Desi Test Automation Story
Jugaadu Automation - The Real Desi Test Automation StoryJugaadu Automation - The Real Desi Test Automation Story
Jugaadu Automation - The Real Desi Test Automation Story
 

Último

Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )Brian Pichman
 
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Will Schroeder
 
IEEE Computer Society’s Strategic Activities and Products including SWEBOK Guide
IEEE Computer Society’s Strategic Activities and Products including SWEBOK GuideIEEE Computer Society’s Strategic Activities and Products including SWEBOK Guide
IEEE Computer Society’s Strategic Activities and Products including SWEBOK GuideHironori Washizaki
 
Governance in SharePoint Premium:What's in the box?
Governance in SharePoint Premium:What's in the box?Governance in SharePoint Premium:What's in the box?
Governance in SharePoint Premium:What's in the box?Juan Carlos Gonzalez
 
Comparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioComparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioChristian Posta
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesMd Hossain Ali
 
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfUiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfDianaGray10
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostMatt Ray
 
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IES VE
 
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1DianaGray10
 
UiPath Studio Web workshop series - Day 5
UiPath Studio Web workshop series - Day 5UiPath Studio Web workshop series - Day 5
UiPath Studio Web workshop series - Day 5DianaGray10
 
VoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXVoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXTarek Kalaji
 
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UbiTrack UK
 
Computer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsComputer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsSeth Reyes
 
Videogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfVideogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfinfogdgmi
 
20230202 - Introduction to tis-py
20230202 - Introduction to tis-py20230202 - Introduction to tis-py
20230202 - Introduction to tis-pyJamie (Taka) Wang
 
Empowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintEmpowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintMahmoud Rabie
 
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAAnypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAshyamraj55
 
Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Commit University
 
99.99% of Your Traces Are (Probably) Trash (SRECon NA 2024).pdf
99.99% of Your Traces  Are (Probably) Trash (SRECon NA 2024).pdf99.99% of Your Traces  Are (Probably) Trash (SRECon NA 2024).pdf
99.99% of Your Traces Are (Probably) Trash (SRECon NA 2024).pdfPaige Cruz
 

Último (20)

Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )
 
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
 
IEEE Computer Society’s Strategic Activities and Products including SWEBOK Guide
IEEE Computer Society’s Strategic Activities and Products including SWEBOK GuideIEEE Computer Society’s Strategic Activities and Products including SWEBOK Guide
IEEE Computer Society’s Strategic Activities and Products including SWEBOK Guide
 
Governance in SharePoint Premium:What's in the box?
Governance in SharePoint Premium:What's in the box?Governance in SharePoint Premium:What's in the box?
Governance in SharePoint Premium:What's in the box?
 
Comparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioComparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and Istio
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
 
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfUiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
 
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
 
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
 
UiPath Studio Web workshop series - Day 5
UiPath Studio Web workshop series - Day 5UiPath Studio Web workshop series - Day 5
UiPath Studio Web workshop series - Day 5
 
VoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXVoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBX
 
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
 
Computer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsComputer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and Hazards
 
Videogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfVideogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdf
 
20230202 - Introduction to tis-py
20230202 - Introduction to tis-py20230202 - Introduction to tis-py
20230202 - Introduction to tis-py
 
Empowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintEmpowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership Blueprint
 
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAAnypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
 
Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)
 
99.99% of Your Traces Are (Probably) Trash (SRECon NA 2024).pdf
99.99% of Your Traces  Are (Probably) Trash (SRECon NA 2024).pdf99.99% of Your Traces  Are (Probably) Trash (SRECon NA 2024).pdf
99.99% of Your Traces Are (Probably) Trash (SRECon NA 2024).pdf
 

Continuous Testing: Preparing for DevOps

  • 1. Continuous Testing Digital Transformation Requires Continuous Testing Ingo Philipp
  • 2. Businesses must continuously exploit digital technologies to both create new sources of customer value and increase operational agility in service of customers. © Forrester Research
  • 3. Digital Disruption Across industries, companies face the challenge of software-led transformation
  • 4. Plan Code Version Build Provision Test Deploy Monitor © Melissa Perri The product is a solution. If the problem isn't solved, the product doesn't work. The goal is to accelerate this value exchange Identify Needs Receive Money Build Solution Your Company Your Customer
  • 5. Requirements Design Implementation Testing Acceptance Deployment Inception Elaboration Construction Transition Development Operations DevOps Agile Horizon Digital Singularity Incremental Rational Unified Process 4 Agile Scrum, Kanban 2 Continuous DevOps 1 6+ Big Bang Waterfall Development Response Flexible Framework Rigid Rules Following a Plan Contract Negotiation Copious Documentation Processes & Tools Individuals & Interactions Working Software Customer Collaboration Responding to Change
  • 6. It’s not the strongest that survive, nor the most intelligent, but the one most responsive to change Charles Darwin Because software is not written, it is rewritten
  • 7. 80% Manual Testing 20% Test Automation The reliance on manual testing is the top technical challenge in app development World Quality Report, 2015/16 Test automation requires developers UI LoadAPI Present Traditional Testing
  • 8. Future Continuous Testing +85% Test Automation Exploratory Tests Automated UI Tests API Tests Load Tests 80% Manual Testing 20% Test Automation Present Traditional Testing Orchestrated Service Virtualization
  • 9. 100% 0% Sprint Start Sprint End API UI Start testing early, shift left test automation Degree of Completion Time Efficiency Gain +4x Creation +6x Maintenance +20x Execution
  • 10. *voke, market snapshot report on service virtualization - 2012 On average, organizations require access to 33systems for development or testing. 18Average # of systems with unrestricted access 96% of testers have restricted test lab access Testing Present Enterprise system landscapes are alike disease gene networks. Welcome to the tester’s hell
  • 11. Automation doesn't make testing easy, it makes testing possible Lesson Learned © Wolfgang Platz
  • 12. A B Visiting London ~100 Billion Billion Billion Billion Billion Billion Billion Combinations = 50
  • 13. Testing harder isn’t the answer, testing smarter is! Lesson Learned © Wolfgang Platz
  • 18. Testing is exactly like washing a pig. Because it’s messy. It has no rules. No clear beginning, middle, or end. It’s kind of a pain in the ass, and when you’re done you’re not sure if the pig is really clean or even why you were washing a pig in the first place. © Luke Sullivan © BBC
  • 19. Testing is exactly like washing a pig. Because it’s messy. It has no rules. No clear beginning, middle, or end. It’s kind of a pain in the ass, and when you’re done you’re not sure if the pig is really clean or even why you were washing a pig in the first place. © Luke Sullivan That’s what we can know
  • 20. Testing is exactly like washing a pig. Because it’s messy. It has no rules. No clear beginning, middle, or end. It’s kind of a pain in the ass, and when you’re done you’re not sure if the pig is really clean or even why you were washing a pig in the first place. © Luke Sullivan That’s what we can know That’s what we know That’s what we check Problem Knowledge Gap © Richard Bradshaw
  • 21. © Cem Kaner © Richard Bradshaw The purpose of testing is to close the knowledge gap The goal is information, not gratuitous automation Testing is and always will be a search for information That’s what we can know That’s what we know That’s what we check Problem Knowledge Gap
  • 22. That’s what we can know That’s what we know That’s what we check Problem Knowledge Gap Checking « Requires Processing » © Rich Rogers Verify through Instructions Pay attention to Deviations Create Test Cases Follow Procedure Examine Requirements Factory Process
  • 23. » Here You Monitor Known Risks That’s what we can know That’s what we know That’s what we check Problem Knowledge Gap Confirmatory Testing « Requires Processing » © Rich Rogers Verify through Instructions Pay attention to Deviations Create Test Cases Follow Procedure Examine Requirements Factory Process MechanicalProcess
  • 24. Confirmatory Testing « Requires Processing » © Rich Rogers Verify through Instructions Pay attention to Deviations Create Test Cases Follow Procedure Examine Requirements Factory Process Exploratory Testing « Requires Thinking » Investigate through Experiments Pay attention to Oracles Create Test Ideas Follow Clues Examine Risks Adaptive Investigation » Here You Analyze Potential Risks CognitiveProcess MechanicalProcess » Here You Monitor Known Risks
  • 25. Checking Efficient Confirmatory Testing Exploring Effective Exploratory Testing Testing Thorough Testing + = Agile Testing Equation © Elisabeth Hendrickson
  • 26. If this doesn’t make an impact, then its absence doesn’t make a difference Our Proof © Trey Smith
  • 27. 100% Manual Testing 0% Automated Testing Testing Present There’s a better way to do it. Find it! Project Charter 11 Manual Testers 4755 Manual Test Cases ? Unknown Coverage 10 Weeks Execution
  • 28. Risk Coverage Optimization Test Data Management UI/API Test Automation Continuous Integration & Distributed Execution 48 Hours Execution Single Agent 8 Hours Execution Multiple Agents 11 Manual Testers 4755 Manual Test Cases ? Unknown Coverage 10 Weeks Execution 89% Business Risk Coverage 1193 Manual Test Cases 37% Business Risk 5 Weeks Execution 2.5 Weeks Execution 92% Automated Test Cases 53% Business Risk Coverage Smoke Testing 34 Minutes Execution Smoke Testing Test Case Count 7% 75% Redundancy 50% Effort Test Data 64% API Tests
  • 29. 8 Hours Execution Multiple Agents 72% Critical Defect Reduction Production Defects 89% Business Risk Coverage 1193 Manual Test Cases 5 Weeks Execution 2.5 Weeks Execution 92% Automated Test Cases 34 Minutes Execution Smoke Testing 53% Business Risk Coverage Smoke Testing Test Case Count 7% Test Case Design & Exploratory Testing 64% API Tests Risk Coverage Optimization Test Data Management UI/API Test Automation Continuous Integration & Distributed Execution
  • 30. The future doesn’t just happen, it gets happened, so make it happen Conclusion © Paul Tagliablue
  • 31. Questions The show is over. It’s your turn. ?