Quality assurance for Metso. How test automation helped Metso gain confidence in the business process quality. SAP Finug Syyseminaari 2018 Qentinel Pace: https://qentinel.com/software-testing/

1. Case Metso – Quality assurance
SAP Finug Syysseminaari 2018
Case Metso – Quality assurance
SAP Finug Syysseminaari 2018
Esko Harjama
Director, Enterprise Architecture
Metso (Product Owner in the project)
Marko Rajala
Consultant
Qentinel (Lead Consultant, User validation specialist in the project)

2. How Qentinel Pace robots do regression tests of business processes in a global SAP/Hybris
implementation.
• Case background
• What has been done?
• Qentinel Pace™
• Qentinel QA solution
• Environment architecture
• Manual vs robotic software testing
• Typical findings from automated testing
• Qentinel Pace – Metso ; facts, insights
• How Metso has seen the service? What are the benefits seen already?
CONFIDENTIAL2
Agenda

3. Metso highlights
Mining Aggregates
Process industries Recycling
12,000
employees
worldwide
FY2017
Orders received EUR 3,0 billion
Sales EUR 2,7 billion
Adjusted EBITA 9,0%
35% equipment, 65% services
PUBLIC
51% 26%
19% 4%
3

4. my.metso.com –an improved digital customer journey for the
entire equipment lifecycle
4

5. Complexities across the online sales process
5
Testing various variables and combinations critical
• Distributor-specific catalogues (items vary)
• Several delivering plants – ’saleability’ defined
by the default plants
• Price & availability specific for customer, plant &
material combinations
• Different checkout parameters for different plants
(= order / quotation types)
• Different currencies
• Different quotation/order types for printouts and
order history

6. Metso needs confidence in digital business
processes
Target: Improving online business with modern easy-to-use online tools
• Over 100 Aggregates distributors, who need to quote and order on Metso offering
based on real-time supply chain data from SAP
• 100 000+ parts and 100+ capital equipment provided online
Business processes must work end-to-end
• Data integrity is a major issue: must test the whole business process with data
• Number of combinations with distributors and parts, equipment: testing is frequent and
laborious
• Regression testing important – core ordering processes have to be robust!
• Equipment are possible to configure based on customer needs; testing needed that
configuration is correct and saved in SAP
• Agile development: need fast feedback to developers
6

7. • Preparations to the first
pilot distributors
• Roll-outs for first group of
users (EMEA)
• Manual quality validation
• Discussion begun about
automatic validation –
because the amount of
distributors was seen to
grow fast
• More distributors added to
production
• Qentinel Pace taken into
use in sprint mode
• Focus in validating the
purchase flow end-to-end
(SAP automation excluded)
• Multibrowser and
multienvironment support
implemented
• Automation to cover SAP
(QVision)
CONFIDENTIAL7
• Testing coverage
increased also to capital
equipment side
• Ensuring the capital
readiness for pilot
• Readiness for Qentinel
Pace achieved to SAP
back-end verification
What has been done?
1Q2018 ? 2Q2018? 3Q2018

8. Qentinel Pace™
Robotic software testing
https://qentinel.com/software-testing/
Test automation
• Cloud-based
• Keyword-based tests require
no programming skills
• Integration with DevOps tools
Business impact
• Cause-effect model
Quality reporting
• Any number of KPIs
• From any source
• In a hierarchy
Usage-based pricing
8

9. Qentinel QA solution
Qentinel Pace™ - Robotic Software Testing
End-to-end testing of the purchase flow
• Ensuring that system operates as it is
planned
• Both in the parts’ and the capital equipment
side
• Main features and data are verified
Qentinel helps in user validation
• Assisting Metso in validating features
Main features are first tested manually
then automatized
• Coordinating business testing
activities at Metso
9

10. Simplified environment architecture
SAPMM
Materials
Management
Hybris Cloud
Shop front-end
my.metso.com
Refresh
Development environments
STG
STG
RMMQMM
QA
QA
DMM
Dev
Dev
PMM
Prod
Prod
Production
Qentinel
Pace
Priority
10

11. Manual vs. Robotic software testing
• Robot won’t take breaks.
• Between test runs some maintenance for scripts is needed.
• Manual testing can focus on possible problematic areas.
Qentinel Pace automation:
Execution time for 1 distributor and 11
cases for Parts: 8min 25sec
We support ~110 distributors in Qentinel Pace
Manually ~ 1 week (37 hours)
With automation ~ 7 hours
Manual testing:
Execution time for 1 distributor and 11 cases
for Parts: 20min 50sec
11

12. Typical findings from automated testing
• Scope related changes
• Configuration issues
• System ”Oops errors” in various situations.
- Changing the sales area of same distributor, corrupted cart
• Functionality errors
• Problems in SAP environment reflects to front-end
• Inconsistency errors in:
- Format how to present different measures, decimal digits etc.
12

13. CONFIDENTIAL13
Insights
• Dashboard of KPI’s and metrics which are
important to project
(Data source can be any relevant system e.g.
JIRA, Azure DevOps etc.)
• Quality Index provides real time status of overall
business benefits

14. • Automated testing creates confidence and pushes quality
• Monthly deployments cause pressure on continuous quality assurance
• Regression testing is critical – core ordering processes cannot tolerate
errors
• Quality assurance feedback for go-live decisions always reviewed
• Larger adoption of test automation among the developers and testers
progressing
CONFIDENTIAL14
How Metso has seen the QA service? What
are the benefits seen already?

15. CONFIDENTIAL15
Metso
www.metso.com
Esko Harjama
esko.harjama@metso.com
Qentinel
www.qentinel.com
Marko Rajala
Marko.rajala@qentinel.com
Thank you !