The document discusses performance monitoring, which analyzes live data beyond initial performance testing. It outlines key parameters to monitor like uptime, page speed, and memory usage. Tools like Splunk can centralize monitoring across distributed systems. The document provides a case study of high page load times and explores potential root causes like slow API calls or custom UI code. Analyzing log messages and database queries can help identify specific problems.

1. ANALYSING
PERFORMANCE
METRICS
Steps beyond performance testing - Adoniram Mishra and
Rupesh Dubey, ThoughtWorks

2. Agenda:
 How is it related and different than performance testing
 What parameters to monitor
 What are the tools we use for performance monitoring
 How to analyze live data

3.  Basics are the same
9 Most important performance monitoring parameters
Uptime
Page Speed
Full page load time
Geographic performance
Disk Free space
Memory Utilization ( Heap memory, overall memory)
Database performance
CPU usage
Internal Jobs / cron jobs

5. • Simpler Architecture
• Centralised Monitoring
• Limited Bottlenecks

6.  Distributed Architecture
 Multi technologies
 Multi vendors
 Scattered Information
 Rich UI

7.  Daily Traffic : 2 million+page views
 7 back end systems
 1 Front end system
 Rich UI + responsiveness
 25 Servers

11. Client
gmetad
gmetad gmetad
gmond
Node
gmond
Node
gmond
Node
gmond
Node
Poll Poll
Poll
Poll
CLUSTER 1
Poll PollPoll
CLUSTER 2

13. Splunk Server
Data Source 1 Data Source 2 Data Source 3
Forwarders Forwarders
Forwarders
Alerts
Search
Dashboard, reports

16.  Case Study
War Room Notes:
Apr 25 9:00 AM -- [Agent
SPLUNK]The Memory on 3 Backend
servers have crossed 90% -- May
hem ! May Hem!

17. • Should we add more memory. -
This will be a temp fix, without
even investigating a root cause
• Let's see what SPLUNK gives
us.Does it had additional
information
• Real Fix lies with ....
Let's refer to our architecture

18.  Recent integration with a 3rd party system
 We wrote our custom UI
 In performance testing the page load time was about 7
sec.
 What's the root cause? Solution:
• Is our API call slow?
• Is our custom UI slow?
• Or is it? ....

20.  For most heavily used db queries
 Need to check which are the slow queries

21.  This is an important aspect to check while doing a functional testing
 How and what message is logged not just the log level

22.  Eg:
127.0.0.1 [01/jan/2014:16:38:24 -0600] "GET /api/books_details HTTP/1.1" 200 476
127.0.0.1 [01/jan/2014:16:39:24 -0600] "GET /api/magazines_details HTTP/1.1" 404
500
We can query the GET calls which is giving 404 as status code from the log server