SlideShare uma empresa Scribd logo
1 de 14
Baixar para ler offline
VictoriaMetrics
Anomaly Detection:
Q1’24 updates
Fred Navruzov, Product Data Scientist
Daria Karavaieva, ML Engineer
Recap: What is anomaly detection?
Anomaly detection (AD hereinafter) refers to the task of identifying unusual
patterns that do not conform to expected behavior
Such “patterns” may introduce significant challenges and losses to your business,
if not properly and timely treated
Monitoring is a hard task, AD itself is a harder task, and AD for time series data -
metrics measured over time - is the hardest due to specific effects, like trends,
seasonality or different anomaly patterns
Recap: Why ML & AI for anomaly detection?
Challenges caused by time series nature of metrics, make simpler approaches, like
threshold-based alerting way less effective for complex patterns (seasonality,
collective anomalies, etc.)
Manual handling of complex metrics doesn’t scale with the growth of your data
Incident management becomes even more complicated
as the volume and complexity of the data increases
Why VictoriaMetrics Anomaly Detection?
● A service that is simple to set up and to debug
● Integrates well with VictoriaMetrics observability ecosystem
● Collective & Contextual anomalies support
(i.e. seasonality, metric interaction)
● Root Cause Analysis for faster and efficient
metric incidents drilldown
5 new releases were shipped (1.8.0 - 1.12.0) in Q1’24. The most notable features:
Flexible configs - now user can define in a single config file
● Multiple model types - combine what’s best for your data yet avoid redundant
resource usage
● Multiple schedulers to set up fit/infer routines of a model
● `queries` arg of a model to define the data it should be run on
● And everything in many-to-many relationship!
What’s new: Flexible Configs
Config section
example
What’s new: Flexible Configs
Model autotune (introduced in v1.12): reduce the cognitive load and let the data
speak for itself
● Select the model class of your choice and let it derive best params from the training data
● Define max anomaly percentage param to encounter in your data - and that’s all
What’s new: AutoTune
● Quickstart - minimalistic guide on how to set up and run
`vmanomaly` (Docker, Kubernetes)
● Model types - explanations and diagrams to understand specifics
of a lifecycle and find the best model for your use case
● AutoTuned model introduction - find out how to set-and-forget
the model of your choice to learn from your data
● VictoriaMetrics Anomaly Detection got its own feature page
What’s new: Docs & site updates
Set up, tweak and evaluate anomaly detection easier and faster to best accommodate
your data patterns!
What’s coming: GUI
● Convenient shipping of forthcoming preset assets is ready - run vmanomaly
with `preset: preset_name` and found the assets under /presets endpoint
● Tweaking is still in progress for node_exporter preset to bring the best value
What’s coming: Presets
Roadmap for 2024
● (Q2) Streaming models support: spend less resources, adapt faster, gradually update the
models after each infer on data. Overall performance optimization.
● (Q2) GUI: Deeper integration with anomaly detection service, “optimize” button to run
autotune, see detected anomalies and get the best params of a model according to your data
● (Q2) Node_exporter preset. Presets for common tasks, like “seasonal_weekly”, “testing”,
“autotuned_daily”. Spend less time setting up the service, receive appropriate results faster.
● (Q3-Q4) Root Cause Analysis: Drill down your incidents faster and more efficient. Finishing
transition from PoC to production.
Try Anomaly Detection
Being a part of Enterprise offering,
VictoriaMetrics Anomaly Detection is available for testing:
contact us or request a trial license here to give it a try!
Thank you!
Have questions left? Contact Us

Mais conteúdo relacionado

Semelhante a VictoriaMetrics Anomaly Detection Updates: Q1 2024

Slides-Артем Коваль-Cloud-Native MLOps Framework - DataFest 2021.pdf
Slides-Артем Коваль-Cloud-Native MLOps Framework - DataFest 2021.pdfSlides-Артем Коваль-Cloud-Native MLOps Framework - DataFest 2021.pdf
Slides-Артем Коваль-Cloud-Native MLOps Framework - DataFest 2021.pdf
vitm11
 
Challenges in Assessing Technical Debt based on Dynamic Runtime Data
Challenges in Assessing Technical Debt based on Dynamic Runtime DataChallenges in Assessing Technical Debt based on Dynamic Runtime Data
Challenges in Assessing Technical Debt based on Dynamic Runtime Data
QAware GmbH
 
AWS re:Invent 2016: Effective Application Data Analytics for Modern Applicati...
AWS re:Invent 2016: Effective Application Data Analytics for Modern Applicati...AWS re:Invent 2016: Effective Application Data Analytics for Modern Applicati...
AWS re:Invent 2016: Effective Application Data Analytics for Modern Applicati...
Amazon Web Services
 
dataR Corporate Overview
dataR Corporate OverviewdataR Corporate Overview
dataR Corporate Overview
fpipolo
 

Semelhante a VictoriaMetrics Anomaly Detection Updates: Q1 2024 (20)

The Automation Firehose: Be Strategic & Tactical With Your Mobile & Web Testing
The Automation Firehose: Be Strategic & Tactical With Your Mobile & Web TestingThe Automation Firehose: Be Strategic & Tactical With Your Mobile & Web Testing
The Automation Firehose: Be Strategic & Tactical With Your Mobile & Web Testing
 
Stitching Shop Management System.pptx
Stitching Shop Management System.pptxStitching Shop Management System.pptx
Stitching Shop Management System.pptx
 
The ZDLC Brief
The ZDLC BriefThe ZDLC Brief
The ZDLC Brief
 
Neev Application Performance Management Services
Neev Application Performance Management ServicesNeev Application Performance Management Services
Neev Application Performance Management Services
 
Slides-Артем Коваль-Cloud-Native MLOps Framework - DataFest 2021.pdf
Slides-Артем Коваль-Cloud-Native MLOps Framework - DataFest 2021.pdfSlides-Артем Коваль-Cloud-Native MLOps Framework - DataFest 2021.pdf
Slides-Артем Коваль-Cloud-Native MLOps Framework - DataFest 2021.pdf
 
Svm Classifier Algorithm for Data Stream Mining Using Hive and R
Svm Classifier Algorithm for Data Stream Mining Using Hive and RSvm Classifier Algorithm for Data Stream Mining Using Hive and R
Svm Classifier Algorithm for Data Stream Mining Using Hive and R
 
Challenges in Assessing Technical Debt based on Dynamic Runtime Data
Challenges in Assessing Technical Debt based on Dynamic Runtime DataChallenges in Assessing Technical Debt based on Dynamic Runtime Data
Challenges in Assessing Technical Debt based on Dynamic Runtime Data
 
Scaling AutoML-Driven Anomaly Detection With Luminaire
Scaling AutoML-Driven Anomaly Detection With LuminaireScaling AutoML-Driven Anomaly Detection With Luminaire
Scaling AutoML-Driven Anomaly Detection With Luminaire
 
Productionizing Predictive Analytics using the Rendezvous Architecture - for ...
Productionizing Predictive Analytics using the Rendezvous Architecture - for ...Productionizing Predictive Analytics using the Rendezvous Architecture - for ...
Productionizing Predictive Analytics using the Rendezvous Architecture - for ...
 
IRJET- A Detailed Analysis on Windows Event Log Viewer for Faster Root Ca...
IRJET-  	  A Detailed Analysis on Windows Event Log Viewer for Faster Root Ca...IRJET-  	  A Detailed Analysis on Windows Event Log Viewer for Faster Root Ca...
IRJET- A Detailed Analysis on Windows Event Log Viewer for Faster Root Ca...
 
AWS re:Invent 2016: Effective Application Data Analytics for Modern Applicati...
AWS re:Invent 2016: Effective Application Data Analytics for Modern Applicati...AWS re:Invent 2016: Effective Application Data Analytics for Modern Applicati...
AWS re:Invent 2016: Effective Application Data Analytics for Modern Applicati...
 
Production machine learning: Managing models, workflows and risk at scale
Production machine learning: Managing models, workflows and risk at scaleProduction machine learning: Managing models, workflows and risk at scale
Production machine learning: Managing models, workflows and risk at scale
 
Mtc strategy-briefing-houston-pd m-05212018-3
Mtc strategy-briefing-houston-pd m-05212018-3Mtc strategy-briefing-houston-pd m-05212018-3
Mtc strategy-briefing-houston-pd m-05212018-3
 
Workshop: Delivering chnages for applications and databases
Workshop: Delivering chnages for applications and databasesWorkshop: Delivering chnages for applications and databases
Workshop: Delivering chnages for applications and databases
 
SenthilKumar_Resume
SenthilKumar_ResumeSenthilKumar_Resume
SenthilKumar_Resume
 
Software Solutions for Energy Communities
Software Solutions for Energy CommunitiesSoftware Solutions for Energy Communities
Software Solutions for Energy Communities
 
Departure Delay Prediction using Machine Learning.
Departure Delay Prediction using Machine Learning.Departure Delay Prediction using Machine Learning.
Departure Delay Prediction using Machine Learning.
 
dataR Corporate Overview
dataR Corporate OverviewdataR Corporate Overview
dataR Corporate Overview
 
The Evolution of a Scrappy Startup to a Successful Web Service
The Evolution of a Scrappy Startup to a Successful Web ServiceThe Evolution of a Scrappy Startup to a Successful Web Service
The Evolution of a Scrappy Startup to a Successful Web Service
 
Workshop BI/DWH AGILE TESTING SNS Bank English
Workshop BI/DWH AGILE TESTING SNS Bank EnglishWorkshop BI/DWH AGILE TESTING SNS Bank English
Workshop BI/DWH AGILE TESTING SNS Bank English
 

Mais de VictoriaMetrics

Large-scale Logging Made Easy: Meetup at Deutsche Bank 2024
Large-scale Logging Made Easy: Meetup at Deutsche Bank 2024Large-scale Logging Made Easy: Meetup at Deutsche Bank 2024
Large-scale Logging Made Easy: Meetup at Deutsche Bank 2024
VictoriaMetrics
 
What’s New in VictoriaMetrics: Q1 2024 Updates
What’s New in VictoriaMetrics: Q1 2024 UpdatesWhat’s New in VictoriaMetrics: Q1 2024 Updates
What’s New in VictoriaMetrics: Q1 2024 Updates
VictoriaMetrics
 
WEDOS & VictoriaMetrics
WEDOS & VictoriaMetricsWEDOS & VictoriaMetrics
WEDOS & VictoriaMetrics
VictoriaMetrics
 
VictoriaMetrics 15/12 Meet Up: 2022 Features Highlights
VictoriaMetrics 15/12 Meet Up: 2022 Features HighlightsVictoriaMetrics 15/12 Meet Up: 2022 Features Highlights
VictoriaMetrics 15/12 Meet Up: 2022 Features Highlights
VictoriaMetrics
 

Mais de VictoriaMetrics (19)

Large-scale Logging Made Easy: Meetup at Deutsche Bank 2024
Large-scale Logging Made Easy: Meetup at Deutsche Bank 2024Large-scale Logging Made Easy: Meetup at Deutsche Bank 2024
Large-scale Logging Made Easy: Meetup at Deutsche Bank 2024
 
VictoriaMetrics Q1 Meet Up '24 - Community & News Update
VictoriaMetrics Q1 Meet Up '24 - Community & News UpdateVictoriaMetrics Q1 Meet Up '24 - Community & News Update
VictoriaMetrics Q1 Meet Up '24 - Community & News Update
 
What’s New in VictoriaMetrics: Q1 2024 Updates
What’s New in VictoriaMetrics: Q1 2024 UpdatesWhat’s New in VictoriaMetrics: Q1 2024 Updates
What’s New in VictoriaMetrics: Q1 2024 Updates
 
VictoriaMetrics December 2023 Meetup: Community Update
VictoriaMetrics December 2023 Meetup: Community UpdateVictoriaMetrics December 2023 Meetup: Community Update
VictoriaMetrics December 2023 Meetup: Community Update
 
VictoriaMetrics for the Atlas Cluster
VictoriaMetrics for the Atlas ClusterVictoriaMetrics for the Atlas Cluster
VictoriaMetrics for the Atlas Cluster
 
WEDOS & VictoriaMetrics
WEDOS & VictoriaMetricsWEDOS & VictoriaMetrics
WEDOS & VictoriaMetrics
 
VictoriaMetrics December 2023 Meetup: Managed VictoriaMetrics Update
VictoriaMetrics December 2023 Meetup: Managed VictoriaMetrics UpdateVictoriaMetrics December 2023 Meetup: Managed VictoriaMetrics Update
VictoriaMetrics December 2023 Meetup: Managed VictoriaMetrics Update
 
December 2024 Meetup: Welcome & VictoriaMetrics Updates
December 2024 Meetup: Welcome & VictoriaMetrics UpdatesDecember 2024 Meetup: Welcome & VictoriaMetrics Updates
December 2024 Meetup: Welcome & VictoriaMetrics Updates
 
What’s new in VictoriaLogs (Q3 2023)
What’s new in VictoriaLogs (Q3 2023)What’s new in VictoriaLogs (Q3 2023)
What’s new in VictoriaLogs (Q3 2023)
 
Q3 Meet Up '23 - Community Update
Q3 Meet Up '23 - Community UpdateQ3 Meet Up '23 - Community Update
Q3 Meet Up '23 - Community Update
 
Managed VictoriaMetrics: Intro & Update
Managed VictoriaMetrics: Intro & UpdateManaged VictoriaMetrics: Intro & Update
Managed VictoriaMetrics: Intro & Update
 
VM Anomaly Detection: Introduction
VM Anomaly Detection: IntroductionVM Anomaly Detection: Introduction
VM Anomaly Detection: Introduction
 
Q3 2023 Meet Up: What's New in VictoriaMetrics
Q3 2023 Meet Up: What's New in VictoriaMetricsQ3 2023 Meet Up: What's New in VictoriaMetrics
Q3 2023 Meet Up: What's New in VictoriaMetrics
 
VictoriaLogs: Open Source Log Management System - Preview
VictoriaLogs: Open Source Log Management System - PreviewVictoriaLogs: Open Source Log Management System - Preview
VictoriaLogs: Open Source Log Management System - Preview
 
VictoriaMetrics: Welcome to the Virtual Meet Up March 2023
VictoriaMetrics: Welcome to the Virtual Meet Up March 2023VictoriaMetrics: Welcome to the Virtual Meet Up March 2023
VictoriaMetrics: Welcome to the Virtual Meet Up March 2023
 
Application Monitoring using Open Source: VictoriaMetrics - ClickHouse
Application Monitoring using Open Source: VictoriaMetrics - ClickHouseApplication Monitoring using Open Source: VictoriaMetrics - ClickHouse
Application Monitoring using Open Source: VictoriaMetrics - ClickHouse
 
VictoriaMetrics 15/12 Meet Up: Updates on Managed VictoriaMetrics
VictoriaMetrics 15/12 Meet Up: Updates on Managed VictoriaMetricsVictoriaMetrics 15/12 Meet Up: Updates on Managed VictoriaMetrics
VictoriaMetrics 15/12 Meet Up: Updates on Managed VictoriaMetrics
 
VictoriaMetrics 2023 Roadmap
VictoriaMetrics 2023 RoadmapVictoriaMetrics 2023 Roadmap
VictoriaMetrics 2023 Roadmap
 
VictoriaMetrics 15/12 Meet Up: 2022 Features Highlights
VictoriaMetrics 15/12 Meet Up: 2022 Features HighlightsVictoriaMetrics 15/12 Meet Up: 2022 Features Highlights
VictoriaMetrics 15/12 Meet Up: 2022 Features Highlights
 

Último

Último (20)

WSO2Con2024 - Facilitating Broadband Switching Services for UK Telecoms Provi...
WSO2Con2024 - Facilitating Broadband Switching Services for UK Telecoms Provi...WSO2Con2024 - Facilitating Broadband Switching Services for UK Telecoms Provi...
WSO2Con2024 - Facilitating Broadband Switching Services for UK Telecoms Provi...
 
Novo Nordisk: When Knowledge Graphs meet LLMs
Novo Nordisk: When Knowledge Graphs meet LLMsNovo Nordisk: When Knowledge Graphs meet LLMs
Novo Nordisk: When Knowledge Graphs meet LLMs
 
WSO2CON2024 - Why Should You Consider Ballerina for Your Next Integration
WSO2CON2024 - Why Should You Consider Ballerina for Your Next IntegrationWSO2CON2024 - Why Should You Consider Ballerina for Your Next Integration
WSO2CON2024 - Why Should You Consider Ballerina for Your Next Integration
 
WSO2CON 2024 - WSO2's Digital Transformation Journey with Choreo: A Platforml...
WSO2CON 2024 - WSO2's Digital Transformation Journey with Choreo: A Platforml...WSO2CON 2024 - WSO2's Digital Transformation Journey with Choreo: A Platforml...
WSO2CON 2024 - WSO2's Digital Transformation Journey with Choreo: A Platforml...
 
WSO2CON 2024 - Designing Event-Driven Enterprises: Stories of Transformation
WSO2CON 2024 - Designing Event-Driven Enterprises: Stories of TransformationWSO2CON 2024 - Designing Event-Driven Enterprises: Stories of Transformation
WSO2CON 2024 - Designing Event-Driven Enterprises: Stories of Transformation
 
WSO2CON 2024 - Building a Digital Government in Uganda
WSO2CON 2024 - Building a Digital Government in UgandaWSO2CON 2024 - Building a Digital Government in Uganda
WSO2CON 2024 - Building a Digital Government in Uganda
 
WSO2CON 2024 - IoT Needs CIAM: The Importance of Centralized IAM in a Growing...
WSO2CON 2024 - IoT Needs CIAM: The Importance of Centralized IAM in a Growing...WSO2CON 2024 - IoT Needs CIAM: The Importance of Centralized IAM in a Growing...
WSO2CON 2024 - IoT Needs CIAM: The Importance of Centralized IAM in a Growing...
 
WSO2Con2024 - Enabling Transactional System's Exponential Growth With Simplicity
WSO2Con2024 - Enabling Transactional System's Exponential Growth With SimplicityWSO2Con2024 - Enabling Transactional System's Exponential Growth With Simplicity
WSO2Con2024 - Enabling Transactional System's Exponential Growth With Simplicity
 
WSO2Con2024 - Low-Code Integration Tooling
WSO2Con2024 - Low-Code Integration ToolingWSO2Con2024 - Low-Code Integration Tooling
WSO2Con2024 - Low-Code Integration Tooling
 
Effective Strategies for Wix's Scaling challenges - GeeCon
Effective Strategies for Wix's Scaling challenges - GeeConEffective Strategies for Wix's Scaling challenges - GeeCon
Effective Strategies for Wix's Scaling challenges - GeeCon
 
WSO2Con2024 - Unleashing the Financial Potential of 13 Million People
WSO2Con2024 - Unleashing the Financial Potential of 13 Million PeopleWSO2Con2024 - Unleashing the Financial Potential of 13 Million People
WSO2Con2024 - Unleashing the Financial Potential of 13 Million People
 
AzureNativeQumulo_HPC_Cloud_Native_Benchmarks.pdf
AzureNativeQumulo_HPC_Cloud_Native_Benchmarks.pdfAzureNativeQumulo_HPC_Cloud_Native_Benchmarks.pdf
AzureNativeQumulo_HPC_Cloud_Native_Benchmarks.pdf
 
WSO2Con2024 - Organization Management: The Revolution in B2B CIAM
WSO2Con2024 - Organization Management: The Revolution in B2B CIAMWSO2Con2024 - Organization Management: The Revolution in B2B CIAM
WSO2Con2024 - Organization Management: The Revolution in B2B CIAM
 
Artyushina_Guest lecture_YorkU CS May 2024.pptx
Artyushina_Guest lecture_YorkU CS May 2024.pptxArtyushina_Guest lecture_YorkU CS May 2024.pptx
Artyushina_Guest lecture_YorkU CS May 2024.pptx
 
BusinessGPT - Security and Governance for Generative AI
BusinessGPT  - Security and Governance for Generative AIBusinessGPT  - Security and Governance for Generative AI
BusinessGPT - Security and Governance for Generative AI
 
What Goes Wrong with Language Definitions and How to Improve the Situation
What Goes Wrong with Language Definitions and How to Improve the SituationWhat Goes Wrong with Language Definitions and How to Improve the Situation
What Goes Wrong with Language Definitions and How to Improve the Situation
 
Devoxx UK 2024 - Going serverless with Quarkus, GraalVM native images and AWS...
Devoxx UK 2024 - Going serverless with Quarkus, GraalVM native images and AWS...Devoxx UK 2024 - Going serverless with Quarkus, GraalVM native images and AWS...
Devoxx UK 2024 - Going serverless with Quarkus, GraalVM native images and AWS...
 
WSO2CON 2024 - Freedom First—Unleashing Developer Potential with Open Source
WSO2CON 2024 - Freedom First—Unleashing Developer Potential with Open SourceWSO2CON 2024 - Freedom First—Unleashing Developer Potential with Open Source
WSO2CON 2024 - Freedom First—Unleashing Developer Potential with Open Source
 
[GeeCON2024] How I learned to stop worrying and love the dark silicon apocalypse
[GeeCON2024] How I learned to stop worrying and love the dark silicon apocalypse[GeeCON2024] How I learned to stop worrying and love the dark silicon apocalypse
[GeeCON2024] How I learned to stop worrying and love the dark silicon apocalypse
 
WSO2CON2024 - It's time to go Platformless
WSO2CON2024 - It's time to go PlatformlessWSO2CON2024 - It's time to go Platformless
WSO2CON2024 - It's time to go Platformless
 

VictoriaMetrics Anomaly Detection Updates: Q1 2024

  • 1.
  • 2. VictoriaMetrics Anomaly Detection: Q1’24 updates Fred Navruzov, Product Data Scientist Daria Karavaieva, ML Engineer
  • 3. Recap: What is anomaly detection? Anomaly detection (AD hereinafter) refers to the task of identifying unusual patterns that do not conform to expected behavior Such “patterns” may introduce significant challenges and losses to your business, if not properly and timely treated Monitoring is a hard task, AD itself is a harder task, and AD for time series data - metrics measured over time - is the hardest due to specific effects, like trends, seasonality or different anomaly patterns
  • 4. Recap: Why ML & AI for anomaly detection? Challenges caused by time series nature of metrics, make simpler approaches, like threshold-based alerting way less effective for complex patterns (seasonality, collective anomalies, etc.) Manual handling of complex metrics doesn’t scale with the growth of your data Incident management becomes even more complicated as the volume and complexity of the data increases
  • 5. Why VictoriaMetrics Anomaly Detection? ● A service that is simple to set up and to debug ● Integrates well with VictoriaMetrics observability ecosystem ● Collective & Contextual anomalies support (i.e. seasonality, metric interaction) ● Root Cause Analysis for faster and efficient metric incidents drilldown
  • 6. 5 new releases were shipped (1.8.0 - 1.12.0) in Q1’24. The most notable features: Flexible configs - now user can define in a single config file ● Multiple model types - combine what’s best for your data yet avoid redundant resource usage ● Multiple schedulers to set up fit/infer routines of a model ● `queries` arg of a model to define the data it should be run on ● And everything in many-to-many relationship! What’s new: Flexible Configs
  • 8. Model autotune (introduced in v1.12): reduce the cognitive load and let the data speak for itself ● Select the model class of your choice and let it derive best params from the training data ● Define max anomaly percentage param to encounter in your data - and that’s all What’s new: AutoTune
  • 9. ● Quickstart - minimalistic guide on how to set up and run `vmanomaly` (Docker, Kubernetes) ● Model types - explanations and diagrams to understand specifics of a lifecycle and find the best model for your use case ● AutoTuned model introduction - find out how to set-and-forget the model of your choice to learn from your data ● VictoriaMetrics Anomaly Detection got its own feature page What’s new: Docs & site updates
  • 10. Set up, tweak and evaluate anomaly detection easier and faster to best accommodate your data patterns! What’s coming: GUI
  • 11. ● Convenient shipping of forthcoming preset assets is ready - run vmanomaly with `preset: preset_name` and found the assets under /presets endpoint ● Tweaking is still in progress for node_exporter preset to bring the best value What’s coming: Presets
  • 12. Roadmap for 2024 ● (Q2) Streaming models support: spend less resources, adapt faster, gradually update the models after each infer on data. Overall performance optimization. ● (Q2) GUI: Deeper integration with anomaly detection service, “optimize” button to run autotune, see detected anomalies and get the best params of a model according to your data ● (Q2) Node_exporter preset. Presets for common tasks, like “seasonal_weekly”, “testing”, “autotuned_daily”. Spend less time setting up the service, receive appropriate results faster. ● (Q3-Q4) Root Cause Analysis: Drill down your incidents faster and more efficient. Finishing transition from PoC to production.
  • 13. Try Anomaly Detection Being a part of Enterprise offering, VictoriaMetrics Anomaly Detection is available for testing: contact us or request a trial license here to give it a try!
  • 14. Thank you! Have questions left? Contact Us