mirror of
https://github.com/VictoriaMetrics/VictoriaMetrics.git
synced 2024-11-21 14:44:00 +00:00
docs: vmanomaly slight improvements (#5637)
* - better messaging - update links to dockerhub in guides - added anomaly_score to FAQ - improve model section (sort + use cases) - slight refactor of a guide * rename guide & change refs * change wording in installation options * - update remaining text reference - add cross-link to component sections in guide * add docs/.jekyll-metadata to .gitignore
This commit is contained in:
Fred Navruzov
GitHub
parent
0a5ffb3bc1
commit
69ae1d30bf
15 changed files with 138 additions and 111 deletions
1
.gitignore
vendored
1
.gitignore
vendored
|
|
@ -22,3 +22,4 @@ Gemfile.lock
|
|||
/_site
|
||||
_site
|
||||
*.tmp
|
||||
/docs/.jekyll-metadata
|
||||
|
|
@ -1,6 +1,6 @@
|
|||
# Docker Compose file for "Getting Started with vmanomaly" guide
|
||||
# Docker Compose file for "vmanomaly integration" guide
|
||||
|
||||
Please read the "Getting Started with vmanomaly" guide first - [https://docs.victoriametrics.com/anomaly-detection/guides/guide-vmanomaly-vmalert.html](https://docs.victoriametrics.com/anomaly-detection/guides/guide-vmanomaly-vmalert.html)
|
||||
Please read the "vmanomaly integration" guide first - [https://docs.victoriametrics.com/anomaly-detection/guides/guide-vmanomaly-vmalert.html](https://docs.victoriametrics.com/anomaly-detection/guides/guide-vmanomaly-vmalert.html)
|
||||
|
||||
To make this Docker compose file work, you MUST replace the content of [vmanomaly_license.txt](https://github.com/VictoriaMetrics/VictoriaMetrics/tree/master/deployment/docker/vmanomaly/vmanomaly-vmalert-guide/vmanomaly_license.txt) with valid license.
|
||||
|
||||
|
|
@ -18,6 +18,13 @@ VictoriaMetrics Anomaly Detection, also known as `vmanomaly`, is a service for d
|
|||
|
||||
Please refer to [our guide section](/anomaly-detection/#practical-guides-and-installation) to find out more.
|
||||
|
||||
## What is anomaly score?
|
||||
Among the metrics produced by `vmanomaly` (as detailed in [vmanomaly output metrics](/anomaly-detection/components/models.html#vmanomaly-output)), `anomaly_score` is a pivotal one. It is **a continuous score > 0**, calculated in such a way that **scores ranging from 0.0 to 1.0 usually represent normal data**, while **scores exceeding 1.0 are typically classified as anomalous**. However, it's important to note that the threshold for anomaly detection can be customized in the alert configuration settings.
|
||||
|
||||
The decision to set the changepoint at `1.0` is made to ensure consistency across various models and alerting configurations, such that a score above `1.0` consistently signifies an anomaly, thus, alerting rules are maintained more easily.
|
||||
|
||||
> Note: `anomaly_score` is a metric itself, which preserves all labels found in input data and (optionally) appends [custom labels, specified in writer](/anomaly-detection/components/writer.html#metrics-formatting) - follow the link detailed output example
|
||||
|
||||
## How does vmanomaly work?
|
||||
`vmanomaly` applies built-in (or custom) [anomaly detection algorithms](/anomaly-detection/components/models.html), specified in a config file. Although a single config file supports one model, running multiple instances of `vmanomaly` with different configs is possible and encouraged for parallel processing or better support for your use case (i.e. simpler model for simple metrics, more sophisticated one for metrics with trends and seasonalities).
|
||||
|
||||
|
|
|
|||
|
|
@ -21,12 +21,13 @@ In the dynamic and complex world of system monitoring, VictoriaMetrics Anomaly D
|
|||
## Practical Guides and Installation
|
||||
Begin your VictoriaMetrics Anomaly Detection journey with ease using our guides and installation instructions:
|
||||
|
||||
- **Quick Start**: Find out what is behind `vmanomaly` [here](/vmanomaly.html)
|
||||
- **Integration**: Simplify the process of integrating anomaly detection into your observability ecosystem. Get started [**here**](/anomaly-detection/guides/guide-vmanomaly-vmalert.html).
|
||||
- **Quick Start**: Find out how `vmanomaly` service operates [here](/vmanomaly.html)
|
||||
- **Integration**: Integrate anomaly detection into your observability ecosystem. Get started [**here**](/anomaly-detection/guides/guide-vmanomaly-vmalert.html).
|
||||
|
||||
- **Installation Options**: Select the method that aligns with your technical requirements:
|
||||
- **Docker Installation**: Suitable for containerized environments. See [Docker guide](/vmanomaly.html#run-vmanomaly-docker-container).
|
||||
- **Helm Chart Installation**: Appropriate for those using Kubernetes. See our [Helm charts](https://github.com/VictoriaMetrics/helm-charts/tree/master/charts/victoria-metrics-anomaly).
|
||||
|
||||
- **Installation Options**: Choose the method that best fits your environment:
|
||||
- **Docker Installation**: Ideal for containerized environments. Follow our [Docker guide](../vmanomaly.md#run-vmanomaly-docker-container) for a smooth setup.
|
||||
- **Helm Chart Installation**: Perfect for Kubernetes users. Deploy using our [Helm charts](https://github.com/VictoriaMetrics/helm-charts/tree/master/charts/victoria-metrics-anomaly) for an efficient integration.
|
||||
|
||||
> Note: starting from [v1.5.0](./CHANGELOG.md#v150) `vmanomaly` requires a [license key](/vmanomaly.html#licensing) to run. You can obtain a trial license key [**here**](https://victoriametrics.com/products/enterprise/trial/index.html).
|
||||
|
||||
|
|
|
|||
|
|
@ -33,52 +33,80 @@ VM Anomaly Detection (`vmanomaly` hereinafter) models support 2 groups of parame
|
|||
|
||||
**Models**:
|
||||
|
||||
* [ARIMA](#arima)
|
||||
* [Holt-Winters](#holt-winters)
|
||||
* [Prophet](#prophet)
|
||||
* [Rolling Quantile](#rolling-quantile)
|
||||
* [Seasonal Trend Decomposition](#seasonal-trend-decomposition)
|
||||
* [Z-score](#z-score)
|
||||
* [MAD (Median Absolute Deviation)](#mad-median-absolute-deviation)
|
||||
* [Isolation forest (Multivariate)](#isolation-forest-multivariate)
|
||||
* [Custom model](#custom-model)
|
||||
* [Prophet](#prophet) - the most versatile one for production usage, especially for complex data ([trends](https://victoriametrics.com/blog/victoriametrics-anomaly-detection-handbook-chapter-1/#trend), [change points](https://victoriametrics.com/blog/victoriametrics-anomaly-detection-handbook-chapter-2/#novelties), [multi-seasonality](https://victoriametrics.com/blog/victoriametrics-anomaly-detection-handbook-chapter-1/#seasonality))
|
||||
* [Z-score](#z-score) - useful for testing and for simpler data ([de-trended](https://victoriametrics.com/blog/victoriametrics-anomaly-detection-handbook-chapter-1/#trend) data without strict [seasonality](https://victoriametrics.com/blog/victoriametrics-anomaly-detection-handbook-chapter-1/#seasonality) and with anomalies of similar magnitude as your "normal" data)
|
||||
* [Holt-Winters](#holt-winters) - well-suited for **data with moderate complexity**, exhibiting distinct [trends](https://victoriametrics.com/blog/victoriametrics-anomaly-detection-handbook-chapter-1/#trend) and/or [seasonal patterns](https://victoriametrics.com/blog/victoriametrics-anomaly-detection-handbook-chapter-1/#seasonality).
|
||||
* [MAD (Median Absolute Deviation)](#mad-median-absolute-deviation) - similarly to Z-score, is effective for **identifying outliers in relatively consistent data** (useful for detecting sudden, stark deviations from the median)
|
||||
* [Rolling Quantile](#rolling-quantile) - best for **data with evolving patterns**, as it adapts to changes over a rolling window.
|
||||
* [Seasonal Trend Decomposition](#seasonal-trend-decomposition) - similarly to Holt-Winters, is best for **data with pronounced [seasonal](https://victoriametrics.com/blog/victoriametrics-anomaly-detection-handbook-chapter-1/#seasonality) and [trend](https://victoriametrics.com/blog/victoriametrics-anomaly-detection-handbook-chapter-1/#trend) components**
|
||||
* [ARIMA](#arima) - use when your data shows **clear patterns or autocorrelation (the degree of correlation between values of the same series at different periods)**. However, good understanding of machine learning is required to tune.
|
||||
* [Isolation forest (Multivariate)](#isolation-forest-multivariate) - useful for **metrics data interaction** (several queries/metrics -> single anomaly score) and **efficient in detecting anomalies in high-dimensional datasets**
|
||||
* [Custom model](#custom-model) - benefit from your own models and expertise to better support your **unique use case**.
|
||||
|
||||
### [ARIMA](https://en.wikipedia.org/wiki/Autoregressive_integrated_moving_average)
|
||||
Here we use ARIMA implementation from `statsmodels` [library](https://www.statsmodels.org/dev/generated/statsmodels.tsa.arima.model.ARIMA.html)
|
||||
|
||||
### [Prophet](https://facebook.github.io/prophet/)
|
||||
Here we utilize the Facebook Prophet implementation, as detailed in their [library documentation](https://facebook.github.io/prophet/docs/quick_start.html#python-api). All parameters from this library are compatible and can be passed to the model.
|
||||
|
||||
*Parameters specific for vmanomaly*:
|
||||
|
||||
* `class` (string) - model class name `"model.arima.ArimaModel"`
|
||||
* `class` (string) - model class name `"model.prophet.ProphetModel"`
|
||||
* `seasonalities` (list[dict], optional) - Extra seasonalities to pass to Prophet. See [`add_seasonality()`](https://facebook.github.io/prophet/docs/seasonality,_holiday_effects,_and_regressors.html#modeling-holidays-and-special-events:~:text=modeling%20the%20cycle-,Specifying,-Custom%20Seasonalities) Prophet param.
|
||||
* `provide_series` (dict, optional) - model resulting metrics. If not specified [standard metrics](#vmanomaly-output) will be provided.
|
||||
|
||||
* `z_threshold` (float, optional) - [standard score](https://en.wikipedia.org/wiki/Standard_score) for calculating boundaries to define anomaly score. Defaults to `2.5`.
|
||||
**Note**: Apart from standard vmanomaly output Prophet model can provide [additional metrics](#additional-output-metrics-produced-by-fb-prophet).
|
||||
|
||||
* `provide_series` (list[string], optional) - List of columns to be produced and returned by the model. Defaults to `["anomaly_score", "yhat", "yhat_lower" "yhat_upper", "y"]`. Output can be **only a subset** of a given column list.
|
||||
|
||||
* `resample_freq` (string, optional) - Frequency to resample input data into, e.g. data comes at 15 seconds resolution, and resample_freq is '1m'. Then fitting data will be downsampled to '1m' and internal model is trained at '1m' intervals. So, during inference, prediction data would be produced at '1m' intervals, but interpolated to "15s" to match with expected output, as output data must have the same timestamps.
|
||||
|
||||
*Default model parameters*:
|
||||
|
||||
* `order` (list[int]) - ARIMA's (p,d,q) order of the model for the autoregressive, differences, and moving average components, respectively.
|
||||
|
||||
* `args` (dict, optional) - Inner model args (key-value pairs). See accepted params in [model documentation](https://www.statsmodels.org/dev/generated/statsmodels.tsa.arima.model.ARIMA.html). Defaults to empty (not provided). Example: {"trend": "c"}
|
||||
**Additional output metrics produced by FB Prophet**
|
||||
Depending on chosen `seasonality` parameter FB Prophet can return additional metrics such as:
|
||||
- `trend`, `trend_lower`, `trend_upper`
|
||||
- `additive_terms`, `additive_terms_lower`, `additive_terms_upper`,
|
||||
- `multiplicative_terms`, `multiplicative_terms_lower`, `multiplicative_terms_upper`,
|
||||
- `daily`, `daily_lower`, `daily_upper`,
|
||||
- `hourly`, `hourly_lower`, `hourly_upper`,
|
||||
- `holidays`, `holidays_lower`, `holidays_upper`,
|
||||
- and a number of columns for each holiday if `holidays` param is set
|
||||
|
||||
*Config Example*
|
||||
<div class="with-copy" markdown="1">
|
||||
|
||||
```yaml
|
||||
model:
|
||||
class: "model.arima.ArimaModel"
|
||||
# ARIMA's (p,d,q) order
|
||||
order: [1, 1, 0]
|
||||
z_threshold: 2.7
|
||||
resample_freq: '1m'
|
||||
# Inner model args (key-value pairs) accepted by statsmodels.tsa.arima.model.ARIMA
|
||||
class: "model.prophet.ProphetModel"
|
||||
seasonalities:
|
||||
- name: 'hourly'
|
||||
period: 0.04166666666
|
||||
fourier_order: 30
|
||||
# Inner model args (key-value pairs) accepted by
|
||||
# https://facebook.github.io/prophet/docs/quick_start.html#python-api
|
||||
args:
|
||||
trend: 'c'
|
||||
# See https://facebook.github.io/prophet/docs/uncertainty_intervals.html
|
||||
interval_width: 0.98
|
||||
country_holidays: 'US'
|
||||
```
|
||||
|
||||
</div>
|
||||
|
||||
Resulting metrics of the model are described [here](#vmanomaly-output)
|
||||
|
||||
### [Z-score](https://en.wikipedia.org/wiki/Standard_score)
|
||||
*Parameters specific for vmanomaly*:
|
||||
|
||||
* `class` (string) - model class name `"model.zscore.ZscoreModel"`
|
||||
* `z_threshold` (float, optional) - [standard score](https://en.wikipedia.org/wiki/Standard_score) for calculation boundaries and anomaly score. Defaults to `2.5`.
|
||||
|
||||
*Config Example*
|
||||
|
||||
<div class="with-copy" markdown="1">
|
||||
|
||||
```yaml
|
||||
model:
|
||||
class: "model.zscore.ZscoreModel"
|
||||
z_threshold: 2.5
|
||||
```
|
||||
|
||||
</div>
|
||||
|
||||
Resulting metrics of the model are described [here](#vmanomaly-output).
|
||||
|
||||
### [Holt-Winters](https://en.wikipedia.org/wiki/Exponential_smoothing)
|
||||
Here we use Holt-Winters Exponential Smoothing implementation from `statsmodels` [library](https://www.statsmodels.org/dev/generated/statsmodels.tsa.holtwinters.ExponentialSmoothing.html). All parameters from this library can be passed to the model.
|
||||
|
||||
|
|
@ -122,48 +150,27 @@ model:
|
|||
|
||||
Resulting metrics of the model are described [here](#vmanomaly-output).
|
||||
|
||||
### [Prophet](https://facebook.github.io/prophet/)
|
||||
Here we utilize the Facebook Prophet implementation, as detailed in their [library documentation](https://facebook.github.io/prophet/docs/quick_start.html#python-api). All parameters from this library are compatible and can be passed to the model.
|
||||
### [MAD (Median Absolute Deviation)](https://en.wikipedia.org/wiki/Median_absolute_deviation)
|
||||
The MAD model is a robust method for anomaly detection that is *less sensitive* to outliers in data compared to standard deviation-based models. It considers a point as an anomaly if the absolute deviation from the median is significantly large.
|
||||
|
||||
*Parameters specific for vmanomaly*:
|
||||
|
||||
* `class` (string) - model class name `"model.prophet.ProphetModel"`
|
||||
* `seasonalities` (list[dict], optional) - Extra seasonalities to pass to Prophet. See [`add_seasonality()`](https://facebook.github.io/prophet/docs/seasonality,_holiday_effects,_and_regressors.html#modeling-holidays-and-special-events:~:text=modeling%20the%20cycle-,Specifying,-Custom%20Seasonalities) Prophet param.
|
||||
* `provide_series` (dict, optional) - model resulting metrics. If not specified [standard metrics](#vmanomaly-output) will be provided.
|
||||
|
||||
**Note**: Apart from standard vmanomaly output Prophet model can provide [additional metrics](#additional-output-metrics-produced-by-fb-prophet).
|
||||
|
||||
**Additional output metrics produced by FB Prophet**
|
||||
Depending on chosen `seasonality` parameter FB Prophet can return additional metrics such as:
|
||||
- `trend`, `trend_lower`, `trend_upper`
|
||||
- `additive_terms`, `additive_terms_lower`, `additive_terms_upper`,
|
||||
- `multiplicative_terms`, `multiplicative_terms_lower`, `multiplicative_terms_upper`,
|
||||
- `daily`, `daily_lower`, `daily_upper`,
|
||||
- `hourly`, `hourly_lower`, `hourly_upper`,
|
||||
- `holidays`, `holidays_lower`, `holidays_upper`,
|
||||
- and a number of columns for each holiday if `holidays` param is set
|
||||
* `class` (string) - model class name `"model.mad.MADModel"`
|
||||
* `threshold` (float, optional) - The threshold multiplier for the MAD to determine anomalies. Defaults to `2.5`. Higher values will identify fewer points as anomalies.
|
||||
|
||||
*Config Example*
|
||||
|
||||
<div class="with-copy" markdown="1">
|
||||
|
||||
```yaml
|
||||
model:
|
||||
class: "model.prophet.ProphetModel"
|
||||
seasonalities:
|
||||
- name: 'hourly'
|
||||
period: 0.04166666666
|
||||
fourier_order: 30
|
||||
# Inner model args (key-value pairs) accepted by
|
||||
# https://facebook.github.io/prophet/docs/quick_start.html#python-api
|
||||
args:
|
||||
# See https://facebook.github.io/prophet/docs/uncertainty_intervals.html
|
||||
interval_width: 0.98
|
||||
country_holidays: 'US'
|
||||
class: "model.mad.MADModel"
|
||||
threshold: 2.5
|
||||
```
|
||||
|
||||
</div>
|
||||
|
||||
Resulting metrics of the model are described [here](#vmanomaly-output)
|
||||
Resulting metrics of the model are described [here](#vmanomaly-output).
|
||||
|
||||
### [Rolling Quantile](https://en.wikipedia.org/wiki/Quantile)
|
||||
|
||||
|
|
@ -216,48 +223,42 @@ Resulting metrics of the model are described [here](#vmanomaly-output).
|
|||
* `trend` - The trend component of the data series.
|
||||
* `seasonal` - The seasonal component of the data series.
|
||||
|
||||
### [MAD (Median Absolute Deviation)](https://en.wikipedia.org/wiki/Median_absolute_deviation)
|
||||
The MAD model is a robust method for anomaly detection that is *less sensitive* to outliers in data compared to standard deviation-based models. It considers a point as an anomaly if the absolute deviation from the median is significantly large.
|
||||
### [ARIMA](https://en.wikipedia.org/wiki/Autoregressive_integrated_moving_average)
|
||||
Here we use ARIMA implementation from `statsmodels` [library](https://www.statsmodels.org/dev/generated/statsmodels.tsa.arima.model.ARIMA.html)
|
||||
|
||||
*Parameters specific for vmanomaly*:
|
||||
|
||||
* `class` (string) - model class name `"model.mad.MADModel"`
|
||||
* `threshold` (float, optional) - The threshold multiplier for the MAD to determine anomalies. Defaults to `2.5`. Higher values will identify fewer points as anomalies.
|
||||
* `class` (string) - model class name `"model.arima.ArimaModel"`
|
||||
|
||||
* `z_threshold` (float, optional) - [standard score](https://en.wikipedia.org/wiki/Standard_score) for calculating boundaries to define anomaly score. Defaults to `2.5`.
|
||||
|
||||
* `provide_series` (list[string], optional) - List of columns to be produced and returned by the model. Defaults to `["anomaly_score", "yhat", "yhat_lower" "yhat_upper", "y"]`. Output can be **only a subset** of a given column list.
|
||||
|
||||
* `resample_freq` (string, optional) - Frequency to resample input data into, e.g. data comes at 15 seconds resolution, and resample_freq is '1m'. Then fitting data will be downsampled to '1m' and internal model is trained at '1m' intervals. So, during inference, prediction data would be produced at '1m' intervals, but interpolated to "15s" to match with expected output, as output data must have the same timestamps.
|
||||
|
||||
*Default model parameters*:
|
||||
|
||||
* `order` (list[int]) - ARIMA's (p,d,q) order of the model for the autoregressive, differences, and moving average components, respectively.
|
||||
|
||||
* `args` (dict, optional) - Inner model args (key-value pairs). See accepted params in [model documentation](https://www.statsmodels.org/dev/generated/statsmodels.tsa.arima.model.ARIMA.html). Defaults to empty (not provided). Example: {"trend": "c"}
|
||||
|
||||
*Config Example*
|
||||
|
||||
<div class="with-copy" markdown="1">
|
||||
|
||||
```yaml
|
||||
model:
|
||||
class: "model.mad.MADModel"
|
||||
threshold: 2.5
|
||||
class: "model.arima.ArimaModel"
|
||||
# ARIMA's (p,d,q) order
|
||||
order: [1, 1, 0]
|
||||
z_threshold: 2.7
|
||||
resample_freq: '1m'
|
||||
# Inner model args (key-value pairs) accepted by statsmodels.tsa.arima.model.ARIMA
|
||||
args:
|
||||
trend: 'c'
|
||||
```
|
||||
|
||||
</div>
|
||||
|
||||
Resulting metrics of the model are described [here](#vmanomaly-output).
|
||||
|
||||
### [Z-score](https://en.wikipedia.org/wiki/Standard_score)
|
||||
*Parameters specific for vmanomaly*:
|
||||
|
||||
* `class` (string) - model class name `"model.zscore.ZscoreModel"`
|
||||
* `z_threshold` (float, optional) - [standard score](https://en.wikipedia.org/wiki/Standard_score) for calculation boundaries and anomaly score. Defaults to `2.5`.
|
||||
|
||||
*Config Example*
|
||||
|
||||
<div class="with-copy" markdown="1">
|
||||
|
||||
```yaml
|
||||
model:
|
||||
class: "model.zscore.ZscoreModel"
|
||||
z_threshold: 2.5
|
||||
```
|
||||
|
||||
</div>
|
||||
|
||||
Resulting metrics of the model are described [here](#vmanomaly-output).
|
||||
|
||||
### [Isolation forest](https://en.wikipedia.org/wiki/Isolation_forest) (Multivariate)
|
||||
Detects anomalies using binary trees. The algorithm has a linear time complexity and a low memory requirement, which works well with high-volume data. It can be used on both univatiate and multivariate data, but it is more effective in multivariate case.
|
||||
|
||||
|
|
@ -317,7 +318,7 @@ The default metrics produced by vmanomaly include:
|
|||
|
||||
## Healthcheck metrics
|
||||
|
||||
Each model exposes [several healthchecks metrics](./../monitoring.html#models-behaviour-metrics) to its `health_path` endpoint:
|
||||
Each model exposes [several healthchecks metrics](/anomaly-detection/components/monitoring.html#models-behaviour-metrics) to its `health_path` endpoint:
|
||||
|
||||
|
||||
## Custom Model Guide
|
||||
|
|
|
|||
|
|
@ -17,4 +17,4 @@ This section holds guides of how to set up and use VictoriaMetrics Anomaly Detec
|
|||
|
||||
Guides:
|
||||
|
||||
* [Getting started with vmanomaly and vmalert](https://docs.victoriametrics.com/anomaly-detection/guides/guide-vmanomaly-vmalert.html)
|
||||
* [vmanomaly integration](/anomaly-detection/guides/guide-vmanomaly-vmalert.html)
|
||||
|
|
@ -1,7 +1,7 @@
|
|||
---
|
||||
weight: 1
|
||||
sort: 1
|
||||
title: Getting started with vmanomaly
|
||||
title: vmanomaly integration
|
||||
menu:
|
||||
docs:
|
||||
parent: "anomaly-detection-guides"
|
||||
|
|
@ -9,9 +9,10 @@ menu:
|
|||
aliases:
|
||||
- /anomaly-detection/guides/guide-vmanomaly-vmalert.html
|
||||
- /guides/guide-vmanomaly-vmalert.html
|
||||
- /gides/vmanomaly-integration.html
|
||||
|
||||
---
|
||||
# Getting started with vmanomaly
|
||||
# vmanomaly integration
|
||||
|
||||
**Prerequisites**:
|
||||
|
||||
|
|
@ -26,7 +27,7 @@ aliases:
|
|||
|
||||
<img width="800" alt="vmanomaly typical setup diagramm" src="guide-vmanomaly-vmalert_overview.webp">
|
||||
|
||||
Example of configuration can be found [here](https://github.com/VictoriaMetrics/VictoriaMetrics/tree/master/deployment/docker/vmanomaly/vmanomaly-vmalert-guide/)
|
||||
> **Note: Configurations used throughout this guide can be found [here](https://github.com/VictoriaMetrics/VictoriaMetrics/tree/master/deployment/docker/vmanomaly/vmanomaly-integration/)**
|
||||
|
||||
## 1. What is vmanomaly?
|
||||
|
||||
|
|
@ -69,8 +70,8 @@ Practical use case is to put anomaly score generated by vmanomaly into alerting
|
|||
- Explore configuration files for [vmanomaly](https://docs.victoriametrics.com/vmanomaly.html) and [vmalert](https://docs.victoriametrics.com/vmalert.html).
|
||||
- Run our own [VictoriaMetrics](https://docs.victoriametrics.com/Single-server-VictoriaMetrics.html) database with data scraped from [Node Exporter](https://prometheus.io/docs/guides/node-exporter/).
|
||||
- Explore data for analysis in [Grafana](https://grafana.com/).
|
||||
- Explore vmanomaly results.
|
||||
- Explore vmalert alerts
|
||||
- Explore `vmanomaly` results.
|
||||
- Explore `vmalert` alerts
|
||||
|
||||
|
||||
## 4. Data to analyze
|
||||
|
|
@ -110,13 +111,15 @@ This query result will generate 8 time series per each cpu, and we will use them
|
|||
**Parameter description**:
|
||||
There are 4 required sections in config file:
|
||||
|
||||
`scheduler` - defines how often to run and make inferences, as well as what timerange to use to train the model.
|
||||
[`scheduler`](/anomaly-detection/components/scheduler.html) - defines how often to run and make inferences, as well as what timerange to use to train the model.
|
||||
|
||||
`model` - specific model parameters and configurations,
|
||||
[`model`](/anomaly-detection/components/models.html) - specific model parameters and configurations,
|
||||
|
||||
`reader` - how to read data and where it is located
|
||||
[`reader`](/anomaly-detection/components/reader.html) - how to read data and where it is located
|
||||
|
||||
`writer` - where and how to write the generated output.
|
||||
[`writer`](/anomaly-detection/components/writer.html) - where and how to write generated output.
|
||||
|
||||
[`monitoring`](/anomaly-detection/components/monitoring.html) (optional) - how to expose healthckeck metrics of `vmanomaly`.
|
||||
|
||||
Let's look into parameters in each section:
|
||||
|
||||
|
|
@ -125,7 +128,7 @@ Let's look into parameters in each section:
|
|||
* `fit_every` - how often to retrain the models. The higher the frequency -- the fresher the model, but the more CPU it consumes. If omitted, the models will be retrained on each infer_every cycle. Format examples: 30s, 4m, 2h, 1d. Time granularity ('s' - seconds, 'm' - minutes, 'h' - hours, 'd' - days).
|
||||
* `fit_window` - what data interval to use for model training. Longer intervals capture longer historical behavior and detect seasonalities better, but is slower to adapt to permanent changes to metrics behavior. Recommended value is at least two full seasons. Format examples: 30s, 4m, 2h, 1d. Time granularity ('s' - seconds, 'm' - minutes, 'h' - hours, 'd' - days). Here is the previous 14 days of data to put into the model training.
|
||||
* `model`
|
||||
* `class` - what model to run. You can use your own model or choose from built-in models: Seasonal Trend Decomposition, Facebook Prophet, ZScore, Rolling Quantile, Holt-Winters, Isolation Forest and ARIMA. Here we use Facebook Prophet (`model.prophet.ProphetModel`).
|
||||
* `class` - what model to run. You can [use your own model](/anomaly-detection/components/models.html#custom-model-guide) or choose from [built-in models](/anomaly-detection/components/models.html#built-in-models). Here we use [Facebook Prophet](/anomaly-detection/components/models.html#prophet) (`model.prophet.ProphetModel`).
|
||||
* `args` - Model specific parameters, represented as YAML dictionary in a simple `key: value` form. For example, you can use parameters that are available in [FB Prophet](https://facebook.github.io/prophet/docs/quick_start.html).
|
||||
* `reader`
|
||||
* `datasource_url` - Data source. An HTTP endpoint that serves `/api/v1/query_range`.
|
||||
|
|
@ -489,4 +492,16 @@ According to the rule configured for vmalert we will see Alert when anomaly scor
|
|||
|
||||
## 10. Conclusion
|
||||
|
||||
Now we know how to set up Victoria Metric Anomaly Detection tool and use it together with vmalert. We also discovered core vmanomaly generated metrics and behaviour.
|
||||
We've explored the integration and practical application of *VictoriaMetrics Anomaly Detection* (`vmanomaly`) in conjunction with `vmalert`. This tutorial has taken you through the necessary prerequisites, setup, and configurations required for anomaly detection in time series data.
|
||||
|
||||
Key takeaways include:
|
||||
|
||||
1. **Understanding vmanomaly and vmalert**: We've discussed the functionalities of `vmanomaly` and `vmalert`, highlighting how they work individually and in tandem to detect anomalies in time series data.
|
||||
|
||||
2. **Practical Configuration and Setup**: By walking through the setup of a docker-compose environment, we've demonstrated how to configure and run VictoriaMetrics along with its associated services, including `vmanomaly` and `vmalert`.
|
||||
|
||||
3. **Data Analysis and Monitoring**: The guide provided insights on how to collect, analyze, and visualize data using Grafana, interpreting the anomaly scores and other metrics generated by `vmanomaly`.
|
||||
|
||||
4. **Alert Configuration**: We've shown how to set up and customize alerting rules in `vmalert` based on produced anomaly scores, enabling proactive monitoring and timely response to potential issues.
|
||||
|
||||
As you continue to use VictoriaMetrics Anomaly Detection and `vmalert`, remember that the effectiveness of anomaly detection largely depends on the appropriateness of the model chosen, the accuracy of configurations and the data patterns observed. This guide serves as a starting point, and we encourage you to experiment with different configurations and models to best suit your specific data needs and use cases. In case you need a helping hand - [contact us](https://victoriametrics.com/contact-us/).
|
||||
|
|
|
|||
|
|
@ -50,7 +50,7 @@ processes in parallel, each using its own config.
|
|||
## Models
|
||||
|
||||
Currently, vmanomaly ships with a set of built-in models:
|
||||
> For a detailed description, see [model section](/anomaly-detection/components/models.html)
|
||||
> For a detailed overview, see [model section](/anomaly-detection/components/models.html)
|
||||
|
||||
1. [**ZScore**](/anomaly-detection/components/models.html#z-score)
|
||||
|
||||
|
|
@ -62,7 +62,7 @@ Currently, vmanomaly ships with a set of built-in models:
|
|||
|
||||
1. [**Prophet**](/anomaly-detection/components/models.html#prophet)
|
||||
|
||||
_(simplest in configuration, recommended for getting starting)_
|
||||
_(simplest in configuration, recommended for getting started)_
|
||||
|
||||
Uses Facebook Prophet for forecasting. The _anomaly score_ is computed of how close the actual time
|
||||
series values follow the forecasted values (_yhat_), and whether it’s within forecasted bounds
|
||||
|
|
@ -216,15 +216,17 @@ This will expose metrics at `http://0.0.0.0:8080/metrics` page.
|
|||
To use *vmanomaly* you need to pull docker image:
|
||||
|
||||
```sh
|
||||
docker pull us-docker.pkg.dev/victoriametrics-test/public/vmanomaly-trial:1.7.2
|
||||
docker pull victoriametrics/vmanomaly:1.7.2
|
||||
```
|
||||
|
||||
> Note: please check what is latest release in [CHANGELOG](/anomaly-detection/CHANGELOG.html)
|
||||
|
||||
> Note: `us-docker.pkg.dev/victoriametrics-test/public/vmanomaly-trial` is deprecated since [v1.6.0](/anomaly-detection/CHANGELOG.html#v160). Use [DockerHub repo](https://hub.docker.com/r/victoriametrics/vmanomaly/tags) instead
|
||||
|
||||
You can put a tag on it for your convinience:
|
||||
|
||||
```sh
|
||||
docker image tag us-docker.pkg.dev/victoriametrics-test/public/vmanomaly-trial vmanomaly
|
||||
docker image tag victoriametrics/vmanomaly:1.7.2 vmanomaly
|
||||
```
|
||||
Here is an example of how to run *vmanomaly* docker container with [license file](#licensing):
|
||||
|
||||
|
|
|
|||
Loading…
Reference in a new issue