mirror of
https://github.com/VictoriaMetrics/VictoriaMetrics.git
synced 2024-11-21 14:44:00 +00:00
docs/vmanomaly: release v1.15.0 (#6768)
### Describe Your Changes
- updated docs on `vmanomaly` with v1.15.0
- additional chapters of FAQ and model pages
### Checklist
The following checks are **mandatory**:
- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
(cherry picked from commit b670fa1a29)
This commit is contained in:
Fred Navruzov
hagen1778
parent
79008b712f
commit
0becae4ad4
3 changed files with 268 additions and 15 deletions
|
|
@ -13,6 +13,19 @@ aliases:
|
|||
Please find the changelog for VictoriaMetrics Anomaly Detection below.
|
||||
|
||||
> **Important note: Users are strongly encouraged to upgrade to `vmanomaly` [v1.9.2](https://hub.docker.com/repository/docker/victoriametrics/vmanomaly/tags?page=1&ordering=name) or newer for optimal performance and accuracy. <br><br> This recommendation is crucial for configurations with a low `infer_every` parameter [in your scheduler](./components/scheduler.md#parameters-1), and in scenarios where data exhibits significant high-order seasonality patterns (such as hourly or daily cycles). Previous versions from v1.5.1 to v1.8.0 were identified to contain a critical issue impacting model training, where models were inadvertently trained on limited data subsets, leading to suboptimal fits, affecting the accuracy of anomaly detection. <br><br> Upgrading to v1.9.2 addresses this issue, ensuring proper model training and enhanced reliability. For users utilizing Helm charts, it is recommended to upgrade to version [1.0.0](https://github.com/VictoriaMetrics/helm-charts/blob/master/charts/victoria-metrics-anomaly/CHANGELOG.md#100) or newer.**
|
||||
|
||||
## v1.15.0
|
||||
Released: 2024-08-06
|
||||
- FEATURE: Introduced models that support [online learning](https://en.wikipedia.org/wiki/Online_machine_learning) for stream-like input. These models significantly reduce the amount of data required for the initial fit stage. For example, they enable reducing `fit_every` from **weeks to hours** and increasing `fit_every` from **hours to weeks** in the [PeriodicScheduler](https://docs.victoriametrics.com/anomaly-detection/components/scheduler/#periodic-scheduler), significantly reducing the **peak amount** of data queried from VictoriaMetrics during `fit` stages. The next models were added:
|
||||
- [`OnlineZscoreModel`](https://docs.victoriametrics.com/anomaly-detection/components/models/#online-z-score) - online version of existing [Z-score](https://docs.victoriametrics.com/anomaly-detection/components/models/#z-score) implementation with the same exact behavior.
|
||||
- [`OnlineMADModel`](https://docs.victoriametrics.com/anomaly-detection/components/models/#online-mad) - online version of existing [MADModel](https://docs.victoriametrics.com/anomaly-detection/components/models/#mad-median-absolute-deviation) implementation with *approximate* behavior, based on [t-digests](https://www.sciencedirect.com/science/article/pii/S2665963820300403) for online quantile estimation.
|
||||
- [`OnlineQuantileModel`](https://docs.victoriametrics.com/anomaly-detection/components/models/#online-seasonal-quantile) - online quantile model, that supports custom ranges for seasonality estimation to cover more complex data patterns.
|
||||
- Find out more about online models specifics in [correspondent section](https://docs.victoriametrics.com/anomaly-detection/components/models/#online-models).
|
||||
|
||||
- FEATURE: Introduced the `optimized_business_params` key (list of strings) to the [`AutoTuned`](https://docs.victoriametrics.com/anomaly-detection/components/models/#autotuned) `optimization_params`. This allows particular business-specific parameters such as [`detection_direction`](https://docs.victoriametrics.com/anomaly-detection/components/models/#detection-direction) and [`min_dev_from_expected`](https://docs.victoriametrics.com/anomaly-detection/components/models/#minimal-deviation-from-expected) to remain **unchanged during optimizations, retaining their default values**.
|
||||
- IMPROVEMENT: Optimized the [`AutoTuned`](https://docs.victoriametrics.com/anomaly-detection/components/models/#autotuned) model logic to minimize deviations from the expected `anomaly_percentage` specified in the configuration and the detected percentage in the data, while also reducing discrepancies between the actual values (`y`) and the predictions (`yhat`).
|
||||
- IMPROVEMENT: Allow [`ProphetModel`](https://docs.victoriametrics.com/anomaly-detection/components/models/#prophet) to fit with multiple seasonalities when used in [`AutoTuned`](https://docs.victoriametrics.com/anomaly-detection/components/models/#autotuned) mode.
|
||||
|
||||
## v1.14.2
|
||||
Released: 2024-07-26
|
||||
- FIX: Patch a bug introduced in [v1.14.1](#v1141), causing `vmanomaly` to crash in `preset` [mode](/anomaly-detection/presets/).
|
||||
|
|
|
|||
|
|
@ -34,6 +34,8 @@ Please see example graph illustrating this logic below:
|
|||
|
||||
|
||||
|
||||
> p.s. please note that additional post-processing logic might be applied to produced anomaly scores, if common arguments like [`min_dev_from_expected`](https://docs.victoriametrics.com/anomaly-detection/components/models/#minimal-deviation-from-expected) or [`detection_direction`](https://docs.victoriametrics.com/anomaly-detection/components/models/#detection-direction) are enabled for a particular model. Follow the links above for the explanations.
|
||||
|
||||
|
||||
## How does vmanomaly work?
|
||||
`vmanomaly` applies built-in (or custom) [anomaly detection algorithms](/anomaly-detection/components/models), specified in a config file.
|
||||
|
|
@ -150,19 +152,85 @@ volumes:
|
|||
vmanomaly_model_dump_dir: {}
|
||||
```
|
||||
|
||||
> **Note**: Starting from [v1.15.0](https://docs.victoriametrics.com/anomaly-detection/changelog#v1150) with the introduction of [online models](https://docs.victoriametrics.com/anomaly-detection/components/models/#online-models), you can additionally reduce resource consumption (e.g., flatten `fit` stage peaks by querying less data from VictoriaMetrics at once).
|
||||
|
||||
**Additional Benefits of Switching to Online Models**:
|
||||
|
||||
- **Reduced Latency**: Online models update incrementally, which can lead to faster response times for anomaly detection since the model continuously adapts to new data without waiting for a batch `fit`.
|
||||
- **Scalability**: Handling smaller data chunks at a time reduces memory and computational overhead, making it easier to scale the anomaly detection system.
|
||||
- **Improved Resource Utilization**: By spreading the computational load over time and reducing peak demands, online models make more efficient use of system resources, potentially lowering operational costs.
|
||||
|
||||
Here's an example of how we can switch from (offline) [Z-score model](https://docs.victoriametrics.com/anomaly-detection/components/models/#z-score) to [Online Z-score model](https://docs.victoriametrics.com/anomaly-detection/components/models/#online-z-score):
|
||||
|
||||
```yaml
|
||||
schedulers:
|
||||
periodic:
|
||||
class: 'periodic'
|
||||
fit_every: '1h'
|
||||
fit_window: '2d'
|
||||
infer_every: '1m'
|
||||
# other schedulers ...
|
||||
models:
|
||||
zscore_example:
|
||||
class: 'zscore'
|
||||
schedulers: ['periodic']
|
||||
# other model params ...
|
||||
# other config sections ...
|
||||
```
|
||||
|
||||
to something like
|
||||
|
||||
```yaml
|
||||
schedulers:
|
||||
periodic:
|
||||
class: 'periodic'
|
||||
fit_every: '180d' # we need only initial fit to start
|
||||
fit_window: '4h' # reduced window, especially if the data doesn't have strong seasonality
|
||||
infer_every: '1m' # the model will be updated during each infer call
|
||||
# other schedulers ...
|
||||
models:
|
||||
zscore_example:
|
||||
class: 'zscore_online'
|
||||
min_n_samples_seen: 120 # i.e. minimal relevant seasonality or (initial) fit_window / sampling_frequency
|
||||
schedulers: ['periodic']
|
||||
# other model params ...
|
||||
# other config sections ...
|
||||
```
|
||||
|
||||
As a result, switching from the offline Z-score model to the Online Z-score model results in significant data volume reduction, i.e. over one week:
|
||||
|
||||
**Old Configuration**:
|
||||
- `fit_window`: 2 days
|
||||
- `fit_every`: 1 hour
|
||||
|
||||
**New Configuration**:
|
||||
- `fit_window`: 4 hours
|
||||
- `fit_every`: 180 days ( >1 week)
|
||||
|
||||
The old configuration would perform 168 (hours in a week) `fit` calls, each using 2 days (48 hours) of data, totaling 168 * 48 = 8064 hours of data for each timeseries returned.
|
||||
|
||||
The new configuration performs only 1 `fit` call in 180 days, using 4 hours of data initially, totaling 4 hours of data, which is **magnitutes smaller**.
|
||||
|
||||
P.s. `infer` data volume will remain the same for both models, so it does not affect the overall calculations.
|
||||
|
||||
**Data Volume Reduction**:
|
||||
- Old: 8064 hours/week (fit) + 168 hours/week (infer)
|
||||
- New: 4 hours/week (fit) + 168 hours/week (infer)
|
||||
|
||||
## Scaling vmanomaly
|
||||
> **Note:** As of latest release we don't support cluster or auto-scaled version yet (though, it's in our roadmap for - better backends, more parallelization, etc.), so proposed workarounds should be addressed manually.
|
||||
> **Note:** As of latest release we do not support cluster or auto-scaled version yet (though, it's in our roadmap for - better backends, more parallelization, etc.), so proposed workarounds should be addressed *manually*.
|
||||
|
||||
`vmanomaly` can be scaled horizontally by launching multiple independent instances, each with its own [MetricsQL](/MetricsQL) queries and [configurations](/anomaly-detection/components/):
|
||||
|
||||
- By splitting **queries**, [defined in reader section](/anomaly-detection/components/reader#vm-reader) and spawn separate service around it. Also in case you have *only 1 query returning huge amount of timeseries*, you can further split it by applying MetricsQL filters, i.e. using "extra_filters" [param in reader](/anomaly-detection/components/reader?highlight=extra_filters#vm-reader)
|
||||
- By splitting **queries**, [defined in reader section](/anomaly-detection/components/reader#vm-reader) and spawn separate service around it. Also in case you have *only 1 query returning huge amount of timeseries*, you can further split it by applying MetricsQL filters, i.e. using "extra_filters" [param in reader](/anomaly-detection/components/reader?highlight=extra_filters#vm-reader). See the example below.
|
||||
|
||||
- or **models** (in case you decide to run several models for each timeseries received i.e. for averaging anomaly scores in your alerting rules of `vmalert` or using a vote approach to reduce false positives) - see `queries` arg in [model config](/anomaly-detection/components/models#queries)
|
||||
|
||||
- or **schedulers** (in case you want the same models to be trained under several schedules) - see `schedulers` arg [model section](/anomaly-detection/components/models#schedulers) and `scheduler` [component itself](/anomaly-detection/components/scheduler)
|
||||
|
||||
|
||||
Here's an example of how to split on `extra_filters`, based on `extra_filters` reader's arg
|
||||
Here's an example of how to split on `extra_filters`, based on `extra_filters` reader's arg:
|
||||
|
||||
```yaml
|
||||
# config file #1, for 1st vmanomaly instance
|
||||
# ...
|
||||
|
|
|
|||
|
|
@ -232,10 +232,12 @@ There are **2 model types**, supported in `vmanomaly`, resulting in **4 possible
|
|||
- [Univariate models](#univariate-models)
|
||||
- [Multivariate models](#multivariate-models)
|
||||
|
||||
Each of these models can also be
|
||||
Each of these models can be of type
|
||||
- [Rolling](#rolling-models)
|
||||
- [Non-rolling](#non-rolling-models)
|
||||
|
||||
Moreover, starting from [v1.15.0](https://docs.victoriametrics.com/anomaly-detection/changelog/#v1150), there exist **[online (incremental) models](#online-models)** subclass. Please refer to the [correspondent section](#online-models) for more details.
|
||||
|
||||
### Univariate Models
|
||||
|
||||
For a univariate type, **one separate model** is fit/used for inference per **each time series**, defined in its [queries](#queries) arg.
|
||||
|
|
@ -276,7 +278,7 @@ As a result, such model instances are **not stored** between consecutive re-fit
|
|||
|
||||
Such models put **more pressure** on your reader's source, i.e. if your model should be fit on large amount of data (say, 14 days with 1-minute resolution) and at the same time you have **frequent inference** (say, once per minute) on new chunks of data - that's because such models require (fit + infer) window of data to be fit first to be used later in each inference call.
|
||||
|
||||
> **Note**: Rolling models require `fit_every` to be set equal to `infer_every` in your [PeriodicScheduler](./scheduler.md#periodic-scheduler).
|
||||
> **Note**: Rolling models require `fit_every` either to be missing or explicitly set equal to `infer_every` in your [PeriodicScheduler](./scheduler.md#periodic-scheduler).
|
||||
|
||||
**Examples:** [RollingQuantile](#rolling-quantile)
|
||||
|
||||
|
|
@ -300,24 +302,65 @@ Produced model instances are **stored in-memory** between consecutive re-fit cal
|
|||
|
||||
|
||||
|
||||
### Online Models
|
||||
|
||||
Introduced in [v1.15.0](https://docs.victoriametrics.com/anomaly-detection/changelog/#v1150), online (incremental) models allow defining a smaller frame `fit_window` and less frequent `fit` calls to reduce the data burden from VictoriaMetrics. They make incremental updates to model parameters during each `infer_every` call, even on a single datapoint.
|
||||
If the model doesn't support online mode, it's called **offline** (its parameters are only updated during `fit` calls).
|
||||
|
||||
Main differences between offline and online:
|
||||
|
||||
Fit stage
|
||||
- Both types have a `fit` stage, run on the `fit_window` data frame.
|
||||
- For offline models, `fit_window` should contain enough data to train the model (e.g., 2 seasonal periods).
|
||||
- For online models, training can start gradually from smaller chunks (e.g., 1 hour).
|
||||
|
||||
Infer stage
|
||||
- Both types have an `infer` stage, run on new datapoints (timestamps > last seen timestamp of the previous `infer` call).
|
||||
- Offline models use a pre-trained (during `fit` call) *static* model to make every `infer` call until the next `fit` call, when the model is completely re-trained.
|
||||
- Online models use a pre-trained (during `fit` call) *dynamic* model, which is gradually updated during each `infer` call with new datapoints. However, to prevent the model from accumulating outdated behavior, each `fit` call resets the model from scratch.
|
||||
|
||||
**Strengths**:
|
||||
|
||||
- The ability to distribute the data load evenly between the initial `fit` and subsequent `infer` calls. For example, an online model can be fit on 10 `1m` datapoints during the initial `fit` stage once per month and then be gradually updated on the same 10 `1m` datapoints during each `infer` call each 10 minutes.
|
||||
- The model can adapt to new data patterns (gradually updating itself during each `infer` call) without needing to wait for the next `fit` call and one big re-training.
|
||||
- Slightly faster training/updating times compared to similar offline models.
|
||||
|
||||
**Limitations**:
|
||||
|
||||
- Until the online model sees enough data (especially if the data shows strong seasonality), its predictions might be unstable, producing more [false positives](https://victoriametrics.com/blog/victoriametrics-anomaly-detection-handbook-chapter-1/#false-positive) (`anomaly_score > 1`) or making [false negative predictions, skipping real anomalies](https://victoriametrics.com/blog/victoriametrics-anomaly-detection-handbook-chapter-1/#false-negative).
|
||||
- Not all models (e.g., complex ones like [Prophet](#prophet)) have a direct online alternative, thus their applicability can be somewhat limited.
|
||||
|
||||
Each of the ([built-in](#built-in-models) or [custom](#custom-model-guide)) online models (like [`OnlineZscoreModel`](#online-z-score)) shares the following common parameters and properties:
|
||||
- `n_samples_seen_` (int) - this model *property* refers to the number of datapoints the model was trained on and increases from 0 (before the first `fit`) with each consecutive `infer` call.
|
||||
- `min_n_samples_seen` (int), optional - this *parameter* defines the minimum number of samples to be seen before reliably computing the [anomaly score](https://docs.victoriametrics.com/anomaly-detection/faq/#what-is-anomaly-score). Otherwise, the anomaly score will be 0 until `n_samples_seen_` > `min_n_samples_seen`, as there is not enough data to trust the model's predictions. For example, if your data has hourly seasonality and '1m' frequency, setting `min_n_samples_seen_` to 288 (1440 minutes in a day / 5 minutes) should be sufficient.
|
||||
|
||||
|
||||
### Offline models
|
||||
|
||||
Every other model that isn't [online](#online-models). Offline models are completely re-trained during `fit` call and isn't updated during consecutive `infer` calls.
|
||||
|
||||
|
||||
## Built-in Models
|
||||
|
||||
### Overview
|
||||
VM Anomaly Detection (`vmanomaly` hereinafter) models support 2 groups of parameters:
|
||||
VictoriaMetrics Anomaly Detection models support 2 groups of parameters:
|
||||
|
||||
- **`vmanomaly`-specific** arguments - please refer to *Parameters specific for vmanomaly* and *Default model parameters* subsections for each of the models below.
|
||||
- Arguments to **inner model** (say, [Facebook's Prophet](https://facebook.github.io/prophet/docs/quick_start.html#python-api)), passed in a `args` argument as key-value pairs, that will be directly given to the model during initialization to allow granular control. Optional.
|
||||
|
||||
**Note**: For users who may not be familiar with Python data types such as `list[dict]`, a [dictionary](https://www.w3schools.com/python/python_dictionaries.asp) in Python is a data structure that stores data values in key-value pairs. This structure allows for efficient data retrieval and management.
|
||||
> **Note**: For users who may not be familiar with Python data types such as `list[dict]`, a [dictionary](https://www.w3schools.com/python/python_dictionaries.asp) in Python is a data structure that stores data values in key-value pairs. This structure allows for efficient data retrieval and management.
|
||||
|
||||
|
||||
**Models**:
|
||||
* [AutoTuned](#autotuned) - designed to take the cognitive load off the user, allowing any of built-in models below to be re-tuned for best params on data seen during each `fit` phase of the algorithm. Tradeoff is between increased computational time and optimized results / simpler maintenance.
|
||||
* [Prophet](#prophet) - the most versatile one for production usage, especially for complex data ([trends](https://victoriametrics.com/blog/victoriametrics-anomaly-detection-handbook-chapter-1/#trend), [change points](https://victoriametrics.com/blog/victoriametrics-anomaly-detection-handbook-chapter-2/#novelties), [multi-seasonality](https://victoriametrics.com/blog/victoriametrics-anomaly-detection-handbook-chapter-1/#seasonality))
|
||||
* [Z-score](#z-score) - useful for testing and for simpler data ([de-trended](https://victoriametrics.com/blog/victoriametrics-anomaly-detection-handbook-chapter-1/#trend) data without strict [seasonality](https://victoriametrics.com/blog/victoriametrics-anomaly-detection-handbook-chapter-1/#seasonality) and with anomalies of similar magnitude as your "normal" data)
|
||||
* [Holt-Winters](#holt-winters) - well-suited for **data with moderate complexity**, exhibiting distinct [trends](https://victoriametrics.com/blog/victoriametrics-anomaly-detection-handbook-chapter-1/#trend) and/or [seasonal patterns](https://victoriametrics.com/blog/victoriametrics-anomaly-detection-handbook-chapter-1/#seasonality).
|
||||
* [MAD (Median Absolute Deviation)](#mad-median-absolute-deviation) - similarly to Z-score, is effective for **identifying outliers in relatively consistent data** (useful for detecting sudden, stark deviations from the median)
|
||||
* [Z-score](#z-score) - useful for initial testing and for simpler data ([de-trended](https://victoriametrics.com/blog/victoriametrics-anomaly-detection-handbook-chapter-1/#trend) data without strict [seasonality](https://victoriametrics.com/blog/victoriametrics-anomaly-detection-handbook-chapter-1/#seasonality) and with anomalies of similar magnitude as your "normal" data)
|
||||
* [Online Z-score](#online-z-score) - [online](#online-models) alternative to [Z-score](#z-score) model with exact same behavior and use cases.
|
||||
* [Holt-Winters](#holt-winters) - well-suited for **data with moderate complexity**, exhibiting distinct [trends](https://victoriametrics.com/blog/victoriametrics-anomaly-detection-handbook-chapter-1/#trend) and/or [single seasonal pattern](https://victoriametrics.com/blog/victoriametrics-anomaly-detection-handbook-chapter-1/#seasonality).
|
||||
* [MAD (Median Absolute Deviation)](#mad-median-absolute-deviation) - similarly to [Z-score](#z-score), is effective for **identifying outliers in relatively consistent data** (useful for detecting sudden, stark deviations from the median).
|
||||
* [Online MAD](#online-mad) - approximate [online](#online-models) alternative to [MAD model](#mad-median-absolute-deviation), appropriate for the same use cases.
|
||||
* [Rolling Quantile](#rolling-quantile) - best for **data with evolving patterns**, as it adapts to changes over a rolling window.
|
||||
* [Online Seasonal Quantile](#online-seasonal-quantile) - best used on **[de-trended](https://victoriametrics.com/blog/victoriametrics-anomaly-detection-handbook-chapter-1/#trend) data with strong (possibly multiple) [seasonalities](https://victoriametrics.com/blog/victoriametrics-anomaly-detection-handbook-chapter-1/#seasonality)**. Can act as a (slightly less powerful) [online](#online-models) replacement to [`ProphetModel`](#prophet).
|
||||
* [Seasonal Trend Decomposition](#seasonal-trend-decomposition) - similarly to Holt-Winters, is best for **data with pronounced [seasonal](https://victoriametrics.com/blog/victoriametrics-anomaly-detection-handbook-chapter-1/#seasonality) and [trend](https://victoriametrics.com/blog/victoriametrics-anomaly-detection-handbook-chapter-1/#trend) components**
|
||||
* [Isolation forest (Multivariate)](#isolation-forest-multivariate) - useful for **metrics data interaction** (several queries/metrics -> single anomaly score) and **efficient in detecting anomalies in high-dimensional datasets**
|
||||
* [Custom model](#custom-model-guide) - benefit from your own models and expertise to better support your **unique use case**.
|
||||
|
|
@ -331,7 +374,8 @@ Tuning hyperparameters of a model can be tricky and often requires in-depth know
|
|||
* `class` (string) - model class name `"model.auto.AutoTunedModel"` (or `auto` starting from [v1.13.0](../CHANGELOG.md#1130) with class alias support)
|
||||
* `tuned_class_name` (string) - Built-in model class to tune, i.e. `model.zscore.ZscoreModel` (or `zscore` starting from [v1.13.0](../CHANGELOG.md#1130) with class alias support).
|
||||
* `optimization_params` (dict) - Optimization parameters for unsupervised model tuning. Control % of found anomalies, as well as a tradeoff between time spent and the accuracy. The more `timeout` and `n_trials` are, the better model configuration can be found for `tuned_class_name`, but the longer it takes and vice versa. Set `n_jobs` to `-1` to use all the CPUs available, it makes sense if only you have a big dataset to train on during `fit` calls, otherwise overhead isn't worth it.
|
||||
- `anomaly_percentage` (float) - expected percentage of anomalies that can be seen in training data, from (0, 0.5) interval.
|
||||
- `anomaly_percentage` (float) - Expected percentage of anomalies that can be seen in training data, from (0, 0.5) interval.
|
||||
- `optimized_business_params` (list[string]) - Starting from [v1.15.0](https://docs.victoriametrics.com/anomaly-detection/v1150) this argument allows particular business-specific parameters such as [`detection_direction`](https://docs.victoriametrics.com/anomaly-detection/components/models/#detection-direction) or [`min_dev_from_expected`](https://docs.victoriametrics.com/anomaly-detection/components/models/#minimal-deviation-from-expected) to remain **unchanged during optimizations, retaining their default values**. I.e. setting `optimized_business_params` to `['detection_direction']` will allow to optimize only `detection_direction` business-specific, while `min_dev_from_expected` will retain default value (0.0). By default and if not set, will be set to `[]` (empty list), meaning no business params will be optimized. **A recommended option is to leave it empty** for more stable results and increased convergence (less iterations needed for a good result).
|
||||
- `seed` (int) - Random seed for reproducibility and deterministic nature of underlying optimizations.
|
||||
- `n_splits` (int) - How many folds to create for hyperparameter tuning out of your data. The higher, the longer it takes but the better the results can be. Defaults to 3.
|
||||
- `n_trials` (int) - How many trials to sample from hyperparameter search space. The higher, the longer it takes but the better the results can be. Defaults to 128.
|
||||
|
|
@ -353,6 +397,7 @@ models:
|
|||
n_trials: 128 # how many configurations to sample from search space during optimization
|
||||
timeout: 10 # how many seconds to spend on optimization for each trained model during `fit` phase call
|
||||
n_jobs: 1 # how many jobs in parallel to launch. Consider making it > 1 only if you have fit window containing > 10000 datapoints for each series
|
||||
optimized_business_params: [] # business-specific params to include in optimization, if not set is empty list
|
||||
# ...
|
||||
```
|
||||
|
||||
|
|
@ -365,6 +410,8 @@ models:
|
|||
### [Prophet](https://facebook.github.io/prophet/)
|
||||
Here we utilize the Facebook Prophet implementation, as detailed in their [library documentation](https://facebook.github.io/prophet/docs/quick_start.html#python-api). All parameters from this library are compatible and can be passed to the model.
|
||||
|
||||
> **Note**: `ProphetModel` is [univariate](#univariate-models), [non-rolling](#non-rolling-models), [offline](#offline-models) model.
|
||||
|
||||
*Parameters specific for vmanomaly*:
|
||||
|
||||
* `class` (string) - model class name `"model.prophet.ProphetModel"` (or `prophet` starting from [v1.13.0](../CHANGELOG.md#1130) with class alias support)
|
||||
|
|
@ -405,6 +452,11 @@ models:
|
|||
Resulting metrics of the model are described [here](#vmanomaly-output)
|
||||
|
||||
### [Z-score](https://en.wikipedia.org/wiki/Standard_score)
|
||||
|
||||
> **Note**: `ZScoreModel` is [univariate](#univariate-models), [non-rolling](#non-rolling-models), [offline](#offline-models) model.
|
||||
|
||||
Model is useful for initial testing and for simpler data ([de-trended](https://victoriametrics.com/blog/victoriametrics-anomaly-detection-handbook-chapter-1/#trend) data without strict [seasonality](https://victoriametrics.com/blog/victoriametrics-anomaly-detection-handbook-chapter-1/#seasonality) and with anomalies of similar magnitude as your "normal" data).
|
||||
|
||||
*Parameters specific for vmanomaly*:
|
||||
|
||||
* `class` (string) - model class name `"model.zscore.ZscoreModel"` (or `zscore` starting from [v1.13.0](../CHANGELOG.md#1130) with class alias support)
|
||||
|
|
@ -412,7 +464,6 @@ Resulting metrics of the model are described [here](#vmanomaly-output)
|
|||
|
||||
*Config Example*
|
||||
|
||||
|
||||
```yaml
|
||||
models:
|
||||
your_desired_alias_for_a_model:
|
||||
|
|
@ -420,10 +471,38 @@ models:
|
|||
z_threshold: 3.5
|
||||
```
|
||||
|
||||
Resulting metrics of the model are described [here](#vmanomaly-output).
|
||||
|
||||
### Online Z-score
|
||||
|
||||
> **Note**: `OnlineZScoreModel` is [univariate](#univariate-models), [non-rolling](#non-rolling-models), [online](#online-models) model.
|
||||
|
||||
Online version of existing [Z-score](#z-score) implementation with the same exact behavior and implications. Introduced in [v1.15.0](https://docs.victoriametrics.com/anomaly-detection/changelog/#v1150)
|
||||
|
||||
*Parameters specific for vmanomaly*:
|
||||
|
||||
* `class` (string) - model class name `"model.online.OnlineZscoreModel"` (or `zscore_online` starting from [v1.15.0](../CHANGELOG.md#1130) with class alias support)
|
||||
* `z_threshold` (float, optional) - [standard score](https://en.wikipedia.org/wiki/Standard_score) for calculation boundaries and anomaly score. Defaults to `2.5`.
|
||||
* `min_n_samples_seen` (int, optional) - the minimum number of samples to be seen (`n_samples_seen_` property) before computing the anomaly score. Otherwise, the **anomaly score will be 0**, as there is not enough data to trust the model's predictions. Defaults to 16.
|
||||
|
||||
*Config Example*
|
||||
|
||||
```yaml
|
||||
models:
|
||||
your_desired_alias_for_a_model:
|
||||
class: "zscore_online" # or 'model.online.OnlineZscoreModel'
|
||||
z_threshold: 3.5
|
||||
min_n_samples_seen: 128 # i.e. calculate it as full seasonality / data freq
|
||||
provide_series: ['anomaly_score', 'yhat'] # common arg example
|
||||
```
|
||||
|
||||
Resulting metrics of the model are described [here](#vmanomaly-output).
|
||||
|
||||
|
||||
### [Holt-Winters](https://en.wikipedia.org/wiki/Exponential_smoothing)
|
||||
|
||||
> **Note**: `HoltWinters` is [univariate](#univariate-models), [non-rolling](#non-rolling-models), [offline](#offline-models) model.
|
||||
|
||||
Here we use Holt-Winters Exponential Smoothing implementation from `statsmodels` [library](https://www.statsmodels.org/dev/generated/statsmodels.tsa.holtwinters.ExponentialSmoothing.html). All parameters from this library can be passed to the model.
|
||||
|
||||
*Parameters specific for vmanomaly*:
|
||||
|
|
@ -465,7 +544,11 @@ models:
|
|||
|
||||
Resulting metrics of the model are described [here](#vmanomaly-output).
|
||||
|
||||
|
||||
### [MAD (Median Absolute Deviation)](https://en.wikipedia.org/wiki/Median_absolute_deviation)
|
||||
|
||||
> **Note**: `MADModel` is [univariate](#univariate-models), [non-rolling](#non-rolling-models), [offline](#offline-models) model.
|
||||
|
||||
The MAD model is a robust method for anomaly detection that is *less sensitive* to outliers in data compared to standard deviation-based models. It considers a point as an anomaly if the absolute deviation from the median is significantly large.
|
||||
|
||||
*Parameters specific for vmanomaly*:
|
||||
|
|
@ -483,11 +566,44 @@ models:
|
|||
threshold: 2.5
|
||||
```
|
||||
|
||||
Resulting metrics of the model are described [here](#vmanomaly-output).
|
||||
|
||||
|
||||
### Online MAD
|
||||
|
||||
> **Note**: `OnlineMADModel` is [univariate](#univariate-models), [non-rolling](#non-rolling-models), [online](#online-models) model.
|
||||
|
||||
The MAD model is a robust method for anomaly detection that is *less sensitive* to outliers in data compared to standard deviation-based models. It considers a point as an anomaly if the absolute deviation from the median is significantly large. This is the online approximate version, based on [t-digests](https://www.sciencedirect.com/science/article/pii/S2665963820300403) for online quantile estimation. introduced in [v1.15.0](https://docs.victoriametrics.com/anomaly-detection/changelog/#v1150)
|
||||
|
||||
*Parameters specific for vmanomaly*:
|
||||
|
||||
* `class` (string) - model class name `"model.online.OnlineMADModel"` (or `mad_online` starting from [v1.13.0](../CHANGELOG.md#1130) with class alias support)
|
||||
* `threshold` (float, optional) - The threshold multiplier for the MAD to determine anomalies. Defaults to `2.5`. Higher values will identify fewer points as anomalies.
|
||||
* `min_n_samples_seen` (int, optional) - the minimum number of samples to be seen (`n_samples_seen_` property) before computing the anomaly score. Otherwise, the **anomaly score will be 0**, as there is not enough data to trust the model's predictions. Defaults to 16.
|
||||
* `compression` (int, optional) - the compression parameter for the T-Digest. Higher values mean higher accuracy but higher memory usage. By default 100.
|
||||
|
||||
*Config Example*
|
||||
|
||||
|
||||
```yaml
|
||||
models:
|
||||
your_desired_alias_for_a_model:
|
||||
class: "mad_online" # or 'model.online.OnlineMADModel'
|
||||
threshold: 2.5
|
||||
min_n_samples_seen: 128 # i.e. calculate it as full seasonality / data freq
|
||||
compression: 100 # higher values mean higher accuracy but higher memory usage
|
||||
provide_series: ['anomaly_score', 'yhat'] # common arg example
|
||||
```
|
||||
|
||||
Resulting metrics of the model are described [here](#vmanomaly-output).
|
||||
|
||||
|
||||
### [Rolling Quantile](https://en.wikipedia.org/wiki/Quantile)
|
||||
|
||||
> **Note**: `RollingQuantileModel` is [univariate](#univariate-models), [rolling](#rolling-models), [offline](#offline-models) model.
|
||||
|
||||
This model is best used on **data with short evolving patterns** (i.e. 10-100 datapoints of particular frequency), as it adapts to changes over a rolling window.
|
||||
|
||||
*Parameters specific for vmanomaly*:
|
||||
|
||||
* `class` (string) - model class name `"model.rolling_quantile.RollingQuantileModel"` (or `rolling_quantile` starting from [v1.13.0](../CHANGELOG.md#1130) with class alias support)
|
||||
|
|
@ -506,7 +622,55 @@ models:
|
|||
|
||||
Resulting metrics of the model are described [here](#vmanomaly-output).
|
||||
|
||||
|
||||
### Online Seasonal Quantile
|
||||
|
||||
> **Note**: `OnlineSeasonalQuantile` is [univariate](#univariate-models), [non-rolling](#non-rolling-models), [online](#online-models) model.
|
||||
|
||||
Online (seasonal) quantile utilizes a set of approximate distributions, based on [t-digests](https://www.sciencedirect.com/science/article/pii/S2665963820300403) for online quantile estimation. Introduced in [v1.15.0](https://docs.victoriametrics.com/anomaly-detection/changelog/#v1150).
|
||||
|
||||
Best used on **[de-trended](https://victoriametrics.com/blog/victoriametrics-anomaly-detection-handbook-chapter-1/#trend) data with strong (possibly multiple) [seasonalities](https://victoriametrics.com/blog/victoriametrics-anomaly-detection-handbook-chapter-1/#seasonality)**. Can act as a (slightly less powerful) replacement to [`ProphetModel`](#prophet).
|
||||
|
||||
It uses the `quantiles` triplet to calculate `yhat_lower`, `yhat`, and `yhat_upper` [output](#vmanomaly-output), respectively, for each of the `min_subseasons` sub-intervals contained in `seasonal_interval`. For example, with '4d' + '2h' seasonality patterns (multiple), it will hold and update 24*4 / 2 = 48 consecutive estimates (each 2 hours long).
|
||||
|
||||
*Parameters specific for vmanomaly*:
|
||||
|
||||
* `class` (string) - model class name `"model.online.OnlineSeasonalQuantile"` (or `quantile_online` starting from [v1.13.0](../CHANGELOG.md#1130) with class alias support)
|
||||
* `quantiles` (list[float], optional) - The quantiles to estimate. `yhat_lower`, `yhat`, `yhat_upper` are the quantile order. By default (0.01, 0.5, 0.99).
|
||||
* `seasonal_interval` (string, optional) - the interval for the seasonal adjustment. If not set, the model will equal to a simple online quantile model. By default not set.
|
||||
* `min_subseason` (str, optional) - the minimum interval to estimate quantiles for. By default None. Note that the minimum interval should be a multiple of the seasonal interval, i.e. if seasonal_interval='2h', then min_subseason='15m' is valid, but '37m' is not.
|
||||
* `use_transform` (bool, optional) - whether to apply a log1p(abs(x)) * sign(x) transformation to the data to stabilize internal quantile estimation. By default False.
|
||||
* `global_smoothing` (float, optional) - the smoothing parameter for the global quantiles. i.e. the output is a weighted average of the global and seasonal quantiles (if `seasonal_interval` and `min_subseason` args are set). Should be from `[0, 1]` interval, where 0 means no smoothing and 1 means using only global quantile values.
|
||||
* `scale` (float, optional) - the scaling factor for the `yhat_lower` and `yhat_upper` quantiles. By default 1.0 (no scaling). if > 1, increases the boundaries [`yhat_lower`, `yhat_upper`] for "non-anomalous" points. Should be > 0.
|
||||
* `season_starts_from` (str, optional) - the start date for the seasonal adjustment, as a reference point to start counting the intervals. By default '1970-01-01'.
|
||||
* `min_n_samples_seen` (int, optional) - the minimum number of samples to be seen (`n_samples_seen_` property) before computing the anomaly score. Otherwise, the **anomaly score will be 0**, as there is not enough data to trust the model's predictions. Defaults to 16.
|
||||
* `compression` (int, optional) - the compression parameter for the T-Digest. Higher values mean higher accuracy but higher memory usage. By default 100.
|
||||
|
||||
*Config Example*
|
||||
|
||||
Suppose you have a data with strong intraday (hourly) and intraweek (daily) seasonality, and data granularity is '5m' with more than 5% expected outliers present in data. Then you can apply similar config:
|
||||
|
||||
```yaml
|
||||
models:
|
||||
your_desired_alias_for_a_model:
|
||||
class: "quantile_online" # or 'model.online.OnlineSeasonalQuantile'
|
||||
quantiles: [0.025, 0.5, 0.975] # lowered to exclude anomalous edges, can be compensated by `scale` param > 1
|
||||
seasonal_interval: '7d' # longest seasonality (week, day) = week, starting from `season_starts_from`
|
||||
min_subseason: '1h' # smallest seasonality (week, day, hour) = hour, will have its own quantile estimates
|
||||
min_n_samples_seen: 288 # 1440 / 5 - at least 1 full day, ideal = 1440 / 5 * 7 - one full week (seasonal_interval)
|
||||
scale: 1.1 # to compensate lowered quantile boundaries with wider intervals
|
||||
season_starts_from: '2024-01-01' # interval calculation starting point, expecially for uncommon seasonalities like '36h' or '12d'
|
||||
compression: 100 # higher values mean higher accuracy but higher memory usage
|
||||
provide_series: ['anomaly_score', 'yhat'] # common arg example
|
||||
```
|
||||
|
||||
Resulting metrics of the model are described [here](#vmanomaly-output).
|
||||
|
||||
|
||||
### [Seasonal Trend Decomposition](https://en.wikipedia.org/wiki/Seasonal_adjustment)
|
||||
|
||||
> **Note**: `StdModel` is [univariate](#univariate-models), [rolling](#rolling-models), [offline](#offline-models) model.
|
||||
|
||||
Here we use Seasonal Decompose implementation from `statsmodels` [library](https://www.statsmodels.org/dev/generated/statsmodels.tsa.seasonal.seasonal_decompose.html). Parameters from this library can be passed to the model. Some parameters are specifically predefined in `vmanomaly` and can't be changed by user(`model`='additive', `two_sided`=False).
|
||||
|
||||
*Parameters specific for vmanomaly*:
|
||||
|
|
@ -536,6 +700,11 @@ Resulting metrics of the model are described [here](#vmanomaly-output).
|
|||
|
||||
|
||||
### [Isolation forest](https://en.wikipedia.org/wiki/Isolation_forest) (Multivariate)
|
||||
|
||||
> **Note**: `IsolationForestModel` is [univariate](#univariate-models), [non-rolling](#non-rolling-models), [offline](#offline-models) model.
|
||||
|
||||
> **Note**: `IsolationForestMultivariateModel` is [multivariate](#multivariate-models), [non-rolling](#non-rolling-models), [offline](#offline-models) model.
|
||||
|
||||
Detects anomalies using binary trees. The algorithm has a linear time complexity and a low memory requirement, which works well with high-volume data. It can be used on both univariate and multivariate data, but it is more effective in multivariate case.
|
||||
|
||||
**Important**: Be aware of [the curse of dimensionality](https://en.wikipedia.org/wiki/Curse_of_dimensionality). Don't use single multivariate model if you expect your queries to return many time series of less datapoints that the number of metrics. In such case it is hard for a model to learn meaningful dependencies from too sparse data hypercube.
|
||||
|
|
@ -616,7 +785,7 @@ Here in this guide, we will
|
|||
- Define VictoriaMetrics Anomaly Detection config file to use our custom model
|
||||
- Run service
|
||||
|
||||
**Note**: The file containing the model should be written in [Python language](https://www.python.org/) (3.11+)
|
||||
> **Note**: The file containing the model should be written in [Python language](https://www.python.org/) (3.11+)
|
||||
|
||||
### 1. Custom model
|
||||
|
||||
|
|
@ -626,7 +795,7 @@ We'll create `custom_model.py` file with `CustomModel` class that will inherit f
|
|||
In the `CustomModel` class, the following methods are required: - `__init__`, `fit`, `infer`, `serialize` and `deserialize`:
|
||||
* `__init__` method should initiate parameters for the model.
|
||||
|
||||
**Note**: if your model relies on configs that have `arg` [key-value pair argument](./models.md#section-overview), do not forget to use Python's `**kwargs` in method's signature and to explicitly call
|
||||
**Note**: if your model relies on configs that have `arg` [key-value pair argument, like Prophet](#prophet), do not forget to use Python's `**kwargs` in method's signature and to explicitly call
|
||||
|
||||
```python
|
||||
super().__init__(**kwargs)
|
||||
|
|
@ -662,7 +831,10 @@ class CustomModel(Model):
|
|||
"""
|
||||
# by default, each `Model` will be created as a univariate one
|
||||
# uncomment line below for it to be of multivariate type
|
||||
#`is_multivariate = True`
|
||||
# is_multivariate = True
|
||||
# by default, each `Model` will be created as offline
|
||||
# uncomment line below for it to be of type online
|
||||
# is_online = True
|
||||
|
||||
def __init__(self, percentage: float = 0.95, **kwargs):
|
||||
super().__init__(**kwargs)
|
||||
|
|
|
|||
Loading…
Reference in a new issue