docs/vmanomaly - release 1.18.0 (#7378)

### Describe Your Changes

docs/vmanomaly - release 1.18.0

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

(cherry picked from commit 5d73b8b866)
This commit is contained in:
Fred Navruzov 2024-10-28 16:25:24 +01:00 committed by hagen1778
parent 517bd9392c
commit 61657e56e8
No known key found for this signature in database
GPG key ID: E92986095E0DD614
8 changed files with 105 additions and 28 deletions

View file

@ -72,7 +72,7 @@ services:
restart: always restart: always
vmanomaly: vmanomaly:
container_name: vmanomaly container_name: vmanomaly
image: victoriametrics/vmanomaly:v1.17.2 image: victoriametrics/vmanomaly:v1.18.0
depends_on: depends_on:
- "victoriametrics" - "victoriametrics"
ports: ports:

View file

@ -11,6 +11,17 @@ aliases:
--- ---
Please find the changelog for VictoriaMetrics Anomaly Detection below. Please find the changelog for VictoriaMetrics Anomaly Detection below.
## v1.18.0
Released: 2024-10-28
- FEATURE: Introduced timezone-aware support in `VmReader` for accurate seasonality modeling, especially during DST shifts. A new `tz` argument enables timezone offset management at both global and [query-specific levels](https://docs.victoriametrics.com/anomaly-detection/components/reader/#per-query-parameters).
- Enhanced [`ProphetModel`](https://docs.victoriametrics.com/anomaly-detection/components/models/#prophet) with a `tz_aware` argument (combined with `tz_seasonalities` and `tz_use_cyclical_encoding`) for timezone-aware timestamps. This addresses a [limitation in Prophet's native design](https://github.com/facebook/prophet/blob/dc1df4cb23a150e14858afb34c9442401c0eb2fc/python/prophet/forecaster.py#L288) that doesn't allow timezone-aware and DST-aware seasonality.
- IMPROVEMENT: Enhanced error handling in [VmReader](https://docs.victoriametrics.com/anomaly-detection/components/reader/#vm-reader) to provide clearer diagnostics and broader coverage.
- FIX: Updated `vmanomaly_version_info` and `vmanomaly_ui_version_info` gauges to correctly set the version label value based on image tags.
- FIX: The `n_samples_seen_` attribute now properly resets to 0 with each new `fit` call in [online model](https://docs.victoriametrics.com/anomaly-detection/components/models/#online-models) classes ([`OnlineMADModel`](https://docs.victoriametrics.com/anomaly-detection/components/models/#online-mad) and [`OnlineQuantileModel`](https://docs.victoriametrics.com/anomaly-detection/components/models/#online-seasonal-quantile)), ensuring accurate tracking of processed sample count.
## v1.17.2 ## v1.17.2
Released: 2024-10-22 Released: 2024-10-22

View file

@ -54,6 +54,33 @@ Respective config is defined in a [`reader`](https://docs.victoriametrics.com/an
## Handling noisy input data ## Handling noisy input data
`vmanomaly` operates on data fetched from VictoriaMetrics using [MetricsQL](https://docs.victoriametrics.com/metricsql/) queries, so the initial data quality can be fine-tuned with aggregation, grouping, and filtering to reduce noise and improve anomaly detection accuracy. `vmanomaly` operates on data fetched from VictoriaMetrics using [MetricsQL](https://docs.victoriametrics.com/metricsql/) queries, so the initial data quality can be fine-tuned with aggregation, grouping, and filtering to reduce noise and improve anomaly detection accuracy.
## Handling timezones
Starting from [v1.18.0](https://docs.victoriametrics.com/anomaly-detection/changelog/#v1180), `vmanomaly` supports timezone-aware anomaly detection through a `tz` argument, available both globally (in the [`reader`](https://docs.victoriametrics.com/anomaly-detection/components/reader#vm-reader) section) and at the [query level](https://docs.victoriametrics.com/anomaly-detection/components/reader/#per-query-parameters).
For models that depend on seasonality, such as [`ProphetModel`](https://docs.victoriametrics.com/anomaly-detection/components/models/#prophet) and [`OnlineQuantileModel`](https://docs.victoriametrics.com/anomaly-detection/components/models/#online-seasonal-quantile), handling timezone shifts is crucial. Changes like Daylight Saving Time (DST) can disrupt seasonality patterns learned by models, resulting in inaccurate anomaly predictions as the periodic patterns shift with time. Proper timezone configuration ensures that seasonal cycles align with expected intervals, even as DST changes occur.
To enable timezone handling:
1. **Globally**: Set `tz` in the [`reader`](https://docs.victoriametrics.com/anomaly-detection/components/reader#vm-reader) section to a specific timezone (e.g., `Europe/Berlin`) to apply this setting to all queries.
2. **Per query**: Override the global setting by specifying `tz` at the individual [query level](https://docs.victoriametrics.com/anomaly-detection/components/reader/#per-query-parameters) for targeted adjustments.
**Example:**
```yaml
reader:
datasource_url: 'your_victoriametrics_url'
tz: 'America/New_York' # global setting for all queries
queries:
your_query:
expr: 'avg(your_metric)'
tz: 'Europe/London' # per-query override
models:
seasonal_model:
class: 'prophet'
queries: ['your_query']
# other model params ...
```
## Output produced by vmanomaly ## Output produced by vmanomaly
`vmanomaly` models generate [metrics](https://docs.victoriametrics.com/anomaly-detection/components/models#vmanomaly-output) like `anomaly_score`, `yhat`, `yhat_lower`, `yhat_upper`, and `y`. These metrics provide a comprehensive view of the detected anomalies. The service also produces [health check metrics](https://docs.victoriametrics.com/anomaly-detection/components/monitoring#metrics-generated-by-vmanomaly) for monitoring its performance. `vmanomaly` models generate [metrics](https://docs.victoriametrics.com/anomaly-detection/components/models#vmanomaly-output) like `anomaly_score`, `yhat`, `yhat_lower`, `yhat_upper`, and `y`. These metrics provide a comprehensive view of the detected anomalies. The service also produces [health check metrics](https://docs.victoriametrics.com/anomaly-detection/components/monitoring#metrics-generated-by-vmanomaly) for monitoring its performance.
@ -132,7 +159,7 @@ services:
# ... # ...
vmanomaly: vmanomaly:
container_name: vmanomaly container_name: vmanomaly
image: victoriametrics/vmanomaly:v1.17.2 image: victoriametrics/vmanomaly:v1.18.0
# ... # ...
ports: ports:
- "8490:8490" - "8490:8490"
@ -230,7 +257,7 @@ P.s. `infer` data volume will remain the same for both models, so it does not af
If you're dealing with a large query in the `queries` argument of [VmReader](https://docs.victoriametrics.com/anomaly-detection/components/reader/#vm-reader) (especially when running [within a scheduler using a long](https://docs.victoriametrics.com/anomaly-detection/components/scheduler/?highlight=fit_window#periodic-scheduler) `fit_window`), you may encounter issues such as query timeouts (due to the `search.maxQueryDuration` server limit) or rejections (if the `search.maxPointsPerTimeseries` server limit is exceeded). If you're dealing with a large query in the `queries` argument of [VmReader](https://docs.victoriametrics.com/anomaly-detection/components/reader/#vm-reader) (especially when running [within a scheduler using a long](https://docs.victoriametrics.com/anomaly-detection/components/scheduler/?highlight=fit_window#periodic-scheduler) `fit_window`), you may encounter issues such as query timeouts (due to the `search.maxQueryDuration` server limit) or rejections (if the `search.maxPointsPerTimeseries` server limit is exceeded).
We recommend upgrading to [v1.17.2](https://docs.victoriametrics.com/anomaly-detection/changelog/#v1171), which introduced the `max_points_per_query` argument (both global and [query-specific](https://docs.victoriametrics.com/anomaly-detection/components/reader/#per-query-parameters)) for the [VmReader](https://docs.victoriametrics.com/anomaly-detection/components/reader/#vm-reader). This argument overrides how `search.maxPointsPerTimeseries` flag handling (introduced in [v1.14.1](https://docs.victoriametrics.com/anomaly-detection/changelog/#v1141)) is used in `vmanomaly` for splitting long `fit_window` queries into smaller sub-intervals. This helps users avoid hitting the `search.maxQueryDuration` limit for individual queries by distributing initial query across multiple subquery requests with minimal overhead. We recommend upgrading to [v1.17.2](https://docs.victoriametrics.com/anomaly-detection/changelog/#v1171) (or newer), which introduced the `max_points_per_query` argument (both global and [query-specific](https://docs.victoriametrics.com/anomaly-detection/components/reader/#per-query-parameters)) for the [VmReader](https://docs.victoriametrics.com/anomaly-detection/components/reader/#vm-reader). This argument overrides how `search.maxPointsPerTimeseries` flag handling (introduced in [v1.14.1](https://docs.victoriametrics.com/anomaly-detection/changelog/#v1141)) is used in `vmanomaly` for splitting long `fit_window` queries into smaller sub-intervals. This helps users avoid hitting the `search.maxQueryDuration` limit for individual queries by distributing initial query across multiple subquery requests with minimal overhead.
By splitting long `fit_window` queries into smaller sub-intervals, this helps avoid hitting the `search.maxQueryDuration` limit, distributing the load across multiple subquery requests with minimal overhead. To resolve the issue, reduce `max_points_per_query` to a value lower than `search.maxPointsPerTimeseries` until the problem is gone: By splitting long `fit_window` queries into smaller sub-intervals, this helps avoid hitting the `search.maxQueryDuration` limit, distributing the load across multiple subquery requests with minimal overhead. To resolve the issue, reduce `max_points_per_query` to a value lower than `search.maxPointsPerTimeseries` until the problem is gone:

View file

@ -229,7 +229,7 @@ This will expose metrics at `http://0.0.0.0:8080/metrics` page.
To use *vmanomaly* you need to pull docker image: To use *vmanomaly* you need to pull docker image:
```sh ```sh
docker pull victoriametrics/vmanomaly:v1.17.2 docker pull victoriametrics/vmanomaly:v1.18.0
``` ```
> Note: please check what is latest release in [CHANGELOG](https://docs.victoriametrics.com/anomaly-detection/changelog/) > Note: please check what is latest release in [CHANGELOG](https://docs.victoriametrics.com/anomaly-detection/changelog/)
@ -239,7 +239,7 @@ docker pull victoriametrics/vmanomaly:v1.17.2
You can put a tag on it for your convenience: You can put a tag on it for your convenience:
```sh ```sh
docker image tag victoriametrics/vmanomaly:v1.17.2 vmanomaly docker image tag victoriametrics/vmanomaly:v1.18.0 vmanomaly
``` ```
Here is an example of how to run *vmanomaly* docker container with [license file](#licensing): Here is an example of how to run *vmanomaly* docker container with [license file](#licensing):

View file

@ -58,13 +58,13 @@ Below are the steps to get `vmanomaly` up and running inside a Docker container:
1. Pull Docker image: 1. Pull Docker image:
```sh ```sh
docker pull victoriametrics/vmanomaly:v1.17.2 docker pull victoriametrics/vmanomaly:v1.18.0
``` ```
2. (Optional step) tag the `vmanomaly` Docker image: 2. (Optional step) tag the `vmanomaly` Docker image:
```sh ```sh
docker image tag victoriametrics/vmanomaly:v1.17.2 vmanomaly docker image tag victoriametrics/vmanomaly:v1.18.0 vmanomaly
``` ```
3. Start the `vmanomaly` Docker container with a *license file*, use the command below. 3. Start the `vmanomaly` Docker container with a *license file*, use the command below.
@ -98,7 +98,7 @@ docker run -it --user 1000:1000 \
services: services:
# ... # ...
vmanomaly: vmanomaly:
image: victoriametrics/vmanomaly:v1.17.2 image: victoriametrics/vmanomaly:v1.18.0
volumes: volumes:
$YOUR_LICENSE_FILE_PATH:/license $YOUR_LICENSE_FILE_PATH:/license
$YOUR_CONFIG_FILE_PATH:/config.yml $YOUR_CONFIG_FILE_PATH:/config.yml

View file

@ -451,16 +451,20 @@ models:
### [Prophet](https://facebook.github.io/prophet/) ### [Prophet](https://facebook.github.io/prophet/)
Here we utilize the Facebook Prophet implementation, as detailed in their [library documentation](https://facebook.github.io/prophet/docs/quick_start.html#python-api). All parameters from this library are compatible and can be passed to the model. `vmanomaly` uses the Facebook Prophet implementation for time series forecasting, with detailed usage provided in the [Prophet library documentation](https://facebook.github.io/prophet/docs/quick_start.html#python-api). All Prophet parameters are supported and can be directly passed to the model via `args` argument.
> **Note**: `ProphetModel` is [univariate](#univariate-models), [non-rolling](#non-rolling-models), [offline](#offline-models) model.
> **Note**: `ProphetModel` is a [univariate](#univariate-models), [non-rolling](#non-rolling-models), [offline](#offline-models) model.
*Parameters specific for vmanomaly*: *Parameters specific for vmanomaly*:
* `class` (string) - model class name `"model.prophet.ProphetModel"` (or `prophet` starting from [v1.13.0](https://docs.victoriametrics.com/anomaly-detection/changelog/#1130) with class alias support) * `class` (string) - model class name `"model.prophet.ProphetModel"` (or `prophet` starting from [v1.13.0](https://docs.victoriametrics.com/anomaly-detection/changelog/#1130) with class alias support)
* `seasonalities` (list[dict], optional) - Extra seasonalities to pass to Prophet. See [`add_seasonality()`](https://facebook.github.io/prophet/docs/seasonality,_holiday_effects,_and_regressors.html#modeling-holidays-and-special-events:~:text=modeling%20the%20cycle-,Specifying,-Custom%20Seasonalities) Prophet param. * `seasonalities` (list[dict], optional): Additional seasonal components to include in Prophet. See Prophets [`add_seasonality()`](https://facebook.github.io/prophet/docs/seasonality,_holiday_effects,_and_regressors.html#modeling-holidays-and-special-events:~:text=modeling%20the%20cycle-,Specifying,-Custom%20Seasonalities) documentation for details.
- `tz_aware` (bool): (Available since [v1.18.0](https://docs.victoriametrics.com/anomaly-detection/changelog/#v1180)) Enables handling of timezone-aware timestamps. Default is `False`. Should be used with `tz_seasonalities` and `tz_use_cyclical_encoding` parameters.
- `tz_seasonalities` (list[str]): (Available since [v1.18.0](https://docs.victoriametrics.com/anomaly-detection/changelog/#v1180)) Specifies timezone-aware seasonal components. Requires `tz_aware=True`. Supported options include `minute`, `hod` (hour of the day), `dow` (day of the week), and `month` (month of the year).
- `tz_use_cyclical_encoding` (bool): (Available since [v1.18.0](https://docs.victoriametrics.com/anomaly-detection/changelog/#v1180)) If set to `True`, applies [cyclical encoding technique](https://www.kaggle.com/code/avanwyk/encoding-cyclical-features-for-deep-learning) to timezone-aware seasonalities. Should be used with `tz_aware=True` and `tz_seasonalities`.
**Note**: Apart from standard `vmanomaly` output, Prophet model can provide [additional metrics](#additional-output-metrics-produced-by-fb-prophet). > **Note**: Apart from standard [`vmanomaly` output](#vmanomaly-output), Prophet model can provide additional metrics.
**Additional output metrics produced by FB Prophet** **Additional output metrics produced by FB Prophet**
Depending on chosen `seasonality` parameter FB Prophet can return additional metrics such as: Depending on chosen `seasonality` parameter FB Prophet can return additional metrics such as:
@ -474,6 +478,8 @@ Depending on chosen `seasonality` parameter FB Prophet can return additional met
*Config Example* *Config Example*
Timezone-unaware example:
```yaml ```yaml
models: models:
your_desired_alias_for_a_model: your_desired_alias_for_a_model:
@ -483,11 +489,27 @@ models:
- name: 'hourly' - name: 'hourly'
period: 0.04166666666 period: 0.04166666666
fourier_order: 30 fourier_order: 30
# Inner model args (key-value pairs) accepted by # inner model args (key-value pairs) accepted by
# https://facebook.github.io/prophet/docs/quick_start.html#python-api # https://facebook.github.io/prophet/docs/quick_start.html#python-api
args: args:
# See https://facebook.github.io/prophet/docs/uncertainty_intervals.html interval_width: 0.98 # see https://facebook.github.io/prophet/docs/uncertainty_intervals.html
interval_width: 0.98 country_holidays: 'US'
```
Timezone-aware example:
```yaml
models:
your_desired_alias_for_a_model:
class: 'prophet' # or 'model.prophet.ProphetModel' until v1.13.0
provide_series: ['anomaly_score', 'yhat', 'yhat_lower', 'yhat_upper', 'trend']
tz_aware: True
tz_seasonalities: ['hod', 'dow'] # intra-day + intra-week seasonality, no intra-year / sub-hour seasonality
tz_use_cyclical_encoding: False
# inner model args (key-value pairs) accepted by
# https://facebook.github.io/prophet/docs/quick_start.html#python-api
args:
interval_width: 0.98 # see https://facebook.github.io/prophet/docs/uncertainty_intervals.html
country_holidays: 'US' country_holidays: 'US'
``` ```
@ -496,7 +518,7 @@ Resulting metrics of the model are described [here](#vmanomaly-output)
### [Z-score](https://en.wikipedia.org/wiki/Standard_score) ### [Z-score](https://en.wikipedia.org/wiki/Standard_score)
> **Note**: `ZScoreModel` is [univariate](#univariate-models), [non-rolling](#non-rolling-models), [offline](#offline-models) model. > **Note**: `ZScoreModel` is a [univariate](#univariate-models), [non-rolling](#non-rolling-models), [offline](#offline-models) model.
Model is useful for initial testing and for simpler data ([de-trended](https://victoriametrics.com/blog/victoriametrics-anomaly-detection-handbook-chapter-1/#trend) data without strict [seasonality](https://victoriametrics.com/blog/victoriametrics-anomaly-detection-handbook-chapter-1/#seasonality) and with anomalies of similar magnitude as your "normal" data). Model is useful for initial testing and for simpler data ([de-trended](https://victoriametrics.com/blog/victoriametrics-anomaly-detection-handbook-chapter-1/#trend) data without strict [seasonality](https://victoriametrics.com/blog/victoriametrics-anomaly-detection-handbook-chapter-1/#seasonality) and with anomalies of similar magnitude as your "normal" data).
@ -518,7 +540,7 @@ Resulting metrics of the model are described [here](#vmanomaly-output).
### Online Z-score ### Online Z-score
> **Note**: `OnlineZScoreModel` is [univariate](#univariate-models), [non-rolling](#non-rolling-models), [online](#online-models) model. > **Note**: `OnlineZScoreModel` is a [univariate](#univariate-models), [non-rolling](#non-rolling-models), [online](#online-models) model.
Online version of existing [Z-score](#z-score) implementation with the same exact behavior and implications. Introduced in [v1.15.0](https://docs.victoriametrics.com/anomaly-detection/changelog/#v1150) Online version of existing [Z-score](#z-score) implementation with the same exact behavior and implications. Introduced in [v1.15.0](https://docs.victoriametrics.com/anomaly-detection/changelog/#v1150)
@ -544,7 +566,7 @@ Resulting metrics of the model are described [here](#vmanomaly-output).
### [Holt-Winters](https://en.wikipedia.org/wiki/Exponential_smoothing) ### [Holt-Winters](https://en.wikipedia.org/wiki/Exponential_smoothing)
> **Note**: `HoltWinters` is [univariate](#univariate-models), [non-rolling](#non-rolling-models), [offline](#offline-models) model. > **Note**: `HoltWinters` is a [univariate](#univariate-models), [non-rolling](#non-rolling-models), [offline](#offline-models) model.
Here we use Holt-Winters Exponential Smoothing implementation from `statsmodels` [library](https://www.statsmodels.org/dev/generated/statsmodels.tsa.holtwinters.ExponentialSmoothing.html). All parameters from this library can be passed to the model. Here we use Holt-Winters Exponential Smoothing implementation from `statsmodels` [library](https://www.statsmodels.org/dev/generated/statsmodels.tsa.holtwinters.ExponentialSmoothing.html). All parameters from this library can be passed to the model.
@ -590,7 +612,7 @@ Resulting metrics of the model are described [here](#vmanomaly-output).
### [MAD (Median Absolute Deviation)](https://en.wikipedia.org/wiki/Median_absolute_deviation) ### [MAD (Median Absolute Deviation)](https://en.wikipedia.org/wiki/Median_absolute_deviation)
> **Note**: `MADModel` is [univariate](#univariate-models), [non-rolling](#non-rolling-models), [offline](#offline-models) model. > **Note**: `MADModel` is a [univariate](#univariate-models), [non-rolling](#non-rolling-models), [offline](#offline-models) model.
The MAD model is a robust method for anomaly detection that is *less sensitive* to outliers in data compared to standard deviation-based models. It considers a point as an anomaly if the absolute deviation from the median is significantly large. The MAD model is a robust method for anomaly detection that is *less sensitive* to outliers in data compared to standard deviation-based models. It considers a point as an anomaly if the absolute deviation from the median is significantly large.
@ -614,7 +636,7 @@ Resulting metrics of the model are described [here](#vmanomaly-output).
### Online MAD ### Online MAD
> **Note**: `OnlineMADModel` is [univariate](#univariate-models), [non-rolling](#non-rolling-models), [online](#online-models) model. > **Note**: `OnlineMADModel` is a [univariate](#univariate-models), [non-rolling](#non-rolling-models), [online](#online-models) model.
The MAD model is a robust method for anomaly detection that is *less sensitive* to outliers in data compared to standard deviation-based models. It considers a point as an anomaly if the absolute deviation from the median is significantly large. This is the online approximate version, based on [t-digests](https://www.sciencedirect.com/science/article/pii/S2665963820300403) for online quantile estimation. introduced in [v1.15.0](https://docs.victoriametrics.com/anomaly-detection/changelog/#v1150) The MAD model is a robust method for anomaly detection that is *less sensitive* to outliers in data compared to standard deviation-based models. It considers a point as an anomaly if the absolute deviation from the median is significantly large. This is the online approximate version, based on [t-digests](https://www.sciencedirect.com/science/article/pii/S2665963820300403) for online quantile estimation. introduced in [v1.15.0](https://docs.victoriametrics.com/anomaly-detection/changelog/#v1150)
@ -643,7 +665,7 @@ Resulting metrics of the model are described [here](#vmanomaly-output).
### [Rolling Quantile](https://en.wikipedia.org/wiki/Quantile) ### [Rolling Quantile](https://en.wikipedia.org/wiki/Quantile)
> **Note**: `RollingQuantileModel` is [univariate](#univariate-models), [rolling](#rolling-models), [offline](#offline-models) model. > **Note**: `RollingQuantileModel` is a [univariate](#univariate-models), [rolling](#rolling-models), [offline](#offline-models) model.
This model is best used on **data with short evolving patterns** (i.e. 10-100 datapoints of particular frequency), as it adapts to changes over a rolling window. This model is best used on **data with short evolving patterns** (i.e. 10-100 datapoints of particular frequency), as it adapts to changes over a rolling window.
@ -668,7 +690,7 @@ Resulting metrics of the model are described [here](#vmanomaly-output).
### Online Seasonal Quantile ### Online Seasonal Quantile
> **Note**: `OnlineQuantileModel` is [univariate](#univariate-models), [non-rolling](#non-rolling-models), [online](#online-models) model. > **Note**: `OnlineQuantileModel` is a [univariate](#univariate-models), [non-rolling](#non-rolling-models), [online](#online-models) model.
Online (seasonal) quantile utilizes a set of approximate distributions, based on [t-digests](https://www.sciencedirect.com/science/article/pii/S2665963820300403) for online quantile estimation. Introduced in [v1.15.0](https://docs.victoriametrics.com/anomaly-detection/changelog/#v1150). Online (seasonal) quantile utilizes a set of approximate distributions, based on [t-digests](https://www.sciencedirect.com/science/article/pii/S2665963820300403) for online quantile estimation. Introduced in [v1.15.0](https://docs.victoriametrics.com/anomaly-detection/changelog/#v1150).
@ -712,7 +734,7 @@ Resulting metrics of the model are described [here](#vmanomaly-output).
### [Seasonal Trend Decomposition](https://en.wikipedia.org/wiki/Seasonal_adjustment) ### [Seasonal Trend Decomposition](https://en.wikipedia.org/wiki/Seasonal_adjustment)
> **Note**: `StdModel` is [univariate](#univariate-models), [rolling](#rolling-models), [offline](#offline-models) model. > **Note**: `StdModel` is a [univariate](#univariate-models), [rolling](#rolling-models), [offline](#offline-models) model.
Here we use Seasonal Decompose implementation from `statsmodels` [library](https://www.statsmodels.org/dev/generated/statsmodels.tsa.seasonal.seasonal_decompose.html). Parameters from this library can be passed to the model. Some parameters are specifically predefined in `vmanomaly` and can't be changed by user(`model`='additive', `two_sided`=False). Here we use Seasonal Decompose implementation from `statsmodels` [library](https://www.statsmodels.org/dev/generated/statsmodels.tsa.seasonal.seasonal_decompose.html). Parameters from this library can be passed to the model. Some parameters are specifically predefined in `vmanomaly` and can't be changed by user(`model`='additive', `two_sided`=False).
@ -744,9 +766,9 @@ Resulting metrics of the model are described [here](#vmanomaly-output).
### [Isolation forest](https://en.wikipedia.org/wiki/Isolation_forest) (Multivariate) ### [Isolation forest](https://en.wikipedia.org/wiki/Isolation_forest) (Multivariate)
> **Note**: `IsolationForestModel` is [univariate](#univariate-models), [non-rolling](#non-rolling-models), [offline](#offline-models) model. > **Note**: `IsolationForestModel` is a [univariate](#univariate-models), [non-rolling](#non-rolling-models), [offline](#offline-models) model.
> **Note**: `IsolationForestMultivariateModel` is [multivariate](#multivariate-models), [non-rolling](#non-rolling-models), [offline](#offline-models) model. > **Note**: `IsolationForestMultivariateModel` is a [multivariate](#multivariate-models), [non-rolling](#non-rolling-models), [offline](#offline-models) model.
Detects anomalies using binary trees. The algorithm has a linear time complexity and a low memory requirement, which works well with high-volume data. It can be used on both univariate and multivariate data, but it is more effective in multivariate case. Detects anomalies using binary trees. The algorithm has a linear time complexity and a low memory requirement, which works well with high-volume data. It can be used on both univariate and multivariate data, but it is more effective in multivariate case.
@ -962,7 +984,7 @@ monitoring:
Let's pull the docker image for `vmanomaly`: Let's pull the docker image for `vmanomaly`:
```sh ```sh
docker pull victoriametrics/vmanomaly:v1.17.2 docker pull victoriametrics/vmanomaly:v1.18.0
``` ```
Now we can run the docker container putting as volumes both config and model file: Now we can run the docker container putting as volumes both config and model file:
@ -976,7 +998,7 @@ docker run -it \
-v $(PWD)/license:/license \ -v $(PWD)/license:/license \
-v $(PWD)/custom_model.py:/vmanomaly/model/custom.py \ -v $(PWD)/custom_model.py:/vmanomaly/model/custom.py \
-v $(PWD)/custom.yaml:/config.yaml \ -v $(PWD)/custom.yaml:/config.yaml \
victoriametrics/vmanomaly:v1.17.2 /config.yaml \ victoriametrics/vmanomaly:v1.18.0 /config.yaml \
--licenseFile=/license --licenseFile=/license
``` ```

View file

@ -48,6 +48,7 @@ reader:
expr: 'avg(vm_blocks)' # initial MetricsQL expression expr: 'avg(vm_blocks)' # initial MetricsQL expression
step: '10s' # individual step for this query, will be filled with `sampling_period` from the root level step: '10s' # individual step for this query, will be filled with `sampling_period` from the root level
data_range: ['-inf', 'inf'] # by default, no constraints applied on data range data_range: ['-inf', 'inf'] # by default, no constraints applied on data range
tz: 'UTC' # by default, tz-free data is used throughout the model lifecycle
# new query-level arguments will be added in backward-compatible way in future releases # new query-level arguments will be added in backward-compatible way in future releases
``` ```
@ -69,6 +70,8 @@ Starting from [v1.13.0](https://docs.victoriametrics.com/anomaly-detection/chang
- `max_points_per_query` (int): Introduced in [v1.17.0](https://docs.victoriametrics.com/anomaly-detection/changelog/#v1170), optional arg overrides how `search.maxPointsPerTimeseries` flag (available since [v1.14.1](#v1141)) impacts `vmanomaly` on splitting long `fit_window` [queries](https://docs.victoriametrics.com/anomaly-detection/components/reader/?highlight=queries#vm-reader) into smaller sub-intervals. This helps users avoid hitting the `search.maxQueryDuration` limit for individual queries by distributing initial query across multiple subquery requests with minimal overhead. Set less than `search.maxPointsPerTimeseries` if hitting `maxQueryDuration` limits. If set on a query-level, it overrides the global `max_points_per_query` (reader-level). - `max_points_per_query` (int): Introduced in [v1.17.0](https://docs.victoriametrics.com/anomaly-detection/changelog/#v1170), optional arg overrides how `search.maxPointsPerTimeseries` flag (available since [v1.14.1](#v1141)) impacts `vmanomaly` on splitting long `fit_window` [queries](https://docs.victoriametrics.com/anomaly-detection/components/reader/?highlight=queries#vm-reader) into smaller sub-intervals. This helps users avoid hitting the `search.maxQueryDuration` limit for individual queries by distributing initial query across multiple subquery requests with minimal overhead. Set less than `search.maxPointsPerTimeseries` if hitting `maxQueryDuration` limits. If set on a query-level, it overrides the global `max_points_per_query` (reader-level).
- `tz` (string): Introduced in [v1.18.0](https://docs.victoriametrics.com/anomaly-detection/changelog/#v1180), this optional argument enables timezone specification per query, overriding the readers default `tz`. This setting helps to account for local timezone shifts, such as [DST](https://en.wikipedia.org/wiki/Daylight_saving_time), in models that are sensitive to seasonal variations (e.g., [`ProphetModel`](https://docs.victoriametrics.com/anomaly-detection/components/models/#prophet) or [`OnlineQuantileModel`](https://docs.victoriametrics.com/anomaly-detection/components/models/#online-seasonal-quantile)).
### Per-query config example ### Per-query config example
```yaml ```yaml
@ -83,6 +86,7 @@ reader:
step: '2m' # overrides global `sampling_period` of 1m step: '2m' # overrides global `sampling_period` of 1m
data_range: [10, 'inf'] # meaning only positive values > 10 are expected, i.e. a value `y` < 10 will trigger anomaly score > 1 data_range: [10, 'inf'] # meaning only positive values > 10 are expected, i.e. a value `y` < 10 will trigger anomaly score > 1
max_points_per_query: 5000 # overrides reader-level value of 10000 for `ingestion_rate` query max_points_per_query: 5000 # overrides reader-level value of 10000 for `ingestion_rate` query
tz: 'America/New_York' # to override reader-wise `tz`
``` ```
### Config parameters ### Config parameters
@ -308,6 +312,17 @@ Introduced in [v1.15.1](https://docs.victoriametrics.com/anomaly-detection/chang
Introduced in [v1.17.0](https://docs.victoriametrics.com/anomaly-detection/changelog/#v1170), optional arg overrides how `search.maxPointsPerTimeseries` flag (available since [v1.14.1](#v1141)) impacts `vmanomaly` on splitting long `fit_window` [queries](https://docs.victoriametrics.com/anomaly-detection/components/reader/?highlight=queries#vm-reader) into smaller sub-intervals. This helps users avoid hitting the `search.maxQueryDuration` limit for individual queries by distributing initial query across multiple subquery requests with minimal overhead. Set less than `search.maxPointsPerTimeseries` if hitting `maxQueryDuration` limits. You can also set it on [per-query](#per-query-parameters) basis to override this global one. Introduced in [v1.17.0](https://docs.victoriametrics.com/anomaly-detection/changelog/#v1170), optional arg overrides how `search.maxPointsPerTimeseries` flag (available since [v1.14.1](#v1141)) impacts `vmanomaly` on splitting long `fit_window` [queries](https://docs.victoriametrics.com/anomaly-detection/components/reader/?highlight=queries#vm-reader) into smaller sub-intervals. This helps users avoid hitting the `search.maxQueryDuration` limit for individual queries by distributing initial query across multiple subquery requests with minimal overhead. Set less than `search.maxPointsPerTimeseries` if hitting `maxQueryDuration` limits. You can also set it on [per-query](#per-query-parameters) basis to override this global one.
</td> </td>
</tr> </tr>
<tr>
<td>
`tz`
</td>
<td>
`UTC`
</td>
<td>
Introduced in [v1.18.0](https://docs.victoriametrics.com/anomaly-detection/changelog/#v1180), this optional argument specifies the [IANA](https://nodatime.org/TimeZones) timezone to account for local shifts, like [DST](https://en.wikipedia.org/wiki/Daylight_saving_time), in models sensitive to seasonal patterns (e.g., [`ProphetModel`](https://docs.victoriametrics.com/anomaly-detection/components/models/#prophet) or [`OnlineQuantileModel`](https://docs.victoriametrics.com/anomaly-detection/components/models/#online-seasonal-quantile)). Defaults to `UTC` if not set and can be overridden on a [per-query basis](#per-query-parameters).
</td>
</tr>
</tbody> </tbody>
</table> </table>
@ -318,11 +333,13 @@ reader:
class: "vm" # or "reader.vm.VmReader" until v1.13.0 class: "vm" # or "reader.vm.VmReader" until v1.13.0
datasource_url: "https://play.victoriametrics.com/" datasource_url: "https://play.victoriametrics.com/"
tenant_id: "0:0" tenant_id: "0:0"
tz: 'America/New_York'
queries: queries:
ingestion_rate: ingestion_rate:
expr: 'sum(rate(vm_rows_inserted_total[5m])) by (type) > 0' expr: 'sum(rate(vm_rows_inserted_total[5m])) by (type) > 0'
step: '1m' # can override global `sampling_period` on per-query level step: '1m' # can override global `sampling_period` on per-query level
data_range: [0, 'inf'] data_range: [0, 'inf']
tz: 'Australia/Sydney' # if set, overrides reader-wise tz
sampling_period: '1m' sampling_period: '1m'
query_from_last_seen_timestamp: True # false by default query_from_last_seen_timestamp: True # false by default
latency_offset: '1ms' latency_offset: '1ms'

View file

@ -385,7 +385,7 @@ services:
restart: always restart: always
vmanomaly: vmanomaly:
container_name: vmanomaly container_name: vmanomaly
image: victoriametrics/vmanomaly:v1.17.2 image: victoriametrics/vmanomaly:v1.18.0
depends_on: depends_on:
- "victoriametrics" - "victoriametrics"
ports: ports: