github-mirrors/VictoriaMetrics
1
0
Fork 0
mirror of https://github.com/VictoriaMetrics/VictoriaMetrics.git synced 2024-11-21 14:44:00 +00:00
Code Issues Projects Releases Packages Wiki Activity

docs/vmanomaly: release v1.17.0 (#7285)

Browse source 
### Describe Your Changes

docs/vmanomaly: release v1.17.0

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
This commit is contained in:
Fred Navruzov 2024-10-17 17:56:04 +02:00 committed by GitHub
parent 635bdd130b
commit bc65c9f399
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
8 changed files with 200 additions and 77 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
deployment/docker/vmanomaly/vmanomaly-integration/docker-compose.yml
View file

@ -73,7 +73,7 @@ services:
restart: always restart: always
vmanomaly: vmanomaly:
container_name: vmanomaly container_name: vmanomaly
image: victoriametrics/vmanomaly:v1.16.3 image: victoriametrics/vmanomaly:v1.17.0
depends_on: depends_on:
- "victoriametrics" - "victoriametrics"
ports: ports:

23
docs/anomaly-detection/CHANGELOG.md
View file

@ -11,6 +11,29 @@ aliases:
--- ---
Please find the changelog for VictoriaMetrics Anomaly Detection below. Please find the changelog for VictoriaMetrics Anomaly Detection below.
## v1.17.0
Released: 2024-10-17
- FEATURE: Added `max_points_per_query` (global and [query-specific](https://docs.victoriametrics.com/anomaly-detection/components/reader/#per-query-parameters)) [VmReader](https://docs.victoriametrics.com/anomaly-detection/components/reader/#vm-reader) arg to control query chunking. This overrides how `search.maxPointsPerTimeseries` flag (introduced in [v1.14.1](#v1141)) is used in `vmanomaly` for splitting long `fit_window` queries into smaller sub-intervals. This helps users avoid hitting the `search.maxQueryDuration` limit for individual queries by distributing initial query across multiple subquery requests with minimal overhead.
- IMPROVEMENT: Enhanced the [self-monitoring](https://docs.victoriametrics.com/anomaly-detection/components/monitoring/#metrics-generated-by-vmanomaly) metrics for consistency across the components. Key changes include:
- Converted several [self-monitoring](https://docs.victoriametrics.com/anomaly-detection/components/monitoring/#metrics-generated-by-vmanomaly) metrics from `Summary` to `Histogram` to enable quantile calculation. This addresses the limitation of the `prometheus_client`'s [Summary](https://prometheus.github.io/client_python/instrumenting/summary/) implementation, which does not support quantiles. The change ensures metrics are more informative for performance analysis. Affected metrics are:
- `vmanomaly_reader_request_duration_seconds` ([VmReader](https://docs.victoriametrics.com/anomaly-detection/components/monitoring/#reader-behaviour-metrics))
- `vmanomaly_reader_response_parsing_seconds` ([VmReader](https://docs.victoriametrics.com/anomaly-detection/components/monitoring/#reader-behaviour-metrics))
- `vmanomaly_writer_request_duration_seconds` ([VmWriter](https://docs.victoriametrics.com/anomaly-detection/components/monitoring/#writer-behaviour-metrics))
- `vmanomaly_writer_request_serialize_seconds` ([VmWriter](https://docs.victoriametrics.com/anomaly-detection/components/monitoring/#writer-behaviour-metrics))
- Added a `query_key` label to the `vmanomaly_reader_response_parsing_seconds` [metric](https://docs.victoriametrics.com/anomaly-detection/components/monitoring/#reader-behaviour-metrics) to provide finer granularity in tracking the performance of individual queries. This metric has also been switched from `Summary` to `Histogram` to align with the other metrics and support quantile calculations.
- Added `preset` and `scheduler_alias` keys to [VmReader](https://docs.victoriametrics.com/anomaly-detection/components/monitoring/#reader-behaviour-metrics) and [VmWriter](https://docs.victoriametrics.com/anomaly-detection/components/monitoring/#writer-behaviour-metrics) metrics for consistency in multi-[scheduler](https://docs.victoriametrics.com/anomaly-detection/components/scheduler/) setups.
- Renamed [Counters](https://prometheus.io/docs/concepts/metric_types/#counter) `vmanomaly_reader_response_count` to `vmanomaly_reader_responses` and `vmanomaly_writer_response_count` to `vmanomaly_writer_responses`.
- Updated [docs](https://docs.victoriametrics.com/anomaly-detection/components/monitoring/#metrics-generated-by-vmanomaly) for better clarity.
- IMPROVEMENT: Accelerated performance of model fitting stages on multicore systems.
- IMPROVEMENT: Optimized query handling in multi-[scheduler](https://docs.victoriametrics.com/anomaly-detection/components/scheduler/) setups by filtering [queries](https://docs.victoriametrics.com/anomaly-detection/components/models/#queries) for each scheduler based on model requirements. This reduces unnecessary data fetching from VictoriaMetrics, ensuring only relevant queries are processed by the [VmReader](https://docs.victoriametrics.com/anomaly-detection/components/reader#vm-reader), leading to better performance and efficiency of configs with multiple active schedulers.
- IMPROVEMENT: Implemented automatic cleanup of files in subdirectories within `/tmp` ([generated by the Stan backend](https://mc-stan.org/cmdstanpy/users-guide/outputs.html) when utilizing [Prophet](https://docs.victoriametrics.com/anomaly-detection/components/models/#prophet) models) after each `fit` operation. This prevents the accumulation of unused data over time in `/tmp`, addressing a potential issue where these files would only be deleted upon termination of the current Python session or service, leading to uncontrolled disk growth.
- FIX: Re-enable the `vmanomaly_reader_response_count` (now called `vmanomaly_reader_responses`) self-monitoring [metric](https://docs.victoriametrics.com/anomaly-detection/components/monitoring/#reader-behaviour-metrics) for the [VmReader](https://docs.victoriametrics.com/anomaly-detection/components/reader/#vm-reader), which was unintentionally disabled in previous releases and now updates correctly as intended.
## v1.16.3 ## v1.16.3
Released: 2024-10-08 Released: 2024-10-08
- IMPROVEMENT: Added `tls_cert_file` and `tls_key_file` arguments to support mTLS (mutual TLS) in `vmanomaly` components. This enhancement applies to the following components: [VmReader](https://docs.victoriametrics.com/anomaly-detection/components/reader/#vm-reader), [VmWriter](https://docs.victoriametrics.com/anomaly-detection/components/writer/#vm-writer), and [Monitoring/Push](https://docs.victoriametrics.com/anomaly-detection/components/monitoring/#push-config-parameters). You can also use these arguments in conjunction with `verify_tls` when it is set as a path to a custom CA certificate file. - IMPROVEMENT: Added `tls_cert_file` and `tls_key_file` arguments to support mTLS (mutual TLS) in `vmanomaly` components. This enhancement applies to the following components: [VmReader](https://docs.victoriametrics.com/anomaly-detection/components/reader/#vm-reader), [VmWriter](https://docs.victoriametrics.com/anomaly-detection/components/writer/#vm-writer), and [Monitoring/Push](https://docs.victoriametrics.com/anomaly-detection/components/monitoring/#push-config-parameters). You can also use these arguments in conjunction with `verify_tls` when it is set as a path to a custom CA certificate file.

72
docs/anomaly-detection/FAQ.md
View file

@ -132,7 +132,7 @@ services:
# ... # ...
vmanomaly: vmanomaly:
container_name: vmanomaly container_name: vmanomaly
image: victoriametrics/vmanomaly:v1.16.3 image: victoriametrics/vmanomaly:v1.17.0
# ... # ...
ports: ports:
- "8490:8490" - "8490:8490"
@ -225,6 +225,76 @@ P.s. `infer` data volume will remain the same for both models, so it does not af
- Old: 8064 hours/week (fit) + 168 hours/week (infer) - Old: 8064 hours/week (fit) + 168 hours/week (infer)
- New: 4 hours/week (fit) + 168 hours/week (infer) - New: 4 hours/week (fit) + 168 hours/week (infer)
## Handling large queries in vmanomaly
If you're dealing with a large query in the `queries` argument of [VmReader](https://docs.victoriametrics.com/anomaly-detection/components/reader/#vm-reader) (especially when running [within a scheduler using a long](https://docs.victoriametrics.com/anomaly-detection/components/scheduler/?highlight=fit_window#periodic-scheduler) `fit_window`), you may encounter issues such as query timeouts (due to the `search.maxQueryDuration` server limit) or rejections (if the `search.maxPointsPerTimeseries` server limit is exceeded).
We recommend upgrading to [v1.17.0](https://docs.victoriametrics.com/anomaly-detection/changelog/#v1170), which introduced the `max_points_per_query` argument (both global and [query-specific](https://docs.victoriametrics.com/anomaly-detection/components/reader/#per-query-parameters)) for the [VmReader](https://docs.victoriametrics.com/anomaly-detection/components/reader/#vm-reader). This argument overrides how `search.maxPointsPerTimeseries` flag handling (introduced in [v1.14.1](https://docs.victoriametrics.com/anomaly-detection/changelog/#v1141)) is used in `vmanomaly` for splitting long `fit_window` queries into smaller sub-intervals. This helps users avoid hitting the `search.maxQueryDuration` limit for individual queries by distributing initial query across multiple subquery requests with minimal overhead.
By splitting long `fit_window` queries into smaller sub-intervals, this helps avoid hitting the `search.maxQueryDuration` limit, distributing the load across multiple subquery requests with minimal overhead. To resolve the issue, reduce `max_points_per_query` to a value lower than `search.maxPointsPerTimeseries` until the problem is gone:
```yaml
reader:
# other reader args
max_points_per_query: 10000 # reader-level constraint
queries:
sum_alerts:
expr: 'sum(ALERTS{alertstate=~'(pending|firing)'}) by (alertstate)'
max_points_per_query: 5000 # query-level override
models:
prophet:
# other model args
queries: [
'sum_alerts',
]
# other config sections
```
### Alternative workaround for older versions
If upgrading is not an option, you can partially address the issue by splitting your large query into smaller ones using appropriate label filters:
For example, such query
```yaml
reader:
# other reader args
queries:
sum_alerts:
expr: 'sum(ALERTS{alertstate=~'(pending|firing)'}) by (alertstate)'
models:
prophet:
# other model args
queries: [
'sum_alerts',
]
# other config sections
```
can be modified to:
```yaml
reader:
# other reader args
queries:
sum_alerts_pending:
expr: 'sum(ALERTS{alertstate='pending'}) by ()'
sum_alerts_firing:
expr: 'sum(ALERTS{alertstate='firing'}) by ()'
models:
prophet:
# other model args
queries: [
'sum_alerts_pending',
'sum_alerts_firing',
]
# other config sections
```
Please note that this approach may not fully resolve the issue if subqueries are not evenly distributed in terms of returned timeseries. Additionally, this workaround is not suitable for queries used in [multivariate models](https://docs.victoriametrics.com/anomaly-detection/components/models#multivariate-models) (especially when using the [groupby](https://docs.victoriametrics.com/anomaly-detection/components/models/#group-by) argument).
## Scaling vmanomaly ## Scaling vmanomaly
> **Note:** As of latest release we do not support cluster or auto-scaled version yet (though, it's in our roadmap for - better backends, more parallelization, etc.), so proposed workarounds should be addressed *manually*. > **Note:** As of latest release we do not support cluster or auto-scaled version yet (though, it's in our roadmap for - better backends, more parallelization, etc.), so proposed workarounds should be addressed *manually*.

4
docs/anomaly-detection/Overview.md
View file

@ -229,7 +229,7 @@ This will expose metrics at `http://0.0.0.0:8080/metrics` page.
To use *vmanomaly* you need to pull docker image: To use *vmanomaly* you need to pull docker image:
```sh ```sh
docker pull victoriametrics/vmanomaly:v1.16.3 docker pull victoriametrics/vmanomaly:v1.17.0
``` ```
> Note: please check what is latest release in [CHANGELOG](https://docs.victoriametrics.com/anomaly-detection/changelog/) > Note: please check what is latest release in [CHANGELOG](https://docs.victoriametrics.com/anomaly-detection/changelog/)
@ -239,7 +239,7 @@ docker pull victoriametrics/vmanomaly:v1.16.3
You can put a tag on it for your convenience: You can put a tag on it for your convenience:
```sh ```sh
docker image tag victoriametrics/vmanomaly:v1.16.3 vmanomaly docker image tag victoriametrics/vmanomaly:v1.17.0 vmanomaly
``` ```
Here is an example of how to run *vmanomaly* docker container with [license file](#licensing): Here is an example of how to run *vmanomaly* docker container with [license file](#licensing):

4
docs/anomaly-detection/components/models.md
View file

@ -962,7 +962,7 @@ monitoring:
Let's pull the docker image for `vmanomaly`: Let's pull the docker image for `vmanomaly`:
```sh ```sh
docker pull victoriametrics/vmanomaly:v1.16.3 docker pull victoriametrics/vmanomaly:v1.17.0
``` ```
Now we can run the docker container putting as volumes both config and model file: Now we can run the docker container putting as volumes both config and model file:
@ -976,7 +976,7 @@ docker run -it \
-v $(PWD)/license:/license \ -v $(PWD)/license:/license \
-v $(PWD)/custom_model.py:/vmanomaly/model/custom.py \ -v $(PWD)/custom_model.py:/vmanomaly/model/custom.py \
-v $(PWD)/custom.yaml:/config.yaml \ -v $(PWD)/custom.yaml:/config.yaml \
victoriametrics/vmanomaly:v1.16.3 /config.yaml \ victoriametrics/vmanomaly:v1.17.0 /config.yaml \
--licenseFile=/license --licenseFile=/license
``` ```

145
docs/anomaly-detection/components/monitoring.md
View file

@ -11,6 +11,16 @@ aliases:
--- ---
There are 2 models to monitor VictoriaMetrics Anomaly Detection behavior - [push](https://docs.victoriametrics.com/keyconcepts/#push-model) and [pull](https://docs.victoriametrics.com/keyconcepts/#pull-model). Parameters for each of them should be specified in the config file, `monitoring` section. There are 2 models to monitor VictoriaMetrics Anomaly Detection behavior - [push](https://docs.victoriametrics.com/keyconcepts/#push-model) and [pull](https://docs.victoriametrics.com/keyconcepts/#pull-model). Parameters for each of them should be specified in the config file, `monitoring` section.
> **Note**: there was an enhancement of [self-monitoring](https://docs.victoriametrics.com/anomaly-detection/components/monitoring/#metrics-generated-by-vmanomaly) metrics for consistency across the components ([v.1.17.0](https://docs.victoriametrics.com/anomaly-detection/changelog/#v1170)). Documentation was updated accordingly. Key changes included:
- Converting several [self-monitoring](https://docs.victoriametrics.com/anomaly-detection/components/monitoring/#metrics-generated-by-vmanomaly) metrics from `Summary` to `Histogram` to enable quantile calculation. This addresses the limitation of the `prometheus_client`'s [Summary](https://prometheus.github.io/client_python/instrumenting/summary/) implementation, which does not support quantiles. The change ensures metrics are more informative for performance analysis. Affected metrics are:
- `vmanomaly_reader_request_duration_seconds` ([VmReader](https://docs.victoriametrics.com/anomaly-detection/components/monitoring/#reader-behaviour-metrics))
- `vmanomaly_reader_response_parsing_seconds` ([VmReader](https://docs.victoriametrics.com/anomaly-detection/components/monitoring/#reader-behaviour-metrics))
- `vmanomaly_writer_request_duration_seconds` ([VmWriter](https://docs.victoriametrics.com/anomaly-detection/components/monitoring/#writer-behaviour-metrics))
- `vmanomaly_writer_request_serialize_seconds` ([VmWriter](https://docs.victoriametrics.com/anomaly-detection/components/monitoring/#writer-behaviour-metrics))
- Adding a `query_key` label to the `vmanomaly_reader_response_parsing_seconds` [metric](https://docs.victoriametrics.com/anomaly-detection/components/monitoring/#reader-behaviour-metrics) to provide finer granularity in tracking the performance of individual queries. This metric has also been switched from `Summary` to `Histogram` to align with the other metrics and support quantile calculations.
- Adding `preset` and `scheduler_alias` keys to [VmReader](https://docs.victoriametrics.com/anomaly-detection/components/monitoring/#reader-behaviour-metrics) and [VmWriter](https://docs.victoriametrics.com/anomaly-detection/components/monitoring/#writer-behaviour-metrics) metrics for consistency in multi-[scheduler](https://docs.victoriametrics.com/anomaly-detection/components/scheduler/) setups.
- Renaming [Counters](https://prometheus.io/docs/concepts/metric_types/#counter) `vmanomaly_reader_response_count` to `vmanomaly_reader_responses` and `vmanomaly_writer_response_count` to `vmanomaly_writer_responses`.
## Pull Model Config parameters ## Pull Model Config parameters
<table class="params"> <table class="params">
@ -241,6 +251,8 @@ Label names [description](#labelnames)
> **Note**: There is a new label key `model_alias` introduced in multi-model support [v1.10.0](https://docs.victoriametrics.com/anomaly-detection/changelog/#v1100). This label key adjustment was made to preserve unique label set production during writing produced metrics back to VictoriaMetrics. > **Note**: There is a new label key `model_alias` introduced in multi-model support [v1.10.0](https://docs.victoriametrics.com/anomaly-detection/changelog/#v1100). This label key adjustment was made to preserve unique label set production during writing produced metrics back to VictoriaMetrics.
> **Note**: as a part of [self-monitoring](https://docs.victoriametrics.com/anomaly-detection/components/monitoring/#metrics-generated-by-vmanomaly) metrics enchancement ([v.1.17.0](https://docs.victoriametrics.com/anomaly-detection/changelog/#v1170)), new metrics, like `vmanomaly_model_run_errors`, was added. Some of them changed the type (`Summary` -> `Histogram`), like `vmanomaly_model_run_duration_seconds`.
<table class="params"> <table class="params">
<thead> <thead>
<tr> <tr>
@ -256,11 +268,11 @@ Label names [description](#labelnames)
`vmanomaly_model_runs` `vmanomaly_model_runs`
</td> </td>
<td>Counter</td> <td>`Counter`</td>
<td>How many times models ran (per model)</td> <td>How many successful `stage` (`fit`, `infer`, `fit_infer`) runs occurred for models of class `model_alias` based on results from the `query_key` query, within the specified scheduler `scheduler_alias`, in the `vmanomaly` service running in `preset` mode.</td>
<td> <td>
`stage, query_key, model_alias, scheduler_alias, preset` `stage`, `query_key`, `model_alias`, `scheduler_alias`, `preset`
</td> </td>
</tr> </tr>
<tr> <tr>
@ -268,11 +280,11 @@ Label names [description](#labelnames)
`vmanomaly_model_run_duration_seconds` `vmanomaly_model_run_duration_seconds`
</td> </td>
<td>Summary</td> <td>`Histogram` (was `Summary` prior to [v1.17.0](https://docs.victoriametrics.com/anomaly-detection/changelog/#v1170)) </td>
<td>How much time (in seconds) model invocations took</td> <td>The total time (in seconds) taken by model invocations during the `stage` (`fit`, `infer`, `fit_infer`), based on the results of the `query_key` query, for models of class `model_alias`, within the specified scheduler `scheduler_alias`, in the `vmanomaly` service running in `preset` mode.</td>
<td> <td>
`stage, query_key, model_alias, scheduler_alias, preset` `stage`, `query_key`, `model_alias`, `scheduler_alias`, `preset`
</td> </td>
</tr> </tr>
<tr> <tr>
@ -280,11 +292,11 @@ Label names [description](#labelnames)
`vmanomaly_model_datapoints_accepted` `vmanomaly_model_datapoints_accepted`
</td> </td>
<td>Counter</td> <td>`Counter`</td>
<td>How many datapoints did models accept</td> <td>The number of datapoints accepted (excluding NaN or Inf values) by models of class `model_alias` from the results of the `query_key` query during the `stage` (`infer`, `fit_infer`), within the specified scheduler `scheduler_alias`, in the `vmanomaly` service running in `preset` mode.</td>
<td> <td>
`stage, query_key, model_alias, scheduler_alias, preset` `stage`, `query_key`, `model_alias`, `scheduler_alias`, `preset`
</td> </td>
</tr> </tr>
<tr> <tr>
@ -292,11 +304,11 @@ Label names [description](#labelnames)
`vmanomaly_model_datapoints_produced` `vmanomaly_model_datapoints_produced`
</td> </td>
<td>Counter</td> <td>`Counter`</td>
<td>How many datapoints were generated by models</td> <td>The number of datapoints generated by models of class `model_alias` during the `stage` (`infer`, `fit_infer`) based on results from the `query_key` query, within the specified scheduler `scheduler_alias`, in the `vmanomaly` service running in `preset` mode.</td>
<td> <td>
`stage, query_key, model_alias, scheduler_alias, preset` `stage`, `query_key`, `model_alias`, `scheduler_alias`, `preset`
</td> </td>
</tr> </tr>
<tr> <tr>
@ -304,11 +316,11 @@ Label names [description](#labelnames)
`vmanomaly_models_active` `vmanomaly_models_active`
</td> </td>
<td>Gauge</td> <td>`Gauge`</td>
<td>How many models are currently inferring</td> <td>The number of model instances of class `model_alias` currently available for inference for the `query_key` query, within the specified scheduler `scheduler_alias`, in the `vmanomaly` service running in `preset` mode.</td>
<td> <td>
`query_key, model_alias, scheduler_alias, preset` `query_key`, `model_alias`, `scheduler_alias`, `preset`
</td> </td>
</tr> </tr>
<tr> <tr>
@ -316,11 +328,22 @@ Label names [description](#labelnames)
`vmanomaly_model_runs_skipped` `vmanomaly_model_runs_skipped`
</td> </td>
<td>Counter</td> <td>`Counter`</td>
<td>How many times a run was skipped (per model)</td> <td>The number of times model runs (of class `model_alias`) were skipped in expected situations (e.g., no data for fitting/inference, or no new data to infer on) during the `stage` (`fit`, `infer`, `fit_infer`), based on results from the `query_key` query, within the specified scheduler `scheduler_alias`, in the `vmanomaly` service running in `preset` mode.</td>
<td> <td>
`stage, query_key, model_alias, scheduler_alias, preset` `stage`, `query_key`, `model_alias`, `scheduler_alias`, `preset`
</td>
</tr>
<tr>
<td>
`vmanomaly_model_run_errors`
</td>
<td>`Counter`</td>
<td>The number of times model runs (of class `model_alias`) failed due to internal service errors during the `stage` (`fit`, `infer`, `fit_infer`), based on results from the `query_key` query, within the specified scheduler `scheduler_alias`, in the `vmanomaly` service running in `preset` mode.</td>
<td>
`stage`, `query_key`, `model_alias`, `scheduler_alias`, `preset`
</td> </td>
</tr> </tr>
</tbody> </tbody>
@ -329,6 +352,8 @@ Label names [description](#labelnames)
### Writer Behaviour Metrics ### Writer Behaviour Metrics
Label names [description](#labelnames) Label names [description](#labelnames)
> **Note**: additional labels (`scheduler_alias`, `preset`) were added to writer and reader metrics in [v1.17.0](https://docs.victoriametrics.com/anomaly-detection/changelog/#v1170) to improve consistency across the components. Also, metrics `vmanomaly_writer_request_duration_seconds` and `vmanomaly_writer_request_serialize_seconds` changed their type to `Histogram` (was `Summary` prior to [v1.17.0](https://docs.victoriametrics.com/anomaly-detection/changelog/#v1170)).
<table class="params"> <table class="params">
<thead> <thead>
<tr> <tr>
@ -344,23 +369,25 @@ Label names [description](#labelnames)
`vmanomaly_writer_request_duration_seconds` `vmanomaly_writer_request_duration_seconds`
</td> </td>
<td>Summary</td> <td>`Histogram` (was `Summary` prior to [v1.17.0](https://docs.victoriametrics.com/anomaly-detection/changelog/#v1170))</td>
<td>How much time (in seconds) did requests to VictoriaMetrics take</td> <td>The total time (in seconds) taken by write requests to VictoriaMetrics `url` for the `query_key` query within the specified scheduler `scheduler_alias`, in the `vmanomaly` service running in `preset` mode.
</td>
<td> <td>
`url, query_key` `url`, `query_key`, `scheduler_alias`, `preset`
</td> </td>
</tr> </tr>
<tr> <tr>
<td> <td>
`vmanomaly_writer_response_count` `vmanomaly_writer_responses` (named `vmanomaly_reader_response_count` prior to [v1.17.0](https://docs.victoriametrics.com/anomaly-detection/changelog/#v1170))
</td> </td>
<td>Counter</td> <td>`Counter`</td>
<td>Response code counts we got from VictoriaMetrics</td> <td>The count of response codes received from VictoriaMetrics `url` for the `query_key` query, categorized by `code`, within the specified scheduler `scheduler_alias`, in the `vmanomaly` service running in `preset` mode.
</td>
<td> <td>
`url, query_key, code` `url`, `code`, `query_key`, `scheduler_alias`, `preset`
</td> </td>
</tr> </tr>
<tr> <tr>
@ -368,11 +395,11 @@ Label names [description](#labelnames)
`vmanomaly_writer_sent_bytes` `vmanomaly_writer_sent_bytes`
</td> </td>
<td>Counter</td> <td>`Counter`</td>
<td>How much bytes were sent to VictoriaMetrics</td> <td>The total number of bytes sent to VictoriaMetrics `url` for the `query_key` query within the specified scheduler `scheduler_alias`, in the `vmanomaly` service running in `preset` mode.</td>
<td> <td>
`url, query_key` `url`, `query_key`, `scheduler_alias`, `preset`
</td> </td>
</tr> </tr>
<tr> <tr>
@ -380,11 +407,11 @@ Label names [description](#labelnames)
`vmanomaly_writer_request_serialize_seconds` `vmanomaly_writer_request_serialize_seconds`
</td> </td>
<td>Summary</td> <td>`Histogram` (was `Summary` prior to [v1.17.0](https://docs.victoriametrics.com/anomaly-detection/changelog/#v1170))</td>
<td>How much time (in seconds) did serializing take</td> <td>The total time (in seconds) taken for serializing data for the `query_key` query within the specified scheduler `scheduler_alias`, in the `vmanomaly` service running in `preset` mode.</td>
<td> <td>
`query_key` `query_key`, `scheduler_alias`, `preset`
</td> </td>
</tr> </tr>
<tr> <tr>
@ -392,23 +419,21 @@ Label names [description](#labelnames)
`vmanomaly_writer_datapoints_sent` `vmanomaly_writer_datapoints_sent`
</td> </td>
<td>Counter</td> <td>`Counter`</td>
<td>How many datapoints were sent to VictoriaMetrics</td> <td>The total number of datapoints sent to VictoriaMetrics for the `query_key` query within the specified scheduler `scheduler_alias`, in the `vmanomaly` service running in `preset` mode.</td>
<td> <td>
`query_key` `query_key`, `scheduler_alias`, `preset`
</td> </td>
</tr> </tr>
<tr> <tr>
<td> <td>
`vmanomaly_writer_timeseries_sent` `vmanomaly_writer_timeseries_sent`
</td> </td>
<td>Counter</td> <td>`Counter`</td>
<td>How many timeseries were sent to VictoriaMetrics</td> <td>The total number of timeseries sent to VictoriaMetrics for the `query_key` query within the specified scheduler `scheduler_alias`, in the `vmanomaly` service running in `preset` mode.</td>
<td> <td>
`query_key`, `scheduler_alias`, `preset`
`query_key`
</td> </td>
</tr> </tr>
</tbody> </tbody>
@ -432,23 +457,23 @@ Label names [description](#labelnames)
`vmanomaly_reader_request_duration_seconds` `vmanomaly_reader_request_duration_seconds`
</td> </td>
<td>Summary</td> <td>`Histogram` (was `Summary` prior to [v1.17.0](https://docs.victoriametrics.com/anomaly-detection/changelog/#v1170))</td>
<td>How much time (in seconds) did queries to VictoriaMetrics take</td> <td>The total time (in seconds) taken by queries to VictoriaMetrics `url` for the `query_key` query within the specified scheduler `scheduler_alias`, in the `vmanomaly` service running in `preset` mode.</td>
<td> <td>
`url, query_key` `url`, `query_key`, `scheduler_alias`, `preset`
</td> </td>
</tr> </tr>
<tr> <tr>
<td> <td>
`vmanomaly_reader_response_count` `vmanomaly_reader_responses` (named `vmanomaly_reader_response_count` prior to [v1.17.0](https://docs.victoriametrics.com/anomaly-detection/changelog/#v1170))
</td> </td>
<td>Counter</td> <td>`Counter`</td>
<td>Response code counts we got from VictoriaMetrics</td> <td>The count of responses received from VictoriaMetrics `url` for the `query_key` query, categorized by `code`, within the specified scheduler `scheduler_alias`, in the `vmanomaly` service running in `preset` mode.</td>
<td> <td>
`url, query_key, code` `url`, `query_key`, `code`, `scheduler_alias`, `preset`
</td> </td>
</tr> </tr>
<tr> <tr>
@ -456,11 +481,11 @@ Label names [description](#labelnames)
`vmanomaly_reader_received_bytes` `vmanomaly_reader_received_bytes`
</td> </td>
<td>Counter</td> <td>`Counter`</td>
<td>How much bytes were received in responses</td> <td>The total number of bytes received in responses for the `query_key` query within the specified scheduler `scheduler_alias`, in the `vmanomaly` service running in `preset` mode.</td>
<td> <td>
`query_key` `query_key`, `scheduler_alias`, `preset`
</td> </td>
</tr> </tr>
<tr> <tr>
@ -468,23 +493,23 @@ Label names [description](#labelnames)
`vmanomaly_reader_response_parsing_seconds` `vmanomaly_reader_response_parsing_seconds`
</td> </td>
<td>Summary</td> <td>`Histogram` (was `Summary` prior to [v1.17.0](https://docs.victoriametrics.com/anomaly-detection/changelog/#v1170))</td>
<td>How much time (in seconds) did parsing take for each step</td> <td>The total time (in seconds) taken for data parsing at each `step` (json, dataframe) for the `query_key` query within the specified scheduler `scheduler_alias`, in the `vmanomaly` service running in `preset` mode.</td>
<td> <td>
`step` `step`, `query_key`, `scheduler_alias`, `preset`
</td> </td>
</tr> </tr>
<tr> <tr>
<td> <td>
`vmanomaly_reader_timeseries_received` `vmanomaly_reader_timeseries_received`
</td> </td>
<td>Counter</td> <td>`Counter`</td>
<td>How many timeseries were received from VictoriaMetrics</td> <td>The total number of timeseries received from VictoriaMetrics for the `query_key` query within the specified scheduler `scheduler_alias`, in the `vmanomaly` service running in `preset` mode.</td>
<td> <td>
`query_key` `query_key`, `scheduler_alias`, `preset`
</td> </td>
</tr> </tr>
<tr> <tr>
@ -492,11 +517,11 @@ Label names [description](#labelnames)
`vmanomaly_reader_datapoints_received` `vmanomaly_reader_datapoints_received`
</td> </td>
<td>Counter</td> <td>`Counter`</td>
<td>How many rows were received from VictoriaMetrics</td> <td>The total number of datapoints received from VictoriaMetrics for the `query_key` query within the specified scheduler `scheduler_alias`, in the `vmanomaly` service running in `preset` mode.</td>
<td> <td>
`query_key` `query_key`, `scheduler_alias`, `preset`
</td> </td>
</tr> </tr>
</tbody> </tbody>
@ -508,7 +533,7 @@ Label names [description](#labelnames)
* `query_key` - query alias from [`reader`](https://docs.victoriametrics.com/anomaly-detection/components/reader/) config section. * `query_key` - query alias from [`reader`](https://docs.victoriametrics.com/anomaly-detection/components/reader/) config section.
* `model_alias` - model alias from [`models`](https://docs.victoriametrics.com/anomaly-detection/components/models/) config section. **Introduced in [v1.10.0](https://docs.victoriametrics.com/anomaly-detection/changelog/#v1100).** * `model_alias` - model alias from [`models`](https://docs.victoriametrics.com/anomaly-detection/components/models/) config section. **Introduced in [v1.10.0](https://docs.victoriametrics.com/anomaly-detection/changelog/#v1100).**
* `scheduler_alias` - scheduler alias from [`schedulers`](https://docs.victoriametrics.com/anomaly-detection/components/scheduler) config section. **Introduced in [v1.11.0](https://docs.victoriametrics.com/anomaly-detection/changelog/#v1110).** * `scheduler_alias` - scheduler alias from [`schedulers`](https://docs.victoriametrics.com/anomaly-detection/components/scheduler) config section. **Introduced in [v1.11.0](https://docs.victoriametrics.com/anomaly-detection/changelog/#v1110).**
* `preset` - preset alias for forthcoming `preset` section compatibility. **Introduced in [v1.12.0](https://docs.victoriametrics.com/anomaly-detection/changelog/#v1120).** * `preset` - preset alias for [`preset`](https://docs.victoriametrics.com/anomaly-detection/presets/) mode of `vmanomaly`. **Introduced in [v1.12.0](https://docs.victoriametrics.com/anomaly-detection/changelog/#v1120).**
* `url` - writer or reader url endpoint. * `url` - writer or reader url endpoint.
* `code` - response status code or `connection_error`, `timeout`. * `code` - response status code or `connection_error`, `timeout`.
* `step` - json or dataframe reading step. * `step` - json or dataframe reading step.

25
docs/anomaly-detection/components/reader.md
View file

@ -67,18 +67,22 @@ Starting from [v1.13.0](https://docs.victoriametrics.com/anomaly-detection/chang
- **High anomaly scores** (>1) when the *data falls outside the expected range*, indicating a data constraint violation. - **High anomaly scores** (>1) when the *data falls outside the expected range*, indicating a data constraint violation.
- **Lowest anomaly scores** (=0) when the *model's predictions (`yhat`) fall outside the expected range*, meaning uncertain predictions. - **Lowest anomaly scores** (=0) when the *model's predictions (`yhat`) fall outside the expected range*, meaning uncertain predictions.
- `max_points_per_query` (int): Introduced in [v1.17.0](https://docs.victoriametrics.com/anomaly-detection/changelog/#v1170), optional arg overrides how `search.maxPointsPerTimeseries` flag (available since [v1.14.1](#v1141)) impacts `vmanomaly` on splitting long `fit_window` [queries](https://docs.victoriametrics.com/anomaly-detection/components/reader/?highlight=queries#vm-reader) into smaller sub-intervals. This helps users avoid hitting the `search.maxQueryDuration` limit for individual queries by distributing initial query across multiple subquery requests with minimal overhead. Set less than `search.maxPointsPerTimeseries` if hitting `maxQueryDuration` limits. If set on a query-level, it overrides the global `max_points_per_query` (reader-level).
### Per-query config example ### Per-query config example
```yaml ```yaml
reader: reader:
class: 'vm' class: 'vm'
sampling_period: '1m' sampling_period: '1m'
max_points_per_query: 10000
# other reader params ... # other reader params ...
queries: queries:
ingestion_rate: ingestion_rate:
expr: 'sum(rate(vm_rows_inserted_total[5m])) by (type) > 0' expr: 'sum(rate(vm_rows_inserted_total[5m])) by (type) > 0'
step: '2m' # overrides global `sampling_period` of 1m step: '2m' # overrides global `sampling_period` of 1m
data_range: [10, 'inf'] # meaning only positive values > 10 are expected, i.e. a value `y` < 10 will trigger anomaly score > 1 data_range: [10, 'inf'] # meaning only positive values > 10 are expected, i.e. a value `y` < 10 will trigger anomaly score > 1
max_points_per_query: 5000 # overrides reader-level value of 10000 for `ingestion_rate` query
``` ```
### Config parameters ### Config parameters
@ -293,6 +297,17 @@ If True, then query will be performed from the last seen timestamp for a given s
Introduced in [v1.15.1](https://docs.victoriametrics.com/anomaly-detection/changelog/#v1151), it allows overriding the default `-search.latencyOffset` [flag of VictoriaMetrics](https://docs.victoriametrics.com/?highlight=search.latencyOffset#list-of-command-line-flags) (30s). The default value is set to 1ms, which should help in cases where `sampling_frequency` is low (10-60s) and `sampling_frequency` equals `infer_every` in the [PeriodicScheduler](https://docs.victoriametrics.com/anomaly-detection/components/scheduler/?highlight=infer_every#periodic-scheduler). This prevents users from receiving `service - WARNING - [Scheduler [scheduler_alias]] No data available for inference.` warnings in logs and allows for consecutive `infer` calls without gaps. To restore the old behavior, set it equal to your `-search.latencyOffset` [flag value]((https://docs.victoriametrics.com/?highlight=search.latencyOffset#list-of-command-line-flags)). Introduced in [v1.15.1](https://docs.victoriametrics.com/anomaly-detection/changelog/#v1151), it allows overriding the default `-search.latencyOffset` [flag of VictoriaMetrics](https://docs.victoriametrics.com/?highlight=search.latencyOffset#list-of-command-line-flags) (30s). The default value is set to 1ms, which should help in cases where `sampling_frequency` is low (10-60s) and `sampling_frequency` equals `infer_every` in the [PeriodicScheduler](https://docs.victoriametrics.com/anomaly-detection/components/scheduler/?highlight=infer_every#periodic-scheduler). This prevents users from receiving `service - WARNING - [Scheduler [scheduler_alias]] No data available for inference.` warnings in logs and allows for consecutive `infer` calls without gaps. To restore the old behavior, set it equal to your `-search.latencyOffset` [flag value]((https://docs.victoriametrics.com/?highlight=search.latencyOffset#list-of-command-line-flags)).
</td> </td>
</tr> </tr>
<tr>
<td>
`max_points_per_query`
</td>
<td>
`10000`
</td>
<td>
Introduced in [v1.17.0](https://docs.victoriametrics.com/anomaly-detection/changelog/#v1170), optional arg overrides how `search.maxPointsPerTimeseries` flag (available since [v1.14.1](#v1141)) impacts `vmanomaly` on splitting long `fit_window` [queries](https://docs.victoriametrics.com/anomaly-detection/components/reader/?highlight=queries#vm-reader) into smaller sub-intervals. This helps users avoid hitting the `search.maxQueryDuration` limit for individual queries by distributing initial query across multiple subquery requests with minimal overhead. Set less than `search.maxPointsPerTimeseries` if hitting `maxQueryDuration` limits. You can also set it on [per-query](#per-query-parameters) basis to override this global one.
</td>
</tr>
</tbody> </tbody>
</table> </table>
@ -313,16 +328,6 @@ reader:
latency_offset: '1ms' latency_offset: '1ms'
``` ```
<!-- ### mTLS protection
Starting from [v1.16.3](https://docs.victoriametrics.com/anomaly-detection/changelog/#v1163), `vmanomaly` supports [mTLS](https://en.wikipedia.org/wiki/Mutual_authentication) requests in its components, like [VmReader](https://docs.victoriametrics.com/anomaly-detection/components/reader/#vm-reader), [VmWriter](https://docs.victoriametrics.com/anomaly-detection/components/writer/#vm-writer), and [Monitoring/Push](https://docs.victoriametrics.com/anomaly-detection/components/monitoring/#push-config-parameters) to query from and write to [VictoriaMetrics Enterprise, configured in the same mode](https://docs.victoriametrics.com/#mtls-protection).
Please see the description of next arguments in a [config](#config-parameters):
- `verify_tls` (if string, acts similar to `-mtlsCAFile` command line arg of VictoriaMetrics).
- `tls_cert_file` (if given, acts similar to `-tlsCertFile` command line arg of VictoriaMetrics).
- `tls_key_file` (if given, acts similar to `-tlsKeyFile` command line arg of VictoriaMetrics). -->
### mTLS protection ### mTLS protection
As of [v1.16.3](https://docs.victoriametrics.com/anomaly-detection/changelog/#v1163), `vmanomaly` supports [mutual TLS (mTLS)](https://en.wikipedia.org/wiki/Mutual_authentication) for secure communication across its components, including [VmReader](https://docs.victoriametrics.com/anomaly-detection/components/reader/#vm-reader), [VmWriter](https://docs.victoriametrics.com/anomaly-detection/components/writer/#vm-writer), and [Monitoring/Push](https://docs.victoriametrics.com/anomaly-detection/components/monitoring/#push-config-parameters). This allows for mutual authentication between the client and server when querying or writing data to [VictoriaMetrics Enterprise, configured for mTLS](https://docs.victoriametrics.com/#mtls-protection). As of [v1.16.3](https://docs.victoriametrics.com/anomaly-detection/changelog/#v1163), `vmanomaly` supports [mutual TLS (mTLS)](https://en.wikipedia.org/wiki/Mutual_authentication) for secure communication across its components, including [VmReader](https://docs.victoriametrics.com/anomaly-detection/components/reader/#vm-reader), [VmWriter](https://docs.victoriametrics.com/anomaly-detection/components/writer/#vm-writer), and [Monitoring/Push](https://docs.victoriametrics.com/anomaly-detection/components/monitoring/#push-config-parameters). This allows for mutual authentication between the client and server when querying or writing data to [VictoriaMetrics Enterprise, configured for mTLS](https://docs.victoriametrics.com/#mtls-protection).

2
docs/anomaly-detection/guides/guide-vmanomaly-vmalert/README.md
View file

@ -385,7 +385,7 @@ services:
restart: always restart: always
vmanomaly: vmanomaly:
container_name: vmanomaly container_name: vmanomaly
image: victoriametrics/vmanomaly:v1.16.3 image: victoriametrics/vmanomaly:v1.17.0
depends_on: depends_on:
- "victoriametrics" - "victoriametrics"
ports: ports: