diff --git a/docs/anomaly-detection/components/reader.md b/docs/anomaly-detection/components/reader.md index 6711b9f6d0..dfaf7a85a5 100644 --- a/docs/anomaly-detection/components/reader.md +++ b/docs/anomaly-detection/components/reader.md @@ -118,145 +118,3 @@ reader: ### Healthcheck metrics `VmReader` exposes [several healthchecks metrics](./monitoring.html#reader-behaviour-metrics). - -<!-- - -# TODO: uncomment and maintain after multimodel config refactor, 2nd priority - -## NDJSON reader -Accepts data in the same format as <code>/export</code>. - -File content example: -``` -{"metric":{"__name__":"metric1","job":"vm"},"values":[745487.56,96334.13,277822.84,159596.94],"timestamps":[1640908800000,1640908802000,1640908803000,1640908804000]} -{"metric":{"__name__":"metric2","job":"vm"},"values":[217822.84,159596.94,745487.56,96334.13],"timestamps":[1640908800000,1640908802000,1640908803000,1640908804000]} - -``` -### Config parameters -<table> - <thead> - <tr> - <th>Parameter</th> - <th>Example</th> - <th>Description</th> - </tr> - </thead> - <tbody> - <tr> - <td><code>class</code></td> - <td><code>"reader.ndjson.NdjsonReader"</code></td> - <td>Name of the class needed to enable reading from JSON line format file.</td> - </tr> - <tr> - <td><code>path</code></td> - <td><code>"tests/reader/export.ndjson"</code></td> - <td>Path to file in JSON line format</td> - </tr> - </tbody> -</table> - -Config file example: -```yaml -reader: - class: "reader.ndjson.NdjsonReader" - path: "tests/reader/export.ndjson" -``` - - -## QueryRange -This datasource is VictoriaMetrics handler for [Prometheus querying API](https://prometheus.io/docs/prometheus/latest/querying/api/). - -[Range query](https://docs.victoriametrics.com/keyConcepts.html#range-query) executes the query expression at the given time range with the given step. - -### Config parameters -<table> - <thead> - <tr> - <th>Parameter</th> - <th>Example</th> - <th>Description</th> - </tr> - </thead> - <tbody> - <tr> - <td><code>class</code></td> - <td><code>"reader.query_range.QueryRangeReader"</code></td> - <td>Name of the class enabling Query Range reader.</td> - </tr> - <tr> - <td><code>path</code></td> - <td><code>"http://localhost:8428/api/v1/query_range?query=sum(rate(vm_rows_inserted_total[30])) by (type)"</code></td> - <td>URL with query</td> - </tr> - </tbody> -</table> - -Config file example: -```yaml -reader: - class: "reader.query_range.QueryRangeReader" - path: "http://localhost:8428/api/v1/query_range?query=sum(rate(vm_rows_inserted_total[30])) by (type)" -``` - - -## CSV reader -### Data format -File should be in `.csv` format and must contain 2 columns with the names: `timestamp` and `y` - metric's datetimes and values accordinally. Order of the columns doesn't matter. - -* `timestamp` can be represented eather in explicit datetime format like `2021-04-21 05:18:19` or in UNIX time in seconds like `1618982299`. - -* `y` should be a numeric value. - - -File content example: -``` -timestamp,y -2020-07-12 23:09:05,61.0 -2020-07-13 23:09:05,63.0 -2020-07-14 23:09:05,63.0 -2020-07-15 23:09:05,66.0 -2020-07-20 23:09:05,68.0 -2020-07-21 23:09:05,69.0 -2020-07-22 23:09:05,69.0 -``` - -### Config parameters -<table> - <thead> - <tr> - <th>Parameter</th> - <th>Type</th> - <th>Example</th> - <th>Description</th> - </tr> - </thead> - <tbody> - <tr> - <td><code>class</code></td> - <td>str</td> - <td><code>"reader.csv.CsvReader"</code></td> - <td>Name of the class enabling CSV reader</td> - </tr> - <tr> - <td><code>path</code></td> - <td>str</td> - <td><code>"data/v1/jumpsup.csv"</code></td> - <td>.csv file location (local path). <b>The file existence is checked during config validation</b></td> - </tr> - <tr> - <td><code>metric_name</code></td> - <td>str</td> - <td><code>"value"</code></td> - <td>Optional. Alias for metric. If not specified, filename without extension will be used. In this example, `jumpsup`.</td> - </tr> - </tbody> -</table> -Config file example: - -```yaml -reader: - class: "reader.csv.CsvReader" - path: "data/v1/jumpsup.csv" - metric_name: "value" -``` ---> \ No newline at end of file diff --git a/docs/anomaly-detection/components/writer.md b/docs/anomaly-detection/components/writer.md index 57e266f62c..e71aedb70a 100644 --- a/docs/anomaly-detection/components/writer.md +++ b/docs/anomaly-detection/components/writer.md @@ -12,9 +12,6 @@ aliases: --- # Writer -<!-- -There are 3 ways to export data from VictoriaMetrics Anomaly Detection: VictoriaMetrics, JSON file, or CSV file. Depending on the chosen option, different parameters should be specified in the config file in the `writer` section. ---> For exporting data, VictoriaMetrics Anomaly Detection (`vmanomaly`) primarily employs the [VmWriter](#vm-writer), which writes produces anomaly scores (preserving initial labelset and optionally applying additional ones) back to VictoriaMetrics. This writer is tailored for smooth data export within the VictoriaMetrics ecosystem. @@ -104,6 +101,7 @@ Future updates will introduce additional export methods, offering users more fle </table> Config example: + ```yaml writer: class: "writer.vm.VmWriter" @@ -125,17 +123,20 @@ writer: `VmWriter` exposes [several healthchecks metrics](./monitoring.html#writer-behaviour-metrics). ### Metrics formatting + There should be 2 mandatory parameters set in `metric_format` - `__name__` and `for`. + ```yaml __name__: PREFIX1_$VAR for: PREFIX2_$QUERY_KEY ``` -* for `__name__` parameter it will name metrics returned by models as `PREFIX1_anomaly_score`, `PREFIX1_yhat_lower`, etc. Vmanomaly output metrics names described [here](anomaly-detection/components/models/models.html#vmanomaly-output) +* for `__name__` parameter it will name metrics returned by models as `PREFIX1_anomaly_score`, `PREFIX1_yhat_lower`, etc. Vmanomaly output metrics names described [here](anomaly-detection/components/models/models.html#vmanomaly-output) * for `for` parameter will add labels `PREFIX2_query_name_1`, `PREFIX2_query_name_2`, etc. Query names are set as aliases in config `reader` section in [`queries`](anomaly-detection/components/reader.html#config-parameters) parameter. It is possible to specify other custom label names needed. For example: + ```yaml custom_label_1: label_name_1 custom_label_2: label_name_2 @@ -145,6 +146,7 @@ Apart from specified labels, output metrics will return labels inherited from in For example if input data contains labels such as `cpu=1, device=eth0, instance=node-exporter:9100` all these labels will be present in vmanomaly output metrics. So if metric_format section was set up like this: + ```yaml metric_format: __name__: "PREFIX1_$VAR" @@ -154,117 +156,10 @@ metric_format: ``` It will return metrics that will look like: + ```yaml {__name__="PREFIX1_anomaly_score", for="PREFIX2_query_name_1", custom_label_1="label_name_1", custom_label_2="label_name_2", cpu=1, device="eth0", instance="node-exporter:9100"} {__name__="PREFIX1_yhat_lower", for="PREFIX2_query_name_1", custom_label_1="label_name_1", custom_label_2="label_name_2", cpu=1, device="eth0", instance="node-exporter:9100"} {__name__="PREFIX1_anomaly_score", for="PREFIX2_query_name_2", custom_label_1="label_name_1", custom_label_2="label_name_2", cpu=1, device="eth0", instance="node-exporter:9100"} {__name__="PREFIX1_yhat_lower", for="PREFIX2_query_name_2", custom_label_1="label_name_1", custom_label_2="label_name_2", cpu=1, device="eth0", instance="node-exporter:9100"} ``` - -<!-- -# TODO: uncomment and maintain after multimodel config refactor, 2nd priority - - -## NDJSON writer -Generates data in the same format as <code>/export</code>. - -### Config parameters -<table> - <thead> - <tr> - <th>Parameter</th> - <th>Example</th> - <th>Description</th> - </tr> - </thead> - <tbody> - <tr> - <td><code>class</code></td> - <td><code>"writer.ndjson.NdjsonWriter"</code></td> - <td>Name of the class needed to enable writing into JSON line format file.</td> - </tr> - <tr> - <td><code>path</code></td> - <td><code>"data/output.ndjson"</code></td> - <td>Path to file in JSON line format</td> - </tr> - <tr> - <td><code>metric_format</code></td> - <td><code>__name__: "vmanomaly_$VAR"</code></td> - <td>Metrics to save the output (in metric names or labels). Must have <code>__name__</code> key. Must have a value with <code>$VAR</code> placeholder in it to distinguish between resulting metrics. Supported placeholders: <code>$VAR</code>, <code>$QUERY_KEY</code> and others as configured by the user.</td> - </tr> - <tr> - <td><code>override</code></td> - <td><code>True</code></td> - <td>Override file flag. Default True</td> - </tr> - </tbody> -</table> - -Config example: -```yaml -writer: - class: "writer.ndjson.NdjsonWriter" - path: 'data/output.ndjson' - metric_format: - __name__: "$VAR" - for: "$QUERY_KEY" - config: "io_ndjson.yaml" -``` - - -## CSV writer - -### Config parameters -<table> - <thead> - <tr> - <th>Parameter</th> - <th>Type</th> - <th>Example</th> - <th>Description</th> - </tr> - </thead> - <tbody> - <tr> - <td><code>class</code></td> - <td>str</td> - <td><code>"writer.csv.CsvWriter"</code></td> - <td>Name of the class enabling CSV writer</td> - </tr> - <tr> - <td><code>header</code></td> - <td>bool</td> - <td><code>True</code></td> - <td>Whether to write header (column names). Default True</td> - </tr> - <tr> - <td><code>path</code></td> - <td>str</td> - <td><code>"data/jumpsup.csv"</code></td> - <td>Where to save the results</td> - </tr> - <tr> - <td><code>override</code></td> - <td>bool</td> - <td><code>True</code></td> - <td>Override file flag. Default True</td> - </tr> - <tr> - <td><code>tz</code></td> - <td>str</td> - <td><code>None</code></td> - <td>Optional. Convert default timestamps in UTC to desired timezone, e.g. 'US/Pacific'. By default local timezone is used</td> - </tr> - </tbody> -</table> -Config example: - -```yaml -writer: - class: "writer.csv.CsvWriter" - path: "data/io_csv_out.csv" - header: True - override: True -``` ---> \ No newline at end of file