From 895cb3e7c60252c4bee3868a159dbfff31ead270 Mon Sep 17 00:00:00 2001 From: Roman Khavronenko Date: Wed, 12 Oct 2022 12:12:37 +0200 Subject: [PATCH] vmalert: update troubleshooting docs (#3228) The default value of `-datasource.queryStep` has changed, so we update the troubleshooting docs accordingly. Signed-off-by: hagen1778 --- app/vmalert/README.md | 7 ++++--- docs/vmalert.md | 7 ++++--- 2 files changed, 8 insertions(+), 6 deletions(-) diff --git a/app/vmalert/README.md b/app/vmalert/README.md index 902f1f42c..726715257 100644 --- a/app/vmalert/README.md +++ b/app/vmalert/README.md @@ -665,9 +665,10 @@ Try the following recommendations in such cases: are delivered to the datasource; * If you know in advance, that data in datasource is delayed - try changing vmalert's `-datasource.lookback` command-line flag to add a time shift for evaluations; -* If time intervals between datapoints in datasource are irregular - try changing vmalert's `-datasource.queryStep` -command-line flag to specify how far search query can lookback for the recent datapoint. By default, this value -is equal to group's evaluation interval. +* If time intervals between datapoints in datasource are irregular or `>=5min` - try changing vmalert's +`-datasource.queryStep` command-line flag to specify how far search query can lookback for the recent datapoint. +The recommendation is to have the step at least two times bigger than `scrape_interval`, since +there are no guarantees that scrape will not fail. Sometimes, it is not clear why some specific alert fired or didn't fire. It is very important to remember, that alerts with `for: 0` fire immediately when their expression becomes true. And alerts with `for > 0` will fire only diff --git a/docs/vmalert.md b/docs/vmalert.md index 5dbb7e3cd..e04ae678b 100644 --- a/docs/vmalert.md +++ b/docs/vmalert.md @@ -669,9 +669,10 @@ Try the following recommendations in such cases: are delivered to the datasource; * If you know in advance, that data in datasource is delayed - try changing vmalert's `-datasource.lookback` command-line flag to add a time shift for evaluations; -* If time intervals between datapoints in datasource are irregular - try changing vmalert's `-datasource.queryStep` -command-line flag to specify how far search query can lookback for the recent datapoint. By default, this value -is equal to group's evaluation interval. +* If time intervals between datapoints in datasource are irregular or `>=5min` - try changing vmalert's +`-datasource.queryStep` command-line flag to specify how far search query can lookback for the recent datapoint. +The recommendation is to have the step at least two times bigger than `scrape_interval`, since +there are no guarantees that scrape will not fail. Sometimes, it is not clear why some specific alert fired or didn't fire. It is very important to remember, that alerts with `for: 0` fire immediately when their expression becomes true. And alerts with `for > 0` will fire only