Commit graph

506 commits

Author SHA1 Message Date
Aliaksandr Valialkin
fc42900eca
lib/pushmetrics: wait until the background goroutines, which push metrics, are stopped at pushmetrics.Stop()
Previously the was a race condition when the background goroutine still could try collecting metrics
from already stopped resources after returning from pushmetrics.Stop().
Now the pushmetrics.Stop() waits until the background goroutine is stopped before returning.

This is a follow-up for https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5549
and the commit fe2d9f6646 .

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5548
2024-01-16 23:18:29 +02:00
hagen1778
07fe834f54
app/all: follow-up after 84d710beab
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5548
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-01-16 23:08:21 +02:00
Hui Wang
186bfa9085
vmalert: validate schema for -external.url (#5450)
Requests with wrong or no schema in  `-external.url` could be rejected by alertmanager.
So we validate schema on start up.
2023-12-17 19:21:22 +02:00
Roman Khavronenko
bfdf9f1e71
app/vmalert: sanitize label names before sending to Alertmanager (#5442)
Before, vmalert would send notifications with labels containing characters
  not supported by Alertmanager validator, resulting into validation errors
  like `msg="Failed to validate alerts" err="invalid label set: invalid name "foo.bar"`

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-12-08 18:09:46 +02:00
Aliaksandr Valialkin
eea354f5bb
go.mod: increase the minimum supported Go version from Go1.19 to Go1.20 2023-11-02 21:26:29 +01:00
Aliaksandr Valialkin
65a37c0113
lib/license: cleanups and prettifications for log messages and docs related to licensing
- Make more clear the docs at docs/enterprise.md, so readers could figure out faster
  on how to obtain enterprise key and how to pass it to VictoriaMetrics Enterprise components.

- Fix examples at docs/enterprise.md, which were referring to non-existing `-license-file` command-line flag.
  The `-licenseFile` command-line flag must be used instead.

- Improve the description of `-license*` command-line flags, so users could understand
  faster how to use them.

- Improve the warning message, which is emitted when the deprecated -eula command-line flag is passed,
  so the user could figure out how to switch faster to -license* command-line flags.

- Disallow running VictoriaMetrics components with both -license and -licenseFile command-line flags.

- Disallow running VictoriaMetrics components when -licensFile points to an empty file.

- Consistently use the phrase "This flag is available only in Enterprise binaries" across
  all the enterprise-specific command-line flags.

- Remove unneeded level of indirection for `noLicenseMessage` and `expiredMessage` string contants
  in order to improve code readability and maintainability.

- Remove unneded `return` statements after `logger.Fatalf()` calls, since these calls exit the app and never return.

- Make sure that the info log message about successful license verification is emitted
  when the license is verified successfully. Previously the error message could be logged
  when the license payload is invalid or if it misses some required features.
2023-10-16 13:18:08 +02:00
Dmytro Kozlov
50ebc3a749
app/vmalert: hide sensetive info in the vmalert (#5059)
Strip sensitive information such as auth headers or passwords from datasource, remote-read,
remote-write or notifier URLs in log messages or UI. This behavior is by default and is controlled via
`-datasource.showURL`, `-remoteRead.showURL`, `remoteWrite.showURL` or `-notifier.showURL` cmd-line flags.

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5044
2023-10-16 01:42:30 +02:00
Roman Khavronenko
cad2c28c6d
vmalert: correctly add duplicated params to the query (#4955)
Fix the bug when Group's `params` fields with multiple values were
overriding each other instead of adding up.
The bug was introduced in this commit eccecdf177
 starting from v1.91.1 https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.91.1

 https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4908

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-09-08 23:34:56 +02:00
Aliaksandr Valialkin
761510c054
Makefile: update golangci-lint from v1.51.2 to v1.54.2
See https://github.com/golangci/golangci-lint/releases/tag/v1.54.2
2023-09-01 11:12:12 +02:00
Roman Khavronenko
0459aaf8f9
vmalert: correctly re-instantinate HTTP req on retries (#4864)
* vmalert: correctly re-instantinate HTTP req on retries

Previosly, request retry to datasource re-used existing HTTP request.
But if request object was already partially processed (body was read),
then retry will be unsuccessful.

The change re-instantinates HTTP request object before retry.

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* vmalert: review fix

Signed-off-by: hagen1778 <roman@victoriametrics.com>

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-08-28 09:56:59 +02:00
Abirdcfly
835c03fb47
vmalert: fix vmalert_remotewrite_send_duration_seconds_total metric value (#4801)
The deferred call's arguments are evaluated immediately, but the function call is not executed until the surrounding function returns.

Signed-off-by: Abirdcfly <fp544037857@gmail.com>
2023-08-10 10:51:44 +08:00
hagen1778
bc5065fd14
vmalert: mention vmalert_iteration_duration_seconds metric in README
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-08-09 22:03:22 +02:00
Haleygo
262c888d4b
vmalert: fix redundant clean up move (#4803)
Follow-up after 55ae2c2d57
2023-08-09 21:16:00 +02:00
Roman Khavronenko
1d4a0796f4
vmalert: cleanup config reload metrics handling (#4790)
* rename `configErr` to `lastConfigErr` to reduce confusion
* add tests to verify metrics and msg are set properly
* fix mistake when config success metric wasn't restored after an error

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-08-07 21:58:40 +02:00
Haleygo
527cfa2f74
vmalert: fix uncleaned tmp files in tests (#4788) 2023-08-07 14:24:56 +02:00
Zakhar Bessarab
2290252119
docs: make phrase about dedup and evaluation interval relation less obscure (#4781)
Value of `-dedup.minScrapeInterval` comand-line flag must be higher than `evaluation_interval` in order to make sure that only one sample on each evaluation will be left after deduplication.
Moreover, value of `-dedup.minScrapeInterval` must be a multiple of vmalert's `evaluation_interval` in order to make sure that samples will be aligned between deduplication window periods.

See: https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4774#issuecomment-1663940811

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2023-08-03 17:04:19 +02:00
Roman Khavronenko
216d4091f7
vmalert: remove deprecated in v1.79.0 web links with */status suffix (#4747)
Links of form `/api/v1/<groupID>/<alertID>/status` were deprecated
in favour of `/api/v1/alerts?group_id=<>&alert_id=<>` links in
v1.79.0. See more details here https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2825

This change removes code responsible for deprecated functionality.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-07-31 16:51:41 +02:00
hagen1778
3b524f671b
docs: rm typo of naming vmalert as a stateless service
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-07-31 16:47:50 +02:00
Roman Khavronenko
9ede3e996b
vmalert: remove deprecated in v1.61.0 -rule.configCheckInterval (#4745)
Use `-configCheckInterval` command-line flag instead.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-07-31 16:39:57 +02:00
Aliaksandr Valialkin
fcf5e33e6a
app/vmalert: use proper timestamp in setConfigSuccess() 2023-07-28 11:15:58 -07:00
Roman Khavronenko
9f1b9b86cc
vmalert: revert unittest feature (#4734)
* Revert "vmalert: unittest support stale datapoint (#4696)"

This reverts commit 0b44df7ec8.

* Revert "docs: specify min version and limitations for vmalert's unit tests"

This reverts commit a24541bd

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* Revert "vmalert: init unit test (#4596)"

This reverts commit da60a68d

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* docs: mention unittest revert in changelog

Signed-off-by: hagen1778 <roman@victoriametrics.com>

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-07-28 10:42:02 +02:00
Aliaksandr Valialkin
f8d30a486e
docs/vmalert.md: fix broken links to Web chapter 2023-07-27 18:05:01 -07:00
Aliaksandr Valialkin
6db8f26e2f
app/vmalert: make golangci-lint happy after ae0e4a8c90 2023-07-27 13:27:20 -07:00
Haleygo
ae0e4a8c90
vmalert: add keep_firing_for field for alerting rule (#4669)
vmalert: support `keep_firing_for` field for alerting rule

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4529

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
2023-07-27 15:13:13 +02:00
hagen1778
03f04afeda
vmalert: clarify docs for state restore with additional details
The important change is to highlight that restore procedure happens
only once and only for already loaded rules. Config hot-reload
doesn't trigger the restore procedure.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-07-27 11:29:17 +02:00
hagen1778
7fe81fe612
vmalert: revert accidental changes to Makefile rule
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-07-27 11:08:44 +02:00
Aliaksandr Valialkin
cc8427f11b
docs: use 1. instead of N. in numbered bullets, so they are automatically adjusted by Github Markdown engine
See https://docs.github.com/en/get-started/writing-on-github/getting-started-with-writing-and-formatting-on-github/basic-writing-and-formatting-syntax#lists
2023-07-26 14:39:44 -07:00
Aliaksandr Valialkin
56c17d16f6
app/vmalert/datasource: substitute golang.org/x/exp/slices.SortFunc with sort.Slice
This removes unnecessary third-party dependency on golang.org/x/exp.

This is a follow-up for da60a68d09
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2945
2023-07-24 18:58:01 -07:00
Haleygo
0b44df7ec8
vmalert: unittest support stale datapoint (#4696)
* vmalert: unittest support stale datapoint

* add stale ut case
2023-07-24 15:40:14 +02:00
Zakhar Bessarab
866b150f0f
app/vmalert/datasource/graphite: allow overriding "from" parameter for datasource queries (#4687)
* app/vmalert/datasource/graphite: allow overriding "from" parameter for datasource queries

Fixes construction of URL parameters for graphite render to allow overriding "from" parameter.

See: https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4685
Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* app/vmalert/datasource/graphite: update flow for building URL parameters

Makes flow of building URL parameters same as Prometheus datasource has:
1) Setting all default values
2) Merging those values with provided `extraParams`

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* Update docs/CHANGELOG.md

Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>

---------

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
2023-07-21 14:28:10 +04:00
hagen1778
b9b1661319
docs: fix the next release version for vmalert
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-07-20 18:48:21 +02:00
hagen1778
a24541bdb7
docs: specify min version and limitations for vmalert's unit tests
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-07-20 15:32:49 +02:00
Haleygo
da60a68d09
vmalert: init unit test (#4596)
vmalert: support unit tests

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2945 
---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
2023-07-20 15:07:10 +02:00
Aliaksandr Valialkin
d25082cbd9
app/vmalert/README.md: sync with docs/vmalert.md after 54b7bd4564 2023-07-19 16:30:21 -07:00
Roman Khavronenko
c32a01c52e
docs: follow-up after aec4b5db81 (#4638)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-07-19 10:10:51 +02:00
Roman Khavronenko
25317b4e70
vmalert: follow-up after d4ac4b7813 (#4659)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-07-18 15:53:37 +02:00
venkatbvc
d4ac4b7813
vmalert: allow to blackhole alerting notifications (#4639)
vmalert: support option to blackhole alerting notifications

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4122

---------

Co-authored-by: Rao, B V Chalapathi <b_v_chalapathi.rao@nokia.com>
2023-07-18 15:06:19 +02:00
Haleygo
b002e2a743
vmalert: fix evalTS after modify group interval (#4629) 2023-07-14 14:45:24 +02:00
Aliaksandr Valialkin
4f6dc25c71
app/vmalert: silence golagci-lint at TestAlertingRule_Template
Add a break if gotAlert is nil

This removes the following golangci-lint warning:

app/vmalert/alerting_test.go:868:8: SA5011(related information): this check suggests that the pointer can be nil (staticcheck)
				if gotAlert == nil {
				   ^
2023-07-13 11:40:44 -07:00
Roman Khavronenko
cbc28ccdb2
vmalert: check for negative offset for missed rounds (#4628)
It could happen for low evaluation intervals and irregular
delays during execution that evaluation time would get
a negative offset. This could result into cumulative
discrepancy between the actual time and evaluation time for rules.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-07-13 17:11:22 +02:00
Zakhar Bessarab
51a9cc9783
docs: make httpAuth.* flags description less ambiguous (#4588)
* docs: make `httpAuth.*` flags description less ambiguous

Currently, it may confuse users whether `httpAuth.*` flags are used by HTTP client or server configuration(see https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4586 for example).

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* docs: fix a typo

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

---------

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2023-07-07 13:50:13 +02:00
Haleygo
bca8ae034f
vmalert:fix query request using rfc3339 format (#4577)
vmalert: consistently use time.RFC3339 format for time in queries

Co-authored-by: hagen1778 <roman@victoriametrics.com>
2023-07-07 10:39:25 +02:00
Aliaksandr Valialkin
9fa8e3895a
docs/vmalert.md: update -help output 2023-07-06 23:13:04 -07:00
Roman Khavronenko
7c8a215a7c
vmalert: allow disabling of step param attached to instant queries (#4574)
vmalert: allow disabling of `step` param attached to instant queries

This might be useful for using vmalert with datasources that to not support this param,
unlike VictoriaMetrics.

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4573

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-07-07 07:44:34 +02:00
Dmytro Kozlov
9bde95bfff
app/vmalert: show on UI groups error after reload config (#4543)
show on UI groups error after reload config

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4076

Co-authored-by: hagen1778 <roman@victoriametrics.com>
2023-07-03 14:59:52 +02:00
Haleygo
a97887a2d9
vmalert: add vmalert_remotewrite_sent_duration_seconds_total metric (#4517)
add `vmalert_remotewrite_sent_duration_seconds_total` metric
2023-06-26 07:34:51 +02:00
Roman Khavronenko
37c9a631ca
vmalert: make linter happy (#4509)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-06-22 17:46:12 +02:00
Roman Khavronenko
5f9ad22884
vmalert: update retry policy for pushing data to -remoteWrite.url (#4504)
By default, vmalert will make multiple retry attempts with exponential delay.
The total time spent during retry attempts shouldn't exceed `-remoteWrite.retryMaxTime` (default is 30s).
When retry time is exceeded vmalert drops the data dedicated for `-remoteWrite.url`.
Before, vmalert dropped data after 5 retry attempts with 1s delay between attempts (not configurable).

See `-remoteWrite.retryMinInterval` and `-remoteWrite.retryMaxTime` cmd-line flags.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: Nikolay <nik@victoriametrics.com>
2023-06-22 15:14:23 +02:00
Roman Khavronenko
4aad7a43df
vmalert: properly interrupt remotewrite retries on shutdown (#4505)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-06-22 15:07:32 +02:00
Roman Khavronenko
94f516df43
docs/vmalert: specify version requirements for new features (#4480)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-06-21 09:44:00 +02:00