Commit graph

1667 commits

Author SHA1 Message Date
Roman Khavronenko
113301308a
vmalert: mention how to build a custom image (#2626)
Thanks to @f41gh7

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-05-23 00:59:34 +02:00
Roman Khavronenko
2cf586da78
vmalert: add new metric vmalert_iteration_interval_seconds (#2623)
The new metric shows the configured evaluation interval per group.
Metric updates its value when group's interval is changed during
hot reload.
The new metric can be used to estimate how close group
is to start missing evaluation rounds. The following query
will show the % of used time by the group to evaluate all rules
before the next round:
```
(max(vmalert_iteration_duration_seconds{quantile="0.99"}) / vmalert_iteration_interval_seconds) * 100
```

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2618
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-05-20 17:31:16 +02:00
Aliaksandr Valialkin
1731fe2ada
docs: update the description for command-line flags according to recent changes 2022-05-20 15:09:43 +03:00
Aliaksandr Valialkin
c6d543e2f9
app/vmselect/vmui: make vmui-update 2022-05-20 14:53:38 +03:00
Boris Petersen
3df8caca15
Add ability to sign requests for all AWS services (#2604)
This adds the ability to utilize sigv4 signing for all AWS services not
just "aps". When the newly introduced property "service" is not set it
will default to "aps".

Signed-off-by: Boris Petersen <boris.petersen@idealo.de>
2022-05-18 14:58:31 +02:00
Roman Khavronenko
5111d850e2
vmalert: remove a line added for debug (#2611)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-05-18 14:57:58 +02:00
Roman Khavronenko
34116882b4
vmalert: support scalar type in response (#2610)
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2607

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-05-18 09:50:46 +02:00
Roman Khavronenko
1fad4dc919
vmalert: support strings in humanize.* templates (#2606)
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2569

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-05-17 15:38:54 +02:00
Yurii Kravets
5c42c1218a
Update vmalert.md (#2580)
docs: update vmalert/README.md

Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
2022-05-17 14:14:18 +02:00
Yurii Bychenok
b7536f2a0a
Updated vmctl documentation, migration from OpenTSDB section (#2595)
Co-authored-by: Yurii Bychenok <ipeacocks@pm.me>
2022-05-17 13:20:35 +02:00
Yury Molodov
c97c1fc1bf
vmui: optimize data fetching (#2584) 2022-05-17 13:13:45 +02:00
Yury Molodov
fcf4190d0b
vmui: add option to customize url params for individual pages (#2582) 2022-05-17 13:13:15 +02:00
Roman Khavronenko
b74c001c92
vmalert: support /rules path for Grafana's ngalert requests (#2593)
Unexpectedly, Grafana makes an extra request to `/rules`
handler in addition to `/api/v1/rules` calls in alerts UI.
This happens only for Grafana versions older than 8.5.*.
Apparently, this is related to support of other monitoring
systems.
Prometheus responds with `text/html` content for UI page `/rules`
to such requests. Actually, returning just a blank page with
SC=200 works as well.

Returning actual response of `/api/v1/rules`
results in error in Grafana since it expects a `yaml` (?) in response.
So we add a placeholder to `vmalert`.

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2583
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-05-16 10:00:24 +02:00
Yury Molodov
4e6b483ef1
fix: change get display type (#2553) 2022-05-16 09:44:13 +02:00
Yury Molodov
ff74472621
vmui: setup predefined dashboards without build (#2541)
vmui: support predefined dashboards in json format

See https://github.com/VictoriaMetrics/VictoriaMetrics/tree/master/app/vmselect/vmui/dashboards
2022-05-16 09:42:37 +02:00
Roman Khavronenko
284bda8746
docs: fix liquid syntax errors (#2592)
For liquid text processor double braces `{{` `}}`
are special chars for templating.
Since we use them in some of our docs with different purpose,
we must escape them to avoid syntax errors from liquid.

For escaping curly braces we use bult-in plugin which helps
to enclose sections of text via `{% raw %}` and `{% endraw %}`.
This approach prevents liquid syntax errors and makes render correct.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-05-16 09:27:19 +02:00
Dima Lazerka
8402231d40 Force up nth-check version 2022-05-15 00:35:20 +03:00
Roman Khavronenko
0d07166eed
vmalert: fix readme formatting (#2587)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-05-14 19:29:09 +02:00
Roman Khavronenko
9bc03f6b04
vmalert: follow-up after 0ac1cdfff5 (#2586)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-05-14 18:56:31 +02:00
Andrii Chubatiuk
a531a96193
added reusable templates support (#2532)
Signed-off-by: Andrii Chubatiuk <andrew.chubatiuk@gmail.com>
2022-05-14 11:38:44 +02:00
Aliaksandr Valialkin
ce47d18052
app/{vmagent,vminsert}: mention port 8089 instead of 8189 in the description for -influxListenAddr flag
InfluxDB uses 8089 port for sending plain Influx line protocol data over TCP and UDP.
See https://docs.influxdata.com/influxdb/v1.8/administration/ports/

This is a follow-up for 20cef877a1
2022-05-13 16:50:45 +03:00
Aliaksandr Valialkin
c448d2fcbb
app/vmalert: apply -remoteRead.disablePathAppend to -datasource.url in the same way as for the -remoteRead.url
This is a follow-up for 0e2486df56

The related pull requests:
- https://github.com/VictoriaMetrics/VictoriaMetrics/pull/1536
- https://github.com/VictoriaMetrics/VictoriaMetrics/pull/1712
2022-05-13 16:44:43 +03:00
Roman Khavronenko
3f0ecee128
vmalert: properly cleanup stale series tracker on rules update (#2577)
Rules executor within group tracks series sent to remote write
in order to mark them as stale if they had disappeared in next
evaluation round.
The executor uses rules ID as a key to identifies series which belong to rule.
On config reload, executor remains active but the set of rules could change.
Hence, we need to properly cleanup the tracker for rules which has been disappeared
on config reload.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-05-13 10:04:49 +02:00
Roman Khavronenko
071f7c24d4
vmctl: make linter happy (#2578)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-05-13 09:55:27 +02:00
Yurii Kravets
20cef877a1
Update docs (#2566)
* deployment/docker: pass `-buildvs=false` to `go build` for production builds

This should resolve the `error obtaining VCS status: exit status 128` error
when the environment contains incorrect version of git or has incorrect access rights
to the directory with VictoriaMetrics source code.

See the following links for additional info:
- https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2508#issuecomment-1117126702 ,
- https://github.com/google/ko/issues/672
- https://github.com/golang/go/issues/49004

* lib/netutil: limit the number of concurrently established connections when calling ConnPool.Get()

This should reduce potential spikes in the number of established connections in the following cases:
- when the connection establishing procedure becomes temporarily slow
- after a temporary spike in the rate of ConnPool.Get() calls

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2552

* docs/CHANGELOG.md: document c8af625bcc

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1322#issuecomment-1120276146

* docs/Cluster-VictoriaMetrics.md: typo fix: `by by` -> `by`

* docs: add `resource usage limits` docs, which describe fine-grained tuning for various resource usage limits

* docs/Cluster-VictoriaMetrics.md: the `/api/v1/label/.../values` query can take CPU and ram at both vmstorage and vmselect

* Update root Readme and root vmagent readme

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-05-12 19:22:13 +02:00
Aliaksandr Valialkin
a0727ab1b1
docs/vmagent.md: typo fix in the description for -promscrape.cluster.replicationFactor command-line flag 2022-05-12 18:50:29 +03:00
Dmytro Kozlov
c8af625bcc
vmctl: fix build for solaris os (#2555)
* vmctl: fix build for solaris os

* vmctl: updated dependency (using Syscall instead of Syscall6)

* vmctl: updated dependency

* vmctl: updated dependency
2022-05-09 21:36:18 +02:00
Aliaksandr Valialkin
a7f18f8cb2
app/vmselect/promql: do not return values from label_value() if the original time series has no values at the selected timestamps 2022-05-09 17:57:39 +03:00
Roman Khavronenko
331a5d9a17
Code check (#2558)
* vmstorage: make gofmt happy

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* vmalert: make linter happy

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-05-09 10:11:56 +02:00
Roman Khavronenko
e9fa363480
Vmalert fix bugs in alerting evaluation (#2557)
* vmalert: calculate time for firing alert based on the given timestamp

Previously, current time was used for checking the `firing` threshold.
This is not correct, since alerts are evaluated at specific timestamps.
Hence, this specific timestamp supposed to be used in the calculation.

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* vmalert: properly calculate evaluation timestamp for rules

Timestamp for rules evaluation should be calculated after
the artifical delay for groups start. Otherwise, evaluation
timestamp can fall back too far in time.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-05-09 10:11:06 +02:00
Artem Navoiev
3f78a609ac
docs: add flags list to vmbackupmanager (#2554)
docs: add flags list to vmbackupmanager docs

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: tenmozes <artem@victoriametrics.com>
2022-05-08 12:18:39 +00:00
Marc Hörsken
8f4f5f1d68
app/vmctl: add flag to handle Prometheus remote_write to InfluxDB (#2545)
Make it possible to migrate timeseries while restoring the
original timeseries name previously written from Prometheus
to InfluxDB v1 via remote_write.

Fixes: https://github.com/VictoriaMetrics/vmctl/issues/8
2022-05-07 19:52:42 +00:00
Aliaksandr Valialkin
d6ad8d090d
app/vmstorage: do not allow to set -retentionPeriod smaller than one day
VictoriaMetrics doesn't support retention periods smaller than one day,
so do not allow to set it to small values.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2496
2022-05-07 00:53:01 +03:00
Marc Hörsken
e726340914
app/vmctl: add flag to skip adding the InfluxDB 'db' label (#2544)
Make it possible to migrate timeseries without changing labels
at all, including not adding the now optional 'db' label.
2022-05-06 18:06:54 +03:00
Dmytro Kozlov
9a63f6c1b8
vmbackup: Prevent save backups to the same folder where TSDB data is (#2547)
* {vmbackup, vmbackup/snapshot}: validate snapshot name

* vmbackup/snapshot: added another checks

* backup/actions: added check that we ignore backup_complete.ignore file

* vmbackup: moved snapshot to lib directory

* lib/snapshot: added functions description

* lib/snapshot: fixed typo

* vmbackup: code cleanup

* wip

* vmbackup: Prevent save backups to the same folder where TSDB data is

* Apply suggestions from code review

* wip

* wip

* wip

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-05-06 18:04:09 +03:00
Roman Khavronenko
20ccf0ba81
vmctl: add tip about safety flags during for native data export (#2540)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-05-06 15:12:29 +02:00
Aliaksandr Valialkin
8d81703635
app/vmagent: add missing _total suffix to vmagent_remotewrite_global_rows_pushed_before_relabel_total counter
This is a follow up for c536139d0b
2022-05-06 15:50:57 +03:00
Aliaksandr Valialkin
c536139d0b
app/vmagent: expose vmagent_remotewrite_global_rows_pushed_before_relabel and vmagent_remotewrite_rows_pushed_after_relabel_total metrics 2022-05-06 15:28:59 +03:00
Aliaksandr Valialkin
51e36fd533
app/vmagent: rename vmagent_remote_write_rate_limit_reached_total to vmagent_remotewrite_rate_limit_reached_total for the sake of consistency with other vmagent_remotewrite_ metrics 2022-05-06 15:01:54 +03:00
Aliaksandr Valialkin
ca4ca4630b
app/vmselect/vmui: make vmui-update after 450d879eaa 2022-05-05 21:26:01 +03:00
Yury Molodov
450d879eaa
vmui: prevent reset relative time (#2543)
* fix: prevent time picker reset to previous time

* fix: add default display type
2022-05-05 21:21:02 +03:00
Yury Molodov
a580efa26a
fix: remove react @types (#2539) 2022-05-05 21:17:41 +03:00
Roman Khavronenko
13efaa42d5
vmstorage: switch to rich duration parser for flag snapshotsMaxAge (#2542)
The switch suppose to allow setting `d`, `w`, `y` duration units.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-05-05 18:43:21 +02:00
Marc Hörsken
bf5e3774cc
app/vmctl: fix empty/skipped labels after db label (#2536)
Do not assume the db label to be the last one and also
make sure we are not skipping it and everything afterwards.
Breaking the loop would cause following labels to be empty.
2022-05-05 13:35:08 +03:00
Aliaksandr Valialkin
d285c2fea7
lib/awsapi: pass filtersQueryString arg to GetEC2APIResponse() function, so the caller could decide whether to use the filters during the AWS API query
The filters shouldn't be passed to DescribeAvailabilityZones API call.
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1626
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1287

Related commits:
0e09fdb8b0
d289ecded1
2022-05-05 10:29:34 +03:00
Aliaksandr Valialkin
c9e57d8000
app/vmstorage/main.go: reduce the difference with cluster version 2022-05-04 23:56:19 +03:00
Aliaksandr Valialkin
fc7c7237e3
app/vmselect: follow-up after 8639e79d38 2022-05-04 23:35:57 +03:00
Aliaksandr Valialkin
2c037ae0d3
docs/vmbackup.md: added missing -storageDataPath argument in the command for creating daily backups 2022-05-04 22:49:37 +03:00
Aliaksandr Valialkin
910f715ffe
docs/vmbackup.md: acutalize docs
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2503
2022-05-04 22:44:15 +03:00
Dmytro Kozlov
7dd9f3b98e
{vmbackup, vmbackup/snapshot}: fixed problem with snapshot backup in another snapshot folder (#2535)
* {vmbackup, vmbackup/snapshot}: validate snapshot name

* vmbackup/snapshot: added another checks

* backup/actions: added check that we ignore backup_complete.ignore file

* vmbackup: moved snapshot to lib directory

* lib/snapshot: added functions description

* lib/snapshot: fixed typo

* vmbackup: code cleanup

* wip

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-05-04 22:12:03 +03:00