Commit graph

3497 commits

Author SHA1 Message Date
Denys Holius
0bca7d49b9
docs/Quick-Start.md: adds missed command to 'Starting VM-Cluster via Docker' section (#4375) 2023-05-31 16:39:38 +02:00
Nikolay
f263031fe9
app/vmauth: properly handle LOCAL proxy protocol command (#4373)
app/vmauth: properly handle LOCAL proxy protocol command

It is required for handling health checks from load balancers

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3335
2023-05-31 15:37:59 +02:00
Roman Khavronenko
a0c040ea58
docs: mention disk space requirements for downsampling (#4369)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-05-31 11:08:18 +02:00
Roman Khavronenko
1b24f4729b
vmalert: mention default value for external.url flag (#4365)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-05-30 12:33:17 +02:00
Artem Navoiev
a8c58fc145
docs: fix enterprise page title to match with official name
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2023-05-29 22:39:56 +02:00
Artem Navoiev
5b0e8c797c
docs: fix markdown headers ordering
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2023-05-29 22:35:05 +02:00
Artem Navoiev
83c1944184
docs: fix markdown for title
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2023-05-29 22:22:25 +02:00
Artem Navoiev
6967c3cb95 fix image path
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2023-05-29 07:11:15 -07:00
Artem Navoiev
8a4c89ea22 docs: change images from markdown tag to html for migration
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2023-05-29 07:11:15 -07:00
Roman Khavronenko
51cea6cad4
vmalert: properly form assets address if httpPrefix set (#4351)
Properly form path to static assets in WEB UI
 if `http.pathPrefix` set.

 https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4349

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-05-29 07:38:13 +02:00
Roman Khavronenko
1c3f50f791
docs: mention multi-tenancy in docs (#4357)
The update should make understanding of multi-tenancy more clear
for influxdb users.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-05-29 07:37:23 +02:00
Artem Navoiev
f791811b15 update docs-sync Makefile command, add hugo front-matter
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2023-05-26 01:23:04 -07:00
Artem Navoiev
434358b551 remove absolute links, fix aliase
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2023-05-25 14:35:21 -07:00
Artem Navoiev
c1ad48edf9
fix title in multi-regional-setup-dedicated-regions.md
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2023-05-25 23:14:29 +02:00
Aliaksandr Valialkin
388ffec262
docs/CHANGELOG.md: document v1.79.13 LTS release 2023-05-18 19:56:28 -07:00
Aliaksandr Valialkin
9e21315315
docs/CHANGELOG.md: document v1.87.6 LTS release 2023-05-18 17:50:12 -07:00
Aliaksandr Valialkin
9fa052c302
deployment/docker: update VictoriaMetrics version from v1.90.0 to v1.91.0 in docker compose files 2023-05-18 15:34:21 -07:00
Aliaksandr Valialkin
f0ed9ab4f2
docs/Release-Guide.md: fix a link to sandbox environment
The link became broken after 5f9d3f9cb5
2023-05-18 15:30:07 -07:00
Aliaksandr Valialkin
52b5498165
docs/CHANGELOG.md: cut v1.91.0 2023-05-18 12:37:12 -07:00
Aliaksandr Valialkin
1f28b46ae9
lib/storage: revert the migration from global to per-day index for (MetricName -> TSID)
This reverts the following commits:
- e0e16a2d36
- 2ce02a7fe6

The reason for revert: the updated logic breaks assumptions made
when fixing https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2698 .
For example, if a time series stop receiving new samples during the first
day after the indexdb rotation, there are chances that the time series
won't be registered in the new indexdb. This is OK until the next indexdb
rotation, since the time series is registered in the previous indexdb,
so it can be found during queries. But the time series will become invisible
for search after the next indexdb rotation, while its data is still there.

There is also incompletely solved issue with the increased CPU and disk IO resource
usage just after the indexdb rotation. There was an attempt to fix it, but it didn't fix
it in full, while introducing the issue mentioned above. See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1401

TODO: to find out the solution, which simultaneously solves the following issues:
- increased memory usage for setups high churn rate and long retention (e.g. what the reverted commit does)
- increased CPU and disk IO usage during indexdb rotation ( https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1401 )
- https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2698

Possible solution - to create the new indexdb in one hour before the indexdb rotation
and to gradually pre-populate it with the needed index data during the last hour before indexdb rotation.
Then the new indexdb will contain all the needed data just after the rotation,
so it won't trigger increased CPU and disk IO.
2023-05-18 11:30:49 -07:00
Aliaksandr Valialkin
4f7f750850
lib/handshake: do not pollute logs with cannot read hello messages on TCP health checks
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1762
2023-05-18 10:41:34 -07:00
Dmytro Kozlov
7cbda6796c
app/vmctl: set default value for --vm-native-step-interval flag (#4327)
* app/vmctl: set default value for `--vm-native-step-interval` flag

* app/vmctl: update CHANGELOG.md

* app/vmctl: update CHANGELOG.md, fix docs

* app/vmctl: fix typo

* app/vmctl: fix typo
2023-05-18 13:43:35 +02:00
Denys Holius
c605d64a95
deployment/docker/Makefile: updated docker compose commands regarding migration from V1 to V2 (#4314)
deployment/docker/Makefile: updated docker compose commands regarding migration from V1 to V2
2023-05-17 13:14:24 +02:00
Aliaksandr Valialkin
63b1cab454
app/vmauth: simplify the code after 4a1d29126c
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4242
2023-05-17 00:37:05 -07:00
Nikolay
4a1d29126c
app/vmauth: retry common network dial errors (#4280)
with tracking request body read calls
it allows us to retry POST and PUT requests
2023-05-17 00:19:33 -07:00
Nikolay
16df18ec14
app/vmauth: do not return invalid credentials (#4288)
at http response by default
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4188

based on https://github.com/VictoriaMetrics/VictoriaMetrics/pull/4190
Thanks @raj-kumar-j  for init implementation
2023-05-17 00:09:47 -07:00
Aliaksandr Valialkin
e0e16a2d36
lib/storage: follow-up after 2ce02a7fe6
- Document the change at docs/CHANGELOG.md
- Clarify comments for non-trivial code touched by the commit
- Improve the logic behind maybeCreateIndexes():
  - Correctly create per-day indexes if the indexdb rotation is performed during
    the first hour or the last hour of the day by UTC.
    Previously there was a possibility of missing index entries on that day.
  - Increase the duration for creating new indexes in the current indexdb for up to 22 hours
    after indexdb rotation. This should reduce the increased resource usage
    after indexdb rotation.
    It is safe to postpone index creation for the current day until the last hour
    of the current day after indexdb rotation by UTC, since the corresponding (date, ...)
    entries exist in the previous indexdb.
- Search for TSID by (date, MetricName) in both the current and the previous indexdb.
  Previously the search was performed only in the current indexdb. This could lead
  to excess creation of per-day indexes for the current day just after indexdb rotation.
- Search for (date, metricID) entries in both the current and the previous indexdb.
  Previously the search was performed only in the current indexdb. This could lead
  to excess creation of per-day indexes for the current day just after indexdb rotation.
2023-05-16 23:19:27 -07:00
Artem Navoiev
39ba4fc1c4
update logo width in cluster doc to 300
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2023-05-16 15:33:15 -07:00
Aliaksandr Valialkin
278278af95
lib/storage: reduce the unimportant logging during Storage start / stop
This should improve the visibility of potentially important logs
2023-05-16 15:14:21 -07:00
Aliaksandr Valialkin
664db964ca
vendor: update github.com/VictoriaMetrics/metrics from v1.23.1 to v1.24.0
This change adds process_* metrics to VictoriaMetrics components under Windows OS

See https://github.com/VictoriaMetrics/metrics/pull/47
2023-05-16 11:37:07 -07:00
Aliaksandr Valialkin
60bdbdaa70
docs/vmbackupmanager.md: run make docs-sync after c7d8dda39225b716ea44df7223db5e4a125d407b 2023-05-16 11:27:42 -07:00
Alexander Marshalov
7b15834cbe
added backup locking/unlocking against retention policy to vmbackupmanager (#558)
* added backup locking/unlocking against retention policy to vmbackupmanager

Signed-off-by: Alexander Marshalov <_@marshalov.org>

* added docs for new commands

Signed-off-by: Alexander Marshalov <_@marshalov.org>

* fix review comments

Signed-off-by: Alexander Marshalov <_@marshalov.org>

---------

Signed-off-by: Alexander Marshalov <_@marshalov.org>
2023-05-16 11:23:36 -07:00
Roman Khavronenko
f68d93cca2
vmalert: follow-up after 669becd011 (#4318)
* vmalert: follow-up after 669becd011

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* vmalert: follow-up after 669becd011

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* vmalert: follow-up after 669becd011

Signed-off-by: hagen1778 <roman@victoriametrics.com>

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-05-16 18:51:38 +02:00
Roman Khavronenko
686bf6c5ad
vmctl: update VictoriaMetrics migration section (#4310)
Remove unnecessary information to simplify the description and tips.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-05-15 16:15:01 +02:00
Artem Navoiev
478258bc4d
fix link in operator quick start docs.2
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2023-05-14 21:20:57 +02:00
Artem Navoiev
c19fc52676
fix link in operator quick start docs
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2023-05-14 19:54:08 +02:00
Aliaksandr Valialkin
2613110f75
deployment/docker: update base docker image from 3.17.3 to 3.18.0
See https://www.alpinelinux.org/posts/Alpine-3.18.0-released.html
2023-05-12 17:31:21 -07:00
Aliaksandr Valialkin
616175b1ce
lib/promutils: properly return error when incorrect Prometheus label names are passed to NewLabelsFromString()
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4284
See also https://github.com/VictoriaMetrics/VictoriaMetrics/pull/4304
2023-05-12 16:52:29 -07:00
Aliaksandr Valialkin
318a87c36f
Revert "lib/promrelabel: show error message if labels not in prometheus exposition format (#4304)"
This reverts commit 193a9c3328.

Reason for revert: the commit doesn't fix the real issue with promutils.NewLabelsFromString()
function, which must return error when improperly formatted Prometheus metric with labels is passed to it.
See https://github.com/prometheus/docs/blob/main/content/docs/instrumenting/exposition_formats.md#text-format-example

E.g. the promutils.NewLabelsFromString() must return error when the following strings are passed to it:

- `{foo:"bar"}`, since `:` is disallowed in Prometheus text exposition format. The corect value is `{foo="bar"}`
- `{"foo":"bar"}`, since label name shouldn't be quoted. The correct value is `{foo="bar"}`.

The reverted commit introduces another set of bugs, which happily accept the following invalid input:

- `{foo=~"bar"}`
- `{foo!="bar"}`
- `{foo!~"bar"}`

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4284
See also https://github.com/VictoriaMetrics/VictoriaMetrics/pull/4304
2023-05-12 16:07:37 -07:00
Aliaksandr Valialkin
160453b86c
lib/protoparser/csvimport: properly parse the last empty column in CSV line
Do not ignore the last empty column in CSV line.
While at it, properly parse CSV columns in single quotes, e.g. `'foo,bar',baz` is parsed as two columns - `foo,bar` and `baz`

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4048

See also https://github.com/VictoriaMetrics/VictoriaMetrics/pull/4298
2023-05-12 15:51:41 -07:00
Aliaksandr Valialkin
b7fe7b801c
Revert "lib/protoparser: fix skip csv line when metric can be collect from the line (#4298)"
This reverts commit 410ae99c2e.

Reason for revert: the commit masks the real issue instead of fixing it.
The real issue is that the scanner.NextColumn() skips the last column if it is empty.

The commit also introduces two bugs:

- a panic if all the metric values in CSV line are empty
- silent import of CSV lines with too small number of columns

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4048
See https://github.com/VictoriaMetrics/VictoriaMetrics/pull/4298
2023-05-12 15:22:27 -07:00
Yury Molodov
f0fad01e8a
vmui: add notification for non-matching queries (#4301)
vmui: add notification for non-matching queries (#4211)

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4211
2023-05-12 13:55:47 +02:00
Roman Khavronenko
5a7159ab2e
docs: mention link to public relabeling playground (#4306)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-05-12 10:58:22 +02:00
Roman Khavronenko
ad2d079ba5
docs: update docs about VMUI pages (#4305)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-05-12 10:51:44 +02:00
Dmytro Kozlov
193a9c3328
lib/promrelabel: show error message if labels not in prometheus exposition format (#4304)
lib/promrelabel: show error message if labels not in prometheus exposition format

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4284
2023-05-12 10:42:56 +02:00
Dmytro Kozlov
410ae99c2e
lib/protoparser: fix skip csv line when metric can be collect from the line (#4298)
* lib/protoparser: fix skip csv line when metric can be collect from the line

* lib/protoparser: fix comment
2023-05-12 11:04:16 +03:00
Yury Molodov
1e4a9a8dfe
vmui: enhancements to top queries page (#4299)
* feat: improvement of the top queries page

* vmui/docs: enhancements to top queries page

* Apply suggestions from code review

---------

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2023-05-11 13:47:32 -07:00
Aliaksandr Valialkin
5d22c36904
docs/CHANGELOG.md: improve the description of the change at 7ea2531db0
Move the change description to the group of vmui changes.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4213
2023-05-11 13:30:33 -07:00
Aliaksandr Valialkin
73812c71a5
lib/promutils: properly parse time strings with timezones at ParseTime() 2023-05-11 13:24:00 -07:00
Yury Molodov
a55d3f6882
vmui: increase font-size and fix the text display (#4273)
vmui: change default font size to 14px for better readability
vmui: fix bug with missing text on buttons in safari

---------

Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
2023-05-11 14:50:09 +02:00
Dmytro Kozlov
7ea2531db0
app/vmui: added table where Labels with the highest number of unique values show (#4271)
* app/vmui: added Labels with the highest number of unique values

* app/vmui: cleanup

* app/vmui: cleanup

* app/vmui: add table description

* app/vmui: fix comment, updated CHANGELOG.md

* app/vmui: disable links

* app/vmui: added actions to the table, it will show values for selected label with the highest number of series

* app/vmui: fix comment
2023-05-11 15:19:36 +03:00
Aliaksandr Valialkin
86424e079e
docs/CHANGELOG.md: fix typo after 2caf0b05c6 2023-05-10 13:03:20 -07:00
Artem Navoiev
0ccb7f51dd
fix typo in changelog
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2023-05-10 19:23:24 +02:00
Roman Khavronenko
51196739af
Vmalert UI updates (#4276)
* vmalert: expand rule groups on anchor click

before, anchor click was only updating the URL.
To expand the group, user had to click on rule's block.
Now, group will toggle automatically.

* vmalert: allow filtering group in web UI

The new filter allows to filter groups and rules within
groups by: errors only or noMatch only.

The filtering supposed to help navigating big numbers of groups/rules.
Filtering is reflected in URL, so can be shared as a link.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-05-10 14:38:13 +02:00
Dmytro Kozlov
060a0cdbf4
docs: Add information about datasource plugin (#4266)
docs: Add information about datasource plugin
2023-05-10 10:26:57 +02:00
Alexander Marshalov
2e494e2375
fixed typos in documentation and commandline flags descriptions (#4275) 2023-05-10 09:50:41 +02:00
Aliaksandr Valialkin
4e0345a5ef
docs/CHANGELOG.md: add a link to docs about never-firing alerts
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4039
2023-05-09 21:58:36 -07:00
Aliaksandr Valialkin
78b23c9a83
docs/CHANGELOG.md: document 8f4de6fa47 2023-05-09 21:39:25 -07:00
Roman Khavronenko
491831df49
vminsert: properly reset labels object on aggregation (#4278)
Without reset, labels duplicates could have been added during stream aggregation.
Since `ctx.Labels` is reused during processing of many series, each series will
add its labels to the context. Even if the same labels were already addeded on prev
iteration. Now, we reset `ctx.Labels` on each iteration to contain so labels from
different series didn't interfere.

This could have cause exceeding of the limit on number of labels per pushed time series.

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4277

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-05-09 08:33:58 -07:00
Aliaksandr Valialkin
b9bb64ce55
lib/promscrape/discovery/consulagent: substitute metaPrefix with the __meta_consulagent_ plaintext string
This simplifies future code navigation and search for the specific meta-label starting from __meta_consulagent_* prefix.
For example, `grep __meta_consulagent_namespace` finds the exact place where this label is defined.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3953
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/4217
2023-05-08 23:40:13 -07:00
Aliaksandr Valialkin
83a1c2484c
docs/CHANGELOG.md: group changelog lines for tip release according to VictoriaMetrics apps 2023-05-08 22:57:23 -07:00
Aliaksandr Valialkin
11eb94d3bc
docs/CHANGELOG.md: document baf456978d
See https://github.com/VictoriaMetrics/VictoriaMetrics/pull/4223
2023-05-08 22:26:04 -07:00
Aliaksandr Valialkin
ff3c90d305
docs/managed-victoriametrics/overview.md: typo fix after 51fbf58d89 2023-05-08 21:58:40 -07:00
Aliaksandr Valialkin
fc42617ecd
docs/CHANGELOG.md: refer to the author and the pull request of the notifier_headers feature at vmalert
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3260
2023-05-08 17:18:48 -07:00
Aliaksandr Valialkin
5a02bc56fb
docs/CHANGELOG.md: document 03150c8973
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4204
2023-05-08 16:30:28 -07:00
Aliaksandr Valialkin
3c0470f91e
docs/vmalert.md: clarify docs regarding the support of recursive globs
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4041
2023-05-08 16:21:44 -07:00
Aliaksandr Valialkin
74155afb71
docs: clarify docs after 5ee344824f
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4183
2023-05-08 16:11:44 -07:00
Aliaksandr Valialkin
185894fc5a
app/vmagent/remotewrite: make more user-friendly the warning message about too small -remoteWrite.maxdiskUsagePerURL value
This is a follow-up for bc17f4828c .
While at it, document the change at docs/CHANGELOG.md .

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4195
2023-05-08 15:42:30 -07:00
Aliaksandr Valialkin
39530903bf
docs/managed-victoriametrics: consistently use setup-notifications prefix for images used at docs/managed-victoriametrics/setup-notifications.md
This should simplify managment of the images belonging to the given docs.
See docs/assets/README.md for details

This is a follow-up for 4052c44ac1
2023-05-08 15:29:53 -07:00
Aliaksandr Valialkin
d906e83e5e
app/vmauth: merge default_url example into multi-url example in order to reduce the amounts of text to read for the user
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4084

This is a follow-up for 041e188df8
2023-05-08 15:12:23 -07:00
Aliaksandr Valialkin
1db9b78b88
app/vmselect: small cleanup after 68e31a6000
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3811
2023-05-08 14:34:37 -07:00
Aliaksandr Valialkin
80946f06c2
app/{vmselect,vmctl}: move ParseTime() to lib/promutils
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4091

This is a follow-up for e2053baf32
2023-05-08 14:17:57 -07:00
Aliaksandr Valialkin
6601fa3e9d
docs/CHANGELOG.md: typo fix after 45a551df9c: 'this doc' -> 'this feature request' 2023-05-08 13:41:58 -07:00
Aliaksandr Valialkin
92a549bccb
app/vmauth/README.md: mention about ip filters and concurrency limiter at Security chapter 2023-05-08 13:35:58 -07:00
Aliaksandr Valialkin
23595465b8
app/vmauth: refer ip_filters option in example auth config
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3491
2023-05-08 13:29:18 -07:00
Aliaksandr Valialkin
8f43f496d7
docs: document IP filters functionality in vmauth
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3491

This is a follow-up for 2f08ed3be2
2023-05-08 12:12:16 -07:00
Aliaksandr Valialkin
1b288e0a05
all: update Go builder from Go1.20.3 to Go1.20.4
See https://github.com/golang/go/issues?q=milestone%3AGo1.20.4+label%3ACherryPickApproved
2023-05-08 09:40:55 -07:00
Aliaksandr Valialkin
24eeb8d9af
docs/CHANGELOG.md: document c77385e78f 2023-05-08 08:50:30 -07:00
Alexander Marshalov
8225a48b56
fixed vm_promscrape_config_last_reload_successful metric value recovery after successful reloading with unchanged content (#4260) (#4268)
Signed-off-by: Alexander Marshalov <_@marshalov.org>
2023-05-08 13:32:51 +02:00
Roman Khavronenko
fa3a17938e
vmalert: follow-up after cae87da (#4269)
* vmalert: follow-up after cae87da

cae87da4bb
Signed-off-by: hagen1778 <roman@victoriametrics.com>

* vmalert: update struct comments

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* vmalert: rm typo

Signed-off-by: hagen1778 <roman@victoriametrics.com>

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-05-08 13:31:54 +02:00
Alexander Marshalov
c0cb3e4f98
update generated docs for operator (#4267)
Signed-off-by: Alexander Marshalov <_@marshalov.org>
2023-05-08 12:39:44 +02:00
Roman Khavronenko
5b8450fc1b
app/vmalert: detect alerting rules which don't match any series at all (#4198)
app/vmalert: detect alerting rules which don't match any series at all

vmalert starts to understand /query responses which contain object:
```
"stats":{"seriesFetched": "42"}
```
If object is present, vmalert parses it and populates a new field
`SeriesFetched`. This field is then used to populate the new metric
`vmalert_alerting_rules_last_evaluation_series_fetched` and to
display warnings in the vmalert's UI.

If response doesn't contain the new object (Prometheus or
VictoriaMetrics earlier than v1.90), then `SeriesFetched=nil`.
In this case, UI will contain no additional warnings.
And `vmalert_alerting_rules_last_evaluation_series_fetched` will
be set to `-1`. Negative value of the metric will help to compile
correct alerting rule in follow-up.

Thanks for the initial implementation to @Haleygo
See https://github.com/VictoriaMetrics/VictoriaMetrics/pull/4056

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4039

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-05-08 09:36:39 +02:00
Roman Khavronenko
01520d3e5d
alerts: update TooHighMemoryUsage threshold (#4256)
It appears that 90% usage for anonymous mem usage
is already concerning. So we lowering the threshold to 80%.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-05-07 22:18:56 +02:00
Zakhar Bessarab
4e71003620
lib/promscrape/discovery/kubernetes: follow-up for d5e94721db (#4255)
- add changelog reference to an author
- fix tests
- add metadata to match Prometheus behavior

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2023-05-05 14:41:17 +02:00
Zakhar Bessarab
aca256735c
lib/storage: fix indexdb rotation infinite loop (#4249)
When using `retentionTimezoneOffset` and having local timezone being more than 4 hours different from UTC indexdb retention calculation could return negative value. This caused indexdb rotation to get in loop.
Fix calculation of offset to use `retentionTimezoneOffset` value properly and add test to cover all legit timezone configs.
See:
- https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4207
- https://github.com/VictoriaMetrics/VictoriaMetrics/pull/4206

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
Co-authored-by: Nikolay <nik@victoriametrics.com>
2023-05-04 17:16:48 +02:00
Alexander Marshalov
56b84140a9
added new consulagent service discovery (#3953) (#4217) 2023-05-04 11:36:21 +02:00
Alexander Marshalov
2eb27ddb22
max value for memory.allowedPercent changed from 200 to 100 (#4171) (#4251)
Signed-off-by: Alexander Marshalov <_@marshalov.org>
2023-05-04 11:34:57 +02:00
Zakhar Bessarab
ddcc3b1e9d
docs: changelog follow-up for 49b77ec01a (#4250)
Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2023-05-03 16:42:22 +04:00
Nikolay
4786f036de
lib/backup: fixes path generation for windows (#4133)
replaces custom fsync function with standard Fsync methods for files.
fixes pattern matching for parts and properly generate backup path for local fs.
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/70
2023-05-03 10:48:53 +02:00
Nikolay
73b6c23271
lib/fs: do not panic at windows at dir deletion (#4132)
Windows doesn't allow to remove dir with opened files. Usually it's a case for snapshots, hard cannot be removed if file is openned.
With this change, dir will be renamed and properly deleted at the next process start.
It's recommended to restart vmstorage/vmsingle for snapshots deletion completion periodically.
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/70
2023-05-03 10:47:02 +02:00
Zakhar Bessarab
bf3b6732bd
lib/promscrape/discovery/kubernetes: add common labels to all ports discovered from endpoints (#4235)
* lib/promscrape/discovery/kubernetes: add common labels to all ports discovered from endpoints

Sets
`__meta_kubernetes_endpoints_name` and `__meta_kubernetes_namespace` labels to all ports of pod.
Prometheus sets those labels to all ports in pod (0ab9553611/discovery/kubernetes/endpoints.go (L267C15-L269)) even if port is not matching any service.

See: #4154

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* lib/promscrape/discovery/kubernetes: fix test for updated discovery logic

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

---------

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2023-05-03 02:17:33 +02:00
Artem Navoiev
4ddfc67d54
remove information of releasing graphite render api from tip section as we released it in 1.90
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2023-05-02 15:30:22 +02:00
Artem Navoiev
5dde81259c prepare operator docs to migration
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2023-05-01 12:32:58 +02:00
Artem Navoiev
51fbf58d89 update managed docs - prepare for migration
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2023-05-01 12:19:57 +02:00
Roman Khavronenko
3d955d1078
docs: note automatic conversion to ms for influx protocol (#4224)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-04-30 17:13:35 +04:00
Artem Navoiev
a5fb8b93a8 add wight do trobubleshooting docs
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2023-04-30 12:33:36 +02:00
Artem Navoiev
ee1da35071 prepare static docs to migration
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2023-04-30 12:33:36 +02:00
Roman Khavronenko
3383f12a4b
vmalert: fix API to return non-nil values (#4222)
Properly return empty slices instead of nil for `/api/v1/rules` and `/api/v1/alerts` API handlers.
This improves compatibility with Grafana.

 https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4221

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-04-28 10:08:29 +02:00
Roman Khavronenko
eb746a4dab
Revert "http server: limit max concurrent requests (#4185)" (#4215)
This reverts commit 77f76371

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-04-27 13:02:47 +02:00
Roman Khavronenko
29e059e49c
app/vmalert: follow-up after 6c322b4a00 (#4214)
6c322b4a00

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-04-27 13:02:21 +02:00
Zakhar Bessarab
b21a55febf
app/vmalert: add support of recursive path globs for rules and templates (#4148)
Supports using `**` for `-rule` and `-rule.templates`: `dir/**/*.tpl` loads contents of dir and all subdirectories recursively.

See: #4041

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
Co-authored-by: Artem Navoiev <tenmozes@gmail.com>
Co-authored-by: Nikolay <nik@victoriametrics.com>
2023-04-26 19:20:22 +02:00
Nikolay
5ee344824f
lib/promscrape: adds filter for consul_sd_configs: (#4184)
* lib/promscrape: adds filter for consul_sd_configs:
it allows advanced filtering for consul service discovery requests
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4183

* typo fix

* removes deprecation mentions since it's not relevant

* Update docs/CHANGELOG.md

Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>

---------

Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
2023-04-26 19:16:27 +02:00
Zakhar Bessarab
89a1c941c2
app/vmalert: return an error when using query function in -external.alert.source flag (#4191)
Templating of `-external.alert.source` is not expected to have access to the query which was causing runtime error when query function was passed as nil.
See: #4181

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2023-04-26 15:31:14 +02:00
Dmytro Kozlov
bc17f4828c
app/vmagent,lib/persistentqueue: show warning message if --remoteWrite.maxDiskUsagePerURL flag lower than 500MB (#4196)
* app/vmagent,lib/persistentqueue: show warning message if `--remoteWrite.maxDiskUsagePerURL` flag lower than 500MB

* app/vmagent,lib/persistentqueue: linter fix

* app/vmagent,lib/persistentqueue: fix comment
2023-04-26 13:23:01 +03:00
dmitryk-dk
368571be00 docs: fix type 2023-04-26 11:50:17 +02:00
dmitryk-dk
0660cc7128 app/vmctl: fix comments 2023-04-26 11:50:17 +02:00
dmitryk-dk
cafad95790 app/vmctl: fix comments 2023-04-26 11:50:17 +02:00
dmitryk-dk
2b34c01f6d docs/managed-victoriametrics: change emails 2023-04-26 11:50:17 +02:00
dmitryk-dk
4052c44ac1 docs/managed-victoriametrics: add notifications setup 2023-04-26 11:50:17 +02:00
Alexander Marshalov
041e188df8
added default_url field in vmauth users config (#4084) (#4156)
* added default url field in vmauth users config (#4084)

---------

Signed-off-by: Alexander Marshalov <_@marshalov.org>
2023-04-26 11:04:35 +02:00
Yury Molodov
4f3f9950d0
vmui: add metric relabel debug (#3889)
* feat: add metric relabel debug (#3807)

* fix: add link to relabeling cookbook

* lib/promrelabel: merge, fix conflicts

* lib/promrelabel: fix diff

* docs/vmui: add metric relabel playground

---------

Co-authored-by: dmitryk-dk <kozlovdmitriyy@gmail.com>
2023-04-26 11:53:29 +03:00
Alexander Marshalov
45a551df9c
changelog for issue #4083 (#4197)
Signed-off-by: Alexander Marshalov <_@marshalov.org>
2023-04-26 10:50:44 +02:00
Yury Molodov
cf567badcf
vmui: display heatmap in the Explore Metrics (#4124)
* feat: display heatmap in the explore metrics (#4111)

* fix: correct calc step for heatmap

* fix: remove spaces in the result of getDurationFromMilliseconds
2023-04-25 13:14:57 +03:00
Yury Molodov
a80b0aebe8
vmui: add a comparison of data to the Cardinality Explorer (#4123)
* feat: add button "show today" to date picker

* feat: add comparison with the prev day (#3967)

* vmui/docs: add comparison of data to cardinality page
2023-04-25 12:21:57 +03:00
Yury Molodov
752895d1ee
docs/vmui: fix CHANGELOG.md about WITH templates (#4194) 2023-04-25 12:05:14 +03:00
Yury Molodov
68e31a6000
vmui: Integrate WITH template playground (#3831)
* feat: add WithTemplate page

* app/vmselect/prometheus: enable json mode for expand with expr API

* app/vmselect/prometheus: enable CORS and add content type

* feat: add api for expand with templates

* fix: remove console from useExpandWithExprs

* app/vmselect/prometheus: fix escaping

* vmui:  integrate WITH template

* app/vmctl: check content type instead of form param

* fix: add content-type for fetch with-exprs

* fix: add a header to the server's response that allows the "Content-Type" header

* app/vmctl: added comment and cleanup

* app/vmctl: use format query param

---------

Co-authored-by: dmitryk-dk <kozlovdmitriyy@gmail.com>
2023-04-25 11:40:01 +03:00
Dmytro Kozlov
e2053baf32
app/vmctl: add support for the different time format in the native binary protocol (#4189)
* app/vmctl: add support for the different time format in the native binary protocol

* app/vmctl: update flag description, update CHANGELOG.md

* app/vmctl: add comment to exported function
2023-04-24 18:33:30 +02:00
Alexander Marshalov
73e22dcf81
added unauthorized_user field in vmauth users config (#4083) (#4157)
added `unauthorized_user` field in vmauth users config (#4083)

---------

Signed-off-by: Alexander Marshalov <_@marshalov.org>
2023-04-24 14:57:13 +02:00
Roman Khavronenko
77f76371d0
http server: limit max concurrent requests (#4185)
* lib/httpserver: introduce `-http.maxConcurrentRequests` command-line flag

Introduce `-http.maxConcurrentRequests` command-line flag to protect
VM components from resource exhaustion during unexpected spikes of HTTP requests.
By default, the new flag's value is set to 0 which means no limits are applied.

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* lib/httpserver: mention http.maxConcurrentRequests in docs

Signed-off-by: hagen1778 <roman@victoriametrics.com>

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-04-24 14:52:06 +02:00
Alexander Marshalov
31e174977e
doc improvements (#4172) (#4186)
- added info about metric `vm_vminsert_metrics_read_total`,
- small doc refactoring
- and added make-command for running docs in docker.

Signed-off-by: Alexander Marshalov <_@marshalov.org>
2023-04-24 13:38:00 +02:00
Yury Molodov
05ab34f2c8
vmui: fix freeze when query regular with heatmap query (#4093)
* fix: fix freeze when query regular with heatmap query

* vmui/docs: fix freeze when query regular with heatmap query
2023-04-21 11:59:09 +03:00
Yury Molodov
3140aa34de
vmui: fix bug where tenant list was not displayed (#4162)
* fix: modify the condition for querying tenants

* fix: change getTenantIdFromUrl output to string
2023-04-21 11:56:08 +03:00
Artem Navoiev
87b925afb2
move note about opensource of graphite in v1.90 release note
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2023-04-19 16:34:00 +02:00
Roman Khavronenko
de61a73c63
vmalert: retry datasource requests with EOF or unexpected EOF errors (#4146)
* vmalert: retry datasource requests with EOF or unexpected EOF errors

Retry failed read request on the closed connection one more time.
This may improve rules execution reliability when connection
between vmalert and datasource closes unexpectedly.

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* vmalert: fix old tests

Signed-off-by: hagen1778 <roman@victoriametrics.com>

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-04-19 10:18:32 +02:00
Zakhar Bessarab
472fe3fd03
lib/httpserver: add handler to serve /robots.txt and deny search indexing (#4143)
This handler will instruct search engines that indexing is not allowed for the content exposed to the internet. This should help to address issues like #4128 when instances are exposed to the internet without authentication.
2023-04-18 16:47:26 +04:00
Balamurugan Krishnamoorthy (Bala)
06aefcec81
Removed duplicate third-party article reference (#4142)
"How do We Keep Metrics for a Long Time in VictoriaMetrics" article is referenced twice in "Third-party articles and slides about VictoriaMetrics" section
2023-04-18 14:03:06 +04:00
Artem Navoiev
413701454c
add graphite render api opensource to changelog
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2023-04-18 11:09:02 +02:00
Aliaksandr Valialkin
4b43c91f8c
vendor: update github.com/VictoriaMetrics/metricsql from v0.56.1 to v0.56.2
This fixes panic when the duration in the query contains `M` suffix.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4120
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3589
2023-04-14 14:05:56 -07:00
Aliaksandr Valialkin
88a4b9c313
docs/vmctl.md: run make docs-sync after 2a5b9ff782 2023-04-13 23:54:41 -07:00
Aliaksandr Valialkin
5e87e03409
docs/CHANGELOG.md: move the bugfix description into the correct place
This is a follow-up for 2a5b9ff782

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4092
2023-04-13 23:49:53 -07:00
Aliaksandr Valialkin
9f8209d593
docs/CHANGELOG.md: run at least 4 background mergers on systems with less than 4 CPU cores
This reduces the probability of sudden spike in the number of small parts when all the background mergers
are busy with big merges.
2023-04-13 23:43:17 -07:00
Dmytro Kozlov
2a5b9ff782
app/vmctl: fix performance degradation, add flag to disable backoff policy (#4097)
* app/vmctl: change api for getting metric names

* app/vmctl: fix tests

* app/vmctl: add flag to enable backoff policy, fix test, performance improvements

* app/vmctl: use one http client

* app/vmctl: made linter happy

* app/vmctl: updated documentation and CHANGELOG.md

* app/vmctl: cleanup

* app/vmctl: rename flag

* app/vmctl: cleanup

* app/vmctl: fix comments

* app/vmctl: fix metrics parser problem, improve tests
2023-04-14 09:34:54 +03:00
Aliaksandr Valialkin
e1211a1187
app/vmstorage: deprecate -bigMergeConcurrency command-line flag
Improperly configured -bigMergeConcurrency command-line flag usually leads to uncontrolled
growth of unmerged parts, which, in turn, increases CPU usage and query durations.

So it is better deprecating this flag. In rare cases -smallMergeConcurrency command-line flag
can be used instead for controlling the concurrency of background merges.
2023-04-13 20:40:24 -07:00
Aliaksandr Valialkin
90b876cd1e
app/vmbackupmanager: sync with enterprise-single-node branch after 41a54c775891c87e3d5ed59ff0769c869dd2fe71 2023-04-13 19:29:06 -07:00
Alexander Marshalov
47e16594dd
Added extra information to docs about total output in stream aggregation. (#4130)
Signed-off-by: Alexander Marshalov <_@marshalov.org>
2023-04-13 16:08:07 +02:00
Zakhar Bessarab
81f28f0f1f
lib/backup/actions: store metadata(creation and completion time) in backup files (#4117)
This makes it easier to understand exact point in time which is included in this backup.

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2023-04-12 18:51:27 +02:00
Aliaksandr Valialkin
7849545e78
docs/Single-server-VictoriaMetrics.md: document automatic switch from graph view to heatmap view for histogram buckets
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3384
2023-04-10 10:49:44 -07:00
Aliaksandr Valialkin
ddb8883817
docs/Troubleshooting.md: mention that it is recommended to use default command-line flag values for VictoriaMetrics components 2023-04-10 10:48:25 -07:00
Aliaksandr Valialkin
9586dabd94
docs/guides: update VictoriaMetrics from v1.89.1 to v1.90.0 2023-04-06 19:07:38 -07:00
Aliaksandr Valialkin
05d44525b7
docs/CHANGELOG.md: formatting fix 2023-04-06 19:04:13 -07:00
Aliaksandr Valialkin
28975067c6
docs/CHANGELOG.md: cut v1.90.0 release 2023-04-06 16:16:42 -07:00
Dmytro Kozlov
244c18fa38
app/vmctl: add multiple filters defined in --vm-native-filter-match flag to discovered metric names (#4063)
* app/vmctl: add multiple filters defined in `--vm-native-filter-match` flag to discovered metric names

* app/vmctl: fix comments

* app/vmctl: move function buildMatchWithFilter to the correct place

* app/vmctl: update CHANGELOG.md

* app/vmctl: fix CI, remove error wrapping

* app/vmctl: fix CI, simplify `Set()`
2023-04-06 15:06:52 -07:00
Aliaksandr Valialkin
ee80e71d17
docs/CHANGELOG.md: document the bugfix, which remove unneeded logger.Errorf() call during stream aggregation with the enabled deduplication
This is a follow-up for ff72ca14b9
2023-04-06 15:00:42 -07:00
Aliaksandr Valialkin
44aad84a53
docs/CHANGELOG.md: document that VictoriaMetrics for Windows cannot delete snapshots
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/70#issuecomment-1491529183
2023-04-06 03:16:06 -07:00
Aliaksandr Valialkin
608d87273d
docs/CHANGELOG.md: document v1.79.12 2023-04-06 03:10:01 -07:00
Aliaksandr Valialkin
7a65329e65
docs/CHANGELOG.md: document v1.87.5 2023-04-06 00:44:07 -07:00
Timur Bakeyev
37a7627254
Fix cut-n-paste error (#4079)
It seems that VMServiceScrape description was c-n-p from vmselect one into all other resources.
2023-04-06 11:02:12 +04:00
Zakhar Bessarab
4b2cc1b32c
docs: fix example operator spec for vmbackupmanager restore usage (#4074)
Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2023-04-05 22:16:39 -07:00
Timur Bakeyev
d87a700528
Fix reference to the imagepullsecrets description (#4080)
Looks like the original document has moved to the https://kubernetes.io/docs/concepts/containers/images/#referring-to-an-imagepullsecrets-on-a-pod.
Alternatively, it could be that https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/ describes the meaning of the parameter in more detail.
2023-04-05 19:56:28 -07:00
Aliaksandr Valialkin
8249451dbb
docs/Troubleshooting.md: add missing help word after c7bcf750c2d031b1259cd8115d7464b67f40cb9eg 2023-04-05 14:28:09 -07:00
Aliaksandr Valialkin
13668fc935
docs/Troubleshooting.md: another typo fixes after c7bcf750c2d031b1259cd8115d7464b67f40cb9eg 2023-04-05 14:14:41 -07:00
Aliaksandr Valialkin
052160dcdc
docs/Troubleshooting.md: fix formatting after c7bcf750c2 2023-04-05 13:46:19 -07:00
Aliaksandr Valialkin
a265da4f53
docs/Troubleshooting.md: fix a typo in the link after c7bcf750c2 2023-04-05 13:40:24 -07:00
Aliaksandr Valialkin
5074cc672a
all: update Go builder from Go1.20.2 to Go1.20.3
See https://github.com/golang/go/issues?q=milestone%3AGo1.20.3+label%3ACherryPickApproved
2023-04-05 13:37:22 -07:00
Aliaksandr Valialkin
c7bcf750c2
docs/Troubleshooting.md: add General troubleshooting checklist
This checklist helps searching for the infromation related to some issue / question
about VictoriaMetrics
2023-04-05 13:29:54 -07:00
Artem Navoiev
59102db4cf
update changelog
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2023-04-05 15:45:38 +02:00
Zakhar Bessarab
a8d1497024
app/vmalert: update Grafana URLs to match latest format (#4061)
See: #4019

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2023-04-04 15:25:29 +04:00
Artem Navoiev
33c6cc2530
fix closing divs in docs
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2023-04-04 13:00:22 +02:00
Aliaksandr Valialkin
55b5276b70
docs/CHANGELOG.md: document edb45d7fc1
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4013
2023-04-02 21:26:12 -07:00
Aliaksandr Valialkin
f638496298
lib/promscrape: do not re-use previously loaded scrape targets on failed attempt to load updated scrape targets at file_sd_configs
The logic employed for re-using the previously loaded scrape target was broken initially.
The commit cc0427897c tried to fix it, but the new logic
became too complex and fragile. So it is better to just remove this logic,
since the targets from temporarily broken file should be eventually loaded on next
attempts every -promscrape.fileSDCheckInterval

This also allows removing fragile hacks around __vm_filepath label.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3989
2023-04-02 21:05:28 -07:00
Dmytro Kozlov
cc0427897c
lib/promscrape: fix the problem with scrape work duplicates when file_sd_config can't be read (#4027)
* lib/promscrape: fix the problem with scrape work duplicates when file_sd_config can't be read

* lib/promscrape: clarified comment

* lib/promscrape: made better approach to handle a problem with growing []*ScrapeWork on each error when loading config

* lib/promscrape: added CHANGELOG.md

* Update docs/CHANGELOG.md

---------

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2023-04-02 20:26:13 -07:00
Yury Molodov
42087518ba
vmui: tips for working with the graph and legend (#4045)
* feat: add tips for working with the graph and legend

* feat: add the ability to collapse the legend

* vmui/docs: add the ability to collapse the legend

---------

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2023-04-01 00:38:18 -07:00
Roman Khavronenko
27b958ba8b
lib/storage: check for free disk space before opening tables (#4035)
* lib/storage: check for free disk space before opening tables

We check for free disk space before call to `openTable`,
so `Storage` can be set to ReadOnly before mergeWorkers start.

Before the change, there was a chance that merges will start
even if Storage has to start in ReadOnly mode because of
`-storage.minFreeDiskSpaceBytes` limit.

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4023
Signed-off-by: hagen1778 <roman@victoriametrics.com>

* lib/storage: chore

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* Update lib/storage/storage.go

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2023-03-31 23:50:27 -07:00
Aliaksandr Valialkin
ffdf430be0
app/vmselect/graphite: open source Graphite Render API 2023-03-31 23:25:04 -07:00
Aliaksandr Valialkin
cddfc4d3f8
deployment/docker: update base Docker image from Alpine 3.17.2 to Alpine 3.17.3
This fixes security issues from https://alpinelinux.org/posts/Alpine-3.17.3-released.html

This is a follow-up for 59c350d0d2
2023-03-31 22:46:27 -07:00
Aliaksandr Valialkin
d577657fb7
lib/streamaggr: follow-up for ff72ca14b9
- Make sure that the last successfully loaded config is used on hot-reload failure
- Properly cleanup resources occupied by already initialized aggregators
  when the current aggregator fails to be initialized
- Expose distinct vmagent_streamaggr_config_reload* metrics per each -remoteWrite.streamAggr.config
  This should simplify monitoring and debugging failed reloads
- Remove race condition at app/vminsert/common.MustStopStreamAggr when calling sa.MustStop() while sa
  could be in use at realoadSaConfig()
- Remove lib/streamaggr.aggregator.hasState global variable, since it may negatively impact scalability
  on system with big number of CPU cores at hasState.Store(true) call inside aggregator.Push().
- Remove fine-grained aggregator reload - reload all the aggregators on config change instead.
  This simplifies the code a bit. The fine-grained aggregator reload may be returned back
  if there will be demand from real users for it.
- Check -relabelConfig and -streamAggr.config files when single-node VictoriaMetrics runs with -dryRun flag
- Return back accidentally removed changelog for v1.87.4 at docs/CHANGELOG.md

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3639
2023-03-31 22:30:38 -07:00
Zakhar Bessarab
5e5fc66e3b
docs/vmctl: add examples of URLs used for migration in different modes (#4042)
Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2023-03-30 17:21:36 +02:00
Roman Khavronenko
4a49577028
vmalert: use missingkey=zero for templating (#4040)
Replace empty labels with "" instead of "<no value>"
during templating, as Prometheus does.

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4012

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-03-30 16:57:00 +04:00
Zakhar Bessarab
ec45f1bc5f
lib/fs: verify response code when reading configuration over HTTP (#4036)
Verifying status code helps to avoid misleading errors caused by attempt to parse unsuccessful response.

Related issue: #4034

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2023-03-30 13:18:00 +02:00
Daria Karavaieva
0945a03843
Vmanomaly guide index fix (#4029)
* name and scrutture change

* fix indexing

* index fix

* name change

* line separator fix
2023-03-29 20:24:06 +02:00
Alexander Marshalov
ff72ca14b9
added hot reload support for stream aggregation configs (#3969) (#3970)
added hot reload support for stream aggregation configs (#3969)

Signed-off-by: Alexander Marshalov <_@marshalov.org>
2023-03-29 18:05:58 +02:00
Eliran Barnoy
9199c23720 Fix operator links to include VMPrometheusConverter for added visibility 2023-03-29 11:44:49 +02:00
Aliaksandr Valialkin
94cabf29b0
lib/flagutil: ArrayString: support commas inside quoted strings and inside [], {} and () braces
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3915
2023-03-28 21:22:55 -07:00
Aliaksandr Valialkin
e094c8e214
docs: mention that VictoriaMetrics rounds time range to UTC days at /api/v1/labels, /api/v1/label/.../values and /api/v1/series handlers
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3107
2023-03-28 17:00:17 -07:00
Aliaksandr Valialkin
aea6df8197
app/vmagent/remotewrite: cosmetic updates after f3a51e8b1d
- Compare directory names instead of paths to directory when determining which persistent queues must be deleted
  This is less error-prone solution, since paths to the same directory can differ, which could lead
  to accidental directory removal for the existing -remoteWrite.url

- Log the `removed %d dangling queues` message when at least a single queue has been removed

- Consistently use filepath.Join() for creating paths to persistent queues.
  This is needed for Windows support (see https://github.com/VictoriaMetrics/VictoriaMetrics/issues/70 )

- Clarify the description of the change at docs/CHANGELOG.md

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4014
2023-03-27 18:33:07 -07:00
Zakhar Bessarab
f3a51e8b1d
app/vmagent: add -remoteWrite.removeDanglingQueues flag (#4017)
* app/vmagent: add `-remoteWrite.removeDanglingQueues` flag which allows to automatically remove dangling persistent queue contents

Related issue: #4014

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* app/vmagent: address review feedback

- remove persistent queues files by default
- rename `remoteWrite.removeDanglingQueues` to `remoteWrite.keepDanglingQueues`
- update docs to reflect changed behaviour

Related issue: #4014

* Apply suggestions from code review

---------

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2023-03-27 18:15:28 -07:00
Aliaksandr Valialkin
02ee4ffd4d
app/vmselect/promql: follow-up for 79e1c6a6fc
- Document the fix at docs/CHANGELOG.md
- Add tests with multiple adjancent zero buckets
- Simplify the fix a bit

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/296
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/4021
2023-03-27 18:03:36 -07:00
Yury Molodov
3214b1c315
vmui: heatmap (#3780)
* fix: add stroke and font for all axes

* feat: add util for generate gradient

* feat: add heatmap plugin

* feat: add heatmap legend

* feat: add heatmap graph (#3384)

* vmui: add heatmap graph (#3384)

* feat: add convert Prometheus to VictoriaMetrics histogram

* fix: prevent re-render graph

* feat: reset step for heatmap

* feat: normalize heatmap data

* fix: format heatmap legend

* wip

* app/vmselect/vmui: run `make vmui-update`

---------

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2023-03-26 00:30:02 -07:00
Aliaksandr Valialkin
92199c964c
docs/MetricsQL.md: quote min, max and avg args for rollup_*() functions in order to reduce the level of confusion when users try to pass the second argument to these functions 2023-03-26 00:01:51 -07:00
Aliaksandr Valialkin
72a0b49330
docs/CHANGELOG.md: document v1.87.4 LTS release 2023-03-25 22:43:59 -07:00
Aliaksandr Valialkin
811f4a9380
app/{vmbackup,vmrestore}: publish vmbackup and vmrestore binaries for Windows
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/70
2023-03-25 15:08:21 -07:00
Aliaksandr Valialkin
e7f46a0aab
app/vmselect/promql: follow-up for 7205c79c5a
- Allocate and initialize seriesByWorkerID slice in a single go instead
  of initializing every item in the list separately.
  This should reduce CPU usage a bit.
- Properly set anti-false sharing padding at timeseriesWithPadding structure
- Document the change at docs/CHANGELOG.md

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3966
2023-03-24 23:34:37 -07:00
Zakhar Bessarab
5ba347bd2c
app/vmbackup: delete created snapshot in case of error during backup (#4008)
Related issue: #2055

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2023-03-24 21:49:58 -07:00
Aliaksandr Valialkin
27f9a1eda2
docs/CHANGELOG.md: cosmetic fixes: remove trailing whitespace and consistently use -flag instead of --flag 2023-03-24 15:44:33 -07:00
Alexander Marshalov
7c86dcc4fa
allowed using dashes and dots in environment variables names (#4009)
* allowed using dashes and dots in environment variables names for templating config files with envtemplate (#3999)

Signed-off-by: Alexander Marshalov <_@marshalov.org>

* Apply suggestions from code review

---------

Signed-off-by: Alexander Marshalov <_@marshalov.org>
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2023-03-24 15:43:05 -07:00
Aliaksandr Valialkin
c1d871a45a
docs/vmauth.md: follow-up for 36edba9bfb
- Document `-configCheckInterval` command-line flag in `quick start` section
- Clarify the addition of `-configCheckInterval` at docs/CHANGELOG.md

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3990
2023-03-24 13:22:37 -07:00
Aliaksandr Valialkin
54796f69db
docs/vmagent.md: clarify that there is no need to specify multiple -remoteWrite.url options when writing data to a single VictoriaMetrics cluster when data replication is needed
Also add a link to https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#url-format from `getting started` section,
so users could quickly find how to write data to VictoriaMetrics cluster
2023-03-24 13:07:33 -07:00
Roman Khavronenko
8db6a71f83
vmalert: mention VMUI example for alert's source (#4005)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-03-24 09:40:55 +01:00
Roman Khavronenko
edeb56d208
docs: mention cluster URL for exporting series (#4002)
docs: mention cluster URL for exporting series

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-03-23 20:55:47 +01:00
Daria Karavaieva
24938872a6
Vmanomaly guide (#3834)
Setting up VMAnomaly on NodeExporter metrics with VictoriaMetrics and AlertManager.


* vmanomaly-guide-draft

* aletr graphs and description

* readme vmanomaly tutorial

* Added back fit_every param for performance

* vmanomaly guide fixes

* added spaces div

* spaces + resize image

* alert example grammar

* quotation marks

* docker link

* typo fixed

* more links

* reader section rephrased

* label change

* lower case for grafana service

* lower case for vm service

* yaml markdown

---------

Co-authored-by: Dima Lazerka <dima@victoriametrics.com>
2023-03-23 21:04:44 +02:00
Dmytro Kozlov
ba505dd357
docs: follow up after dc2c712a29 (#4001) 2023-03-23 18:27:55 +01:00
Yury Molodov
023c65968f
vmui: display errors for each query individually (#3987) (#3994) 2023-03-23 13:10:59 +01:00
Alexander Marshalov
36edba9bfb
added configCheckInterval flag for vmauth (#3990) (#3991)
* added configCheckInterval flag for vmauth (#3990)
Signed-off-by: Alexander Marshalov <_@marshalov.org>
2023-03-23 09:34:12 +01:00
Nikolay
a2f716b6cc
lib/netutil: log only parsing errors for proxy-protocol (#3985)
* lib/netutil: log only parsing errors for proxy-protocol

Previosly every error was logged. With configured TCP health checks at load-balancer or kubernetes, vmauth spams a lot of false positive error message into logs

* Update docs/CHANGELOG.md

Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>

* Update lib/netutil/tcplistener.go

Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>

---------

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
2023-03-21 10:22:39 -07:00
Aliaksandr Valialkin
6a78755b66
docs/vmagent.md: mention in docs that the target relabel debug page shows target url now 2023-03-20 22:20:03 -07:00
Dmytro Kozlov
e79cd24807
lib/promrelabel: make target url from labels on target relabel page (#3882)
* lib/promrelabel: make target url from labels on target relabel page

* wip

---------

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2023-03-20 22:07:52 -07:00
Aliaksandr Valialkin
8d709f3483
docs/CHANGELOG.md: cosmetic fixes 2023-03-20 14:14:20 -07:00
Aliaksandr Valialkin
91533531f5
docs/Troubleshooting.md: document an additional case, which could result in slow inserts
If `-cacheExpireDuration` is lower than the interval between ingested samples for the same time series,
then vm_slow_row_inserts_total` metric is increased.

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3976#issuecomment-1476883183
2023-03-20 13:28:36 -07:00
Dmytro Kozlov
8da9502df6
app/vmctl: automatically check tty (#3938)
app/vmctl: automatically detect if TTY is available
2023-03-20 11:16:08 +01:00
Yury Molodov
d4525bd2d0
vmui: support for drag'n'drop in the "Trace analyzer" page (#3971)
vmui: add drag-and-drop support for the trace analyzer page
2023-03-20 11:07:18 +01:00
Yury Molodov
a2af2e5a1b
vmui: improve usability of date/time picker (#3968)
* vmui: allow manually set input date and time
* vmui/docs: improve usability of date/time picker
2023-03-20 09:22:49 +01:00
Aliaksandr Valialkin
43b24164ef
all: add Windows build for VictoriaMetrics
This commit changes background merge algorithm, so it becomes compatible with Windows file semantics.

The previous algorithm for background merge:

1. Merge source parts into a destination part inside tmp directory.
2. Create a file in txn directory with instructions on how to atomically
   swap source parts with the destination part.
3. Perform instructions from the file.
4. Delete the file with instructions.

This algorithm guarantees that either source parts or destination part
is visible in the partition after unclean shutdown at any step above,
since the remaining files with instructions is replayed on the next restart,
after that the remaining contents of the tmp directory is deleted.

Unfortunately this algorithm doesn't work under Windows because
it disallows removing and moving files, which are in use.

So the new algorithm for background merge has been implemented:

1. Merge source parts into a destination part inside the partition directory itself.
   E.g. now the partition directory may contain both complete and incomplete parts.
2. Atomically update the parts.json file with the new list of parts after the merge,
   e.g. remove the source parts from the list and add the destination part to the list
   before storing it to parts.json file.
3. Remove the source parts from disk when they are no longer used.

This algorithm guarantees that either source parts or destination part
is visible in the partition after unclean shutdown at any step above,
since incomplete partitions from step 1 or old source parts from step 3 are removed
on the next startup by inspecting parts.json file.

This algorithm should work under Windows, since it doesn't remove or move files in use.
This algorithm has also the following benefits:

- It should work better for NFS.
- It fits object storage semantics.

The new algorithm changes data storage format, so it is impossible to downgrade
to the previous versions of VictoriaMetrics after upgrading to this algorithm.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3236
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3821
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/70
2023-03-19 01:36:51 -07:00
Aliaksandr Valialkin
6460475e3b
lib/{mergeset,storage}: prevent from long wait time when creating a snapshot under high data ingestion rate
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3551
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/3873
2023-03-19 00:15:30 -07:00
Alexander Marshalov
57b00bafc9
updated api doc for operator (#3972)
Signed-off-by: Alexander Marshalov <_@marshalov.org>
2023-03-17 10:42:30 +01:00
Zakhar Bessarab
ac3043ff74
doc/vmgateway-grafana-openid-guide: fix formatting, add reproducible example and example results (#3964) 2023-03-17 09:57:10 +01:00
Aliaksandr Valialkin
fdbb819195
all: typo fix in the same way as in e566d49e3a: 8248 -> 8428 2023-03-16 22:06:38 -07:00
Artem Navoiev
f8a2a3784b
managed quickstart fix typo
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2023-03-15 22:38:59 +01:00
Artem Navoiev
20aa707979
fix anchor after chaning manager quick start
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2023-03-15 22:32:20 +01:00
Nikolay
91cbb9063d
Vmagent kafka updates (#535)
* app/vmagent: allow vm proto for kafka consumer and producer
it should reduce network usage up to 50%.
According to benchmarks without any encoding at kafka topic, it reduces traffic up to 50%.
With enabled zstd at kafka topic, it shows no diffence in traffic. So it
doesn't make much sense to use it.
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1225

* mention eb61a7dd68b834b08d01727a918f207700348ada at changelog

* app/vmagent: bumps kafka lib version
it allows compiling vmagent for arm64 machines
fixes https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2271

* mention d19b1a888248c96cfd7ccee00ba6f596d89be1d7 at change log

* app/vmagent: adds natural concurrency for kafka consumer
it should improve performance for data consumption
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1957

* mention change 0c143bb22ca2e7e0b7eec9bc84a94ee2b41626ca

* Update app/vmagent/kafka/consumer.go

Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>

* Update app/vmagent/kafka/consumer_cgo.go

Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>

---------

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
2023-03-15 13:03:44 -07:00
Alexander Marshalov
55afae8641
updated vars doc for operator (#3960)
Signed-off-by: Alexander Marshalov <_@marshalov.org>
2023-03-15 17:28:01 +01:00
dmitryk-dk
0691e115b1 docs: cleanup 2023-03-15 11:54:59 +01:00
dmitryk-dk
32266aaea2 app/vmctl: update managed quickstart guide 2023-03-15 11:54:59 +01:00
Zakhar Bessarab
8f6d5217d1
doc: add guide for vmgateway configuration with OpenID and Grafana (#3951)
docs: add guide for vmgateway configuration with OpenID and Grafana
2023-03-14 16:19:29 +01:00
Zakhar Bessarab
6a5d236245
lib/storage: log original labels set when label value is truncated (#3952)
lib/storage: log original labels set when label value is truncated
2023-03-14 10:59:40 +01:00
Aliaksandr Valialkin
6f6333831e
docs/Single-server-VictoriaMetrics.md: clarify that the cache directory can be removed manually when VictoriaMetrics is stopped 2023-03-13 00:23:40 -07:00
Aliaksandr Valialkin
3e7bfe1200
docs/CHANGELOG.md: document v1.87.3 2023-03-13 00:20:51 -07:00
Aliaksandr Valialkin
02ffe05750
docs/CHANGELOG.md: document v1.79.11 LTS release 2023-03-12 23:22:53 -07:00
Aliaksandr Valialkin
388d6ee16e
docs/CHANGELOG.md: cut v1.89.1 2023-03-12 19:14:19 -07:00
Aliaksandr Valialkin
e8225d7d6b
app/vmselect/promql: prevent from cannot unmarshal timeseries from rollupResultCache panic after the upgrade to v1.89.0
The issue has been introduced in 0af9e2b693
2023-03-12 19:09:39 -07:00
Aliaksandr Valialkin
911bab4f6a
docs/CHANGELOG.md: cut v1.89.0 2023-03-12 17:29:44 -07:00
Aliaksandr Valialkin
468de76e9a
app/vmselect: remove data race on updating EvalConfig.IsPartialResponse from concurrently running goroutines
This properly returns `is_partial: true` for partial responses.
2023-03-12 16:54:08 -07:00
Aliaksandr Valialkin
0af9e2b693
app/vmselect/promql: prevent from SIGBUS crash on architecures, which deny unaligned access to 8-byte words (e.g. ARM)
Thanks to @oliverpool for nailing down the root cause of the issue and for the initial attempt to fix it
at https://github.com/VictoriaMetrics/VictoriaMetrics/pull/3927
2023-03-12 16:32:08 -07:00
Artem Navoiev
7257a2a97f
fix typos on image
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2023-03-12 17:12:57 +01:00
Yury Molodov
01367faa39
vmui: remove send step param for instant queries (#3931)
* fix: remove step param for instant queries (#3896)

* vmui: remove send step param for instant queries

* Update docs/CHANGELOG.md

---------

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2023-03-12 03:09:56 -07:00
Aliaksandr Valialkin
a52413ce0a
docs/CHANGELOG.md: document 113a89904d 2023-03-12 01:58:18 -08:00
Aliaksandr Valialkin
b19de3fa12
docs/CHANGELOG.md: yet another typo fix 2023-03-12 01:06:40 -08:00
Aliaksandr Valialkin
2f1d24fccf
docs/CHANGELOG.md: typo fix 2023-03-12 01:04:14 -08:00
Aliaksandr Valialkin
b5db69fe05
app/vmselect/netstorage: do not intern string representation of MetricName for time series received from vmstorage
It has been appeared that this interning may lead to increased memory usage and increased CPU usage
when vmselect performs queries, which select big number of time series.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3692
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3863
2023-03-12 00:52:35 -08:00
Aliaksandr Valialkin
babc9e9815
docs/CHANGELOG.md: document 927d9da270 2023-03-12 00:25:00 -08:00
Aliaksandr Valialkin
e3488c6cbc
docs/CHANGELOG.md: typo fixes 2023-03-12 00:09:26 -08:00
Aliaksandr Valialkin
48e32b325e
docs/CHANGELOG.md: document c9f44daaee8f4282d9ed41e3ba799c7a33841313 2023-03-11 23:55:13 -08:00
Roman Khavronenko
856c2db144
vmalert: support concurrent reading from object storage (#532)
* vmalert: support concurrent reading from object storage

Config reading from GCS or S3 can be slow if object storage
contains a big number of files. Object storages are usually
fast for downloading and are slow for individual operations.
If there would be thousands of files to read, vmalert could
spend significant time for retrieving those because it is
done sequentially.

The change introduces ability to read configs from object
storage concurrently. By default, both GCS and S3 are now
read with 50 concurrent readers. This significantly reduces
the load time:
* loading 500 files with concurrency=1 takes 27s
* loading 500 files with concurrency=50 takes <1s

* vmalert: add note to Changelog

* vmalert: cleanup

* vmalert: use ticker properly

* app/vmalert: improve status reporting during config loading

* vmalert: support concurrent reading from object storage

Config reading from GCS or S3 can be slow if object storage
contains a big number of files. Object storages are usually
fast for downloading and are slow for individual operations.
If there would be thousands of files to read, vmalert could
spend significant time for retrieving those because it is
done sequentially.

The change introduces ability to read configs from object
storage concurrently. By default, both GCS and S3 are now
read with 50 concurrent readers. This significantly reduces
the load time:
* loading 500 files with concurrency=1 takes 27s
* loading 500 files with concurrency=50 takes <1s

* app/vmalert: make linter happy
2023-03-11 23:51:23 -08:00
Alexander Marshalov
c0c3dc02cf
Stream aggregation doc improvements based on users feedback (#3934)
docs: stream aggregation doc improvements based on users feedback
2023-03-10 21:39:58 +01:00
Zakhar Bessarab
f7a7eb8f3e
docs: add a note about cache reset for vmalert backfilling docs (#3940)
docs: add a note about cache reset for vmalert backfilling docs
2023-03-10 13:45:11 +01:00
Dmytro Kozlov
4c3bc04efa
app/vmctl: update importing tips when migrating data with overlapping time range (#3941)
app/vmctl: update importing tips when migrating data with overlapping time range
2023-03-10 11:29:08 +01:00
Dmytro Kozlov
3c9058c168
app/vmctl: add support of basic auth and barer token (#3921)
app/vmctl: add support of basic auth and bearer token
2023-03-09 14:53:29 +01:00
Roman Khavronenko
d66bae212b
app/vmalert: log number of configration files found for each specified -rule (#3936)
The change also introduces `List` method to `FS` interface.
The `List` method can be used for wildcard support in object storage FS.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: Nikolay <nik@victoriametrics.com>
2023-03-09 14:46:19 +01:00
Dmytro Kozlov
7f54c181bb
app/vmctl: follow up after 09e3742a82 (#3937)
app/vmctl: follow up after 09e3742a82
2023-03-09 13:28:55 +01:00
Roman Khavronenko
3de7fc5c71
security: bump go version to 1.20.2 (#3935)
upgrade Go builder from Go1.20.1 to Go1.20.2
See the list of issues addressed in Go1.20.2 here (https://github.com/golang/go/issues?q=milestone%3AGo1.20.2+label%3ACherryPickApproved).

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-03-09 13:20:54 +01:00
Denys Holius
cbba6bd3db
Adds snap badge to README.md (#3930)
docs: adds snap badge to README.md
2023-03-08 16:31:58 +01:00
Aliaksandr Valialkin
1b5dc9f91d
all: follow-up for 7a3e16e774
- Sync the description for -httpListenAddr.useProxyProtocol command-line flag at vmagent and vmauth,
  so it is consistent with the description at vmauth and victoria-metrics
- Add a sample of panic text to docs/CHANGELOG.md, so it could be googled
- Mention the -httpListenAddr.useProxyProtocol command-line flag in the description for the bugfix

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3335
2023-03-08 01:26:55 -08:00
Aliaksandr Valialkin
ed73317622
docs/CHANGELOG.md: improve description for 4b136abff8 2023-03-08 01:05:33 -08:00
Aliaksandr Valialkin
70b831e684
docs/CHANGELOG.md: improve the description of the bugfix at 62beea23f7
- Make the description easier to read by humans :)
- Add a link to VictoriaMetrics datasource plugin for Grafana, so users could easily discover it
2023-03-08 00:59:42 -08:00
Aliaksandr Valialkin
6e8e64f695
docs/CHANGELOG.md: clarify the description for 6bfe9cc733
- Add the panic message to the description, so it is easier to google
- Add a link to the corresponding bugreport

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3897
2023-03-08 00:39:34 -08:00
Aliaksandr Valialkin
9e71462cee
all: typo fixes of the same type as in the d056be710b 2023-03-08 00:30:21 -08:00
Aliaksandr Valialkin
884e58d58d
docs/CHANGELOG.md: clarify the description for the change at 8bab50dc29
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3600
2023-03-08 00:23:35 -08:00
Aliaksandr Valialkin
1bb529e23e
app/vmagent/remotewrite: follow-up for e3a756d82869f8c357b072f6e635ebfc7d65dd2c
- Document the fix
- Move the detection of VictoriaMetrics remoteWrite protocol from client.init() to newHTTPClient()
  This simplifies the fix to the following diff:

diff --git a/app/vmagent/remotewrite/client.go b/app/vmagent/remotewrite/client.go
index 099899c19..70b904af4 100644
--- a/app/vmagent/remotewrite/client.go
+++ b/app/vmagent/remotewrite/client.go
@@ -151,10 +151,6 @@ func newHTTPClient(argIdx int, remoteWriteURL, sanitizedURL string, fq *persiste
        }
        c.sendBlock = c.sendBlockHTTP

-       return c
-}
-
-func (c *client) init(argIdx, concurrency int, sanitizedURL string) {
        useVMProto := forceVMProto.GetOptionalArg(argIdx)
        usePromProto := forcePromProto.GetOptionalArg(argIdx)
        if useVMProto && usePromProto {
@@ -173,6 +169,10 @@ func (c *client) init(argIdx, concurrency int, sanitizedURL string) {
        }
        c.useVMProto = useVMProto

+       return c
+}
+
+func (c *client) init(argIdx, concurrency int, sanitizedURL string) {
2023-03-07 23:54:24 -08:00
Aliaksandr Valialkin
202083f38c
docs/CHANGELOG.md: document ec2abf9b69 2023-03-07 23:33:19 -08:00
Alexander Marshalov
ccb08fc28b
added documentation about new templates field of vmalertmanager specification in operator (https://github.com/VictoriaMetrics/operator/issues/592) (#3924)
Signed-off-by: Alexander Marshalov <_@marshalov.org>
2023-03-07 17:51:26 +01:00
Nikolay
7a3e16e774
lib/netutil: fixes panic at proxy protocol (#3905)
it may occur if non proxy protocol message received by tcp server.
Listener Accept method must return only non-recoverable errors.
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3335
2023-03-07 08:50:18 -08:00
Yury Molodov
bbf8e459a0
vmui: fix display of selected value in the selector (#3919)
vmui: fix selected value in dropdowns for Explore page
2023-03-07 16:23:02 +01:00
Nikolay
02f13d5681
docs: updates operator api.md (#3922) 2023-03-07 09:09:05 +01:00
Craig Rodrigues
1b194bc6de
docs: move installation methods further up in README (#3904) 2023-03-06 12:44:33 +01:00
Dmytro Kozlov
cc5b916237
docs: follow up after 4b136abff8 (#3918)
docs: follow up after 4b136abff8
2023-03-06 12:41:48 +01:00
Alexander Marshalov
75977a4d02
added doc for placeholder support in vmagent specification for operator (https://github.com/VictoriaMetrics/operator/issues/592) (#3916) 2023-03-06 11:58:31 +01:00
Roman Khavronenko
95dc65e7b3
docs: follow-up after 62beea23f7 (#3907)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-03-03 22:45:19 +01:00
Artem Makhortov
943243321a
doc: vmctl vm-native-step-interval supported values (#3899) 2023-03-03 16:05:33 +01:00
Nikolay
6bfe9cc733
lib{mergset,storage}: prevent possible race condition with logging st… (#3900)
lib{mergset,storage}: prevent possible race condition with logging stats for merges

Previously partwrapper could be release by background process and reference for part may be invalid 
during logging stats. It will lead to panic at vmstorage
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3897
2023-03-03 12:33:42 +01:00
Artem Navoiev
de621c0cb7
remove image width for vmalert managed vm guide
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2023-03-02 21:51:34 +01:00
Artem Navoiev
14ef5311f0
change title
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2023-03-02 18:06:43 +01:00
Artem Navoiev
5872b5711d add vmalert managed vm integration guide
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2023-03-02 18:04:17 +01:00
Dmytro Kozlov
8bab50dc29
app/vmctl: add backoff retries to native protocol (#3859)
app/vmctl: vm-native - split migration on per-metric basis

`vm-native` mode now splits the migration process on per-metric basis. 
This allows to migrate metrics one-by-one according to the specified filter. 
This change allows to retry export/import requests for a specific metric and provides a better 
understanding of the migration progress.

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
2023-03-02 13:19:45 +01:00
Dmytro Kozlov
ec2abf9b69
app/vmctl: Increase http request timeout made by remote read client, add importing tips (#3879)
app/vmctl: Increase default http request timeout made by remote read client
2023-02-28 09:50:33 +01:00
Aliaksandr Valialkin
95e1173423
docs/CHANGELOG.md: document v1.79.10 release 2023-02-27 17:35:48 -08:00
Aliaksandr Valialkin
8288e327ee
docs/CHANGELOG.md: cut v1.88.1 2023-02-27 15:28:18 -08:00
Aliaksandr Valialkin
dfe3939665
docs/CHANGELOG.md: link to the issue, which may benefit from -internStringDisableCache command-line flag
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3863
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3692
2023-02-27 14:55:40 -08:00
Aliaksandr Valialkin
46127b432d
lib/bytesutil: add -internStringDisableCache and -internStringCacheExpireDuration command-line flags
This commit is based on https://github.com/VictoriaMetrics/VictoriaMetrics/pull/3872
2023-02-27 14:16:49 -08:00
Aliaksandr Valialkin
0d3f31f60e
lib/storage: follow-up for 39cdc546dd
- Use flag.Duration instead of flagutil.Duration for -snapshotCreateTimeout,
  since the flagutil.Duration is intended mostly for big durations, e.g. days, months and years,
  while the -snapshotCreateTimeout is usually smaller than one hour.
- Add links to https://docs.victoriametrics.com/#how-to-work-with-snapshots in docs/CHANGELOG.md,
  so readers could easily find the corresponding docs when reading the changelog.
- Properly remove all the created directories on unsuccessful attempt to create
  snapshot in Storage.CreateSnapshot().

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3551
2023-02-27 13:07:38 -08:00
Zakhar Bessarab
39cdc546dd
lib/storage: enhancements for snapshots process (#3873)
* lib/{fs,mergeset,storage}: skip `.must-remove.` dirs when creating snapshot (#3858)

* lib/{mergeset,storage}: add timeout configuration for snapshots creation, remove incomplete snapshots from storage

* docs: fix formatting

* app/vmstorage: add metrics to track status of snapshots

* app/vmstorage: use `vm_http_requests_total` metric for snapshot endpoints metrics, rename new flag to make name more clear

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* app/vmstorage: update flag name in docs

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* app/vmstorage: reflect new metrics names change in docs

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

---------

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2023-02-27 12:12:03 -08:00
Zakhar Bessarab
5fadd58cf6
lib/promscrape: correctly register vm_promscrape_config_* metrics (#3876)
* lib/promscrape: set `vm_promscrape_config_last_reload_successful` to 1 if there was no promscrape config provided

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* lib/promscrape: register `vm_promscrape_config_*` metrics only in case promscrape config is used

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

---------

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2023-02-27 11:53:53 -08:00
Zakhar Bessarab
ac6d937372
doc: add changelog reference for vmgateway OpenID discovery (#3877)
* doc: add changelog reference for vmgateway OpenID discovery

* doc: add vmgateway docs

---------

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2023-02-27 11:50:22 -08:00
Aliaksandr Valialkin
b3bb18d674
app/vmselect/promql: fix panic when calculating aggr_func(rollup*())
The panic has been introduced in dac21d874b
2023-02-27 11:48:27 -08:00
Zakhar Bessarab
ac4c7adec6
app/vmgateway: add new flag doc
Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2023-02-27 11:18:34 -08:00
Zakhar Bessarab
5d7da8479f
app/vmgateway: fix typo in docs
Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2023-02-27 11:18:08 -08:00
Zakhar Bessarab
4ee73f54a6
app/vmgateway: add OpenID discovery of JWKS endpoints
Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2023-02-27 11:17:39 -08:00
Aliaksandr Valialkin
23871fb0bf
app/vmagent: add -remoteWrite.vmProtoCompressLevel command-line flag for tuning the compression level for VictoriaMetrics remote write protocol 2023-02-27 11:03:49 -08:00
Dmytro Kozlov
27c9446520
app/vmctl: skip series if measurement not found (#3869)
app/vmctl: skip measurements with no fields for influxdb mode
2023-02-27 14:28:47 +01:00
Aliaksandr Valialkin
ffa327d6d1
app/vmagent: use the provided auth options when checking whether the remote storage supports VictoriaMetrics remote write protocol
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3847
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1225
2023-02-26 12:07:47 -08:00
Aliaksandr Valialkin
2d36dbcfa9
docs/CHANGELOG.md: cut v1.88.0 2023-02-24 17:54:21 -08:00
Aliaksandr Valialkin
b4bf8dc2f0
docs: update -help output after e1c3267e34 2023-02-24 17:16:13 -08:00
Roman Khavronenko
e1c3267e34
vmselect/promql: check for deadline in count_values fn (#3806)
* vmselect/promql: check for deadline in `count_values` fn

`count_values` could be very slow during the data processing.
Checking for deadline between iterations supposed to reduce
probability of exceeding `search.maxQueryDuration`.

The change also adds a new trace record, which captures the time
spent in aggregation function. Before that, the trace for aggr funcs
could be confusing since it doesn't account for all the places where
time was spent.

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* wip

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2023-02-24 16:59:26 -08:00
Aliaksandr Valialkin
c33cc4322c
docs/CHANGELOG.md: document v1.87.2 release 2023-02-24 16:14:28 -08:00
Aliaksandr Valialkin
6a88d5402d
docs/CHANGELOG.md: document v1.79.9 release 2023-02-24 15:10:48 -08:00
Roman Khavronenko
dac21d874b
metricsql: support optional 2nd argument for rollup functions (#3841)
* metricsql: support optional 2nd argument for rollup functions

Support optional 2nd argument `min`, `max` or `avg` for rollup functions:
 * rollup
 * rollup_delta
 * rollup_deriv
 * rollup_increase
 * rollup_rate
 * rollup_scrape_interval

 If second argument is passed, then rollup function will return only the selected aggregation type.
 This change can be useful for situations where only one type of rollup calculation is needed.
 For example, `rollup_rate(requests_total[5m], "max")`.

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* wip

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2023-02-24 13:47:52 -08:00
Aliaksandr Valialkin
87aeeec3e8
docs/CHANGELOG.md: document d8eaa511b0 2023-02-24 12:42:02 -08:00
Aliaksandr Valialkin
a2340e6c95
docs/CHANGELOG.md: typo fix: scrape scrape -> scrape 2023-02-24 12:33:29 -08:00
Aliaksandr Valialkin
c6ad3692ad
lib/promscrape: follow-up for 43e104a83f
- Return immediately on context cancel during the backoff sleep.
  This should help with https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3747

- Add a comment describing why the second attempt to obtain the response from remote side
  is perfromed immediately after the first attempt.

- Remove fasthttp dependency from lib/promscrape/discoveryutils

- Set context deadline before calling doRequestWithPossibleRetry().
  This simplifies the doRequestWithPossibleRetry() a bit.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3293
2023-02-24 12:20:42 -08:00
Zakhar Bessarab
43e104a83f
fix: do not use exponential backoff for first retry of scrape request (#3824)
* fix: do not use exponential backoff for first retry of scrape request (#3293)

* lib/promscrape: refactor `doRequestWithPossibleRetry` backoff to simplify logic

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* Update lib/promscrape/client.go

Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>

* lib/promscrape: refactor `doRequestWithPossibleRetry` to make it more straightforward

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

---------

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
2023-02-24 11:39:56 -08:00
Aliaksandr Valialkin
8b7a828c65
app/vmselect/vmui: make vmui-update after d4fc0ed874 2023-02-23 19:25:52 -08:00
Aliaksandr Valialkin
a02cf92fd1
docs: update --help descriptions after recent changes 2023-02-23 19:02:27 -08:00
Aliaksandr Valialkin
b285207aa7
app/vmselect: add -search.logQueryMemoryUsage command-line flag for logging queries, which take big amounts of memory
Thanks to @michal-kralik for initial attempts for this feature:

- https://github.com/VictoriaMetrics/VictoriaMetrics/pull/3651
- https://github.com/VictoriaMetrics/VictoriaMetrics/pull/3715

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3553
2023-02-23 18:47:08 -08:00
Aliaksandr Valialkin
c080443fef
app/vmagent: automatically detect whether the remote storage supports VictoriaMetrics remote write protocol
Substitute -remoteWrite.useVMProto with -remoteWrite.forcePromProto command-line flag,
which can be used for forcing Prometheus remote write protocol in cases when the remote storage
supports VictoriaMetrics remote write protocol.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3847
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1225
2023-02-23 17:36:55 -08:00
Aliaksandr Valialkin
637043a40e
docs/CHANGELOG.md: document 6d019a3c37
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3830
2023-02-22 19:24:00 -08:00
Alexander Marshalov
aaef0bac00
fix interpolate function for filling only intermediate gaps (#3816) (#3857)
* fix interpolate function for filling only intermediate gaps (#3816)

* Update docs/CHANGELOG.md

---------

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2023-02-22 18:38:43 -08:00
Aliaksandr Valialkin
383bf9689e
docs/CHANGELOG.md: document d2b92d3264
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3747
2023-02-22 17:51:52 -08:00
Aliaksandr Valialkin
9fbd45a22f
lib/promscrape/discovery/kuma: follow-up for 317fef95f9
- Do not generate __meta_server label, since it is unavailable in Prometheus.
- Add a link to https://docs.victoriametrics.com/sd_configs.html#kuma_sd_configs to docs/CHANGELOG.md,
  so users could click it and read the docs without the need to search the corresponding docs.
- Remove kumaTarget struct, since it is easier generating labels for discovered targets
  directly from the response returned by Kuma. This simplifies the code.
- Store the generated labels for discovered targets inside atomic.Value. This allows reading them
  from concurrent goroutines without the need to use mutex.
- Use synchronouse requests to Kuma instead of long polling, since there is a little sense
  in the long polling when the Kuma server may return 304 Not Modified response every -promscrape.kumaSDCheckInterval.
- Remove -promscrape.kuma.waitTime command-line flag, since it is no longer needed when long polling isn't used.
- Set default value for -promscrape.kumaSDCheckInterval to 30s in order to be consistent with Prometheus.
- Remove unnecessary indirections for string literals, which are used only once, in order to improve code readability.
- Remove unused fields from discoveryRequest and discoveryResponse.
- Update tests.
- Document why fetch_timeout and refresh_interval options are missing in kuma_sd_config.
- Add docs to discoveryutils.RequestCallback and discoveryutils.ResponseCallback,
  since these are public types.

Side notes: it is weird that Prometheus implementation for kuma_sd_configs sets `instance` label,
since usually this label is set by the Prometheus itself to __address__ after the relabeling phase.
See https://www.robustperception.io/life-of-a-label/

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3389

See https://github.com/prometheus/prometheus/issues/7919
and https://github.com/prometheus/prometheus/pull/8844
as a reference implementation in Prometheus
2023-02-22 17:51:51 -08:00
Aliaksandr Valialkin
4f649a0573
docs/CHANGELOG.md: document 110c3896e7
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3600
2023-02-22 17:51:51 -08:00
Alexander Marshalov
b6845951a5
fixed typo in dns+srv documentation (#3861) 2023-02-23 02:40:00 +01:00
Artem Navoiev
84c82e988d Add Dig Security case study
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2023-02-22 16:38:57 +01:00
Alexander Marshalov
317fef95f9
add kuma_sd_config for Kuma Control Plane targets discovery (#3389) (#3840) 2023-02-22 13:59:56 +01:00
Artem Navoiev
6b4ccc17c2
docs: fix typo OpentTSDB -> OpenTSDB (#3854)
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2023-02-22 11:24:26 +01:00
Aliaksandr Valialkin
10cf6c9781
app/vmselect: allow zero value for -search.latencyOffset command-line flag
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2061#issuecomment-1299109836

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/218
2023-02-21 18:06:55 -08:00
Aliaksandr Valialkin
f04ec714c2
docs/Articles.md: mention rules backfilling via vmalert article
This is a follow-up for 5446ce0018
2023-02-21 17:44:08 -08:00
Roman Khavronenko
5446ce0018
docs: mention rules replay blogpost in vmalert docs (#3851)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-02-21 22:16:15 +01:00
panguicai
21546e6922
docs: update operator release name to be consistent with the following (#3845)
Signed-off-by: panguicai008 <1121906548@qq.com>
2023-02-21 16:57:54 +01:00
Aliaksandr Valialkin
7be8a8a37b
docs/CHANGELOG.md: fix a link to VictoriaMetrics remote write protocol 2023-02-20 19:59:44 -08:00
Aliaksandr Valialkin
6c38702212
docs/Single-server-VictoriaMetrics.md: remove + from start and end query args, since curl substitutes them with whitespace and breaks the query 2023-02-20 19:32:09 -08:00
Aliaksandr Valialkin
ef1ef0598b
docs/guides/migrate-from-influx.md: remove misleading doublequotes from the query example 2023-02-20 19:25:51 -08:00
Zakhar Bessarab
9d91d8fc91
vmgateway: add support of JWKS endpoint usage for JWT keys verification (#521) 2023-02-20 19:22:55 -08:00
Aliaksandr Valialkin
f4f1f2f976
docs/vmagent.md: remove the claim that VictoriaMetrics remote write protocol reduces the network bandwidth usage by up to 10x comparing to Prometheus remote write protocol
The 10x savings are reproduced only on artificial data.
The savings on production data are usually in the range 2x-4x.
2023-02-20 19:11:31 -08:00
Aliaksandr Valialkin
a678cbe62e
docs/vmagent.md: mention that Mimir doesnt support backfilling 2023-02-20 19:11:30 -08:00
Aliaksandr Valialkin
76f2c70be3
app/vmagent: add support for VictoriaMetrics remote write protocol, which allows saving up to 10x on network bandwidth costs under high load
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1225
2023-02-20 19:11:30 -08:00
Roman Khavronenko
74f122294d
docs: update vmalert docs (#3843)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-02-20 16:39:31 +01:00
Corporte Gadfly
d5171c155c
docs: typo fix (#3839) 2023-02-20 10:06:19 +01:00
Aliaksandr Valialkin
2cea923ff7
app/vmselect/promql: add share(q) aggregate function for normalizing results across multiple time series in [0..1] value range per each timestamp and aggregation group 2023-02-18 22:42:01 -08:00
Aliaksandr Valialkin
c86f1f1d1b
app/vmselect/promql: add range_zscore(q) and range_trim_zscore(z, q) functions
These functions may be useful for dropping outliers at https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3759
2023-02-18 22:41:59 -08:00
Aliaksandr Valialkin
406822a16c
app/vmselect/promql: add range_mad(q) and range_trim_outliers(k, q) functions
These functions may help trimming outliers during query time
for the use case described at https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3759
2023-02-18 15:19:10 -08:00
Haleygo
6ef6f3a771
vmalert: fix maxResolveDuration flag note (#3827)
Signed-off-by: Haleygo <hui.wang@daocloud.io>
2023-02-16 19:26:17 +01:00
Aliaksandr Valialkin
11ce30820b
all: update Go builder from Go1.20.0 to Go1.20.1
See https://github.com/golang/go/issues?q=milestone%3AGo1.20.1+label%3ACherryPickApproved
2023-02-14 23:05:16 -08:00
Aliaksandr Valialkin
f3be9483f4
docs/CHANGELOG.md: improve the docs for 8ea02eaa8e 2023-02-13 09:41:46 -08:00
Roman Khavronenko
a645a95bd6
docs: improve troubleshooting docs for vmalert (#3812)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-02-13 17:29:30 +01:00
Roman Khavronenko
934646ccf7
follow-up after d1cbc35cf6 (#3813)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-02-13 16:23:50 +01:00
Artem Navoiev
2e2b1ba87e
change docs for VictoriaMetrics Managed (#3803)
* change docs for VictoriaMetrics Managedd

Signed-off-by: Artem Navoiev <tenmozes@gmail.com>

* Update docs/managed-victoriametrics/user-managment.md

Co-authored-by: Max Golionko <8kirk8@gmail.com>

---------

Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
Co-authored-by: Max Golionko <8kirk8@gmail.com>
2023-02-13 13:29:03 +01:00
Aliaksandr Valialkin
e5070c0bcd
docs/sd_configs.md: properly escape __address__ string 2023-02-11 14:45:54 -08:00
Aliaksandr Valialkin
c5b9f7f751
docs/sd_configs.md: document how the __address__ label is generated per each discovered target 2023-02-11 14:41:59 -08:00
Aliaksandr Valialkin
25a9017a72
app/vmui: show median instead of avg on graph tooltip and line legend
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3706
2023-02-11 12:51:12 -08:00
Aliaksandr Valialkin
3ec8a4dc80
lib/{mergeset,storage}: allow at least 3 concurrent flushes during background merges on systems with 1 or 2 CPU cores
This should prevent from data ingestion slowdown and query performance degradation
on systems with small number of CPU cores (1 or 2), when big merge is performed.

This should help https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3790

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3337
2023-02-11 12:08:52 -08:00
Aliaksandr Valialkin
4156e78e11
docs/Articles.md: add a link to https://www.techetio.com/2022/08/21/evaluating-backend-options-for-prometheus-metrics/ 2023-02-11 12:08:52 -08:00
Aliaksandr Valialkin
b11bdc46be
app/vmselect/promql: add mad_over_time(m[d]) function
See https://github.com/prometheus/prometheus/issues/5514
2023-02-11 01:06:20 -08:00
Aliaksandr Valialkin
ed4492ddd5
all: update alpine base docker image from 1.17.1 to 1.17.2
See https://alpinelinux.org/posts/Alpine-3.17.2-released.html
2023-02-11 00:37:17 -08:00
Aliaksandr Valialkin
776391917f
app/vmauth: improve load balancing by sending incoming requests to backends with the lowest number of concurrent requests
While at it, stop sending requests to unavailable backend for 3 seconds
before the next attempt. This should reduce the amounts of useless work
and the number of useless network packets when the backend is temporarily unavailable.
2023-02-11 00:30:31 -08:00
Aliaksandr Valialkin
f3625e4f3f
app/vmauth: add -maxConcurrentPerUserRequests command-line option for limiting the number of concurrent requests on a per-user basis
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3346
2023-02-10 21:58:21 -08:00
Aliaksandr Valialkin
70f8911ca7
app/vmauth: automatically retry failing GET requests on the remaining backends 2023-02-09 21:05:55 -08:00
Dmytro Kozlov
f582f9e8ab
app/vmauth: add concurrent requests limit per auth record (#3749)
* app/vmauth: add concurent requests limit per auth record

* app/vmauth: added clarification comment

* app/vmauth: remove unused code

* app/vmauth: move read from limiter

* app/vmauth: fix text

* app/vmauth: fix comments

* - Clarify the docs for the max_concurrent_requests option at docs/vmauth.md
- Clarify the description of the change at docs/CHANGELOG.md
- Make sure that the -maxConcurrentRequests takes precedence over per-user max_concurrent_requests
- Update tests for verifying that the max_concurrent_requests option is parsed properly

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3346

---------

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2023-02-09 20:03:01 -08:00
Aliaksandr Valialkin
73358571ee
app/vmalert: follow-up after d3c64aae8768d58781ee7e358bd7f3d8e0eb836d
- Document the change at docs/CHANGELOG.md
- Add `Reading rules from object storage` section to docs/vmalert.md
- Add `s3` prefix to command-line flags related to the configuration of s3 and gcs clients
- Explicitly mention that reading rules from object storage is supported only in enterprise version
2023-02-09 18:52:00 -08:00
Aliaksandr Valialkin
d4b97b69bf
docs/CHANGELOG.md: document d621d50d4fb3b43a0bcb4419bee979f0192d38fe 2023-02-09 14:40:07 -08:00
Aliaksandr Valialkin
a42e3e8dfb
docs/CHANGELOG.md: cut v1.87.1 and mark 1.87.x as LTS release 2023-02-09 11:20:57 -08:00
Zakhar Bessarab
f13a255918
lib/promscrape: fix cancelling in-flight scrape requests during configuration reload (#3791)
* lib/promscrape: fix cancelling in-flight scrape requests during configuration reload when using `streamParse` mode (see #3747)

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* Update docs/CHANGELOG.md

---------

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2023-02-09 11:13:06 -08:00
Aliaksandr Valialkin
513707a8c7
app/vmui: UX enhancements for https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3706
- Display `min` value additionally to `avg`, `max` and `last`
- Allow copy-n-pasting metric name with its labels from both legend and tooltup
2023-02-09 11:04:51 -08:00
Aliaksandr Valialkin
f40661e7b7
docs/vmagent.md: clarify that automatically generated metrics contain all the target-specific labels, including instance and job 2023-02-09 11:04:51 -08:00
Air
a1432e6b0a Possibly spelling in the Quick start 2023-02-09 15:50:20 +02:00
Yury Molodov
e1063ce3c1
vmui: improve tenant selector (#3794)
* fix: change styles tenant selector (#3792)

* docs/CHANGELOG.md: document the change

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3792

---------

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2023-02-09 00:08:59 -08:00
Aliaksandr Valialkin
46a521191f
docs/CHANGELOG.md: document changes at v1.79.8 LTS release 2023-02-08 23:38:46 -08:00
Yury Molodov
8afc0aef8d
vmui: add last/max/avg values (#3789)
* feat: add last/max/avg values (#3706)

* fix: change filter exclude values

* app/vmui: wip

- improve the visualization for avg/max/last values
- make getAvgFromArray() function resilient against inf/undefined/nil
- export getLastFromArray() function, which is resilient against inf/undefined/nil
- run `make vmui-update`

---------

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2023-02-08 22:41:20 -08:00
Aliaksandr Valialkin
114c14febf
docs/CHANGELOG.md: document 75bcf86a31
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3740
2023-02-08 11:24:07 -08:00
Aliaksandr Valialkin
a1ee679042
docs/CHANGELOG.md: add more context to the bugfix description in Nomad service discovery
See 146fd2eca3
2023-02-08 09:24:48 -08:00
Aliaksandr Valialkin
6c21b6ec09
docs/CHANGELOG.md: document the change at 67b01329a0
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3761
2023-02-08 08:42:19 -08:00
Roman Khavronenko
e83f14210d
Vmalert fixes (#3788)
* vmalert: use group's ID in UI to avoid collisions

Identical group names are allowed. So we should used IDs
for various groupings and aggregations in UI.

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* vmalert: prevent disabling state updates tracking

The minimum number of update states to track is now set to 1.

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* vmalert: properly update `debug` and `update_entries_limit` params on hot-reload

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* vmalert: display `debug` field for rule in UI

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* vmalert: exclude `updates` field from json marhsaling

This field isn't correctly marshaled right now.
And implementing the correct marshaling for it doesn't
seem right, since json representation is mostly used
by systems like Grafana. And Grafana doesn't expect this
field to be present.

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* fix test for disabled state

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* fix test for disabled state

Signed-off-by: hagen1778 <roman@victoriametrics.com>

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-02-08 14:34:03 +01:00
Roman Khavronenko
86e47177dc
docs: follow-up after 2e4bfcce63 (#3785)
2e4bfcce63

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-02-08 09:48:05 +01:00
Aliaksandr Valialkin
1901fbf19b
docs/CHANGELOG.md: fix formatting for the change from 6fd10e8871 2023-02-07 09:34:57 -08:00
earthgecko
3a51a3bc42
Clarifications between standalone/cluster ingestion endpoints (#3771)
docs: clarifications between standalone/cluster ingestion endpoints

This is an attempt to make it a bit clearer to the user that the cluster version ingestion URLs are different from the standalone ones.  I have also changed the order of the list items to make it a bit clearer and hopefully stop the user simply inferring that `/prometheus/api/v1` is only related to Prometheus data.
2023-02-07 15:17:12 +01:00
Max Golionko
977c642934
docs: update formatiing for k8s monitoring with Managed VictoriaMetrics (#3768)
* jekyll formatting madness
2023-02-06 18:33:40 +08:00
Roman Khavronenko
c32d8ea29e
vmalert: update docs (#3770)
vmalert: update flags description

Signed-off-by: hagen1778 <roman@victoriametrics.com>

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-02-06 09:51:30 +01:00
Roman Khavronenko
6fd10e8871
vmalert: speed up state restore procedure on start (#3758)
* vmalert: speed up state restore procedure on start

Alerts state restore procedure has been changed to become asynchronous.
It doesn't block groups start anymore which significantly improves vmalert's startup time.
Instead, state restore is called by each group in their goroutines after the first rules
evaluation.

While previously state restore attempt was made for all loaded alerting rules,
now it is called only for alerts which became active after the first evaluation.
This reduces the amount of API calls to the configured remote read URL.

This also means that `remoteRead.ignoreRestoreErrors` command-line flag becomes deprecated now
and will have no effect if configured.

See relevant issue https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2608

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* make lint happy

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* Apply suggestions from code review

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2023-02-03 19:46:13 -08:00
Aliaksandr Valialkin
98dc968920
docs/CHANGELOG.md: document f63f487787
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3707
2023-02-03 19:30:12 -08:00
Aliaksandr Valialkin
7440f971ab
docs/MetricsQL.md: add links to "rollup results" explanation 2023-02-03 11:09:42 -08:00
Aliaksandr Valialkin
12cf8d9f69
docs/managed-victoriametrics/how-to-monitor-k8s.md: rename image files according to docs/assets/README.md 2023-02-03 10:43:02 -08:00
Max Golionko
e18f8e9413
docs: move managed victoria metics guide into right folder (#3750)
* move guide folder
* image width control
2023-02-03 03:13:36 +08:00
Max Golionko
07b7fe83c4 Update docs/managed_victoriametrics/how-to-monitor-k8s.md
Co-authored-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2023-02-02 15:43:55 +02:00
Max Golionko
a2ba1f09e4 add guide to list of guides 2023-02-02 15:43:55 +02:00
Max Golionko
79527441ec added k8s guide for managed VM 2023-02-02 15:43:55 +02:00
Aliaksandr Valialkin
dc142867b8
docs/CHANGELOG.md: typo fixes 2023-02-01 20:40:44 -08:00
Aliaksandr Valialkin
fe736c5388
docs/CHANGELOG.md: cut v1.87.0 2023-02-01 13:03:11 -08:00
Dmytro Kozlov
7b87fac8e7
lib/promscrape: fix honor_labels behavior (#3739)
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2023-02-01 11:21:44 -08:00
Aliaksandr Valialkin
7b1caf1db3
docs/CHANGELOG.md: document 9254e494f9 2023-02-01 09:56:52 -08:00
Zakhar Bessarab
68985455f1
fix: vmselect multi-level setup panic (#3738)
* app/vmselect/netstorage: fix panic for multi-level cluster setup when `replicationFactor` was set and request contained `trace` parameter (#3734)

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* app/vmselect/netstorage: use correct context for retry

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

---------

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2023-02-01 08:59:30 -08:00
Zakhar Bessarab
4cf37c5e70
app/vmbackup: fix deleting snapshot after backup completion (#3735) (#3736)
Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2023-02-01 11:23:58 +01:00
Aliaksandr Valialkin
4ecb61e247
docs/CHANGELOG.md: document 442a9f16b4
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3661
2023-01-31 13:03:46 -08:00
Roman Khavronenko
1cbdcd391c
docs: mention -vmalert.proxyURL in vmalert docs (#3730)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-01-30 16:28:33 +01:00
Aliaksandr Valialkin
0788be35eb
lib/promscrape/discovery/azure: add __meta_azure_machine_size label in the same way as Prometheus does
See https://github.com/prometheus/prometheus/pull/11650
2023-01-27 17:07:12 -08:00
Aliaksandr Valialkin
ab57b92932
lib/promscrape/discovery/kubernetes: add support for __meta_kubernetes_pod_container_id
See https://github.com/prometheus/prometheus/issues/11843
and https://github.com/prometheus/prometheus/pull/11844
2023-01-27 16:34:06 -08:00
Yury Molodov
ac14d50c18
vmui: add select of Tenant ID (#3673)
* feat: add select of tenantID

* feat: replace tenantID to default url

* fix: move the tenantID selector to the top header

* fix: hide tenantID selector by condition

* fix: correct z-index

* app/vmselect/vmui: `make vmui-update`

---------

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2023-01-27 15:53:14 -08:00
Aliaksandr Valialkin
51ad94677c
docs: update security chapters after bd716d1b0c 2023-01-27 15:45:35 -08:00
Denys Holius
bd716d1b0c
Improving docs by adding additional security sections (#3713)
* docs/Cluster-VictoriaMetrics.md: adds security section

* docs/Quick-Start.md: adds Security recommendation section
2023-01-27 15:22:21 -08:00
Aliaksandr Valialkin
06f6b76521
app/vmagent: properly return 200 response code when importing data via Prometheus PushGateway protocol
This is the same fix as has been already applied to app/vminsert at cdb6d651e9

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3636
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1415
2023-01-27 14:40:02 -08:00
Aliaksandr Valialkin
a0c8b86eab
docs/vmauth.md: update docs after ff39a91147
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3346
2023-01-27 14:10:19 -08:00
Aliaksandr Valialkin
ff39a91147
app/vmauth: limit the number of concurrent requests served by vmauth with the -maxConcurrentRequests command-line flag
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3346

This commit is based on the https://github.com/VictoriaMetrics/VictoriaMetrics/pull/3486
2023-01-27 14:07:30 -08:00
Aliaksandr Valialkin
372b1688d7
app/vmauth: do not use net/http/httputil.ReverseProxy
This allows better controlling requests to backends and providing better error logging.
For example, if the backend was unavailable, then the ReverseProxy was logging the error
message without client ip and the initial request uri. This could harden debugging.

This is based on https://github.com/VictoriaMetrics/VictoriaMetrics/pull/3486
2023-01-27 13:40:05 -08:00
Aliaksandr Valialkin
2afa4ae00a
docs/managed-victoriametrics: typo fix in links to images 2023-01-27 11:36:20 -08:00
Aliaksandr Valialkin
6342eb5523
docs/assets/README.md: mention that locally placed doc-specific images simplify referring them from various views without the need to deal with folder prefixes 2023-01-27 11:29:23 -08:00
Aliaksandr Valialkin
54a0ccbaca
docs/managed-victoriametrics/user-management.md: move the associated images to docs/managed-victoriametrics/ folder with user-management_ prefix according to docs/assets/README.md 2023-01-27 11:25:22 -08:00
Aliaksandr Valialkin
b28cf0faa8
docs/managed-victoriametrics/quickstart.md: move the associated images to docs/managed-victoriametrics/ folder with quickstart_ prefix according to docs/assets/README.md 2023-01-27 11:16:03 -08:00
Aliaksandr Valialkin
3251d392b5
docs/assets: add README.md with the explanation on which files can be put into the docs/assets folder 2023-01-27 11:02:16 -08:00
Aliaksandr Valialkin
119010f7f2
docs/Cluster-VictoriaMetrics.md: move Naive_cluster_scheme.png from the docs/assets/images/ folder into docs/ folder and add Cluster-VictoriaMetrics_ prefix to the image name
The docs/assets folder should be used only for assets specific to docs generation at https://docs.victoriametrics.com, e.g. css, js and images.

All the other assets related to specific docs should be placed in the same folder as the corresponding *.md file.
These assets should have the same name prefix as the corresponding doc file name. This simplifies tracking the lifetime of these assets.
For example, if the doc is removed, it is very easy to remove all assets associated with it with a simple `rm -rf docs/doc-name*` command.

This also simplifies generating correct urls for doc-specific assets from both https://docs.victoriametrics.com
and from https://github.com/VictoriaMetrics/VictoriaMetrics/blob/master/docs/ - just refer to the asset name without any directory prefixes.
2023-01-27 10:54:27 -08:00
dmitryk-dk
4250b67fe8 docs: move how to register from dbaas to docs 2023-01-27 14:07:09 +02:00
dmitryk-dk
465c889f7f docs: use absolute path 2023-01-27 14:07:09 +02:00
dmitryk-dk
834c18a346 docs: add documentation of user management on managed-vm 2023-01-27 14:07:09 +02:00
Aliaksandr Valialkin
558165521b
docs/Cluster-VictoriaMetrics.md: update command-line descriptions after ebebaecd94 2023-01-27 00:04:41 -08:00
Aliaksandr Valialkin
0890adde67
docs: update command-line descriptions after 73256fe438 2023-01-27 00:00:37 -08:00
Nikolay
73256fe438
lib/netutil: init implimentation of proxy protocol (#3687)
* lib/netutil: init implimentation of proxy protocol
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3335

* wip

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2023-01-26 23:08:35 -08:00
Aliaksandr Valialkin
2c1419c687
docs/CHANGELOG.md: make the description for the bugfix from 465a285324 more reader-friendly 2023-01-26 10:08:13 -08:00
Nikolay
465a285324
lib/storage: properly release parts inMerge lock (#3711)
if storage doesn't have enough disk space, finalDedupWatcher holds inMerge lock for all parts and never release it until storage restart
2023-01-26 08:05:20 -08:00
Roman Khavronenko
95ee86b600
docs: specify the time window for series_limit (#3708)
Signed-off-by: hagen1778 <roman@victoriametrics.com>

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-01-25 09:30:20 -08:00
Aliaksandr Valialkin
28f66f0079
docs: update the list of command-line flags according to the latest changes 2023-01-25 09:20:24 -08:00
Aliaksandr Valialkin
d655d6b047
lib/streamaggr: add ability to de-duplicate input samples before aggregation 2023-01-25 09:14:49 -08:00
Yury Molodov
99d49e3ceb
vmui: include fonts in its bundle (#3705)
* feat: include fonts in the build

* fix: reduce size fonts

* wip

- Document the change at docs/CHANGELOG.md
- Run `make vmui-update`

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2023-01-24 09:30:56 -08:00
Yury Molodov
20ad848c5d
vmui: improvements to the UI styles (#3704)
* feat: add dark theme

* update packages

* feat: add multilevel menu (#3678)

* fix: correct styles

* fix: update link to cardinality-explorer

* fix: remove unused scss variables

* docs/CHANGELOG.md: document the changes

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2023-01-24 09:20:31 -08:00
Roman Khavronenko
1a8875b417
discover/ec2: follow-up after e2b4ab8384 (#3703)
Signed-off-by: hagen1778 <roman@victoriametrics.com>

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-01-24 11:02:18 +01:00
Aliaksandr Valialkin
a971bcc3fe
lib/bytesutil: do not intern long strings, since they may need big amounts of additional memory for the cache
Allow users fine-tuning the maximum string length for interning via -internStringMaxLen command-line flag.
This may be used for fine-tuning RAM vs CPU usage for certain workloads.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3692
2023-01-23 23:36:22 -08:00
Aliaksandr Valialkin
babecd8363
lib/promscrape: follow-up for 393876e52a
- Document the change in docs/CHANGELOG.md
- Reduce memory usage when sending stale markers even more by parsing the response in stream parsing mode
- Update the TestSendStaleSeries

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3668
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3675
2023-01-23 21:52:59 -08:00
Denys Holius
f3cb412508
Fix/remove vmanomaly from release guide (#3699)
* docs/Release-Guide.md: remove vmanomaly from release guide because it has own release cycle

* fixed a typo
2023-01-23 14:32:26 +01:00
Aliaksandr Valialkin
edc51b3119
docs/Articles.md: added missing third-party articles about VictoriaMetrics 2023-01-22 14:23:45 -08:00
Aliaksandr Valialkin
cb5d073bc9
docs/Single-server-VictoriaMetrics.md: make it clear that VictoriaMetrics supports both pull and push protocols at how to import time series data chapter 2023-01-22 14:04:29 -08:00
Aliaksandr Valialkin
864c651c03
docs/Articles.md: add https://dev.to/aws-builders/ultra-monitoring-with-victoria-metrics-1p2 2023-01-22 13:51:53 -08:00
Aliaksandr Valialkin
4e25bc2087
lib/vmselectapi: propagate timeout errors from vmselect to vmstorage instead of closing the connection established from vmselect to vmstorage
This is a follow-up for 20e9598254
2023-01-20 19:33:42 -08:00
Aliaksandr Valialkin
74df30456b
app/vmselect: make vmui-update after df7b81b44d 2023-01-20 12:07:13 -08:00
Yury Molodov
df7b81b44d
vmui: add support for time zone selection for older versions of browsers (#3680)
* fix: add check for support of getting time zones

* vmui: add support for time zone selection for older versions of browsers
2023-01-20 11:47:53 -08:00
Denys Holius
d513230d43
Adds some improvements to release guide docs (#3679)
* docs/Release-Guide.md: fixed a typo

* Release-Guide.md: adds missed steps for updating vmanomaly and vmgateway helm charts
2023-01-19 16:03:42 +01:00
Aliaksandr Valialkin
9785b3f57f
docs/CHANGELOG.md: cut v1.86.2 2023-01-18 12:01:07 -08:00
Aliaksandr Valialkin
19e28ce7b6
docs/CHANGELOG.md: document 777038fe44
See https://github.com/VictoriaMetrics/VictoriaMetrics/pull/3672
2023-01-18 11:39:48 -08:00
Aliaksandr Valialkin
68463c9e87
lib/promscrape: follow-up for d79f1b106c
- Document the fix at docs/CHANGELOG.md
- Limit the concurrency for sendStaleMarkers() function in order to limit its memory usage
  when big number of targets disappear and staleness markers are sent
  for all the metrics exposed by these targets.
- Make sure that the writeRequestCtx is returned to the pool
  when there is no need to send staleness markers.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3668
2023-01-17 23:11:56 -08:00
Zakhar Bessarab
322d96bfe5
discovery/{consul,nomad}: fix cancelling serviceWatcher in-flight requests (#3658)
* lib/promscrape/discovery/{consul,nomad}: fix background service update watches not canceling requests on serviceWatcher stop

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* lib/promscrape/discovery/{consul,nomad}: fix closing serviseWatcher during scrape job restart

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* wip

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3468

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2023-01-17 21:47:11 -08:00
Yury Molodov
fcef2ff6b2
vmui: correctly display range results in Table view (#3657)
* fix: properly display range results

* fix: set range values to empty array

* wip

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2023-01-17 21:03:28 -08:00
Aliaksandr Valialkin
1ffa793322
docs/CHANGELOG.md: fix the link to feature request implemented at e58921aa8f
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3571
2023-01-17 20:27:43 -08:00
Yury Molodov
e58921aa8f
vmui: give more visually different colors to graph lines (#3656)
* feat: make more different colors of graph lines

* docs/CHANGELOG.md: give more visually different colors to graph lines
2023-01-17 20:25:37 -08:00
Aliaksandr Valialkin
006af394ff
vendor: update github.com/VictoriaMetrics/metricsql from v0.51.1 to v0.51.2
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3664
2023-01-17 11:26:41 -08:00
Aliaksandr Valialkin
289af65071
lib/promscrape: properly apply series limit
Fix the following issues:

- Series limit wasn't applied when staleness tracking was disabled.
- Series limit didn't prevent from sending staleness markers for new series exceeding the limit.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3660

Thanks to @hagen1778 for the initial attempt to fix the issue
at https://github.com/VictoriaMetrics/VictoriaMetrics/pull/3665
2023-01-17 10:14:49 -08:00
Aliaksandr Valialkin
e06168f489
docs/CHANGELOG.md: document the 74f196c37813a18c2c8f28831696ed7d9f535604 2023-01-17 09:10:41 -08:00
Aliaksandr Valialkin
09d7fa2737
lib/{mergeset,storage}: do not slow down concurrently executed queries during assisted merges
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3647
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3641
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/648
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/291
2023-01-16 14:31:52 -08:00
Yury Molodov
04c408e986
vmui: make the step input field global across all the tabs and views (#3644)
* feat: make the step input field global

* fix: correct get step from url

* fix: set minimumSignificantDigits to 1

* app/vmselect/vmui: `make vmui-update`

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2023-01-15 13:47:08 -08:00
Aliaksandr Valialkin
cdb6d651e9
app/vminsert: return 200 OK status code when importing data in pushgateway format
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1415
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3636
2023-01-15 13:29:53 -08:00
Denys Holius
ffe6e6fe59
guides/guide-delete-or-replace-metrics.md: fixed wrong curl command (#3652) 2023-01-15 13:02:52 -08:00
Corporte Gadfly
f32a79189b
docs: fix typo (#3648) 2023-01-13 09:39:11 +01:00
Aliaksandr Valialkin
be8fba9b6a
app/vmselect/netstorage: tune the number of blocks per series which should be unpacked by a single goroutine instead of spinning up multiple goroutines
This reduces overhead on time series data unpacking for typical cases,
this reducing CPU usage at vmselect
2023-01-12 09:31:44 -08:00
Aliaksandr Valialkin
98de06ff38
docs/CHANGELOG.md: update the description of the change at 20f28eb9d6 2023-01-12 08:58:52 -08:00
Nikolay
20f28eb9d6
/lib/promscrape: use correct err logger for scrape unmarshalling (#3645)
/lib/promscrape: use correct err logger for scrape unmarshalling
It correctly suppresses scrape errors and adds correct context for err msg
2023-01-12 17:40:42 +01:00
Aliaksandr Valialkin
9fdd1a10c6
docs/Cluster-VictoriaMetrics.md: mention the -vmui.customDashboardsPath command-line flag at vmselect
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3322
2023-01-11 23:39:55 -08:00
Aliaksandr Valialkin
a194982117
app/vmselect: follow-up after 820312a2b1
- Move the feature description at the correct place at docs/CHANGELOG.md
- Run `make vmui-update`
- Various cosmetic fixes

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3322
2023-01-11 23:28:00 -08:00
Dmytro Kozlov
820312a2b1
app/vmui: define custom path for dashboards json file (#3545)
* app/vmui: define custom path for dashboards json file

* app/vmui: remove unneeded code

* app/vmui: move handler to own file, fix show dashboards,

* app/vmui: move flag to handler, add flag description

* app/vmauth: fix part of the comments

* feat: add store for dashboards

* fix: prevent fetch dashboards for app mode

* app/vmauth: use simple cache for predefined dashboards

* app/vmauth: update dashboards doc

* app/vmauth: fix ci

* app/vmui: decrease timeout

* app/vmselect: removed cache, fix comments

* app/vmselect: remove unused const

* app/vmselect: fix error log, use slice byte instead of struct

Co-authored-by: Yury Moladau <yurymolodov@gmail.com>
2023-01-11 23:06:07 -08:00
Zakhar Bessarab
8537533beb
lib/promscrape/discovery/dockerswarm: fix discovery filters being applied to all objects (#3632)
* lib/promscrape/discovery/dockerswarm: fix discovery filters being applied to all objects

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* Update docs/CHANGELOG.md

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2023-01-11 22:50:34 -08:00
Artem Navoiev
13a1a7f826
fix operator docs hugo schema
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2023-01-11 17:28:34 +01:00
Artem Navoiev
83c87f822a docs: prepare operator docs to migration
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2023-01-11 18:19:40 +02:00
Artem Navoiev
54aec2b1ba
docs: operator *.MD -> *.md
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2023-01-11 16:29:29 +01:00
Aliaksandr Valialkin
351fc152e0
docs/CHANGELOG.md: cut v1.86.1 2023-01-11 01:26:43 -08:00
Aliaksandr Valialkin
975bb8722f
lib/vmselectapi: properly calculate query timeout
vmselect passes query timeout to vmstorage in seconds.
The commit 20e9598254 treated it as timeout in nanoseconds.
Fix this in order to prevent from the following errors under vmstorage load:

cannot process vmselect request: cannot execute "search_v7": couldn't start executing the request in 0.000 seconds,
since -search.maxConcurrentRequests=... concurrent requests are already executed.
2023-01-11 01:22:33 -08:00
Aliaksandr Valialkin
0771d57860
app/vmselect/promql: make a copy of per-series timestamps before their modification
The per-series timestamps are usually shared among series, so it is unsafe modifying them.

The issue has been appeared after the optimization at 2f3ddd4884
2023-01-11 00:59:13 -08:00
Aliaksandr Valialkin
a0f9cb27f9
docs/CHANGELOG.md: document v1.79.7 LTS release 2023-01-10 21:24:28 -08:00
Aliaksandr Valialkin
37fe04999c
docs/CHANGELOG.md: add release date for v1.86.0 2023-01-10 17:45:57 -08:00
Aliaksandr Valialkin
28df7c2a96
docs/CHANGELOG.md: cut v1.86.0 2023-01-10 16:22:26 -08:00
Zakhar Bessarab
4225a0bd75
Use httpAuth.* flags as a fallback for endpoints protected by *AuthKey flags (#3582)
* {lib/server, app/}: use `httpAuth.*` flag as fallback for `*AuthKey` if it is not set

* lib/ingestserver/opentsdbhttp: fix opentdb HTTP handler not respecting `httpAuth.*` flags

* Apply suggestions from code review

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2023-01-10 15:46:13 -08:00
Dmytro Kozlov
0811000bb0
app/vmctl: Add insecure skip verify flag for remote read protocol (#3611)
* app/vmctl: Add insecure skip verify flag for remote read protocol
2023-01-10 23:18:49 +01:00
Artem Navoiev
37c52ccaf4
vmagent: add minimal scrape file exampe for vmagent quick start (#3627)
* vmagent: add minimal scrape file exampe for vmagent quick start

Signed-off-by: Artem Navoiev <tenmozes@gmail.com>

* replace example with link to your prometheus.yml in docker

Signed-off-by: Artem Navoiev <tenmozes@gmail.com>

Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2023-01-10 23:16:10 +01:00
Aliaksandr Valialkin
cbe62f23ba
lib/promscrape/discovery/gce: follow-up for b2ccdaaa2f
- Use promutils.Labels.GetLabels() instead of comparing promutils.Labels.Labels to nil.
  This make the code more consistent with other places.

- Mention the release where the issue has been introduced at docs/CHANGELOG.md.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3624
2023-01-10 13:51:03 -08:00
Zakhar Bessarab
b2ccdaaa2f
lib/promscrape/discovery/gce: fix crash in case instance does not have any labels set (#3625)
Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2023-01-10 11:07:11 +01:00
Denys Holius
b06e795a1e
docs/Release-Guide.md: added missed link to rpm repository (#3623) 2023-01-10 10:14:56 +01:00
Aliaksandr Valialkin
e5eca54951
lib/promscrape/discovery/nomad: sync nomad_sd_configs fields with the Prometheus implementation
See the list of configs supported by Prometheus at f88a0a7d83/discovery/nomad/nomad.go (L76-L84)

- Removed "token" option. In can be set either via NOMAD_TOKEN env var or via `bearer_token` config option.
- Removed "scheme" option. It is automatically detected depending on whether the `tls_config` is set.
- Removed "services" and "tags" options, since they aren't supported by Prometheus.
- Added "region" option. If it is missing, then the region is read from NOMAD_REGION env var.
  If this var is empty, then it is set to "global" in the same way as Nomad client does.
  See 865ee8d37c/api/api.go (L297)
  and 865ee8d37c/api/api.go (L555-L556)
- If the "server" option is missing, then it is read from NOMAD_ADDR in the same way
  as Nomad client does - see 865ee8d37c/api/api.go (L294-L296)

This is a follow-up for 8aee209c53

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3367
2023-01-09 21:14:48 -08:00
Aliaksandr Valialkin
1f9d605988
app/vmselect/netstorage: consistently select the sample with the biggest value out of samples with identical timestamps
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3333

This fix is based on https://github.com/VictoriaMetrics/VictoriaMetrics/pull/3620 ,
but doesn't slow down the common case with merging replicated data blocks so significantly.

Benchmark results:

Before the change:

BenchmarkMergeSortBlocks/replicationFactor-1-4         	   13968	     85643 ns/op	 956.53 MB/s	    1700 B/op	       1 allocs/op
BenchmarkMergeSortBlocks/replicationFactor-2-4         	   10806	    109171 ns/op	1500.77 MB/s	    2191 B/op	       1 allocs/op
BenchmarkMergeSortBlocks/replicationFactor-3-4         	    8887	    130623 ns/op	1881.45 MB/s	    2660 B/op	       1 allocs/op
BenchmarkMergeSortBlocks/replicationFactor-4-4         	    7440	    157348 ns/op	2082.52 MB/s	    3174 B/op	       1 allocs/op
BenchmarkMergeSortBlocks/replicationFactor-5-4         	    6534	    184473 ns/op	2220.38 MB/s	    3612 B/op	       1 allocs/op
BenchmarkMergeSortBlocks/overlapped-blocks-bestcase-4  	   13419	     85205 ns/op	 961.44 MB/s	    2213 B/op	       1 allocs/op
BenchmarkMergeSortBlocks/overlapped-blocks-worstcase-4 	     579	   1894900 ns/op	  43.23 MB/s	   46760 B/op	       1 allocs/op

After the change:

BenchmarkMergeSortBlocks/replicationFactor-1-4         	   13832	     85298 ns/op	 960.40 MB/s	    1716 B/op	       1 allocs/op
BenchmarkMergeSortBlocks/replicationFactor-2-4         	    8833	    134222 ns/op	1220.66 MB/s	    2675 B/op	       1 allocs/op
BenchmarkMergeSortBlocks/replicationFactor-3-4         	    6487	    184830 ns/op	1329.65 MB/s	    3636 B/op	       1 allocs/op
BenchmarkMergeSortBlocks/replicationFactor-4-4         	    4977	    236318 ns/op	1386.61 MB/s	    4733 B/op	       1 allocs/op
BenchmarkMergeSortBlocks/replicationFactor-5-4         	    4088	    296734 ns/op	1380.36 MB/s	    5761 B/op	       1 allocs/op
BenchmarkMergeSortBlocks/overlapped-blocks-bestcase-4  	   14083	     84067 ns/op	 974.47 MB/s	    2110 B/op	       1 allocs/op
BenchmarkMergeSortBlocks/overlapped-blocks-worstcase-4 	     536	   2043534 ns/op	  40.09 MB/s	   50511 B/op	       1 allocs/op
2023-01-09 13:01:48 -08:00
Roman Khavronenko
8aee209c53
lib/promscrape: remove datacenter field from nomad_sd_config (#3612)
Looks like `datacenter` field isn't part of `/v1/services` API.
See https://developer.hashicorp.com/nomad/api-docs/services#list-services
and https://developer.hashicorp.com/nomad/api-docs/services#read-service

Related issues:
https://github.com/traefik/traefik/issues/9109
https://github.com/prometheus/prometheus/issues/11776

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-01-09 09:07:40 +01:00
Dmytro Kozlov
488940502c
lib/storage: fix returning camelcase label names (#3608)
* lib/storage: fix returning camelcase label names

* doc: add change log

* Update docs/CHANGELOG.md

* Update docs/CHANGELOG.md

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2023-01-07 00:50:14 -08:00
Aliaksandr Valialkin
af263fe881
all: small improvements in error messages and command-line flag descriptions related to concurrency limiters 2023-01-07 00:11:44 -08:00
Aliaksandr Valialkin
986a05e18d
lib/promscrape: limit the concurrency during parsing and relabeling the scraped samples
This should reduce memory usage when scraping big number of targets,
since this limits the summary memory usage during concurrent parsing and relabeling
by the number of available CPU cores.
2023-01-06 22:59:17 -08:00
Aliaksandr Valialkin
c63755c316
lib/writeconcurrencylimiter: improve the logic behind -maxConcurrentInserts limit
Previously the -maxConcurrentInserts was limiting the number of established client connections,
which write data to VictoriaMetrics. Some of these connections could be idle.
Such connections do not consume big amounts of CPU and RAM, so there is a little sense in limiting
the number of such connections. So now the -maxConcurrentInserts command-line option
limits the number of concurrently executed insert requests, not including idle connections.

It is recommended removing -maxConcurrentInserts command-line option, since the default value
for this option should work good for most cases.
2023-01-06 22:20:19 -08:00
Aliaksandr Valialkin
f299d2ca1a
lib/vmselectapi: limit the number of concurrently executed requests
This should prevent from out of memory errors when big number of vmselect
nodes send many concurrent requests to vmstorage

The limit can be controlled at vmstorage via the following command-line flags:
- search.maxConcurrentRequests
- search.maxQueueDuration

See https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#resource-usage-limits
2023-01-06 22:11:34 -08:00
Aliaksandr Valialkin
f392913d00
lib/promscrape: follow-up after bced9fb978
- Document the bugfix at docs/CHANGELOG.md
- Wait until all the worker goroutines are done in consulWatcher.mustStop()
- Do not log `context canceled` errors when discovering consul serviceNames
- Removed explicit handling of gzipped responses at lib/promscrape/discoveryutils.Client,
  since this handling is automatically performed by net/http.Transport.
  See DisableCompression option at https://pkg.go.dev/net/http#Transport .
- Remove explicit handling of the proxyURL, since it is automatically handled
  by net/http.Transport. See Proxy option at https://pkg.go.dev/net/http#Transport .
- Expliticly set MaxIdleConnsPerHost, since its default value equals to 2.
  Such a small value may result in excess tcp connection churn
  when more than 2 concurrent requests are processed by lib/promscrape/discoveryutils.Client.
- Do not set explicitly the `Host` request header, since it is automatically set by net/http.Client.
- Backport the bugfix to the recently added nomad_sd_configs - see https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3367

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3468
2023-01-05 21:13:06 -08:00
Aliaksandr Valialkin
9f348cf8a1
lib/promscrape/discovery/nomad: follow-up after 48f371a46c
- Remove undocumented `username` and `password` config options from `nomad_sd_config`.
  TODO: probably, remove these options from `consul_sd_config` too?
  These options exist there for backwards compatibility purposes.

- Add __meta_nomad_service_alloc_id and __meta_nomad_service_job_id meta-labels
  These labels contain AllocID and JobID fields for the discovered Nomad services.

- Various typo fixes.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3367
2023-01-05 18:07:20 -08:00
Karan Sharma
48f371a46c
lib/promscrape: add Prometheus-compatible service discovery for Nomad (#3549)
Add nomad_sd_config support for service discovery
2023-01-05 23:03:58 +01:00
Zakhar Bessarab
01bc0c94ab
doc: add vmbackupmanager monitoring section (#3605)
* doc: add vmbackupmanager monitoring section

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2023-01-05 16:03:06 +01:00
Artem Navoiev
8b763175ff
Add Understand Your Setup Size Guide (#3572)
docs: add Understand Your Setup Size Guide

Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2023-01-05 14:56:50 +01:00
Aliaksandr Valialkin
2ee81a5dbb
docs/CHANGELOG.md: add missing dot 2023-01-05 03:35:02 -08:00
Zakhar Bessarab
185cdcd813
lib/promscrape/discovery/dockerswarm: fix query encoding of filters (#3586)
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2023-01-05 03:34:25 -08:00
Aliaksandr Valialkin
0dea3b71da
lib/promscrape: pre-fetch metric_relabel_configs rules when debugging metric relabeling for a particular target
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3407
2023-01-05 03:26:49 -08:00
Aliaksandr Valialkin
a1076abcbf
lib/promscrape: follow-up for a7e29c38bc
- Document the bugfix at docs/CHANGELOG.md
- Make the fix more durable against future changes when droppedTargetsMap.Register may be called from other places.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3580
2023-01-05 02:52:08 -08:00
Aliaksandr Valialkin
0e1f0ade31
lib/streamaggr: sort by and without labels in the aggregate output metric name
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3460
2023-01-05 02:08:44 -08:00
Aliaksandr Valialkin
04dff34de4
vendor: update github.com/VictoriaMetrics/metricsql from v0.50.0 to v0.51.0
Updates https://github.com/VictoriaMetrics/metricsql/pull/7
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3589
2023-01-05 01:50:10 -08:00
Denys Holius
09fe346d18
docs/Release-Guide.md: adds release scenario for RPM LTS packages (#3588) 2023-01-04 14:36:14 +01:00
Aliaksandr Valialkin
fd175ad80b
docs: update -help outputs for vm* tools 2023-01-03 23:27:06 -08:00
Aliaksandr Valialkin
fa13bbc48a
app/{vmagent,vminsert}: add support for streaming aggregation
See https://docs.victoriametrics.com/stream-aggregation.html

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3460
2023-01-03 22:19:21 -08:00
Aliaksandr Valialkin
d3a1c36842
docs/Single-server-VictoriaMetrics.md: impromve formatting for prominent features chapter 2023-01-03 21:58:21 -08:00
Aliaksandr Valialkin
189f85b24c
docs/vmbackupmanager.md: run make docs-sync after fa842d6534 2023-01-03 21:50:28 -08:00
Roman Khavronenko
4b3d8eb573
vmalert: mention specifics of Alertmanager HA mode (#3573)
Stress the importance of specifying of all Alertmanager
URLs in vmalert's `-notifier.url` or `notifier.config`
if it runs in cluster mode.

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3547

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-12-30 09:54:03 +01:00
Aliaksandr Valialkin
59e1e84a92
docs/CHANGELOG.md: document 1720bddb4f
Updats https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3530
2022-12-29 12:31:48 -08:00
Roman Khavronenko
2cedb3e883
csvimport: support empty values (#3565)
Before, if the imported line contained multiple metrics and one
or more of them had an empty values - the whole line was ignored.

Now, only metrics with empty values are ignored, and the rest
of the metrics are accepted successfully.

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3540

Signed-off-by: hagen1778 <roman@victoriametrics.com>

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-12-29 11:52:10 -08:00
Aliaksandr Valialkin
7af8857b68
docs/CHANGELOG.md: add a link to the docs and the pull request for update_entries_limit option in vmalert
This is a follow up for 6588fcbfca
2022-12-29 10:38:26 -08:00
Aliaksandr Valialkin
c90752a8be
vendor: update github.com/valyala/fastjson/fastfloat from v1.6.3 to v1.6.4
This should properly parse floating-point numbers with missing integer or fractional parts.
For example, 123. or .123

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3544
2022-12-29 10:34:11 -08:00
Roman Khavronenko
6588fcbfca
vmalert: allow configuring the default number of stored rule's update states (#3556)
Allow configuring the default number of stored rule's update states in memory
 via global `-rule.updateEntriesLimit` command-line flag or per-rule via rule's
 `update_entries_limit` configuration param.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-12-29 12:36:44 +01:00
Aliaksandr Valialkin
3dc684634e
app/vmselect/searchutils: accept partial RFC3339 values at time, start and end query args
This simplifies manual usage of the APIs. For example, the following query
would return the results over the 2022 year.

  /api/v1/query_range?start=2022&end=2023&step=1d&query=...

This is equivalent to:

  /api/v1/query_range?start=2022-01-01T00:00:00Z&end=2023-01-01T00:00:00Z&step=1d&query=...
2022-12-28 19:41:54 -08:00
Aliaksandr Valialkin
ed9161a04f
docs: remove a case study for Dreamteam.gg, since it looks like the company no longer exists 2022-12-28 11:34:40 -08:00
Artem Navoiev
34c705988e fix broken links
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2022-12-27 13:17:22 +02:00
Artem Navoiev
7d9c4bebc0
update links to grafana dashboards (#3534)
docs: update links to grafana dashboards

Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2022-12-25 17:36:20 +01:00
Aliaksandr Valialkin
0dca224ec3
app/vmui: small improvements for header panel
- Rename `Custom panel` tab to more clear `Query` tab
- Rename `Cardinality` tab to `Explore cardinality`, so it becomes consistent with `Explore metrics` tab
- Move `Dashboards` tab to the end, since it isn't used too much
2022-12-23 22:24:31 -08:00
Aliaksandr Valialkin
4deea604bf
app/vmui: follow-up after f6d31f5216
- Document the feature at docs/CHANGELOG.md.
- Document the metrics explorer at https://docs.victoriametrics.com/#metrics-explorer .
- Properly set `start` and `end` args for the selected time range
  when performing the request, which returns metric names.
- Improve queries, so they return lower number of lines and labels.
  This should improve metrics' exploration.
- Properly encode label filters and query args before passing them to VictoriaMetrics.
- Various cosmetic fixes.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3386
2022-12-22 17:17:01 -08:00
Yury Molodov
f6d31f5216
vmui: add explore tab for exploration of metrics, which belong to a particular job/instance (#3470)
* feat: add "Explore" page

* feat: add graphs for explore page

* vmui: add explore tab for exploration of metrics, which belong to a particular job/instance

* refactor: rename variables

* refactor: extract graph to ExploreMetricItemGraph.tsx

* feat: add searchable for Select.tsx

* feat: improve metrics explorer

* feat: set document title by page

* feat: add page to view icons

* fix: improve styles

* fix: add encodeURIComponent to query
2022-12-22 15:24:40 -08:00
Aliaksandr Valialkin
f6ac045933
docs/CHANGELOG.md: document 1df87807fd
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3513
2022-12-22 15:22:09 -08:00
Aliaksandr Valialkin
31886aef3d
lib/promrelabel: add support for keepequal and dropequal relabeling actions
These actions are supported by Prometheus starting from v2.41.0

See https://github.com/prometheus/prometheus/pull/11564 ,
https://github.com/prometheus/prometheus/issues/11556
and https://github.com/prometheus/prometheus/issues/3756

Side note:

It's a pity that Prometheus developers decided inventing `keepequal` and `dropequal`
relabeling actions instead of adding support for `keep_if_equal` and `drop_if_equal` relabeling
actions supported by VictoriaMetrics since June 2020 - see 2a39ba639d .
2022-12-21 20:04:55 -08:00
Yury Molodov
0f8ffc7df9
vmui: fix change timezone (#3519)
vmui: fix time picker with changed time zone
2022-12-21 17:22:37 +01:00
Aliaksandr Valialkin
f443fad56d
docs/CHANGELOG.md: cut v1.85.3 2022-12-20 14:51:23 -08:00
Roman Khavronenko
e4e9dfb785
vmagent: respect -usePromCompatibleNaming if no relabeling is set (#3511)
* vmagent: respect `-usePromCompatibleNaming` if no relabeling is set

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3493

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* vmagent: upd test

Signed-off-by: hagen1778 <roman@victoriametrics.com>

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-12-20 14:32:45 -08:00
Yury Molodov
731d189fa9
fix: change the logic for hide query (#3514) 2022-12-20 14:29:46 -08:00
Zakhar Bessarab
4be4645142
app/vmbackupmanager: add metrics for better observability (#488)
* app/vmbackupmanager: add metrics for better observability, include more information to `/api/v1/backups` API call response

* app/vmbackupmanager: drop old metrics before creating new ones

* app/vmbackupmanager: use `_total` postfix for counter metrics

* app/vmbackupmanager: remove `_total` postfix for gauge-like metrics

* app/vmbackupmanager: add `_last_run_failed` metrics for backups and retention

* app/vmbackupmanager: address review feedback

* app/vmbackupmanager: fix metric name

* app/vmbackupmanager: address review feedback, remove background updates of metrics, add restoring state of `_last_run_failed` metric from remote storage

* app/vmbackupmanager: improve performance for backup size calculation

* app/vmbackupmanager: refactor backup and retention runs to deduplicate each run logic

* {app/vmbackupmanager,lib/formatutil}: move HumanizeBytes into lib package

* app/vmbackupmanager: fix creating new metrics instead of reusing existing ones

* lit/formatutil: add comment to make linter happy

* app/vmbackupmanager: address review feedback
2022-12-20 14:18:06 -08:00
Aliaksandr Valialkin
4e55b67a44
lib/storage: clear the err if it is set to io.EOF when searching for the TSID by metricID
This is expected error after when recently added indexdb data isn't available for search yet
or wasn't flushed to disk after unclean shutdown of VictoriaMetrics.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3515
2022-12-20 14:05:29 -08:00
Aliaksandr Valialkin
a48510573e
Revert "docs/Release-Guide.md: add LATEST_TAG=stable env var for make publish-release in order to create stable tag for the published components at DockerHub"
This reverts commit 8afa7ef837.
2022-12-20 12:24:27 -08:00
Aliaksandr Valialkin
8afa7ef837
docs/Release-Guide.md: add LATEST_TAG=stable env var for make publish-release in order to create stable tag for the published components at DockerHub
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2911
2022-12-20 12:22:42 -08:00
Aliaksandr Valialkin
1d7b5cb83c
docs/CHANGELOG.md: document the change at 547f07463b29c09c62c9af35eac9cee6764b3286
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2612
2022-12-20 10:22:07 -08:00
Roman Khavronenko
8122191368
docs: fix link typo in operator docs (#3508) 2022-12-20 14:52:47 +01:00
Aliaksandr Valialkin
6bf46c7bf5
docs/CHANGELOG.md: formatting fix 2022-12-20 01:06:34 -08:00
Aliaksandr Valialkin
9fa3f1dc57
app/vmselect/promql: do not extend too short lookbehind window for rate() function if it is set explicitly
Previously too short lookbehind window d for rate(m[d]) could be automatically extended
if it didn't cover at least two raw samples. This was needed in order to guarantee
non-empty results from rate(m[d]) on short time ranges.

Now the lookbehind window isn't extended if it is set explicitly,
since it is expected that the user knows what he is doing.

The lookbehind window continues to be extended when needed if it isn't set explicitly.
For example, in the case of rate(m).

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3483
2022-12-20 00:18:20 -08:00
Aliaksandr Valialkin
eabb8762ee
docs/Articles.md: add a link to https://www.youtube.com/watch?v=Mesc6JBFNhQ 2022-12-19 21:40:51 -08:00
Aliaksandr Valialkin
680925f872
docs/CHANGELOG.md: add a warning for releases between v1.83.0 and v1.85.1 that it is recommended upgrading to v1.85.2 because of the https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3502
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3502
2022-12-19 13:36:49 -08:00
Aliaksandr Valialkin
d2ad184377
docs/CHANGELOG.md: consistently use YYYY-MM-DD format for release dates
The previously used DD-MM-YYYY format could be confused with the MM-DD-YYYY format.
The YYYY-MM-DD format reduces this confusion.
2022-12-19 13:33:05 -08:00
Aliaksandr Valialkin
6530344a8f
docs/CHANGELOG.md: cut v1.85.2 2022-12-19 13:09:29 -08:00
Aliaksandr Valialkin
0bf3ae9559
lib/promscrape/discovery/consul: expose service tags in individual labels __meta_consul_tag_<tagname>
This simplifies copying service tags to target labels with the following relabeling rule:

- action: labelmap
  regex: __meta_consul_tag_(.+)

See https://stackoverflow.com/questions/44339461/relabeling-in-prometheus
2022-12-19 13:08:11 -08:00
Aliaksandr Valialkin
6c98b56935
lib/storage: search for TSIDs for the given metricIDs in the previous indexdb if they aren't found in the current indexdb
The issue triggers after the indexdb rotation for time series, which stop receiving new samples.
This results in missing data for such time series in query responses.

This commit should address the https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3502

The issue has been introduced in 2dd93449d8
2022-12-19 12:03:09 -08:00
Aliaksandr Valialkin
4cb83f0f4a
lib/logger: follow-up for 72f8fce107
- Document the change at docs/CHANELOG.md
- Log fatal errors if the -loggerJSONFields contains unexpected values
- Rename -loggerJsonFields to -loggerJSONFields for the sake of consistency naming commonly used in Go

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2348
2022-12-16 17:42:07 -08:00
Aliaksandr Valialkin
56a9ea3753
docs/vmalert.md: mention latency_offset query arg, which has been added in 86dae56bd0
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3481
2022-12-16 17:20:37 -08:00
Aliaksandr Valialkin
65f8fc527f
lib/promscrape: stop dropping metric name if relabeling rules do not instruct to do this on the /metric-relabel-debug page 2022-12-16 17:02:41 -08:00
Roman Khavronenko
86dae56bd0
vmselect: support overriding of -search.latencyOffset (#3489)
support overriding of `-search.latencyOffset` value via
URL param `latency_offset`.

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3481

Signed-off-by: hagen1778 <roman@victoriametrics.com>

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-12-16 16:54:57 -08:00
Aliaksandr Valialkin
17a244571b
docs/keyConcepts.md: update the list of supported data ingestion protocols
- Add DataDog protocol
- Remove native protocol, since it isn't intended for general-purpose usage by external clients
2022-12-15 12:01:59 -08:00
Aliaksandr Valialkin
4de9d35458
lib/flagutil/bytes.go: properly handle values bigger than 2GiB on 32-bit architectures
This fixes handling of values bigger than 2GiB for the following command-line flags:

- -storage.minFreeDiskSpaceBytes
- -remoteWrite.maxDiskUsagePerURL
2022-12-14 19:26:31 -08:00
Aliaksandr Valialkin
5d30080555
lib/flagutil: support for TB and TiB suffixes for command-line flags, which accept byte sizes 2022-12-14 17:52:32 -08:00
Aliaksandr Valialkin
38341802c2
app/vmselect: add /expand-with-exprs page 2022-12-14 16:18:55 -08:00
Aliaksandr Valialkin
231569f89b
docs/vmagent.md: small formatting fix 2022-12-14 14:25:35 -08:00
Aliaksandr Valialkin
0c54ff20eb
docs/CHANGELOG.md: fix the link to the issue https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3466 2022-12-14 14:17:17 -08:00
Aliaksandr Valialkin
76445bfd98
docs/CHANGELOG.md: add release date for v1.85.1 2022-12-14 14:12:07 -08:00
Aliaksandr Valialkin
ea44c39377
docs/CHANGELOG.md: cut v1.85.1 2022-12-14 11:57:49 -08:00
Aliaksandr Valialkin
e0009ec466
docs/Cluster-VictoriaMetrics.md: mention the vm_storage_is_read_only metric, which can help debugging readonly mode at vmstorage 2022-12-14 09:31:28 -08:00
Aliaksandr Valialkin
7a61bafe59
docs/vmagent.md: clarify that relabeling is actually a debugging at relabel debug section 2022-12-13 15:46:21 -08:00
Aliaksandr Valialkin
b89d862aa5
docs/CHANGELOG.md: document the bugfix at a50120a212 2022-12-13 09:36:24 -08:00
Dima Lazerka
bde93844de
Add Anomaly Detection to enterprise features list (#3476)
* Add Anomaly Detection to enterprise features list

* Update docs/enterprise.md

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-12-12 17:42:44 -08:00
Aliaksandr Valialkin
d1af6046c7
lib/{mergeset,storage}: do not block small merges by pending big merges - assist with small merges instead
Blocked small merges may result into big number of small parts, which, in turn,
may result in increased CPU and memory usage during queries, since queries need to inspect
all the existing small parts.

The issue has been introduced in 8189770c50

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3337
2022-12-12 17:00:50 -08:00
Aliaksandr Valialkin
3b18931050
lib/bytesutil: cache results for all the input strings, which were passed during the last 5 minutes from FastStringMatcher.Match(), FastStringTransformer.Transform() and InternString()
Previously only up to 100K results were cached.
This could result in sub-optimal performance when more than 100K unique strings were actually used.
For example, when the relabeling rule was applied to a million of unique Graphite metric names
like in the https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3466

This commit should reduce the long-term CPU usage for https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3466
after all the unique Graphite metrics are registered in the FastStringMatcher.Transform() cache.

It is expected that the number of unique strings, which are passed to FastStringMatcher.Match(),
FastStringTransformer.Transform() and to InternString() during the last 5 minutes,
is limited, so the function results fit memory. Otherwise OOM crash can occur.
This should be the case for typical production workloads.
2022-12-12 14:41:13 -08:00
Zakhar Bessarab
ebcb4ab617
{app/{vmbackup/vmrestore}: update path example to use Azure terminology for consistency (#3475) 2022-12-12 22:17:22 +03:00
Roman Khavronenko
8286f9608f
vmalert: support $for or .For template variables (#3474)
support `$for` or `.For` template variables  in alert's annotations.

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3246

Signed-off-by: hagen1778 <roman@victoriametrics.com>

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-12-12 22:16:10 +03:00
Aliaksandr Valialkin
7ae744fce6
lib/protoparser/datadog: do not re-use previously parsed field values if they are missing in the currently parsed message
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3432
2022-12-11 13:09:25 -08:00
Aliaksandr Valialkin
9f642d10ff
docs/CHANGELOG.md: cut v1.85.0 2022-12-11 02:01:11 -08:00
Aliaksandr Valialkin
38f8e8adc3
docs/CHANGELOG.md: document changes at v1.79.6 2022-12-11 01:51:36 -08:00
Aliaksandr Valialkin
d141cb28a6
docs/CHANGELOG.md: document 461158a437
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3427
2022-12-10 23:42:16 -08:00
Aliaksandr Valialkin
88597f187b
app/{vmagent,vminsert}/datadog: make the host label optional in DataDog data ingestion protocol
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3432
2022-12-10 23:32:31 -08:00
Aliaksandr Valialkin
e272a0ec78
app/vmselect/promql: allow passing inf arg into functions, which accept numeric limit on the number of output time series
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3461
2022-12-10 22:47:47 -08:00
Aliaksandr Valialkin
b7aec1be4d
docs/CHANGELOG.md: add a link to the issue related to reduced CPU and memory usage at vmalert
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3464

The related commit - b97bd01605
2022-12-10 22:21:19 -08:00
Aliaksandr Valialkin
b01607e3fb
docs: clarify that single-node VictoriaMetrics also provides functionality for relabel debugging
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3407
2022-12-10 20:49:52 -08:00
Aliaksandr Valialkin
3f4cb9a142
docs: sync with cluster branch after 97b41e727c 2022-12-10 02:32:24 -08:00
Aliaksandr Valialkin
a8b8e23d68
lib/promscrape: implement target-level and metric-level relabel debugging
Target-level debugging is performed by clicking the 'debug' link at the corresponding target
on either http://vmagent:8429/targets page or on http://vmagent:8428/service-discovery page.

Metric-level debugging is perfromed at http://vmagent:8429/metric-relabel-debug page.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3407

See https://docs.victoriametrics.com/vmagent.html#relabel-debug
2022-12-10 02:09:44 -08:00
Aliaksandr Valialkin
c5dd973f9c
docs/url-examples.md: add missing whitespace for proper heading for the example on how to send data via OpenTSDB protocol 2022-12-09 17:33:44 -08:00
Aliaksandr Valialkin
765ee5f7ba
docs/CHANGELOG.md: document b97bd01605 2022-12-09 11:49:29 -08:00
Aliaksandr Valialkin
2406c0dcfd
docs/CHANGELOG.md: document the bugfix at 05b42601c3
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3247
2022-12-08 18:35:28 -08:00
Aliaksandr Valialkin
5b9e6b9d24
lib/storage: follow-up after 7c0ae3a86a
- Update docs at https://docs.victoriametrics.com/#deduplication
- Optimize the deduplication loop a bit

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3333
2022-12-08 18:16:57 -08:00
Roman Khavronenko
7c0ae3a86a
lib/storage: keep sample with the biggest value on timestamp conflict (#3421)
The change leaves raw sample with the biggest value for identical
timestamps per each `-dedup.minScrapeInterval` discrete interval
when the deduplication is enabled.

```
benchstat old.txt new.txt
name                                         old time/op    new time/op    delta
DeduplicateSamples/minScrapeInterval=1s-10      817ns ± 2%     832ns ± 3%      ~     (p=0.052 n=10+10)
DeduplicateSamples/minScrapeInterval=2s-10     1.56µs ± 1%    2.12µs ± 0%   +35.19%  (p=0.000 n=9+7)
DeduplicateSamples/minScrapeInterval=5s-10     1.32µs ± 3%    1.65µs ± 2%   +25.57%  (p=0.000 n=10+10)
DeduplicateSamples/minScrapeInterval=10s-10    1.13µs ± 2%    1.50µs ± 1%   +32.85%  (p=0.000 n=10+10)

name                                         old speed      new speed      delta
DeduplicateSamples/minScrapeInterval=1s-10   10.0GB/s ± 2%   9.9GB/s ± 3%      ~     (p=0.052 n=10+10)
DeduplicateSamples/minScrapeInterval=2s-10   5.24GB/s ± 1%  3.87GB/s ± 0%   -26.03%  (p=0.000 n=9+7)
DeduplicateSamples/minScrapeInterval=5s-10   6.22GB/s ± 3%  4.96GB/s ± 2%   -20.37%  (p=0.000 n=10+10)
DeduplicateSamples/minScrapeInterval=10s-10  7.28GB/s ± 2%  5.48GB/s ± 1%   -24.74%  (p=0.000 n=10+10)
```

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3333
Signed-off-by: hagen1778 <roman@victoriametrics.com>

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-12-08 18:06:11 -08:00
Aliaksandr Valialkin
eeacbaf0b6
all: update Go builder from v1.19.3 to v1.19.4
See https://github.com/golang/go/issues?q=milestone%3AGo1.19.4+label%3ACherryPickApproved
2022-12-08 16:41:24 -08:00
Aliaksandr Valialkin
56b8980915
lib/promscrape: allow using sample_limit and series_limit options in stream parsing mode
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3458
2022-12-08 16:33:38 -08:00
Aliaksandr Valialkin
bce1c5d572
docs/Cluster-VictoriaMetrics.md: typo fix 2022-12-07 12:26:34 -08:00
Aliaksandr Valialkin
189217a069
docs/CHANGELOG.md: document the addition of file-based discovery of vmstorage nodes 2022-12-07 12:04:57 -08:00
Aliaksandr Valialkin
59430e4274
docs/Cluster-VictoriaMetrics.md: make docs-sync after 5de8330ce00adfc5ac794070d30a2617ddc14bf2 2022-12-07 12:02:38 -08:00
Aliaksandr Valialkin
49ebc48809
lib/querytracer: put the version of VictoriaMetrics in the first message of query trace
This should simplify further debugging, since the first thing to start the debugging by query trace
is to know the version of VictoriaMetrics, which produced this trace.
2022-12-07 09:46:39 -08:00
Aliaksandr Valialkin
3f82e3fa36
docs: follow-up after e1bf2a85d0559d112908ce81597f3261d3a085c0
- Document the change at docs/CHANGELOG.md
- Run `make docs-sync` for copying app/vmgateway/README.md to docs/vmgateway.md
  in order to propagate docs' changes to https://docs.victoriametrics.com/vmgateway.html
2022-12-06 21:05:22 -08:00
Aliaksandr Valialkin
35a3170d97
docs: document the addition of -storageNode.discoveryInterval command-line flag in VictoriaMetrics cluster enterprise
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3417
2022-12-06 19:57:06 -08:00
Aliaksandr Valialkin
5cc19b1f7e
docs/Articles.md: change the link to How we tried using VictoriaMetrics and Thanos at the same time from Russian to English article 2022-12-06 16:30:10 -08:00
Roman Khavronenko
3dec847c93
vmalert: correctly return error for RW failures (#3452)
* vmalert: correctly return error for RW failures

By mistake, in 0989649ad0 the error
for remote write failures weren't return to user.
This change fixes it.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-12-06 15:36:46 +01:00
Aliaksandr Valialkin
9a7c36e645
docs/Articles.md: add a link to the talk "How do We Keep Metrics for a Long Time in VictoriaMetrics" 2022-12-06 00:55:38 -08:00
Aliaksandr Valialkin
50ea632bfe
docs/Articles.md: link to a video for the talk 'VictoriaMetrics: scaling to 100 million metrics per second' 2022-12-06 00:53:05 -08:00
Aliaksandr Valialkin
a40c50f4fe
docs/CHANGELOG.md: document 1e0666abb4 2022-12-05 23:10:17 -08:00
Aliaksandr Valialkin
caa1c43166
docs: follow-up for 7645d9ae00
- Document the change at docs/CHANGELOG.md
- Document the feature at https://docs.victoriametrics.com/#vmui

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3446
2022-12-05 22:50:41 -08:00
Yury Molodov
01a9b36a95
vmui: timezone select (#3414)
* feat: add timezone selection

* vmui: provide feature timezone select

* fix: correct timezone with relative time

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-12-05 22:44:31 -08:00
Aliaksandr Valialkin
718d1d90b6
docs/CHANGELOG.md: document fd43b5bad0
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3444
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3445
2022-12-05 22:01:42 -08:00
Aliaksandr Valialkin
5eae9a9914
app/vmselect/promql: add range_trim_spikes(phi, q) function for trimming phi percent of largest spikes per each time series returned by q 2022-12-05 21:55:01 -08:00
Aliaksandr Valialkin
eed32b368c
docs/vmctl.md: make docs-sync after 86c31f2955
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2930
2022-12-05 17:24:10 -08:00
Zakhar Bessarab
86c31f2955
app/vmctl: add option to migrate between clusters with automatic tenants discovery (#3450) 2022-12-05 17:18:09 -08:00
Aliaksandr Valialkin
f3e84b4dea
{dashboards,alerts}: subtitute {type="indexdb"} with {type=~"indexdb.*"} inside queries after 8189770c50
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3337
2022-12-05 16:00:22 -08:00
Aliaksandr Valialkin
8189770c50
all: add -inmemoryDataFlushInterval command-line flag for controlling the frequency of saving in-memory data to disk
The main purpose of this command-line flag is to increase the lifetime of low-end flash storage
with the limited number of write operations it can perform. Such flash storage is usually
installed on Raspberry PI or similar appliances.

For example, `-inmemoryDataFlushInterval=1h` reduces the frequency of disk write operations
to up to once per hour if the ingested one-hour worth of data fits the limit for in-memory data.

The in-memory data is searchable in the same way as the data stored on disk.
VictoriaMetrics automatically flushes the in-memory data to disk on graceful shutdown via SIGINT signal.
The in-memory data is lost on unclean shutdown (hardware power loss, OOM crash, SIGKILL).

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3337
2022-12-05 15:16:14 -08:00
Aliaksandr Valialkin
ddc3d6b5c3
lib/mergeset: drop the crufty code responsible for direct upgrade from releases prior v1.28.0
Upgrade to v1.84.0, wait until the "finished round 2 of background conversion" message
appears in the log and then upgrade to newer release.
2022-12-03 21:17:31 -08:00
Aliaksandr Valialkin
c1cd4a9101
docs/CHANGELOG.md: consistently add - prefix in front of command-line flags
This is a follow-up for bcba5d2a78
2022-12-02 19:08:26 -08:00
Aliaksandr Valialkin
b6712ac08e
docs: follow-up after 30fea30685
- Run `make docs-sync`, so app/vmalert/README.md is copied to docs/vmalert.md
- Clarify the feature description in the docs/CHANGELOG.md

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3408
2022-12-02 19:03:11 -08:00
Denys Holius
54741f6f38
docs/Articles.md: fir broken link (#3433) 2022-12-02 10:35:40 +01:00
Roman Khavronenko
cd5c451ea3
docs: fix typo in cluster's README (#3430)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-12-01 16:24:16 +01:00
Roman Khavronenko
bcba5d2a78
vmalert: fix replay step param (#3428)
The recent change in modifying default value
of `datasource.queryStep` flag resulted in situation
where replay mode was always running queries with
step=`datasource.queryStep`. When it should always
use rule's evaluation interval.

The fix is related not to replay mode only, but
for all Range requests. Now step param is set
individually for each mode.

Signed-off-by: hagen1778 <roman@victoriametrics.com>

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-12-01 13:57:53 +01:00
Zakhar Bessarab
30fea30685
app/vmalert: add remoteWrite.sendTimeout command-line flag to configure timeout for sending data to remoteWrite.url (#3423)
* app/vmalert: add `remoteWrite.sendTimeout` command-line flag to configure timeout for sending data to `remoteWrite.url`

* vmalert: remove WriteTimeout from clients Cfg
No need to have it as a part of configuration struct:
* the client isn't used by other packages;
* there are no internal tests to check the WriteTimeout.

* vmalert: remove DisablePathAppend from clients Cfg
No need to have it as a part of configuration struct:
* the client isn't used by other packages;
* there are no internal tests to check the DisablePathAppend.

Co-authored-by: hagen1778 <roman@victoriametrics.com>
2022-12-01 09:57:19 +01:00
Roman Khavronenko
8cc4f7eac6
vmalert: properly pass headers during the restore procedure (#3420)
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3418

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-12-01 09:27:39 +01:00
Aliaksandr Valialkin
3cdff3de23
docs/vmagent.md: update after 959f06d175 2022-11-29 21:33:49 -08:00
Aliaksandr Valialkin
f325410c26
lib/promscrape: optimize service discovery speed
- Return meta-labels for the discovered targets via promutils.Labels
  instead of map[string]string. This improves the speed of generating
  meta-labels for discovered targets by up to 5x.

- Remove memory allocations in hot paths during ScrapeWork generation.
  The ScrapeWork contains scrape settings for a single discovered target.
  This improves the service discovery speed by up to 2x.
2022-11-29 21:26:00 -08:00
Aliaksandr Valialkin
c7ce4979ec
all: follow-up after 05cf8a6ecc 2022-11-29 21:03:59 -08:00
Aliaksandr Valialkin
4822406b64
app/vmalert: substitute -datasource.disablePathAppend with -remoteRead.disablePathAppend in the description for -datasource.url command-line flag
This is a follow-up for 959f06d175
2022-11-29 20:36:41 -08:00
Dmytro Kozlov
05cf8a6ecc
vmctl: support of the remote read protocol (#3232)
vmctl: support of the remote read protocol

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
2022-11-29 22:53:28 +01:00
Max Golionko
959f06d175
vmalert: flag reference update (#3415)
* flag reference update

there is no flag `-datasource.disablePathAppend` and datasource actually checking for `-remoteRead.disablePathAppend`

* update source for doc as well
2022-11-29 19:22:57 +01:00
Zakhar Bessarab
2f837c1b23
doc/operator: update formatting for backup section, add FAQ section (#3416)
* doc/operator: update formatting for backup section, add FAQ section

* doc/operator: address review feedback

* doc/operator: add note about difference between `VMRestore` and `VMBackupmanager` as init containers

* Update docs/operator/backups.MD

Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>

Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
2022-11-29 19:19:46 +01:00
Aliaksandr Valialkin
0002de937b
docs/CHANGELOG.md: document 027ab74efb
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3402
2022-11-28 18:55:01 -08:00
Aliaksandr Valialkin
5c906beea2
docs/url-examples.md: allow linking to how to send OpenTSDB/Graphite data chapters 2022-11-28 18:42:49 -08:00
Aliaksandr Valialkin
654e94f420
lib/promscrape: add exported_ prefix to metric names exported by scrape targets if they clash with automatically generated metrics
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3406
2022-11-28 18:37:09 -08:00
Aliaksandr Valialkin
090343ff50
docs/Articles.md: add a link to slides about scaling to 100 million metrics per second 2022-11-28 16:41:14 -08:00
Aliaksandr Valialkin
ad105147dd
docs/CHANGELOG.md: cut v1.84.0 2022-11-25 19:53:29 -08:00
Aliaksandr Valialkin
e014467f42
docs/README.md: make docs-sync after 58d459e8a8 2022-11-25 16:55:47 -08:00
Aliaksandr Valialkin
58d459e8a8
app/{vminsert,vmagent}: follow-up after 53a63c6c4c
Extend /api/v1/import/prometheus with the support for Pushgateway way of specifying additional labels.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1415
2022-11-25 16:48:14 -08:00
Zakhar Bessarab
8b6d528fbd
{app/vmstorage,app/vmselect}: add API to get list of existing tenants (#3348)
* {app/vmstorage,app/vmselect}: add API to get list of existing tenants

* {app/vmstorage,app/vmselect}: add API to get list of existing tenants

* app/vmselect: fix error message

* {app/vmstorage,app/vmselect}: fix error messages

* app/vmselect: change log level for error handling

* wip

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-11-25 11:05:47 -08:00
Aliaksandr Valialkin
da13d36af9
app/vmselect/vmui: make vmui-update after eb772aa50e 2022-11-24 17:37:18 -08:00
Yury Molodov
eb772aa50e
vmui: improve table view (#3377)
* vmui: add compact table view (#3365)

* feat: add compact table view

* fix: add overflow table

* fix: change table styles

* vmui: compact table view

* Update docs/CHANGELOG.md

Co-authored-by: Michal Kralik <michal.kralik@percona.com>
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-11-24 17:33:07 -08:00
Aliaksandr Valialkin
399ed9a3b9
docs/MetricsQL.md: document that histogram_share() accepts optional boundsLabel arg 2022-11-24 17:27:30 -08:00
Aliaksandr Valialkin
045fec631b
docs/vmagent.md: typo fix 2022-11-24 17:19:45 -08:00
Aliaksandr Valialkin
04bb2e14dd
docs/Articles.md: add a link to "The cost of scale in Prometheus ecosystem" talk 2022-11-23 00:11:51 +02:00
Yury Molodov
7dc2349913
vmui: add set up series custom limits (#3368)
* feat: add set up series custom limits

* feat: add button for show series without limits

* fix: resolve merge conflicts
2022-11-22 15:31:17 +02:00
Aliaksandr Valialkin
b1622ad63e
docs/Articles.md: add a link to https://www.youtube.com/watch?v=_zORxrgLtec (OSA Con 2022: Specifics of data analysis in Time Series Databases) 2022-11-22 01:08:21 +02:00
Aliaksandr Valialkin
9498f871e7
app/vminsert: add missing vm_relabel_config_* metrics after 03d88bc066
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3345
2022-11-22 00:47:49 +02:00
Roman Khavronenko
03d88bc066
vmagent: expose metrics for tracking config state (#3375)
Expose `vm_relabel_config_*` and `vm_promscrape_config_*` metrics
for tracking relabel and scrape configuration hot-reloads.

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3345
Signed-off-by: hagen1778 <roman@victoriametrics.com>

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-11-22 00:38:43 +02:00
Yury Molodov
7d1b3e7e14
vmui: add copy button to row on Table view (#3363)
* feat: add copy button to row on Table view

* vmui: add copy button to row on Table view

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-11-22 00:31:33 +02:00
Yury Molodov
82d254af08
vmui: sticky tooltip (#3376)
* feat: add ability to make tooltip "sticky"

* vmui: add ability to make tooltip "sticky"
2022-11-22 00:26:53 +02:00
Aliaksandr Valialkin
ee1479bac6
docs/CHANGELOG.md: link to the related issue for range_normalize() function 2022-11-21 23:27:00 +02:00
Aliaksandr Valialkin
d9c3a2b605
app/vmselect/promql: add range_normalize(q1, ..., qN) function for normalizing query results into [0..1] value range
This may be useful for analyzing correlation between time series with different value ranges
2022-11-21 23:25:00 +02:00
Aliaksandr Valialkin
95f0266558
lib/promscrape/discovery/gce: do not pass filter arg when discovering zones
The filter arg isn't supported by zones API in GCE.

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3202
2022-11-21 22:32:05 +02:00
Aliaksandr Valialkin
05ed98c98b
app/vmselect/promql: allow using SI and IEC suffixes in numeric values inside queries
For example, 10Ki is equivalent to 10*1024, while 5.3M is equivalent to 5.3*1000*1000
2022-11-21 21:27:55 +02:00
Aliaksandr Valialkin
2c9e403d5f
app/vmselect/promql: properly return an empty result from limit_offset() if offset exceeds the number of inner time series
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3312
2022-11-21 16:47:37 +02:00
Aliaksandr Valialkin
b796a0dc3f
app/vmselect/promql: optimize e1 op e2 when e1 returns an empty result
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3349
2022-11-21 16:09:10 +02:00
Roman Khavronenko
84742f229a
vmalert: add default list of alerting rules (#3373)
The default list of alerting rules contains the basic
rules for checking vmalert's health state and is recommended
to use for monitoring vmalert deployments.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-11-21 14:45:45 +01:00
Aliaksandr Valialkin
20d758e3e4
all: add a link to https://docs.victoriametrics.com/enterprise.html into description for enterprise flags 2022-11-21 15:42:01 +02:00
Aliaksandr Valialkin
cb1a621d63
app/{vminsert,vmselect}: add -storageNode.filter command-line flag for filtering the discovered storage nodes
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3353
2022-11-21 15:20:14 +02:00
Aliaksandr Valialkin
65b4e96a80
docs/Cluster-VictoriaMetrics.md: sync with cluster branch 2022-11-18 14:06:23 +02:00
Aliaksandr Valialkin
a061d33400
app/vmselect: clarify that it isnt recommended setting -replicationFactor at vmselect nodes even if the replication is enabled at vminsert nodes 2022-11-18 14:04:53 +02:00
Yury Molodov
519bd2af7b
vmui: add trace analyzer (#3310)
* refactor: change structure project

* refactor: change structure project

* fix: add hooks for set query params

* refactor: add index for pages

* docs: add TESTCASES.md

* refactor: restructure components

* feat: add page with trace analyzer

* fix: change detect trace data

* Update app/vmui/packages/vmui/src/pages/TracePage/index.tsx

Co-authored-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* Update app/vmui/packages/vmui/src/pages/TracePage/index.tsx

Co-authored-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* fix: change descriptions on trace page

* Update app/vmui/packages/vmui/src/pages/TracePage/index.tsx

Co-authored-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* feat: add base components

* feat: add reset styles

* docs: add description about trace analyzer

* feat: add styles for custom panel page

* feat: add styles for predefined panels

* feat: add style for TracingsView.tsx

* feat: add Alerts

* feat: add Tooltip.tsx

* fix: correct styles

* feat: add DatePicker.tsx

* feat: add tables

* feat: add theme provider

* fix: replace using callbacks as props to handlers

* fix: correct update time

* fix: change TimePicker.tsx

* fix: correct styles

* fix: update packages

* vmui: refactor code, remove material-ui

* feat: add paste json for trace analyzer

* vmui: update trace analyzer docs

* app/vmselect/vmui: `make vmui-update`

Co-authored-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-11-17 22:22:01 +02:00
Aliaksandr Valialkin
e79bfdf4b8
docs/CHANGELOG.md: document the fix for CPU usage spikes
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3343
2022-11-17 22:02:02 +02:00
Aliaksandr Valialkin
578bb58ea9
app/vmselect/vmui: make vmui-update after 51bfd1ab80 2022-11-17 18:54:55 +02:00
Yury Molodov
51bfd1ab80
vmui: add ability hide query (#3359)
* feat: add ability hide query

* fix: change logic hide query

* fix: remove console.log
2022-11-17 18:42:33 +02:00
Yury Molodov
bc8a782f74
vmui/refactor (#3298)
* refactor: change structure project

* refactor: change structure project

* fix: add hooks for set query params

* refactor: add index for pages

* docs: add TESTCASES.md

* refactor: restructure components

* feat: add base components

* feat: add reset styles

* feat: add styles for custom panel page

* feat: add styles for predefined panels

* feat: add style for TracingsView.tsx

* feat: add Alerts

* feat: add Tooltip.tsx

* fix: correct styles

* feat: add DatePicker.tsx

* feat: add tables

* feat: add theme provider

* fix: replace using callbacks as props to handlers

* fix: correct update time

* fix: change TimePicker.tsx

* fix: correct styles

* fix: update packages

* vmui: refactor code, remove material-ui

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-11-17 01:09:14 +02:00
Aliaksandr Valialkin
a260e2659e
app/vmselect/promql: add range_stdvar() and range_stddev() functions for calculating variance and deviation over time series on the selected time range 2022-11-17 01:03:40 +02:00
Aliaksandr Valialkin
c1a3192d8b
app/vmselect/promql: add range_linear_regression(q) function for calculating simple linear regression for the selected time series on the selected time range 2022-11-17 00:38:48 +02:00
Aliaksandr Valialkin
c3362e3db4
lib/workingsetcache: add -prevCacheRemovalPercent command-line flag for tuning memory usage vs CPU usage ratio
Reduce the default value of this flag from 1% to 0.2% after 71335e6024

This flag should help determining the best ratio for https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3343
2022-11-16 12:39:39 +02:00
Aliaksandr Valialkin
09b79d74a7
docs/CHANGELOG.md: document changes in v1.79.5 release 2022-11-11 01:27:45 +02:00
Aliaksandr Valialkin
bbe1a1472c
docs/CHANGELOG.md: cut v1.83.1 2022-11-10 14:06:57 +02:00
Aliaksandr Valialkin
1b9dff133a
docs/CHANGELOG.md: document the fix at 71335e6024 2022-11-10 13:46:51 +02:00
Aliaksandr Valialkin
b8839df32c
lib/protoparser/opentsdb: follow-up after 04b0e4e7bf
- Simplify the parser code to be less error prone
- Document the change
- Add a test for OpenTSDB put line with trailing whitespace without tags

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3290
2022-11-09 15:35:05 +02:00
Aliaksandr Valialkin
e17a1acf4a
docs/CHANGELOG.md: document 7130af7fd2
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2814
2022-11-09 12:15:49 +02:00
Denys Holius
aebe21e2c8
guides/README.md: fix link to guide-delete-or-replace-metrics.html (#3331) 2022-11-09 12:00:46 +02:00
Aliaksandr Valialkin
c973aca617
app/vminsert/netstorage: move nodesHash from global state to storageNodesBucket
This should prevent from panics when the list of discovered vmstorage nodes changes.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/3329
2022-11-09 11:51:10 +02:00
Roman Khavronenko
9f8bf524ad
bump go version to 1.19.3 (#3327)
Signed-off-by: hagen1778 <roman@victoriametrics.com>

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-11-08 16:43:59 +01:00
Zakhar Bessarab
91dd79f40f
docs/operator: fix description for SA at VMClusterSpec (#3313) 2022-11-07 16:51:01 +01:00
Aliaksandr Valialkin
7fa5d043f5
lib/promscrape/discovery/consul: add __meta_consul_partition label in the same way as Prometheus does
See https://github.com/prometheus/prometheus/pull/11482
2022-11-07 15:25:53 +02:00
Aliaksandr Valialkin
daa70e6560
lib/storage: follow-up for 790768f20b
- Document the bugfix at docs/CHANGELOG.md
- Simplify the bugfix a bit
2022-11-07 14:04:08 +02:00
Aliaksandr Valialkin
23c79e2e49
docs/Single-server-VictoriaMetrics.md: mention about security certifications at Security chapter
This is a follow-up for 94bd49402e
2022-11-05 10:07:29 +02:00
Aliaksandr Valialkin
4ef5fe1317
docs/guides/guide-vmcluster-multiple-retention-setup.md: clarify docs after a75d85b11e 2022-11-05 10:02:48 +02:00
Artem Navoiev
94bd49402e
docs: Add link to security page from Readme (#3286)
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2022-11-04 09:49:12 +01:00
Dmytro Kozlov
99d8fcb332
docs/operator: change VMAgentRemoteWriteSettings.MaxDiskUsagePerURL type from int32 to int64 (#3307)
* docs/operator: change VMAgentRemoteWriteSettings.MaxDiskUsagePerURL type from int32 to int64

* docs: operator updates api description

Co-authored-by: f41gh7 <nik@victoriametrics.com>
2022-11-04 01:12:57 +01:00
Artem Navoiev
a75d85b11e
update multi-tenancy guide. Add infromation about Enterprise and Rete… (#3285)
docs: update multi-tenancy guide

Add information about Enterprise and Retention Filters

Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2022-11-01 15:19:09 +01:00
Aliaksandr Valialkin
df88832c86
docs/Single-server-VictoriaMetrics.md: follow-up after a1a97b9321
- Remove trailing whitespace at the end of lines
- Remove redundant sentence stating that time series matching the given selector will be deleted.
  It should be clear from the surrounding context.
2022-11-01 10:49:10 +02:00
Dmytro Kozlov
a1a97b9321
docs: clarify information about usage of the api/v1/admin/tsdb/delete_series API (#3287)
docs: clarify information about usage of the `api/v1/admin/tsdb/delete_series` API
2022-11-01 09:36:28 +01:00
Aliaksandr Valialkin
a1011931ac
docs/Release-Guide.md: instruct to update VictoriaMetrics version in deployment/docker/docker-compose*.yml files after creating new release
This is a follow-up for d1509f4559
2022-11-01 10:31:02 +02:00
Aliaksandr Valialkin
619b3c926d
docs/enterprise.md: mention that feature requests from enterprise customers are prioritized 2022-11-01 10:28:12 +02:00
Aliaksandr Valialkin
0f8f36de24
docs: typo fixes 2022-10-29 04:52:18 +03:00
Aliaksandr Valialkin
740f7ac5e0
docs/CHANGELOG.md: cut v1.83.0 2022-10-29 02:54:54 +03:00
Aliaksandr Valialkin
cdd3443806
app/vmbackupmanager: add functionality for automated restore from backup 2022-10-29 02:30:52 +03:00
Aliaksandr Valialkin
76e8888272
lib/promscrape: properly add exported_ prefix to labels, which clash with target labels if honor_labels: true option isn't set.
The issue was in the `labels := dst[offset:]` line in the beginning of appendExtraLabels() function.
The `dst` may be re-allocated when adding extra labels to it. In this case the addition of `exported_`
prefix to labels inside `labels` slice become invisible in the returned `dst` labels.

While at it, properly handle some corner cases:

- Add additional `exported_` prefix to clashing metric labels with already existing `exported_` prefix.
- Store scraped metric names in `exported___name__` label if scrape target contains `__name__` label.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3278

Thanks to @jplanckeel for the initial attempt to fix this issue
at https://github.com/VictoriaMetrics/VictoriaMetrics/pull/3281
2022-10-28 22:14:26 +03:00
Aliaksandr Valialkin
9aee303ca1
docs/Cluster-VictoriaMetrics.md: improve docs for dns+srv service discovery 2022-10-28 14:24:48 +03:00
Aliaksandr Valialkin
a72c5f76eb
app/{vminsert,vmselect}: add support for automatic discovery and update of vmstorage nodes
Thanks to @dmitryk-dk for the initial implemenation at https://github.com/VictoriaMetrics/VictoriaMetrics-enterprise/pull/446
2022-10-28 13:13:45 +03:00
Aliaksandr Valialkin
a018b1d75e
app/vmalert/templates: properly escape all the special chars in quotesEscape function
Previously the `quotesEscape` function was escaping only double quotes.
This wasn't enough, since the input string could contain other special chars,
which must be escaped when put inside JSON string. For example, carriage return and line feed chars (\n\r),
backslash char, etc. This led to the following issues, which were improperly fixed:

- https://github.com/VictoriaMetrics/VictoriaMetrics/issues/890 - this issue
  was "fixed" by introducing the `crlfEscape` function, which led to unnecessary
  complications in user templates, while not fixing various corner cases
  such as backslash chars in the input string.
  See 1de15ad490

- https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3139 - this issue
  was "fixed" by urlencoding the whole string passed to -external.alert.source
  command-line flag. This led to invalid urls, which couldn't be parsed by Grafana.
  See 00c838353d
  and 4bd0244599

This commit properly encodes the input string passed to `quotesEscape`, so it can be safely embedded inside JSON strings.

This commit deprecates crlfEscape template function and adds the following new template functions:

- strvalue and stripDomain - these functions are supported by Prometheus, so they were added
  for compatibility purposes.
- jsonEscape and htmlEscape for converting the input string to valid quoted JSON string
  and for html-escaping the input string, so it could be safely embedded as a plaintext
  into html.

This commit also documents all supported template functions at https://docs.victoriametrics.com/vmalert.html#template-functions
The deprecated crlfEscape function isn't documented on purpose, since its usefulness is negative in general case.
2022-10-28 00:01:16 +03:00
Aliaksandr Valialkin
4bd0244599
Revert "vmalert: escape query params if external alert source defined (#3267)"
This reverts commit 00c838353d.

Reason for revert: it incorrectly fixes the issue https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3139 .
Now `-external.alert.source=explore?orgId=1&left=...` is converted to the following invalid url, which cannot be handled by Grafana:

https://grafana.example.com/explore%3ForgId%3D1%26left%3D...

The next commit will contain the correct fix of the issue - the `quotesEscape` function must
properly escape the string, so it could be embedded into JSON string. This function must
properly escape \n\r chars too. In this case the `crlfEscape` function becomes unnecessary.
Actually, the next commit makes the `crlfEscape` function deprecated.
2022-10-27 22:30:27 +03:00
Dmytro Kozlov
00c838353d
vmalert: escape query params if external alert source defined (#3267)
vmalert: escape query args if external alert source defined
2022-10-26 10:00:14 -04:00
Aliaksandr Valialkin
518c340ae3
lib/envtemplate: allow referring env vars from other env vars via %{ENV_VAR} syntax
This is a follow-up for 02096e06d0
2022-10-26 14:49:33 +03:00
Yury Molodov
eae6063450
fix: change step setting field (#3270)
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-10-26 02:46:46 +03:00
Yury Molodov
bc7456841f
vmui: add responsive styles for small screens (#3256)
* fix: add responsive styles for small screens

* fix: correct additional settings margins

* docs/CHANGELOG.md: add responsive styles
2022-10-26 02:39:54 +03:00
Aliaksandr Valialkin
685433b8da
docs/Cluster-VictoriaMetrics.md: update docs about env vars usage in command-line flags
This is a follow-up for 02096e06d0
2022-10-26 01:54:15 +03:00
Aliaksandr Valialkin
02096e06d0
lib/envflag: allow referring environment variables in command-line flags 2022-10-26 01:52:05 +03:00
Aliaksandr Valialkin
5d82e7d64a
docs: run make docs-sync after 2ed3d49c26 2022-10-26 01:14:31 +03:00
Aliaksandr Valialkin
db8abd000e
docs/Makefile: fix docs-up Makefile command after 9ccd22c1f6 2022-10-25 17:53:24 +03:00
Denys Holius
9ccd22c1f6
Docs: add guide "How to delete and replace metrics" (#2829)
docs: add guide how to delete and replace metrics
2022-10-25 09:02:14 -04:00
Aliaksandr Valialkin
15849cb571
docs/guides/migrate-from-influx.md: properly display images at https://github.com/VictoriaMetrics/VictoriaMetrics/blob/master/docs/guides/migrate-from-influx.md
This is a follow-up for 42375679db
2022-10-24 21:28:31 +03:00
Nguyen Van Duc
42375679db
Fix images not display on key concepts document (#3266) 2022-10-24 21:22:41 +03:00
Aliaksandr Valialkin
b1324631b1
docs/CHANGELOG.md: document 274e235bf7
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3240
2022-10-24 21:02:17 +03:00
Aliaksandr Valialkin
c52c23c272
docs/enterprise.md: describe all the enteprise features in a short doc at https://docs.victoriametrics.com/enterprise.html 2022-10-24 18:02:03 +03:00
Aliaksandr Valialkin
cac28ae0ae
docs/CHANGELOG.md: typo fixes 2022-10-24 16:59:55 +03:00
Aliaksandr Valialkin
8e998aa1a1
lib/storage: add support for retention filters (aka multiple retentions for distinct sets of time series)
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/143
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/289
2022-10-24 16:40:20 +03:00
Aliaksandr Valialkin
d7e657e5f9
docs/CHANGELOG.md: document 69b27275d2b2bf1bdae0d8b887b44bde2a787649 2022-10-24 16:08:15 +03:00
Aliaksandr Valialkin
5e4dfe50c6
lib/storage: subsitute searchTSIDs functions with more lightweight searchMetricIDs function
The searchTSIDs function was searching for metricIDs matching the the given tag filters
and then was locating the corresponding TSID entries for the found metricIDs.

The TSID entries aren't needed when searching for time series names (aka MetricName),
so this commit removes the uneeded TSID search from the implementation of /api/v1/series API.
This improves perfromance of /api/v1/series calls.

This commit also improves performance a bit for /api/v1/query and /api/v1/query_range calls,
since now these calls cache small metricIDs instead of big TSID entries
in the indexdb/tagFilters cache (now this cache is named indexdb/tagFiltersToMetricIDs)
without the need to compress the saved entries in order to save cache space.

This commit also removes concurrency limiter during searching for matching time series,
which was introduced in 8f16388428, since the concurrency
for all the read queries is already limited with -search.maxConcurrentRequests command-line flag.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/648
2022-10-23 12:23:47 +03:00
Aliaksandr Valialkin
a10647e0bf
app/vmselect/promql: expose missing metric vm_cache_size_max_bytes{type="promql/rollupResult"} 2022-10-23 12:13:47 +03:00
Aliaksandr Valialkin
4cd173bbaa
lib/procutil: stop immediately after receiving the second SIGINT or SIGTERM signal
Previously VictoriaMetrics apps could stop responding to SIGINT and SIGTERM signals
if they hang for some reason in graceful shutdown procedure.
2022-10-20 21:40:20 +03:00
Aliaksandr Valialkin
cba9696a14
docs/CHANGELOG.md: move the BUGFIX line for 1059c4d84a into correct place 2022-10-18 20:38:57 +03:00
Nikolay
1059c4d84a
lib/promscrape/discovery/kubernetes: correctly wrap error (#3250)
* lib/promscrape/discovery/kubernetes: correctly wrap error
follow-up after 1304824201

* Update docs/CHANGELOG.md

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-10-18 20:37:42 +03:00
Roman Khavronenko
4e0ea95f26
vmalert: lower severity level for RW retries (#3237)
The message about dropped data still remains at `error` level.
The change supposed to make log message more clear about how
serious it is.

Signed-off-by: hagen1778 <roman@victoriametrics.com>

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-10-18 14:27:20 +02:00
Aliaksandr Valialkin
069401a304
all: log error when environment variables referred from -promscrape.config are missing
This should prevent from using incorrect config files
2022-10-18 10:47:16 +03:00
Aliaksandr Valialkin
646fb17237
docs/MetricsQL.md: document that input histograms must have the same set of buckets when calculating the quantile over them
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3231
2022-10-14 13:59:58 +03:00
Aliaksandr Valialkin
adf3419699
docs/CHANGELOG.md: add a note that it is recommended to use v1.82.1 instead of v1.82.0 2022-10-14 13:41:01 +03:00
Aliaksandr Valialkin
3a0c69651a
docs/CHANGELOG.md: release v1.82.1 2022-10-14 11:33:25 +03:00
Aliaksandr Valialkin
2e3be68617
lib/bytesutil: make sure that the string passed to FastStringMather.Match() is copied before using it as a key in the internal cache map
This prevents from possible corruption of the internal cache map
when the underlying byte slice used by the string key is modified.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3227
2022-10-14 09:51:19 +03:00
Denys Holius
b1de74a6ba
k8s-monitoring-via-vm-cluster.md: remove extra spaces (#3234) 2022-10-13 17:25:35 +02:00
Yury Molodov
ff6151fa49
vmui: limit number of plotted series (#3229)
* feat: add maximum display series by tabs

* feat: add warning on PredefinedPanels.tsx

* docs/CHANGELOG.md: vmui limit number of plotted series

* docs/CHANGELOG.md: vmui limit number of plotted series

* wip

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-10-13 12:13:47 +03:00
Aliaksandr Valialkin
e6fd33044f
app/vmselect/promql: follow-up for 930f1ee153
Document the change at docs/CHANGELOG.md
Apply it to histogram_quantile() in the same way as to histogram_share()

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/3225
2022-10-13 12:03:08 +03:00
Aliaksandr Valialkin
76e275ddef
docs/CHANGELOG.md: document b856581ad3 2022-10-13 10:35:48 +03:00
Dan Fredell
42ce4364fc
Multi retention standardization of docs (#3223)
* Multi retention standardization of docs

While reading through the docs the implementation details had different formatting for storageNode. This standardizes them and adds a link to the retention docs.

* Update docs/guides/guide-vmcluster-multiple-retention-setup.md

Co-authored-by: Aliaksandr Valialkin <valyala@gmail.com>
2022-10-13 10:27:16 +03:00
Roman Khavronenko
96a106eab2
vmalert: update troubleshooting docs (#3228)
The default value of `-datasource.queryStep` has changed, so we update
the troubleshooting docs accordingly.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-10-12 12:12:37 +02:00
Aliaksandr Valialkin
cfae887c75
docs/CHANGELOG.md: document a4975ace86
The original commit, which led to the issue - 877940a131
2022-10-12 09:30:36 +03:00
Aliaksandr Valialkin
41925a9500
docs/CHANGELOG.md: document 9544b5cacf7446203fac19eb7c575779dc9b280e 2022-10-12 09:25:17 +03:00
Aliaksandr Valialkin
a79272db1f
docs/vmbackup.md: run make docs-sync after 076e721c22 2022-10-10 21:57:46 +03:00
Aliaksandr Valialkin
875abf0ef4
docs/CHANGELOG.md: document e384d88abf 2022-10-10 21:52:06 +03:00
Aliaksandr Valialkin
04a05f161c
app/vmselect: return back the logic for limits the amounts of memory occupied by concurrently executed queries if -search.maxMemoryPerQuery isn't set
This is needed for preserving backwards compatibility with the previous releases of VictoriaMetrics.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3203
2022-10-10 21:45:13 +03:00
Aliaksandr Valialkin
921918cb49
docs/sd_configs.md: document __scrape_timeout__, __scrape_interval__ and __series_limit__ labels 2022-10-09 15:05:30 +03:00
Aliaksandr Valialkin
50f5eae0e0
lib/promrelabel: remove unconditional sorting of the labels in ParsedConfigs.Apply(), since the sorting isnt needed in many places
Sort labels explicitly after calling the ParsedConfigs.Apply() when needed.

This reduces CPU usage when performing metric-level relabeling, where labels' sorting isn't needed.
2022-10-09 14:51:16 +03:00
Aliaksandr Valialkin
8e1ccecd97
docs: mention -search.maxMemoryPerQuery in the description to -search.maxConcurrentQueries command-line flag
This is a follow-up for 5138eaeea0

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3203
2022-10-09 13:56:59 +03:00
Aliaksandr Valialkin
4f4d591ccb
docs/vmbackupmanager.md: update docs after adding the support to make backups to Azure blob storage
This is a follow-up for 262ce77e2d
2022-10-08 10:30:39 +03:00
Aliaksandr Valialkin
8f8ce5e238
docs: add description for -search.maxMemoryPerQuery command-line flag 2022-10-08 01:16:55 +03:00
Aliaksandr Valialkin
5138eaeea0
app/vmselect: allow limiting per-query memory usage via -search.maxMemoryPerQuery command-line flag
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3203
2022-10-08 01:08:05 +03:00
Aliaksandr Valialkin
3aafbc3624
docs/CHANGELOG.md: add a link to a feature request for the feature, which allows specifying full scrape urls in targets list and in the __address__ label
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3208
2022-10-07 23:45:57 +03:00
Aliaksandr Valialkin
5269b1ad77
lib/promscrape: allow controlling staleness tracking on a per-scrape_config basis
Add support for no_stale_markers option at scrape_config section.
See https://docs.victoriametrics.com/sd_configs.html#scrape_configs and
https://docs.victoriametrics.com/vmagent.html#prometheus-staleness-markers
2022-10-07 23:36:14 +03:00
Aliaksandr Valialkin
93811da76d
docs/CHANGELOG.md: document the 27ed4b853e
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3169
See https://github.com/VictoriaMetrics/VictoriaMetrics/pull/3196#issuecomment-1269765205
2022-10-07 23:06:32 +03:00
Aliaksandr Valialkin
b47caa86db
all: update the minimum required Go verson from 1.19.1 to 1.19.2
This is needed because of security vulnerabilities found in Go 1.19.1
See https://go.dev/doc/devel/release#go1.19.2
2022-10-07 22:43:37 +03:00
Aliaksandr Valialkin
f9df0cae16
lib/promscrape: allow specifying full target url in __address__ label
Previously the `__address__` label could contain only `host:port` part of the target url,
while the scheme and metrics path were obtained from `__scheme__` and `__metrics_path__`
labels. Now it is possible to set the full url in `__address__` label.

This makes valid the following scrape config, which is frequently used by novice users:

scrape_configs:
- job_name: foo
  static_configs:
  - targets:
    - http://host1/metrics1
    - https://host2/metrics2
2022-10-07 22:43:04 +03:00
Aliaksandr Valialkin
8bc840358f
docs/CHANGELOG.md: cut v1.82.0 2022-10-07 03:15:23 +03:00
Aliaksandr Valialkin
7b6ce3f75e
docs/CHANGELOG.md: typo fix 2022-10-07 03:11:22 +03:00
Aliaksandr Valialkin
ba4050ab1f
docs/CHANGELOG.md: add a changelog for v1.79.4 LTS release (copied from lts-1.79 branch) 2022-10-07 02:50:59 +03:00
Aliaksandr Valialkin
285e92706d
docs/MetricsQL.md: formatting improvements
Also put a link to the function type in docs for every function.
This should simplify understanding of MetricsQL functions for novice users.
2022-10-07 00:53:14 +03:00
Aliaksandr Valialkin
f452c84579
app/vmselect/promql: properly calculate vm_rows_scanned_per_query histogram for rollup functions, which take into account only a few samples on the provided lookbehind window 2022-10-06 23:22:24 +03:00
Aliaksandr Valialkin
40e899fd67
app/vmselect/promql: properly calculate quantiles_over_time() over a single raw sample 2022-10-06 22:37:21 +03:00
Aliaksandr Valialkin
8ae713253e
docs/CHANGELOG.md: remove duplicate description of the bugfix, which has been included in v1.81.2 2022-10-06 15:53:05 +03:00
Roman Khavronenko
5b10fa87b2
vmalert: fix misleading line regarding multitenancy (#3206)
Signed-off-by: hagen1778 <roman@victoriametrics.com>

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-10-06 15:03:45 +03:00
Aliaksandr Valialkin
8d7910a463
docs/CHANGELOG.md: add missing link to /api/v1/export docs
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3161
2022-10-06 15:02:13 +03:00
Aliaksandr Valialkin
d9282027e6
app: follow-up after ec04fcac93
* Optimize fast path for /api/v1/import when importing numeric values
* Move the docs about the change from features to bugfixes at docs/CHANGELOG.md
* Update tests at lib/protoparser/vmimport

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3161
2022-10-06 14:52:02 +03:00
Dmytro Kozlov
ec04fcac93
Properly parse json when export import metric (#3180)
* app/vmselect: properly work when export import json from `api/v1/{export, import}` API

* app/vmselect: update convert function

* app/vmselect: export null if `math.IsNaN(v)`

* app/vmselect: get float from json

* lib/protoparser: add test

* docs: add change log

* lib/protoparser: make export import api compatible
2022-10-06 13:54:20 +03:00
Aliaksandr Valialkin
dba49943d3
docs/Single-server-VictoriaMetrics.md: update docs after the a54987f671 2022-10-06 13:29:38 +03:00
Yury Molodov
a54987f671
vmui: maximum queries (#3196)
vmui: allow using up 4 queries at the same time

The change also introduces UI updates to make using 
multiple queries more conveniently.
2022-10-06 07:10:21 +02:00
Aliaksandr Valialkin
5307cf068f
docs: follow-up after 262ce77e2d
* Document the addition of Azure blob storage support in vmbackup / vmrestore
* List the supported storage system types at docs/vmrestore.md
* Mention about azblob storage system support at -src and -dst command-line flags
  for vmbackup / vmrestore tools.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1029
2022-10-06 00:35:34 +03:00
Aliaksandr Valialkin
f596e49881
docs/CHANGELOG.md: document f8ac55d70ada9ef8490b322abefb05f28f75e2e9 2022-10-06 00:05:37 +03:00
Aliaksandr Valialkin
c45c61cf93
app/vmalert: follow-up after f8ac55d70ada9ef8490b322abefb05f28f75e2e9
* Use vm_account_id and vm_project_id labels to be consistent with https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#multitenancy-via-labels
* Document the feature that vmalert now exposes vm_account_id and vm_project_id
  labels if -clusterMode is set.
* Use literal strings instead of string constants for vm_account_id and vm_project_id.
  This improves code readability.
2022-10-06 00:05:33 +03:00
Aliaksandr Valialkin
703094a37a
app/vmbackupmanager: expose vm_backup_in_flight metrics (follow-up after c6bca4a0b47b4f5626e1913d3480e62d657ed4cf) 2022-10-05 23:30:21 +03:00
Aliaksandr Valialkin
f9fc838b7b
app/vmalert: update -external.alert.source command-line flag description after 61544e13ad 2022-10-05 22:52:38 +03:00
Aliaksandr Valialkin
440495df52
Makefile: fix sed command and if condition for Linux bash after 2d11896486 2022-10-05 22:40:04 +03:00
Roman Khavronenko
61544e13ad
vmalert: allow using {{$labels}} for templating in -external.alert.source (#3194)
The change is supposed to provide additional flexibility for generating alert's
source link based on label values.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-10-05 19:25:03 +02:00
Dmytro Kozlov
e0ea76db62
docs: add cardinality explorer blog post (#3198)
docs: add cardinality explorer blog post
2022-10-05 13:19:41 +02:00
Roman Khavronenko
2d11896486
docs: udpate Datadog section (#3190)
The `docs-sync` command was updated to modify images path
for assets in `docs/` folder. The change allows to refer images from `docs`
without copying them to the root folder.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-10-03 13:13:07 +02:00
Yurii Kravets
272f00dbb6
docs: Update How to send data from DataDog agent (#3168)
docs: Update How to send data from DataDog agent
2022-10-03 11:45:05 +02:00
Aliaksandr Valialkin
c53b7e66ef
app/vmselect: improve performance scalability on multi-CPU systems for /api/v1/export/... endpoints 2022-10-01 22:05:43 +03:00
Aliaksandr Valialkin
fcc7ab71b3
app/vmselect: do not export NaN values for stale metrics at /federate endpoint
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3185
2022-10-01 19:47:37 +03:00
Aliaksandr Valialkin
7812761ab4
docs/Cluster-VictoriaMetrics.md: add a security note for multitenant support via labels 2022-10-01 18:57:28 +03:00
Aliaksandr Valialkin
d7327d2f02
docs/vmagent.md: fix incorrect url for multitenant writes to VictoriaMetrics cluster 2022-10-01 18:51:37 +03:00
Aliaksandr Valialkin
0dc93cca7f
app/vmagent/remotewrite: allow specifying per--remoteWrite.url disk limits for persistent queue with pending data
This commit is based on https://github.com/VictoriaMetrics/VictoriaMetrics/pull/3071

Related issue: https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2970
2022-10-01 18:40:59 +03:00
Aliaksandr Valialkin
a296994fed
app/vmauth: do not remove trailing slash from the proxied path
This should fix the issue with opening VMUI at /vmui/ page.

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1752
2022-10-01 16:52:30 +03:00
Aliaksandr Valialkin
4998402004
lib/promscrape: add external_labels from global section of -promscrape.config after the relabeling is applied to the scraped metrics
This aligns with Prometheus behaviour.

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3137
2022-10-01 16:13:19 +03:00
Aliaksandr Valialkin
973ce4b561
app/vmagent: accept requests with /prometheus and /influx path prefixes in the same way as VictoriaMetrics does
This allows using vmagent as a drop-in replacement for VictoriaMetrics
for push protocols.
2022-10-01 14:10:48 +03:00
Aliaksandr Valialkin
63a94b1d54
docs/vmagent.md: formatting fixes 2022-10-01 13:36:24 +03:00
lionelee
1fda517af9
docs/vmagent.md: fix typo (#3186) 2022-10-01 12:30:01 +03:00
Aliaksandr Valialkin
db16759c68
lib/storage: optimize matching speed for non-trivial regexp filters
Wrap re.Match into bytesutil.FastStringMatcher.

This increases performance for `{foo=~"complex_regex_here"}` filters
by up to 4x.
2022-10-01 12:06:06 +03:00
Aliaksandr Valialkin
9e8fbef27e
docs/CHANGELOG.md: clarify the description of the improvement in relabeling performance 2022-10-01 11:54:48 +03:00
Aliaksandr Valialkin
c54e14cdec
lib/promscrape/discovery/ec2: expose __meta_ec2_region label in the same way as Prometheus 2.39 does
See https://github.com/prometheus/prometheus/pull/11326
2022-09-30 20:48:32 +03:00
Aliaksandr Valialkin
d0b7172316
app/vminsert: remove support for undocumented VictoriaMetrics_AccountID and VictoriaMetrics_ProjectID labels in tcp-based data ingestion endpoints
These labels are substituted by documented vm_account_id and vm_project_id labels.

See https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#multitenancy-via-labels

This is a follow up for 505d359b39
2022-09-30 18:35:53 +03:00
Nikolay
33f40f4a5f
app/vminsert: allows parsing tenant id from labels (#3009)
* app/vminsert: allows parsing tenant id from labels
it should help mitigate issues with vmagent's multiTenant mode, which works incorrectly at heavy load
and it cannot handle more then 100 different tenants.
This functional hidden with flag and do not change vminsert default behaviour
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2970

* Update docs/Cluster-VictoriaMetrics.md

Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>

* wip

* app/vminsert/netstorage: clean remaining labels in order to free up GC

* docs/Cluster-VictoriaMetrics.md: typo fix

* wip

* wip

Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-09-30 18:35:53 +03:00
Roman Khavronenko
740bb2cc00
vmalert: support auth configs per static_target (#3188)
Allow configuring authorization params per list of targets
in vmalert's notifier config for `static_configs`.

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2690

Signed-off-by: hagen1778 <roman@victoriametrics.com>

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-09-30 17:10:17 +02:00
Aliaksandr Valialkin
a18d6d5ccc
lib/promrelabel: optimize action: replace for non-trivial regex values
Cache `action: replace` results for non-trivial regexs and return them next time
instead of performing CPU-intensive regex replacement.

Optimize also `action: labelmap_all` and `action: replace_all` in the same way.
2022-09-30 12:25:05 +03:00
Dmytro Kozlov
e220bc3cd5
docs, app/vmgateway: add description about new auth.httpHeader flag (#3134)
* docs, app/vmgateway: add description about new `auth.httpHeader` flag

* Update docs/CHANGELOG.md

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-09-30 08:55:12 +03:00
Aliaksandr Valialkin
b70f815dc4
app/vmselect/promql: remove empty series before applying aggregate function
Previously empty series (e.g. series with all NaN samples) were passed to aggregate functions.
Such series must be ingored by all the aggregate functions.
So it is better from consistency PoV filtering out empty series before applying aggregate functions.
2022-09-30 08:39:54 +03:00
Roman Khavronenko
b64b9b9fec
app/vmselect: ignore empty series for limit_offset (#3178)
* app/vmselect: ignore empty series for `limit_offset`

VictoriaMetrics doesn't return empty series (with all NaN values) to
the user. But such series are filtered after transform functions.
It means `limit_offset` will account for empty series as well.

For example, let's consider following data set:
```
time series:
foo{label="1"} NaN, NaN, NaN, NaN // empty series
foo{label="2"} 1, 2, 3, 4
foo{label="3"} 4, 3, 2, 1
```

When user requests all series for metric `foo` the empty series
will be filtered out:
```
/query=foo:
foo{label="v2"} 1, 2, 3, 4
foo{label="v3"} 4, 3, 2, 1
```

But `limit_offset(1, 1, foo)` is applied to original series, not filtered yet.
So it will return `foo{label="v2"}` (skips the first in list)
```
/query=limit_offset(1, 1, foo):
foo{label="v2"} 1, 2, 3, 4
```

Expected result would be to apply `limit_offset` to already filtered list,
so in result we receive `foo{label="v3"}`:
```
/query=limit_offset(1, 1, foo):
foo{label="v3"} 4, 3, 2, 1
```

The change does exactly that - filters empty series before applying `limit_offset`.

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* app/vmselect: ignore empty series for `limit_offset`

Signed-off-by: hagen1778 <roman@victoriametrics.com>

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-09-30 08:20:34 +03:00
Aliaksandr Valialkin
fda60b3d4d
lib/promrelabel: properly parse regex with escaped $ at the end
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3131

Thanks to @dmitryk-dk for the initial fix at https://github.com/VictoriaMetrics/VictoriaMetrics/pull/3179
2022-09-30 08:15:43 +03:00
Aliaksandr Valialkin
bf2f14a3a6
docs/CHANGELOG.md: document 39f559d22b
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3013
2022-09-30 07:48:25 +03:00
Nikolay
f61b8cec69
lib/awsapi: fixes sign encoding (#3183)
* lib/awsapi: fixes sign encoding

previously white spaces at filter were incorrectly encoded
encoding tip was copied from aws signing lib
For example, the space character must be encoded as %20 (not using '+', as some encoding schemes do)
https://docs.aws.amazon.com/general/latest/gr/sigv4-create-canonical-request.html
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3171

* Update lib/awsapi/sign.go

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-09-30 07:43:44 +03:00
nemobis
6a818dbddf
docs: fix typo in Cluster-VictoriaMetrics.md (#3172) 2022-09-28 09:18:29 +02:00
panguicai
fbc85e654c
docs: fix typo for vmalert docs (#3173)
Signed-off-by: panguicai008 <1121906548@qq.com>
2022-09-28 09:15:05 +02:00
Roman Khavronenko
4ad3b36630
docs: update img for key concepts (#3176)
Make dots bigger for range_query, so it will be easier
to read the graph.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-09-27 21:22:59 +02:00
Dmytro Kozlov
6ab1cede62
lib/{httpserver,netutil}: allow to define min and max TLS version of the http server (#3109)
* lib/{httpserver,netutil}: allow to define min and max TLS version of the http server

* lib/httpserver: added descriptions about tls supported versions

* lib/netutil: check minimal tls version, added supported tls versions to error

* wip

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-09-26 17:35:45 +03:00
Denys Holius
d63410bf6f
docs: managed_victoriametrics/quickstart.md fix image link (#3165) 2022-09-26 16:35:18 +02:00
Roman Khavronenko
36a9a834b3
docs: clarify numeric label values for label_value (#3129)
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3114
Signed-off-by: hagen1778 <roman@victoriametrics.com>

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-09-26 16:48:38 +03:00
Max Golionko
d67948b8e3
added security policy (#3140)
* added security policy

* Update SECURITY.md

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-09-26 16:47:02 +03:00
Roman Khavronenko
e96ccf3f71
lib/mergeset: follow-up after a0e7432e42 (#3145)
* lib/mergeset: follow-up after a0e7432e42

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* Apply suggestions from code review

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-09-26 16:39:56 +03:00
Aliaksandr Valialkin
72c29d762e
docs/CHANGELOG.md: document f022296d96
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3147
2022-09-26 16:32:19 +03:00
Aliaksandr Valialkin
a2431c2a88
docs/CHANGELOG.md: document 166d444159 2022-09-26 15:39:14 +03:00
Aliaksandr Valialkin
41f8c2987d
lib/protoparser/graphite: accept whitespace in metric names and tags according to the specification
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/99
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3102

See the specification https://graphite.readthedocs.io/en/latest/tags.html
2022-09-26 15:17:25 +03:00
Aliaksandr Valialkin
5983ecf4d1
docs/Single-server-VictoriaMetrics.md: mention that VictoriaMetrics sanitizes metric names in DataDog data by default
This is a follow-up for 7c2474dac7
2022-09-26 14:02:43 +03:00
Aliaksandr Valialkin
7c2474dac7
lib/protoparser/datadog: sanitize metric names by default in the same way as DataDog does
This commit is based on the pull request https://github.com/VictoriaMetrics/VictoriaMetrics/pull/3105

Thanks to @PerGon for the idea and initial implementation.
2022-09-26 13:57:23 +03:00
Aliaksandr Valialkin
fcffdba9dc
app/{vmagent,vminsert}: add -usePromCompatibleNaming command-line flag for normalizing metric names and label names in the ingested samples
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3113

Thanks to @erkexzcx for the idea and the initial pull request at https://github.com/VictoriaMetrics/VictoriaMetrics/pull/3146
2022-09-26 13:11:39 +03:00
Aliaksandr Valialkin
819aa95552
docs/vmalert.md: follow-up for 0c95f928ae
- Clarify the description for -datasource.queryStep command-line flag
- Consistently use a single dash in front of -datasource.queryStep command-line flag
- Update -help output at docs/vmalert.md
2022-09-26 08:47:31 +03:00
Aliaksandr Valialkin
6b71f33c8b
docs/vmalert.md: follow-up after 7748a9d629
- Consistently use single dash in front of command-line flags instead of double dashes.
- Add a warning that too small -search.latencyOffset may lead to incomplete query results.
2022-09-26 08:36:24 +03:00
Aliaksandr Valialkin
b68cd810fc
docs: follow-up for 03d54ac890
- Typo fix: 'adn' -> 'and'
- Remove duplicate link to https://victoriametrics.com/blog/victoriametrics-monitoring/
  from docs/Articles.md . The initial link has been already added in 27254096b2
2022-09-26 08:26:38 +03:00
Roman Khavronenko
0c95f928ae
vmalert: set default value for datasource.queryStep to 5m (#3149)
Change default value for command-line flag `datasource.queryStep` from `0s` to `5m`.
Param `step` is added by vmalert to every rule evaluation request sent to datasource.
Before this change, `step` was equal to group's evaluation interval by default.
Param `step` for instant queries defines how far VM can look back for the last written data point.
The change supposed to improve reliability of the rules evaluation when evaluation interval
is lower than scraping interval.

Signed-off-by: hagen1778 <roman@victoriametrics.com>

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-09-23 10:22:53 +02:00
Roman Khavronenko
7748a9d629
vmalert: add info about search.latencyOffset to Troubleshooting (#3151)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-09-23 09:49:14 +02:00
Roman Khavronenko
03d54ac890
docs: mention VictoriaMetrics Monitoring blog post (#3152)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-09-23 07:12:15 +02:00
Aliaksandr Valialkin
27254096b2
docs/Articles.md: add a link to https://victoriametrics.com/blog/victoriametrics-monitoring/ 2022-09-22 19:17:10 +03:00
Aliaksandr Valialkin
a3e536c0e7
docs/Articles.md: add a link to https://dataswamp.org/~solene/2022-09-11-exploring-monitoring-stacks.html 2022-09-22 19:17:10 +03:00
Roman Khavronenko
5714a68ac6
deployment/docker: move cluster compose env to master branch (#3130)
* deployment/docker: move cluster compose env to master branch

The change supposed to simplify the process of maintaining for
single/cluster docker-compose envs, alerts, dashboards. It also
supposes to reduce confusion for users when looking for cluster
related alerts/configs.

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* deployment/docker: move cluster compose env to master branch

Review updates.

Signed-off-by: hagen1778 <roman@victoriametrics.com>

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-09-21 11:48:38 +03:00
Aliaksandr Valialkin
1aef635de4
docs/CHANGELOG.md: clarify the change at 622bbedbe1
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/3119
2022-09-19 15:02:05 +03:00
Aliaksandr Valialkin
584a5d6b1f
docs/managed_victoriametrics/quickstart.md: small fixes after 606166ef68 2022-09-19 14:52:46 +03:00
Roman Khavronenko
622bbedbe1
vmalert: always re-evaluate Annotations (#3119)
* vmalert: always re-evaluate Annotations

Previously, Annotations were evaluated only:
1. On alert creating.
2. On alert's value change.

This is premature optimization. It was assumed that since annotations
could contain only text with alert's labels or value - there is no need
in spending resources to re-compile Annotations.

Later, template function `query` was added, which can execute
arbitrary queries and return different results on every evaluation.
So if it was used in annotations, it would be executed only on init
or value change.

Another case when optimization caused an issue - annotations hot reload.
In this case, annotations of the active alert won't change even if Rule's
annotations were changed.

This fix enables Annotations re-evaluation on each iteration to resolve
issues above. It would have some impact on performance, but it is unlikely
it will be noticeable.

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* vmalert: add tp Changelog

Signed-off-by: hagen1778 <roman@victoriametrics.com>

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-09-16 16:19:10 +02:00
Roman Khavronenko
3484673566
vmalert: add Troubleshooting section to docs (#3115)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-09-15 16:15:39 +02:00
Dmytro Kozlov
606166ef68
docs/managed_victoriametrics: add how to restore password section (#3116)
docs/managed_victoriametrics: add how to restore password section
2022-09-15 16:02:31 +02:00
Aliaksandr Valialkin
455002922e
docs/Cluster-VictoriaMetrics.md: update -help output after explicit marking of enterprise flags 2022-09-15 13:22:58 +03:00
Aliaksandr Valialkin
e7995375b5
docs: update -help output after explicit mentioning of enterprise flags 2022-09-15 13:22:57 +03:00
Aliaksandr Valialkin
4193af4571
docs/vmauth.md: update -help output after explicit marking of enterprise flags 2022-09-15 13:22:57 +03:00
Aliaksandr Valialkin
3b2599c659
docs/vmalert.md: update -help output after explicit marking of enterprise flags 2022-09-15 13:22:57 +03:00
Aliaksandr Valialkin
bbecd27557
docs: update docs with explicitly marked enterprise command-line flags for VictoriaMetrics and vmagent 2022-09-15 13:22:56 +03:00
Dmytro Kozlov
a9629cc32d
app/vmctl: add description about influx-skip-database-label flag (#3111) 2022-09-15 10:46:54 +02:00
Aliaksandr Valialkin
b869c757a9
docs/FAQ.md: add an answer for What is the difference between single-node and cluster versions of VictoriaMetrics? 2022-09-15 09:56:41 +03:00
Dmytro Kozlov
b75f1854c5
vmselect/promql: add alphanumeric sort by label (sort_by_label_numeric) (#2982)
* vmselect/promql: add alphanumeric sort by label (sort_by_label_numeric)

* vmselect/promql: fix tests, add documentation

* vmselect/promql: update test

* vmselect/promql: update for alphanumeric sorting, fix tests

* vmselect/promql: remove comments

* vmselect/promql: cleanup

* vmselect/promql: avoid memory allocations, update functions descriptions

* vmselect/promql: make linter happy (remove ineffectual assigment)

* vmselect/promql: add test case, fix behavior when strings are equal

* vendor: update github.com/VictoriaMetrics/metricsql from v0.44.1 to v0.45.0

this adds support for sort_by_label_numeric and sort_by_label_numeric_desc functions

* wip

* lib/promscrape: read response body into memory in stream parsing mode before parsing it

This reduces scrape duration for targets returning big responses.

The response body was already read into memory in stream parsing mode before this change,
so this commit shouldn't increase memory usage.

* wip

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-09-14 17:41:09 +03:00
Aliaksandr Valialkin
5306f79fd1
docs/Articles.md: add a link to https://www.groundcover.com/blog/prometheus-grafana-kubernetes 2022-09-14 15:11:19 +03:00
Roman Khavronenko
877940a131
vmalert: add experimental feature of storing Rule's evaluation state (#3106)
vmalert: add experimental feature of storing Rule's evaluation state

The new feature keeps last 20 state changes of each Rule
in memory. The state are available for view on the Rule's
view page. The page can be opened by clicking on `Details`
link next to Rule's name on the `/groups` page.

States change suppose to help in investigating cases when Rule
doesn't generate alerts or records.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-09-14 14:04:24 +02:00
Roman Khavronenko
efea51a9ee
bump Go version to 1.19.1 (#3108)
The reason is to cover vulnerability GO-2022-0969
Found in: net/http@go1.18.5
Fixed in: net/http@go1.19.1
More info: https://pkg.go.dev/vuln/GO-2022-0969

Signed-off-by: hagen1778 <roman@victoriametrics.com>

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-09-14 12:29:19 +02:00
Aliaksandr Valialkin
74c00a8762
lib/promscrape: read response body into memory in stream parsing mode before parsing it
This reduces scrape duration for targets returning big responses.

The response body was already read into memory in stream parsing mode before this change,
so this commit shouldn't increase memory usage.
2022-09-14 13:15:29 +03:00
Aliaksandr Valialkin
3af24a5b7c
docs: consistently use docs.victoriametrics.com instead of victoriametrics.github.io in all the links 2022-09-13 16:50:13 +03:00
Aliaksandr Valialkin
0ead64b6cf
app/vmalert: follow-up after 8441375da2
- Rename logDebug() to logDebugf() and pass format string together
  with format args directly to logDebugf(). This eliminates fmt.Sprintf()
  overhead at logDebug() call site when debugging is disabled.

- Format labels in debug message in Prometheus format, e.g. {label1="value1",...labelN="valueN"}

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3025
2022-09-13 16:35:14 +03:00
Roman Khavronenko
8441375da2
vmalert: add debug mode for alerting rules (#3055)
* vmalert: add `debug` mode for alerting rules

Debug information includes alerts state changes and requests
sent to the datasource. Debug can be enabled only on rule's
level. It might be useful for debugging unexpected
behaviour of alerting rule.

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3025

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* vmalert: review fixes

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* Update app/vmalert/alerting.go

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>

* vmalert: go fmt

Signed-off-by: hagen1778 <roman@victoriametrics.com>

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-09-13 16:25:43 +03:00
Aliaksandr Valialkin
25c9a1604a
docs/CHANGELOG.md: document atomic directory deletion
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3038
2022-09-13 16:17:39 +03:00
Aliaksandr Valialkin
db4f0fe6fc
docs/ExtendedPromQL.md: move the page to the bottom of the contents list at http://docs.victoriametrics.com 2022-09-13 16:17:37 +03:00
Aliaksandr Valialkin
e041c913bd
Revert "docs/ExtendedPromQL.md: remove outdated doc"
This reverts commit 971e3d83f7.

Reason for revert: the ExtendedPromQL doc is still referred from third-party sites.
2022-09-13 16:17:37 +03:00
Craig Rodrigues
57ea8dbb36
docs: Use headings in DataDog section to make things easier to read (#3089)
Signed-off-by: Craig Rodrigues <rodrigc@crodrigues.org>

Signed-off-by: Craig Rodrigues <rodrigc@crodrigues.org>
2022-09-12 10:58:25 +03:00
Yury Molodov
1b41169415
vmui: fix data processing (#3092)
* fix: change data processing

* app/vmselect/vmui: `make vmui-update`

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-09-12 10:55:11 +03:00
Aliaksandr Valialkin
c48ff746c6
docs/FAQ.md: add a link to https://victoriametrics.com/blog/mimir-benchmark/ to VictoriaMetrics vs Mimir chapter 2022-09-12 10:37:28 +03:00
Aliaksandr Valialkin
c4af0e833a
docs/Articles.md: add a link to https://victoriametrics.com/blog/mimir-benchmark/ 2022-09-11 15:23:35 +03:00
Yury Molodov
9541ef2e9e
vmui: add lists of top queries (#3065)
* feat: add lists of top queries

* fix: change the field label

* refactor: add handlers for readability

* app/vmselect: `make vmui-update`

* docs: document `top queries` tab

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-09-08 21:43:37 +03:00
John Belmonte
defced2599
MetricsQL doc spellcheck (#3080) 2022-09-08 21:28:03 +03:00
Aliaksandr Valialkin
53b0c2eee4
docs/CHANGELOG.md: document ec273eafef
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3076
2022-09-08 21:25:30 +03:00
Aliaksandr Valialkin
b9231c715a
docs/Release-Guide.md: require manual push of the created release tags to public Github
The automated push of release tags to Github require specifying the remote repository name
when doing `git push <remote-repo-name> v1.xx.y`.
The remote repository name can differ in different environments,
so it cannot be put into Makefile rule.

TODO: create a Makefile rule, which generates standard remote names for public
and private repositories in Git, so `git push` for release tags could be automated then.
2022-09-08 15:00:18 +03:00
Aliaksandr Valialkin
07441b1cee
app/vmselect/netstorage: fix a typo, which leads to incorrect query results in VictoriaMetrics cluster
The typo has been introduced in the commit 1a254ea20c

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3067
2022-09-08 13:48:20 +03:00
Aliaksandr Valialkin
9f20d01a81
Revert "docs: add Monitoring at scale with Victoria Metrics (#3078)"
This reverts commit f24572fa65,
because the article https://tech.bedrockstreaming.com/2022/09/06/monitoring-at-scale-with-victoriametrics.html
has been already added at 09ff3f1928
2022-09-08 11:05:25 +03:00
Roman Khavronenko
f24572fa65
docs: add Monitoring at scale with Victoria Metrics (#3078)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-09-06 16:29:48 +02:00
Max Golionko
7da9443686
moved cluster dashboard to master (#3074)
dashboards: move cluster dashboard to master branch

This change should simplify dashboards management.
2022-09-06 16:19:43 +02:00
Aliaksandr Valialkin
ef7fdbb63c
docs/Single-server-VictoriaMetrics.md: add a note that cardinality explorer at cluster version of VictoriaMetrics may return lower than expected number of unique label values
See the corresponding comment in the code:

5a6e617b5e/app/vmselect/netstorage/netstorage.go (L1039-L1045)
2022-09-06 14:46:35 +03:00
Aliaksandr Valialkin
09ff3f1928
docs/Articles.md: add https://tech.bedrockstreaming.com/2022/09/06/monitoring-at-scale-with-victoriametrics.html 2022-09-06 13:32:41 +03:00
Aliaksandr Valialkin
651ace6ce4
docs/vmctl.md: make docs-sync after c5261d5f56
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2733
2022-09-06 13:19:48 +03:00
Aliaksandr Valialkin
5fa9525498
lib/storage: verify that timestamps in block are in the range specified by blockHeader.{Min,Max}Timestamp when upacking the block
This should reduce chances of unnoticed on-disk data corruption.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2998
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3011

This change modifies the format for data exported via /api/v1/export/native -
now this data contains MaxTimestamp and PrecisionBits fields from blockHeader.

This is OK, since the native export format is undocumented.
2022-09-06 13:08:09 +03:00
Zakhar Bessarab
c5261d5f56
vmctl: implement support of chunking data for vm-native export (#3044)
vmctl: implement support of chunking data for vm-native export process

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2733
2022-09-06 09:09:34 +02:00
Craig Rodrigues
462fc7b394
docs: Clarify DataDog examples for VictoriaMetrics cluster (#3048)
Signed-off-by: Craig Rodrigues <rodrigc@crodrigues.org>
2022-09-05 16:25:05 +02:00
Yurii Kravets
7b04112352
docs: Update keyConcepts (#3040)
docs: update keyConcepts

Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
2022-09-05 13:59:28 +02:00
Aliaksandr Valialkin
ae31b2363f
app/vmselect/prometheus: follow-up after 50e2524bc2
- Add getCommonParamsWithDefaultDuration function and use it at /api/v1/series, /api/v1/labels and /api/v1/label/.../values
- Document the default behaviour for setting 5 minutes time range if start arg isn't passed to /api/v1/series, /api/v1/labels and /api/v1/label/.../values
- Document the change at docs/CHANGELOG.md

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/3052
2022-09-05 11:55:48 +03:00
Aliaksandr Valialkin
7dc632719d
app/vmselect/promql: consistently calculate rate_over_sum(m[d]) as sum_over_time(m[d])/d
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3045
2022-09-02 23:18:04 +03:00
Aliaksandr Valialkin
ac98ecdc1d
docs/CHANGELOG.md: cut v1.81.1 2022-09-02 21:58:28 +03:00
Aliaksandr Valialkin
08538ff82a
app/vmselect/netstorage: fix potential panic under high load
The panic may trigger during data blocks' processing received
from vmstorage nodes when some of vmstorage nodes return an error
or when `-replicationFactor` is set to values higher than 2 at `vmselect`.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3058
2022-09-02 21:38:27 +03:00
Aliaksandr Valialkin
4076277cf0
app/vmselect/promql: evaluate union() args in parallel in order to increase query performance
Note that the parallel execution of `union()` args may take more memory and CPU time
than the sequential execution if args contain heavy queries, which may load all the available CPU,
disk and memory resources and vmselect and vmstorage levels.
2022-09-02 19:46:27 +03:00
Aliaksandr Valialkin
f9d4ade35a
docs/Articles.md: add a link to https://aatarasoff.medium.com/optimizing-linkerd-metrics-in-prometheus-de607ec10f6b 2022-08-31 05:03:22 +03:00
Aliaksandr Valialkin
b26de84b4a
docs/FAQ.md: mention that single-node vm is easier to setup and operate than the cluster version of vm 2022-08-31 05:02:15 +03:00
Aliaksandr Valialkin
6b0050a028
docs/CHANGELOG.md: cut v1.81.0 2022-08-31 02:33:23 +03:00
Max Golionko
c685afebb2
simplify release process (#3012)
* simplify release process

* address comments

* address comments

* wip

* wip

* wip

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-08-31 02:27:24 +03:00
Aliaksandr Valialkin
6c5d5bc7e6
docs/CHANGELOG.md: add missing link to vmagent 2022-08-31 02:21:44 +03:00
Aliaksandr Valialkin
c84e6429a0
docs/CHANGELOG.md: document v1.79.3 LTS release 2022-08-31 00:22:31 +03:00
Aliaksandr Valialkin
022bb62fa1
docs: clarify why cluster version is recommended to use for serving high loads 2022-08-31 00:22:30 +03:00
Craig Rodrigues
a54d7c24ff
docs: Add DataDog to list of ingestion methods (#3047)
Signed-off-by: Craig Rodrigues <rodrigc@crodrigues.org>

Signed-off-by: Craig Rodrigues <rodrigc@crodrigues.org>
2022-08-31 00:21:53 +03:00
Craig Rodrigues
e68a97f8d4
docs: add https://github.com/VictoriaMetrics/ansible-playbooks to README.md (#3046)
Signed-off-by: Craig Rodrigues <rodrigc@crodrigues.org>
Co-authored-by: Roman Khavronenko <hagen1778@gmail.com>
2022-08-30 20:52:24 +02:00
Aliaksandr Valialkin
bcdba8be18
docs/FAQ.md: add chapters explaining main differences between single-node and cluster versions of VictoriaMetrics
- Which type of VictoriaMetrics is better to use in production?
- How to migrate data from single-node VictoriaMetrics to cluster?
2022-08-30 13:12:03 +03:00
Artem Navoiev
5d4b1bc742
update multi-region guide, specify multi-level vmselect option (#3039)
* update multi-region guide, specify multi-level vmselect option

Signed-off-by: Artem Navoiev <tenmozes@gmail.com>

* add more references

* Apply suggestions from code review

Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-08-30 12:26:23 +03:00
Aliaksandr Valialkin
1eec2460ba
docs: fix links to DogStatsD
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3042
2022-08-30 10:49:26 +03:00
Aliaksandr Valialkin
d2e94ee91a
docs/CHANGELOG.md: document the 044d51b668 2022-08-30 09:42:19 +03:00
Aliaksandr Valialkin
ad11b8d83d
app/vmselect/promql: follow-up after 2d71b4859c
- Use getScalar() function for obtaining the expected scalar from phi arg
- Reduce the error message returned to the user when incorrect phi is passed to histogram_quantiles
- Improve the description of this bugfix in the docs/CHANGELOG.md

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3026
2022-08-27 01:35:49 +03:00
Dmytro Kozlov
2d71b4859c
vmselect/promql: fix panic in histogram_quantiles function (#3029)
* vmselect/promql: fix panic in histogram_quantiles function

* Update docs/MetricsQL.md

Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
2022-08-27 01:33:56 +03:00
Aliaksandr Valialkin
c06e7a142c
lib/promscrape: optimize discoveryutils.SanitizeLabelName()
Cache sanitized label names and return them next time.
This reduces the number of allocations and speeds up the SanitizeLabelName()
function for common case when the number of unique label names is smaller than 100k
2022-08-27 00:17:45 +03:00
Aliaksandr Valialkin
7afe8450fc
lib/promrelabel: optimize matching for commonly used regex patterns in if option
The following regex patterns are optimized:

- literal string match, e.g. "foo"
- prefix match, e.g. "foo.*" and "foo.+"
- substring match, e.g. ".*foo.*" and ".+foo.+"
- alternate values match, e.g. "foo|bar|baz"
2022-08-26 14:53:06 +03:00
Aliaksandr Valialkin
0d4ea03a73
lib/promrelabel: optimize action: {labeldrop,labelkeep,keep,drop} with regex containing alternate values
For example, the following relabeling rule must work much faster now:

- action: labeldrop
  regex: "foo|bar|baz"
2022-08-24 17:54:29 +03:00
Aliaksandr Valialkin
cdffe401e4
app/vmagent: follow-up after 2b22aa1537
- Document the change at docs/CHANGELOG.md
- Move auth token parsing from app/vmagent/opentsdbhttp/ to app/vmagent/main.go,
  since it must be parsed only when multitenancy support is enabled at vmagent side.
  See https://docs.victoriametrics.com/vmagent.html#multitenancy
2022-08-24 16:18:59 +03:00
Dmytro Kozlov
463ea6897b
vmselect/promql: enable search.maxPointsSubqueryPerTimeseries for sub-queries (#2963)
* vmselect/promql: enable search.maxPointsPerTimeSeriesSubquery for sub-queries

* vmselect/promql: cleanup

* vmselect/promql: rename config flag

* vmselect/promql: add tests

* vmselect/promql: use test object instead of log

* vmselect/promql: fix posible panic is subquery has more points. add description

* vmselect/promql: update tests descriptions

* vmselect/promql: update doInternal validation

* vmselect/promql: fix linter

* vmselect/promql: fix linter

* vmselect/promql: update documentation and release notes

* wip

- Properly apply -search.maxPointsSubqueryPerTimeseries limit to subqueries.
  Previously the -search.maxPointsPerTimeseries limit was unexpectedly applied to subqueries
  if it was smaller than the -search.maxPointsSubqueryPerTimeseries .
- Clarify docs for -search.maxPointsSubqueryPerTimeseries command-line flag .
- Document -search.maxPointsPerTimeseries and -search.maxPointsSubqueryPerTimeseries flags at https://docs.victoriametrics.com/#resource-usage-limits .
- Update docs/CHANGELOG.md .

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2922

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-08-24 15:25:18 +03:00
Aliaksandr Valialkin
3d12ee47f9
docs: mention that it is safe sharing the collected profiles from security PoV
The collected profiles do not contain sensitive information
2022-08-24 14:07:36 +03:00
Aliaksandr Valialkin
796aa310c2
app/vmstorage: expose vm_{hourly,daily}_series_limit_{max,current}_series metrics if -storage.max{Hourly,Daily}Series limits are set
These metrics allow alerting when the number of unique series approach the limit.
For example, the following query alerts when the number of series reaches 90% of the configured limit:

    vm_hourly_series_limit_current_series / vm_hourly_series_limit_max_series > 0.9
2022-08-24 13:44:04 +03:00
Aliaksandr Valialkin
b1e1c50627
all: update Google Analytics tracking code from Unversal Analytics to v4
This is needed because Google Analytics devs decided to force their users
to update all their tracking codes to GA v4.

See https://support.google.com/analytics/answer/9744165

The link to the new tracking property ( G-N9SVT8S3HK ) - https://analytics.google.com/analytics/web/?authuser=1#/a129683199p328513681/admin/streams/table/4004610168
2022-08-24 12:16:37 +03:00
Aliaksandr Valialkin
c011fb0f30
docs/vmagent.md: fix alerting query when scraped samples are dropped because of exceeded series limit
This is a follow-up after 7d26414b2e
2022-08-24 01:18:31 +03:00
Roman Khavronenko
8d0f5b9e60
docs: follow-up after 88425bb285 (#3007)
Signed-off-by: hagen1778 <roman@victoriametrics.com>

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-08-22 13:49:17 +02:00
laixintao
88425bb285
vmalert: add $activeAt into template variables. (#3000)
vmalert: add `$activeAt` template variable for annotations

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2999
2022-08-22 13:32:36 +02:00
Aliaksandr Valialkin
6e4e3fae63
docs/CHANGELOG.md: document d59d829cdb
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/2673
2022-08-21 23:35:47 +03:00
Ivan Dudin
f69990ba10
Update Cluster-VictoriaMetrics.md (#3005)
Flags `-search.maxTagKeys` and `-search.maxTagValues` are present at vmstorage and not at vmselect
```
# ./vmselect-prod -h | egrep 'search.maxTagKeys|search.maxTagValues'
# ./vmstorage-prod -h | egrep 'search.maxTagKeys|search.maxTagValues'
  -search.maxTagKeys int
  -search.maxTagValues int
```
2022-08-21 23:28:40 +03:00
Aliaksandr Valialkin
22aa07b8c9
docs/Cluster-VictoriaMetrics.md: document the best strategies for cluster update / upgrade 2022-08-21 23:21:02 +03:00
Aliaksandr Valialkin
8550c44e31
app/vmagent: add ability to construct a label from multiple existing labels by referring them in the replacement field during relabeling
For example:

- target_label: composite-label
  replacement: "{{source_label1}}-{{source_label2}}"
2022-08-21 22:50:01 +03:00
Aliaksandr Valialkin
f2043d53ad
docs: change links to Prometheus docs about instant and range queries to links to VictoriaMetrics docs 2022-08-21 19:01:54 +03:00
Aliaksandr Valialkin
e94b4622f3
docs/keyConcepts.md: more fixes 2022-08-21 18:53:37 +03:00
Aliaksandr Valialkin
af8201cbcc
docs/FAQ.md: add a question on differences between Grafana Mimir and VictoriaMetrics 2022-08-21 17:00:33 +03:00
Aliaksandr Valialkin
1b6f66d566
docs/keyConcepts.md - clarify docs a bit 2022-08-21 12:00:21 +03:00
Aliaksandr Valialkin
c6736a3ad2
docs/Cluster-VictoriaMetrics.md: clarify required conditions for cluster availability 2022-08-20 09:40:34 +03:00
Yurii Kravets
4ca189bf94
doc: udpate Cluster-VictoriaMetrics (#3003)
added note about resource usage for data re-routing
2022-08-20 09:37:27 +03:00
Aliaksandr Valialkin
3a1eb471a3
app/vmalert/README.md: sync with docs/vmalert.md after a229182dbe 2022-08-20 08:54:01 +03:00
laixintao
a229182dbe
doc: fix broken url of template function for vmalert. (#3002) 2022-08-20 08:50:26 +03:00
Aliaksandr Valialkin
b90103d950
docs/CHANGELOG.md: document 10402459d8 2022-08-19 11:43:59 +03:00
Roman Khavronenko
f451e0eabe
docs: fix docs formatting related to vmalert (#2994)
Signed-off-by: hagen1778 <roman@victoriametrics.com>

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-08-18 09:53:46 +02:00
Aliaksandr Valialkin
dff6314c87
docs/relabeling.md: fix heading for useful tips for target relabeling 2022-08-18 01:32:49 +03:00
Aliaksandr Valialkin
149eb59546
docs/relabeling.md: typo fixes 2022-08-18 01:30:03 +03:00
Aliaksandr Valialkin
c1b43afdcd
docs/relabeling.md: improve relabeling tips for scrape targets 2022-08-18 01:24:50 +03:00
Aliaksandr Valialkin
21e63e1518
docs/relabeling.md: add a link to VictoriaMetrics enhancements for relabeling 2022-08-18 01:20:01 +03:00
Aliaksandr Valialkin
421aca7b2f
docs: fix ordering after adding the docs/relabeling.md 2022-08-18 01:17:27 +03:00
Aliaksandr Valialkin
43c6f81b90
docs/relabeling.md: add a cookbook for common relabeling tasks 2022-08-18 01:13:52 +03:00
Aliaksandr Valialkin
22c47e97a5
docs: follow-up after 68e56b6fc5 2022-08-17 21:24:00 +03:00
Roman Khavronenko
68e56b6fc5
vmalert: set alert's source link to UI instead of JSON source (#2986)
We switch default alert's source link to redirect user
to vmalert's UI instead of previous JSON object. While it breaks
compatibility, it also supposed to improve user's experience.
The old behavior can be achieved by updating `-external.alert.source`
command-line flag.

Signed-off-by: hagen1778 <roman@victoriametrics.com>

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-08-17 14:46:28 +02:00
Aliaksandr Valialkin
eaa5d6cbd7
docs/sd_configs.md: clarify a bit honor_labels config option 2022-08-17 14:25:55 +03:00
Aliaksandr Valialkin
7d26414b2e
lib/promscrape: automatically generate additional per-target labels for targets with non-zero series limit
The following metrics are generated:

- scrape_series_limit
- scrape_series_current
- scrape_series_limit_samples_dropped

These metrics simplify alerting on targets, which expose too many time series

See https://docs.victoriametrics.com/vmagent.html#automatically-generated-metrics
and https://docs.victoriametrics.com/vmagent.html#cardinality-limiter for more details
2022-08-17 13:19:33 +03:00
Aliaksandr Valialkin
5b449649b6
docs/CHANGELOG.md: group vmalert features closer to each other 2022-08-17 11:50:23 +03:00
Roman Khavronenko
04c174a11e
docs: add new article link (#2989)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-08-17 10:01:01 +02:00
Aliaksandr Valialkin
bb68ab99fa
lib/promscrape: retry http requests if the server returns 429 status code
The 429 status code means that the server is overwhelmed with requests.
The client can retry the request after some wait time.
Implement this strategy for service discovery and scrape requests.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2940
2022-08-16 15:01:08 +03:00
Aliaksandr Valialkin
f830edc0bc
lib/storage: improve performance for /api/v1/labels and /api/v1/label/.../values endpoints when match[] filter matches small number of time series
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2978
2022-08-16 13:32:40 +03:00
Aliaksandr Valialkin
e0e7c14788
docs/MetricsQL.md: add the list of supported ... functions lines just before the corresponding lists
This improves the readability a bit
2022-08-16 12:08:57 +03:00
Roman Khavronenko
8b3989ba39
docs: update vmalert docs (#2987)
* mention recently added `$alertID` and `$groupID` variables in the changelog
* properly escape template examples in the vmalert's README

Signed-off-by: hagen1778 <roman@victoriametrics.com>

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-08-16 11:44:08 +03:00
Roman Khavronenko
f1b2273d13
vmalert: support $alertID and $groupID in template variables (#2983)
Support of these two variables allows building custom URLs with
alert's ID and group ID params.

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/517#issuecomment-1207141432

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-08-16 08:08:27 +02:00
Artem Navoiev
edd7a92e8b
fix typo in influx guide (#2984) 2022-08-15 22:59:35 +02:00
Aliaksandr Valialkin
4ac79d29ad
app/vmselect: follow-up after 63e0f16062
* Explicitly store a pointer to UserReadableError in the error interface.
  Previously Go automatically converted the value to a pointer before storing in the error interface.

* Add Unwrap() method to UserReadableError, so it can be used transparently with the other code,
  which calls errors.Is() and errors.As().

* Document the change in docs/CHANGELOG.md
2022-08-15 13:50:16 +03:00
Roman Khavronenko
5792cae0ab
docs: mention default vmalert's behavior change in Update notes (#2981)
Signed-off-by: hagen1778 <roman@victoriametrics.com>

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-08-15 13:31:24 +03:00
Yury Molodov
1513866d51
vmui: shortcut keys legend (#2971)
* feat: add shortcut modal

* feat: add shortcut descriptions

* app/vmselect/vmui: `make vmui-update`

* docs/CHANGELOG.md: document the change

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-08-15 11:54:47 +03:00
Yurii Kravets
88c4f29ea5
doc: add upgrade/downgrade without downtime to FAQ (#2973)
* doc: add upgrade/downgrade without downtime to FAQ

Added info on how to upgrade or downgrade VictoriaMetrics without downtime to FAQ
Based on reply https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2944#issuecomment-1207502038

* Udate pr

Fixing format
2022-08-15 11:28:39 +03:00
Aliaksandr Valialkin
c3f8481011
lib/promscrape: update links to sd_configs from Prometheus site to https://docs.victoriametrics.com/sd_configs.html 2022-08-15 01:40:20 +03:00
Aliaksandr Valialkin
95d36da358
lib/promscrape/discovery/kubernetes: add __meta_kubernetes_pod_container_image label in the same way as Prometheus 2.38 does
See https://github.com/prometheus/prometheus/pull/11034
2022-08-15 01:18:23 +03:00
Aliaksandr Valialkin
c4fcd9f1c5
lib/promscrape/discovery/kubernetes: add __meta_kubernetes_service_port_number label to role: service in the same way as Prometheus 2.38 does
See https://github.com/prometheus/prometheus/pull/11002
2022-08-15 01:06:34 +03:00
Aliaksandr Valialkin
d335694add
app/vmalert/templates: add toTime() template function in the same way as Prometheus 2.38 does
See https://github.com/prometheus/prometheus/pull/10993
2022-08-15 00:49:31 +03:00
Aliaksandr Valialkin
511805d88d
lib/promscrape/discovery/dns: add support for resolving MX records
See https://github.com/prometheus/prometheus/pull/10099
2022-08-15 00:32:34 +03:00
Aliaksandr Valialkin
e1cb15807e
app/vmselect/netstorage: improve scalability of blocks processing on systems with multiple CPU cores
Previously a single syncwg.WaitGroup was used for tracking the lifetime of processBlock callbacks
across all the per-vmstorage goroutines. This could be slow on systems with many CPU cores
because of inter-CPU synchronization overhead.

Use a separate per-vmstorage sync.WaitGroup instead in order to reduce inter-CPU synchronization overhead.
This should imrpove performance for heavy queries over big number of blocks on multi-CPU systems.
2022-08-11 23:53:58 +03:00
Aliaksandr Valialkin
1e4fc20486
docs/CHANGELOG.md: clarify the change at 28441711e6 2022-08-11 23:49:22 +03:00
Roman Khavronenko
fa51c76ef9
vmalert: follow-up after 28441711e6 (#2972)
Signed-off-by: hagen1778 <roman@victoriametrics.com>

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-08-11 13:30:32 +02:00
Roman Khavronenko
a0e7432e42
lib/storage: prevent excessive loops when storage is in RO (#2962)
* lib/storage: prevent excessive loops when storage is in RO

Returning nil error when storage is in RO mode results
into excessive loops and function calls which could
result into CPU exhaustion. Returning an err instead
will trigger delays in the for loop and save some resources.

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* document the change

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-08-09 12:17:00 +03:00
Aliaksandr Valialkin
ad00f4aaaa
docs/CHANGELOG.md: cut v1.80.0 2022-08-08 19:58:50 +03:00
Aliaksandr Valialkin
901300aea2
docs/CHANGELOG.md: add changes for v1.79.2 2022-08-08 18:00:00 +03:00
Aliaksandr Valialkin
06b6063a36
docs/CHANGELOG.md: link to the issue related to vmselect panic in multi-level cluster setup
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2961
2022-08-08 15:09:09 +03:00
Aliaksandr Valialkin
44c4c1a8cb
docs/vmagent.md: typo fix in __meta_kubernetes_annotation_prometheus_io_tenant label name
It must be __meta_kubernetes_pod_annotation_prometheus_io_tenant
2022-08-08 14:52:49 +03:00
Aliaksandr Valialkin
46d7792b72
lib/promscrape: follow-up after 2c553d5a2f
- fix broken tests
- cosmetic code cleanup
- document the change at https://docs.victoriametrics.com/vmagent.html#multitenancy
- document the change at https://docs.victoriametrics.com/CHANGELOG.html
2022-08-08 14:46:26 +03:00
Aliaksandr Valialkin
7a87251ff5
docs: sync -help output with the latest changes 2022-08-08 14:04:30 +03:00
Aliaksandr Valialkin
7b8bc8ad59
all: bump the minimum supported version of Go from 1.17 to 1.18
This is needed because some dependencies uses generics, which have been appeared in Go1.18

This is a follow-up for caf3dd4fa2
2022-08-08 13:39:38 +03:00
Aliaksandr Valialkin
730c39876e
app/vmselect/netstorage: prevent from calling processBlocks callback after the exit from ProcessBlocks function
This should prevent from panic at multi-level vmselect
when the top-level vmselect is configured with -replicationFactor > 1
2022-08-08 13:35:25 +03:00
Roman Khavronenko
a086e48964
vmalert: remove notions of vmalert being compatible with VM only (#2954)
vmalert can be successfully used with datasources
compatible with Prometheus HTTP API. So we remove comments or
notes in Readme which are saying opposite.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-08-08 09:45:21 +02:00
Aliaksandr Valialkin
34f0341b36
docs/CHANGELOG.md: link to the issue with improper handling of enpoint option at ec2_sd_configs
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2917
2022-08-08 03:32:50 +03:00
Aliaksandr Valialkin
68c4ec9472
lib/promrelabel: do not split regex into multiple lines if it contains groups
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2928
2022-08-08 03:15:26 +03:00
Aliaksandr Valialkin
51debcdf6d
docs/CHANGELOG.md: link to the issue regarding the increased load on Consul
This is a follow-up for 68de1f4e4a

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2940
2022-08-08 02:22:31 +03:00
Aliaksandr Valialkin
566d12d9ef
docs/_includes/img.html: fix https://github.com/VictoriaMetrics/VictoriaMetrics/security/code-scanning/28 2022-08-08 01:03:47 +03:00
Yury Molodov
3cb013aeb8
vmui: graph action on moush hold and move (#2915)
* fix: change event for graph panning

* fix: change detect key

* feat: add zoom in with mouse selection

* - document the change
- run `make vmui-update`

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-08-07 23:55:09 +03:00
Aliaksandr Valialkin
9f37935819
Makefile: remove redundant -mod=vendor option when running Go tools
The `-mod=vendor` is automatically set when there is a `vendor` directory
starting from Go1.14 - see https://go.dev/doc/go1.14#go-command

Since the minimum supported Go version for VictoriaMetrics is Go1.17,
then the `-mod=vendor` option is no longer needed.
2022-08-07 20:39:08 +03:00
Aliaksandr Valialkin
004e683c55
docs/sd_configs.md: mention that http client configs can contain headers and proxy_headers options 2022-08-07 18:27:26 +03:00
Aliaksandr Valialkin
0e3f21bc25
docs/sd_configs.md: mention that http client options can be specified in scrape_configs section 2022-08-07 00:21:00 +03:00
Aliaksandr Valialkin
f36de0ecc9
docs: add docs for scrape_configs section 2022-08-07 00:04:31 +03:00
Aliaksandr Valialkin
46ed82b894
docs/sd_configs.md: add docs for static_configs 2022-08-06 23:17:17 +03:00
Aliaksandr Valialkin
b8fc2d356f
docs/sd_configs.md: add docs for openstack_sd_configs 2022-08-06 23:07:01 +03:00
Aliaksandr Valialkin
34d5eda904
docs/sd_configs.md: document kubernetes_sd_configs 2022-08-06 22:38:39 +03:00
Aliaksandr Valialkin
0ef29ceb14
lib/promscrape/discovery/kubernetes: add missing __meta_kubernetes_ingress_class_name label for role: ingress
See 7e65ad3e43
and 7e1111ff14
2022-08-05 20:55:00 +03:00
Aliaksandr Valialkin
b8fdac4bd7
docs/sd_configs.md: document http_sd_configs 2022-08-05 19:50:06 +03:00
Aliaksandr Valialkin
d8b9cb909a
docs/sd_configs.md: document gce_sd_configs 2022-08-05 19:36:57 +03:00
Aliaksandr Valialkin
e3b427ea54
docs/sd_configs.md: document file_sd_configs 2022-08-05 19:16:54 +03:00