Commit graph

6420 commits

Author SHA1 Message Date
Nikolay
e3ce736ce2
app/vmselect/graphite: fixes tests for arm (#4348)
at arm based CPUs only 9 digits after comma matches for tests.
Especially at holtWinters functions. Since it only takes effect at tests
it makes no sense for changing float prescision at actual functions

(cherry picked from commit 228ea03bda)
2023-06-02 13:19:34 +02:00
Artem Navoiev
ec97b851bb
remove absolute links, fix aliase
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
(cherry picked from commit 434358b551)
2023-06-02 13:19:33 +02:00
Artem Navoiev
db3d34ca9a
fix title in multi-regional-setup-dedicated-regions.md
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
(cherry picked from commit c1ad48edf9)
2023-06-02 13:19:33 +02:00
Roman Khavronenko
033e1ac3d2
vmalert: do not return nil rules for /api/v1/rules (#4344)
The fix addresses a case when vmalert is configured with a group
which has `name`, but doesn't have `rules` configured. In this
case it still returns a `nil` instead of `[]` slice.

Fixing this via current commit.

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4221

Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 66ed6fe62f)
2023-06-02 13:19:33 +02:00
Roman Khavronenko
5950a94e63
vmalert: fix the typo in popup (#4331)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 76f7e66d8e)
2023-06-02 13:19:32 +02:00
Roman Khavronenko
576e59d82c
cluster: standardize default HTTP responses (#4368)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-06-01 10:26:52 +02:00
Haleygo
6c84b61893
vmselect:fix init sn take too much time (#4366)
* vmselect: descrease start time for vmselect

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4364
2023-05-30 13:04:31 +02:00
Aliaksandr Valialkin
c30f0e51d7
lib/promrelabel: use monospace font at textarea for writing relabel configs on /metric-relabel-debug and /target-relabel-debug pages
This simplifies visual inspection of indentation in yaml configs
2023-05-18 20:49:47 -07:00
Aliaksandr Valialkin
934a7f485c
app/vmselect: log locations of sendPrometheusError() calls
Previously the location inside the sendPrometheusError() was logged.
This could make hard investigating error locations via `vm_log_messages_total` metric.
2023-05-18 20:39:50 -07:00
Aliaksandr Valialkin
f67a2f1e0f
docs/CHANGELOG.md: document v1.79.13 LTS release 2023-05-18 19:56:43 -07:00
Aliaksandr Valialkin
3c4aeddfde
docs/CHANGELOG.md: document v1.87.6 LTS release 2023-05-18 17:50:22 -07:00
Aliaksandr Valialkin
b73a2336f0
deployment/docker: update VictoriaMetrics version from v1.90.0 to v1.91.0 in docker compose files 2023-05-18 15:34:54 -07:00
Aliaksandr Valialkin
63540f3ff1
docs/Release-Guide.md: fix a link to sandbox environment
The link became broken after 5f9d3f9cb5
2023-05-18 15:34:52 -07:00
Aliaksandr Valialkin
93f934eae3
deployment/docker/Makefile: use alpine 3.17.3 instead of alpine 3.18.0 for certs image, since alpine 3.18.0 doesnt work for cross-platform builds 2023-05-18 14:11:02 -07:00
Aliaksandr Valialkin
deab51e708
docs/CHANGELOG.md: cut v1.91.0 2023-05-18 12:37:20 -07:00
Aliaksandr Valialkin
131a1146d9
vendor: make vendor-update 2023-05-18 12:22:16 -07:00
Aliaksandr Valialkin
1ff67bb036
app/vmselect/vmui: run make vmui-update after 39c1b0f8d1 2023-05-18 12:15:22 -07:00
Aliaksandr Valialkin
0ebfb91aba
lib/storage: revert the migration from global to per-day index for (MetricName -> TSID)
This reverts the following commits:
- e0e16a2d36
- 2ce02a7fe6

The reason for revert: the updated logic breaks assumptions made
when fixing https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2698 .
For example, if a time series stop receiving new samples during the first
day after the indexdb rotation, there are chances that the time series
won't be registered in the new indexdb. This is OK until the next indexdb
rotation, since the time series is registered in the previous indexdb,
so it can be found during queries. But the time series will become invisible
for search after the next indexdb rotation, while its data is still there.

There is also incompletely solved issue with the increased CPU and disk IO resource
usage just after the indexdb rotation. There was an attempt to fix it, but it didn't fix
it in full, while introducing the issue mentioned above. See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1401

TODO: to find out the solution, which simultaneously solves the following issues:
- increased memory usage for setups high churn rate and long retention (e.g. what the reverted commit does)
- increased CPU and disk IO usage during indexdb rotation ( https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1401 )
- https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2698
2023-05-18 11:28:54 -07:00
Dmytro Kozlov
6da5cfa3a8
app/vmctl: set default value for --vm-native-step-interval flag (#4327)
* app/vmctl: set default value for `--vm-native-step-interval` flag

* app/vmctl: update CHANGELOG.md

* app/vmctl: update CHANGELOG.md, fix docs

* app/vmctl: fix typo

* app/vmctl: fix typo
2023-05-18 11:03:26 -07:00
Denys Holius
211f48fed6
deployment/docker/Makefile: updated docker compose commands regarding migration from V1 to V2 (#4314)
deployment/docker/Makefile: updated docker compose commands regarding migration from V1 to V2
2023-05-18 10:48:35 -07:00
Aliaksandr Valialkin
0397b3f0f7
lib/handshake: do not pollute logs with cannot read hello messages on TCP health checks
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1762
2023-05-18 10:37:59 -07:00
Aliaksandr Valialkin
b22bcb6f0a
app/vmauth: allow -auth.config without users section of unauthorized_user section is present here
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4083
2023-05-18 09:44:07 -07:00
Aliaksandr Valialkin
6aa8029f30
app/vmauth: simplify the code after 4a1d29126c
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4242
2023-05-17 00:38:04 -07:00
Nikolay
da115b5170
app/vmauth: retry common network dial errors (#4280)
with tracking request body read calls
it allows us to retry POST and PUT requests
2023-05-17 00:37:39 -07:00
Nikolay
953a5df43f
app/vmauth: do not return invalid credentials (#4288)
at http response by default
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4188

based on https://github.com/VictoriaMetrics/VictoriaMetrics/pull/4190
Thanks @raj-kumar-j  for init implementation
2023-05-17 00:10:51 -07:00
Aliaksandr Valialkin
67beb8c856
lib/storage: follow-up after 2ce02a7fe6
- Document the change at docs/CHANGELOG.md
- Clarify comments for non-trivial code touched by the commit
- Improve the logic behind maybeCreateIndexes():
  - Correctly create per-day indexes if the indexdb rotation is performed during
    the first hour or the last hour of the day by UTC.
    Previously there was a possibility of missing index entries on that day.
  - Increase the duration for creating new indexes in the current indexdb for up to 22 hours
    after indexdb rotation. This should reduce the increased resource usage
    after indexdb rotation.
    It is safe to postpone index creation for the current day until the last hour
    of the current day after indexdb rotation by UTC, since the corresponding (date, ...)
    entries exist in the previous indexdb.
- Search for TSID by (date, MetricName) in both the current and the previous indexdb.
  Previously the search was performed only in the current indexdb. This could lead
  to excess creation of per-day indexes for the current day just after indexdb rotation.
- Search for (date, metricID) entries in both the current and the previous indexdb.
  Previously the search was performed only in the current indexdb. This could lead
  to excess creation of per-day indexes for the current day just after indexdb rotation.
2023-05-16 23:31:59 -07:00
Roman Khavronenko
c3b1d9ee21
lib/storage: introduce per-day MetricName=>TSID index (#4252)
The new index substitutes global MetricName=>TSID index
used for locating TSIDs on ingestion path.
For installations with high ingestion and churn rate, global
MetricName=>TSID index can grow enormously making
index lookups too expensive. This also results into bigger
than expected cache growth for indexdb blocks.

New per-day index supposed to be much smaller and more efficient.
This should improve ingestion speed and reliability during
re-routings in cluster.

The negative outcome could be occupied disk size, since
per-day index is more expensive comparing to global index.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-05-16 23:18:11 -07:00
Aliaksandr Valialkin
bc98ea9a8d
lib/storage: reduce the unimportant logging during Storage start / stop
This should improve the visibility of potentially important logs
2023-05-16 15:32:35 -07:00
Aliaksandr Valialkin
05113bba09
lib/mergeset: remove superflouos logging when opening and closing the Table
The logged messages had little useful info, while they were polluting log output during VictoriaMetrics start/stop
2023-05-16 15:32:35 -07:00
Aliaksandr Valialkin
4a5b5c5020
lib/mergeset: close and open the table before making snapshots at TestTableCreateSnapshotAt()
This gives guarantees that all the in-memory data is written to disk at the snapshot time.

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4272
See https://github.com/VictoriaMetrics/VictoriaMetrics/pull/4316
2023-05-16 15:32:34 -07:00
Artem Navoiev
036ab185bf
update logo width in cluster doc to 300
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2023-05-16 23:05:30 +02:00
Aliaksandr Valialkin
f09745f613
lib/{mergeset,storage}: make it clear that DebugFlush() doesn't store all the recently ingested data to disk
DebugFlush() makes sure that the recently ingested data becomes visible to search.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4272
2023-05-16 11:55:58 -07:00
Aliaksandr Valialkin
9af04e4140
vendor: update github.com/VictoriaMetrics/metrics from v1.23.1 to v1.24.0
This change adds process_* metrics to VictoriaMetrics components under Windows OS

See https://github.com/VictoriaMetrics/metrics/pull/47
2023-05-16 11:37:18 -07:00
Aliaksandr Valialkin
4f6680f399
docs/vmbackupmanager.md: run make docs-sync after c7d8dda39225b716ea44df7223db5e4a125d407b 2023-05-16 11:27:39 -07:00
Alexander Marshalov
ad35081066
backup metadata are written in separate file (#560)
Signed-off-by: Alexander Marshalov <_@marshalov.org>
2023-05-16 11:24:44 -07:00
Alexander Marshalov
e895358939
added backup locking/unlocking against retention policy to vmbackupmanager (#558)
* added backup locking/unlocking against retention policy to vmbackupmanager

Signed-off-by: Alexander Marshalov <_@marshalov.org>

* added docs for new commands

Signed-off-by: Alexander Marshalov <_@marshalov.org>

* fix review comments

Signed-off-by: Alexander Marshalov <_@marshalov.org>

---------

Signed-off-by: Alexander Marshalov <_@marshalov.org>
2023-05-16 11:23:43 -07:00
Roman Khavronenko
42b90e5e9a
vmalert: follow-up after 669becd011 (#4318)
* vmalert: follow-up after 669becd011

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* vmalert: follow-up after 669becd011

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* vmalert: follow-up after 669becd011

Signed-off-by: hagen1778 <roman@victoriametrics.com>

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-05-16 10:14:15 -07:00
Zakhar Bessarab
a2fc912c43
lib/storage: follow-up after a50d63c376 (#4289)
* lib/storage: follow-up after a50d63c376

- ensure retentionMsecs is rounded to day
- remove localTimeOffset in test as localOffset is ignored when using `UnixMilli`

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* lib/storage: restore retention timezone offset effect on retention deadline

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

---------

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2023-05-16 10:13:20 -07:00
Michael Hoffmann
a99918085d
vmalert: improve retry logic for remote write (#4134)
vmalert should not retry on 4xx status codes
according to https://prometheus.io/docs/concepts/remote_write_spec/
2023-05-16 10:10:39 -07:00
Yury Molodov
5c2ed85eb9
vmui: refactor code using custom hooks (#4145)
* refactor: replace boolean useState with useBoolean

* refactor: replace useResize with useWindowSize/useElementSize

* refactor: replace addEventListener with useEventListener

* refactor: replace navigator.clipboard.writeText with useCopyToClipboard

* fix: prevent redirect loop
2023-05-16 10:09:24 -07:00
Roman Khavronenko
afd03f87fe
vmctl: update VictoriaMetrics migration section (#4310)
Remove unnecessary information to simplify the description and tips.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-05-16 10:07:47 -07:00
Artem Navoiev
b4aaeda72f
fix link in operator quick start docs.2
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2023-05-16 10:06:52 -07:00
Artem Navoiev
c771f4eccf
fix link in operator quick start docs
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2023-05-16 10:05:36 -07:00
Nikolay
5514b5d552
app/vminsert: allow parsing tenant id from (#4144)
VictoriaMetrics_ProjectID and VictoriaMetrics_AccountID labels.
It should help to migrate for new labels vm_account_id vm_project_id without service downtime
2023-05-16 08:16:37 -07:00
Aliaksandr Valialkin
9461d3fdfa
lib/promutils: add ParseTimeAt() function 2023-05-13 20:12:55 -07:00
Aliaksandr Valialkin
e6648d14b2
deployment/docker: update base docker image from 3.17.3 to 3.18.0
See https://www.alpinelinux.org/posts/Alpine-3.18.0-released.html
2023-05-12 17:31:35 -07:00
Aliaksandr Valialkin
3b1e40d73f
lib/promutils: properly return error when incorrect Prometheus label names are passed to NewLabelsFromString()
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4284
See also https://github.com/VictoriaMetrics/VictoriaMetrics/pull/4304
2023-05-12 17:02:06 -07:00
Aliaksandr Valialkin
b24da0f901
Revert "lib/promrelabel: show error message if labels not in prometheus exposition format (#4304)"
This reverts commit 193a9c3328.

Reason for revert: the commit doesn't fix the real issue with promutils.NewLabelsFromString()
function, which must return error when improperly formatted Prometheus metric with labels is passed to it.
See https://github.com/prometheus/docs/blob/main/content/docs/instrumenting/exposition_formats.md#text-format-example

E.g. the promutils.NewLabelsFromString() must return error when the following strings are passed to it:

- `{foo:"bar"}`, since `:` is disallowed in Prometheus text exposition format. The corect value is `{foo="bar"}`
- `{"foo":"bar"}`, since label name shouldn't be quoted. The correct value is `{foo="bar"}`.

The reverted commit introduces another set of bugs, which happily accept the following invalid input:

- `{foo=~"bar"}`
- `{foo!="bar"}`
- `{foo!~"bar"}`

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4284
See also https://github.com/VictoriaMetrics/VictoriaMetrics/pull/4304
2023-05-12 17:01:23 -07:00
Aliaksandr Valialkin
4df7573858
lib/protoparser/csvimport: properly parse the last empty column in CSV line
Do not ignore the last empty column in CSV line.
While at it, properly parse CSV columns in single quotes, e.g. `'foo,bar',baz` is parsed as two columns - `foo,bar` and `baz`

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4048

See also https://github.com/VictoriaMetrics/VictoriaMetrics/pull/4298
2023-05-12 16:59:50 -07:00
Aliaksandr Valialkin
6fd39e2000
Revert "lib/protoparser: fix skip csv line when metric can be collect from the line (#4298)"
This reverts commit 410ae99c2e.

Reason for revert: the commit masks the real issue instead of fixing it.
The real issue is that the scanner.NextColumn() skips the last column if it is empty.

The commit also introduces two bugs:

- a panic if all the metric values in CSV line are empty
- silent import of CSV lines with too small number of columns

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4048
See https://github.com/VictoriaMetrics/VictoriaMetrics/pull/4298
2023-05-12 16:59:11 -07:00