Commit graph

7635 commits

Author SHA1 Message Date
Aliaksandr Valialkin
1831c731a3
app/vmagent/remotewrite: do not drop persistent queues when -remoteWrite.multitenantURL is set
It is unsafe to drop persistent queues when -remoteWrite.multitenantURL command-line flag is set,
since these queues are created on demand when a new sample for the given tenant is pushed
to the remote storage.

This addresses https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5357
The issue has been appeared in the commit f3a51e8b1d
when implementing https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4014
2023-11-23 20:40:39 +02:00
Github Actions
e1fb9d9230
Automatic update operator docs from VictoriaMetrics/operator@45bfa36 (#5373) 2023-11-22 20:24:50 +02:00
Aliaksandr Valialkin
348482c575
app/vmalert/notifier: remove backticks from the description for -notifier.blackhole command-line flag
Backticks in flag description are automatically converted to flag type. See https://pkg.go.dev/flag#PrintDefaults

This is a follow-up for 20025d4fd6 and 25317b4e70
2023-11-22 20:17:01 +02:00
Aliaksandr Valialkin
0b0776a440
docs/Cluster-VictoriaMetrics.md: sync with cluster branch 2023-11-22 19:31:21 +02:00
Aliaksandr Valialkin
334a739ff6
docs: convert png images to webp in all the docs except of docs/operator/*
This reduces the size of docs/* folder from 33MB to 18MB

Images inside docs/operator/* must be converted at the https://github.com/VictoriaMetrics/operator/tree/master/docs
and then the updated images must be automatically propagated to the docs/operator/*

This is a follow-up for d3f919df3e

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5206
2023-11-22 19:21:00 +02:00
Github Actions
0715b4e121
Automatic update operator docs from VictoriaMetrics/operator@b96131b (#5371) 2023-11-22 03:59:07 -08:00
Aliaksandr Valialkin
389f34cb57
deployment/docker: remove built binaries at bin folder after creating docker image from them at make publish-via-docker 2023-11-21 14:33:24 +02:00
Aliaksandr Valialkin
3a15c9ffb3
.github/workflows/codeql-analysis.yml: cache Go artifacts 2023-11-21 13:04:57 +02:00
Aliaksandr Valialkin
2b420b5c0a
.github/workflows/main.yml: ignore changes inside dashboards and deployment/**.yml
The dashaboards/ and deployment/**.yml do not contain files, which may change main workflow results,
so it is better to ignore them.
2023-11-21 12:57:42 +02:00
Aliaksandr Valialkin
fb835ad658
.github/workflows: run build and test jobs in parallel in order to speed up the workflow run 2023-11-21 12:22:01 +02:00
hagen1778
d493da562e
lib/storage: fix typo
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-11-21 11:20:43 +01:00
Aliaksandr Valialkin
5d1ce9891b
.github/workflows: add Go version to Go artifacts cache key
When Go version changes, artifacts for the previous Go version may becomes useless,
so there is a little sense in re-using them.
2023-11-21 12:09:44 +02:00
Aliaksandr Valialkin
74fda0b311
.github/workflows: take into account Makefile contents when generating cache key for Go build aftifacts
The cache key must change when the corresponding 'make ...' command changes inside Makefile.
2023-11-21 12:09:43 +02:00
Aliaksandr Valialkin
95f12c7e28
.github/workflows: use stable Go release - it should always point to the latest stable release
This eliminates the need to update .github/workflows/* files whenever new Go stable release is out,
like in the 2db1a664e1 .

See https://github.com/actions/setup-go#using-stableoldstable-aliases
2023-11-21 12:09:43 +02:00
hagen1778
e96b4410a1
lib/storage: fix typo
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-11-21 10:52:53 +01:00
Aliaksandr Valialkin
c160a49908
.github/workflows/main.yml: try improving caching for Go artifacts
The default caching for Go artifacts from actions/setup-go@v4 uses the hash of go.sum file
as a cache key - see https://github.com/actions/cache/blob/main/examples.md#go---modules .
This isn't enough for VictoriaMetrics case, since different makefile actions build different the Go artifacts,
which need to be cached. So embed the action name in the cache key.
2023-11-21 01:11:08 +02:00
Aliaksandr Valialkin
a007a5a8a4
.github/workflows/codeql-analysis.yml: remove check-latest and cache inputs for actions/setup-go
- The `cache: true` is no longer needed starting from actions/setup-go@v4 - see https://github.com/actions/setup-go#v4
- The `check-latest: true` may slow down the action, so it is better to disalbe it - see https://github.com/actions/setup-go#check-latest-version
2023-11-21 00:56:43 +02:00
Aliaksandr Valialkin
f3d47c3dc3
Makefile: allow specifying the needed concurrency for make via MAKE_CONCURRENCY env var 2023-11-21 00:55:19 +02:00
Aliaksandr Valialkin
ba803a7cd2
Makefile: localize parallel make to release, release-victoria-logs and crossbuild commands
This is a follow-up for 81ddee4f3a
2023-11-20 23:15:17 +02:00
Aliaksandr Valialkin
81ddee4f3a
Makefile: speedup release, publish and crossbuild rules by using parallel make 2023-11-20 22:53:23 +02:00
Aliaksandr Valialkin
fbab838dc0
app/vmagent/README.md: sync with docs/vmagent.md after cbe4a5c251 , so make docs-sync properly works 2023-11-20 22:42:58 +02:00
Nikolay
cbe4a5c251
app/vmagent: adds google pubsub as remoteWrite dst and ingest consumer (#713)
it allows to push and receive metrics from google pubsub queue
Adds needed documentation and examples for it
2023-11-20 22:42:30 +02:00
hagen1778
1ef6b7f32b
docs: calrify version when vminsertConnsShutdownDuration was added
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-11-20 17:15:37 +01:00
hagen1778
20025d4fd6
docs: typo after 3f5a41e35e
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-11-20 17:05:15 +01:00
hagen1778
3ffa8975d4
docs: follow-up after d3f919df3e
d3f919df3e
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-11-20 11:52:09 +01:00
Dmytro Kozlov
d3f919df3e
docs/managed-victoriametrics: use webp format to reduce image size (#5206) 2023-11-20 11:28:47 +01:00
Khanh Quoc Le
4b7e6b36ce
Add _stream fields log (#5068) 2023-11-17 15:58:52 +01:00
Hui Wang
ae3107153c
lib/protoparser/promremotewrite: fall back to zstd decoding if Snappy-decoding fails (#5344)
This case is possible after the following steps:
1. vmagent successfully performed handshake with the -remoteWrite.url and the remote storage supports zstd-compressed data.
2. remote storage became unavailable or slow to ingest data, vmagent compressed the collected data into blocks with zstd and puts these blocks to persistent queue on disk.
3. vmagent restarts and the remote storage is unavailable during the handshake, then vmagent falls back to Snappy compression.
4. vmagent starts sending zstd-compressed data from persistent queue to the remote storage, while falsely advertizing it sends Snappy-compressed data.
5. The remote storage receives zstd-compressed data and fails unpacking it with Snappy.

The solution is the same as 12cd32fd75, just fall back to zstd decompression if Snappy decompression fails.
2023-11-17 15:51:09 +01:00
Aliaksandr Valialkin
34a26397d7
docs/enterprise.md: update VictoriaMetrics version in examples from v1.95.0 to v1.95.1
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.95.1
2023-11-17 15:46:57 +01:00
Github Actions
75059f3feb
Automatic update operator docs from VictoriaMetrics/operator@388745c (#5340) 2023-11-17 15:45:25 +01:00
Aliaksandr Valialkin
727709da67
deployment: update VictoriaMetrics docker image tag from v1.95.0 to v1.95.1
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.95.1
2023-11-17 15:44:15 +01:00
Aliaksandr Valialkin
faee0e43d1
app/vmselect/promql: reduce the number of memory allocations inside copyTimeseriesShallow()
Previously the number of memory allocations inside copyTimeseriesShallow() was equal to 1+len(tss)
Reduce this number to 2 by pre-allocating a slice of timeseries structs with len(tss) length.
2023-11-17 15:40:03 +01:00
luckyxiaoqiang
c8e6e47e2a
docs/metricsql: remove duplicate sentence (#5349) 2023-11-17 15:18:17 +01:00
Artem Navoiev
657f3bdd21
gh action bump pagefind version to 1.0.4
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2023-11-17 14:26:17 +01:00
Aliaksandr Valialkin
3545633934
docs/CHANGELOG.md: cut v1.95.1 2023-11-16 20:31:59 +01:00
Aliaksandr Valialkin
e9d86d7e52
vendor: run make vendor-update 2023-11-16 20:20:27 +01:00
Aliaksandr Valialkin
aefd744abb
dashboards: remove path!="/favicon.ico" filter from requests rate graphs
The `path!="/favicon.ico"` filter has little sense, since there are many other special paths,
which may be filtered out - /metrics, /flags, /health, /ping, /robots.txt, /-/healthy, /-/ready, /reload, etc.
See /lib/httpserver/httpserver.go for more details.
It will be hard or impossible to maintain filters for all these paths, so it is better to drop this filter
in order to simplify queries and improve the consistency of these queries.
2023-11-16 19:28:49 +01:00
Aliaksandr Valialkin
aae06e003e
app/vmselect: simplify code a bit after 63e0f16062
Use only a single call to prometheus.WriteErrorResponse() inside sendPrometheusError
2023-11-16 18:14:33 +01:00
Aliaksandr Valialkin
84b3c1d3bc
docs/FAQ.md: add a link to https://docs.victoriametrics.com/#monitoring in questions where this is needed 2023-11-16 17:45:30 +01:00
Aliaksandr Valialkin
2ea03cf80d
lib/handshake: add SetReadDeadline and SetWriteDeadline implementations additionally to SetDeadline
This is a follow-up for 27a5461785

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5327
2023-11-16 16:48:05 +01:00
Roman Khavronenko
1fbd0dd9d8
lib/handshake: check for deadline in Read and Write methods (#5327)
The buffered connection could have exceeded the underlying connection
deadline during reading or writing to an internal buffer.
With this change, buffered connection struct additionally checks
for a deadline in Read/Write methods.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-11-16 16:47:46 +01:00
Github Actions
8924fea33a
Automatic update operator docs from VictoriaMetrics/operator@bc8b02f (#5331) 2023-11-16 16:29:37 +01:00
Roman Khavronenko
8dfc874be3
docs/vmalert: clarify deduplication recommendations for HA setup (#5336)
Please see discussion here https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5279

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-11-16 16:26:57 +01:00
Aliaksandr Valialkin
61035419d5
docs/CHANGELOG.md: remove duplicate word query after 2cbdb1db22 2023-11-16 16:24:03 +01:00
Aliaksandr Valialkin
2cbdb1db22
app/vmselect/promql: properly handle duplicate series when merging cached results with the results obtained from the database
evalRollupFuncNoCache() may return time series with identical labels (aka duplicate series)
when performing queries satisfying all the following conditions:

- It must select time series with multiple metric names. For example, {__name__=~"foo|bar"}
- The series selector must be wrapped into rollup function, which drops metric names. For example, rate({__name__=~"foo|bar"})
- The rollup function must be wrapped into aggregate function, which has no streaming optimization.
  For example, quantile(0.9, rate({__name__=~"foo|bar"})

In this case VictoriaMetrics shouldn't return `cannot merge series: duplicate series found` error.
Instead, it should fall back to query execution with disabled cache.

Also properly store the merged results. Previously they were incorrectly stored because of a typo
introduced in the commit 41a0fdaf39

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5332
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5337
2023-11-16 16:01:40 +01:00
hagen1778
d389a4fcf3
dashboards: use version instead of short_version in annotations
`version` label won't show the difference if various flavors of the same
version were deployed. But `short_version` will.

For example, on the sandbox env we test VM builds before new version release.
Without this change, the version update won't be visible on dashboard.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-11-16 09:26:47 +01:00
Github Actions
ec7cac3641
Automatic update Grafana datasource docs from VictoriaMetrics/grafana-datasource@95d0711 (#5329) 2023-11-16 11:12:32 +04:00
Aliaksandr Valialkin
0a28c8e91b
docs/Articles.md: add https://hackernoon.com/unleashing-vm-histograms-for-ruby-migrating-from-prometheus-to-victoriametrics-with-vm-client 2023-11-16 00:53:44 +01:00
Yury Molodov
98e73a4022
vmui: change autocomplete hotkey to Alt/Option + A (#5328) 2023-11-15 23:33:10 +01:00
Aliaksandr Valialkin
17c45d1206
docs/vmbackup.md: fix links to https://docs.victoriametrics.com/vmbackup.html#permanent-deletion-of-objects-in-s3-compatible-storages
This is a follow-up for 2fc7e9f47e
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5121
2023-11-15 23:26:32 +01:00