Commit graph

6807 commits

Author SHA1 Message Date
Zakhar Bessarab
f0c06b428e
app/vmselect: fix panic when using /select/multitenant endpoint (#4912)
app/vmselect: fix panic when using `/select/multitenant` endpoint

Such requests must be rejected as not found since vmselect does not support multitenant endpoint.

See: https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4910

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2023-08-29 21:48:12 +02:00
Nikolay
e711c8877e
deployment/docker: disable provenance in buildx (#4911)
* deployment/docker: disable provenance in buildx
it must fix an issue with multi-platform manifest generation
at buildx >= 0.10 backward compatibility was broken and generated image cannot be used with docker systems that doesn't support oci.
disabling attestat temporary fixes it.
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4907
https://docs.docker.com/build/attestations/slsa-provenance/

* Update docs/CHANGELOG.md

---------

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2023-08-29 16:32:13 +02:00
Aliaksandr Valialkin
4dcd2ff415
docs/VictoriaLogs/Roadmap.md: add alerting to TODO list 2023-08-29 13:26:32 +02:00
Aliaksandr Valialkin
503798da53
vendor: run make vendor-update 2023-08-29 13:16:20 +02:00
Aliaksandr Valialkin
b56294c174
lib/proxy: consistently use gopkg.in/yaml.v2 across all the code 2023-08-29 13:16:20 +02:00
Aliaksandr Valialkin
0609d8c8c5
Makefile: cleanup bin/ directory at the beginning of make publish-release command
This is needed in order to prevent from non-build artifacts to be uploaded to Github release page
2023-08-29 13:16:20 +02:00
Aliaksandr Valialkin
5e8dfcf65e
app/{vmselect,vlselect}: run make vmui-update vmui-logs-update after recent changes to app/vmui 2023-08-29 12:58:58 +02:00
Aliaksandr Valialkin
339879edd0
lib/netutil/tcpdialer.go: reduce the code difference with enterprise branch 2023-08-29 12:45:57 +02:00
Aliaksandr Valialkin
19d61737c1
app/{vminsert,vmselect}: follow-up after 2b7b3293c1
- Document the change at docs/CHANGELOG.md
- Set the default value for -vmstorageUserTimeout to 3 seconds. This is much better
  than the 0 value, which means that TCP connection to unreachable vmstorage could block
  for up to 16 minutes.
- Document -vmstorageUserTimeout at docs/Cluster-VictoriaMetrics.md
2023-08-29 12:17:39 +02:00
Will Jordan
2b7b3293c1
Add vmstorageUserTimeout flags to configure TCP user timeout (Linux) (#4423)
`TCP_USER_TIMEOUT` (since Linux 2.6.37) specifies the maximum amount of
time that transmitted data may remain unacknowledged before TCP will
forcibly close the connection and return `ETIMEDOUT` to the application.

Setting a low TCP user timeout allows RPC connections quickly reroute
around unavailable storage nodes during network interruptions.
2023-08-29 11:46:39 +02:00
Aliaksandr Valialkin
0aa0435d17
docs/VictoriaLogs/CHANGELOG.md: document the fix 8d50032dd6
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4895
2023-08-29 11:29:11 +02:00
crossoverJie
64b27c9217
lib/logstorage: Set ptwHot to nil when the partition pointed by ptwHot is dropped (#4902) 2023-08-29 11:22:53 +02:00
Roman Khavronenko
c81e90223c
dashboards: provide copies of Grafana dashboards alternated with Vict… (#4905)
dashboards: provide copies of Grafana dashboards alternated with VictoriaMetrics datasource

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-08-29 11:20:16 +02:00
hagen1778
d70b346623
lib/promscrape: follow-up after eabcfc9bcd
`-promscrape.cluster.membersCount` by default should be `1`, like every
single vmagent is a cluster of one member on its own.
The change additionally validates that user can't set `-promscrape.cluster.membersCount`
to value lower than `1`.

eabcfc9bcd
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-08-29 11:18:12 +02:00
Haleygo
685c3d95e7
fix clusterMembersCount check (#4900) 2023-08-29 11:15:50 +02:00
Aliaksandr Valialkin
1e1aa94ffb
lib/logstorage: eliminate data race when clearing s.ptwHot after deleting the corresponding partition
The previous code could result in the following data race:
1. The s.ptwHot partition is marked to be deleted
2. ptw.decRef() is called on it
3. ptw.pt is set to nil
4. s.ptwHot.pt is accessed from concurrent goroutine, which leads to panic.

The change clears s.ptwHot under s.partitionsLock in order to prevent from the data race.

This is a follow-up for 8d50032dd6

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4895
2023-08-29 11:12:07 +02:00
Aliaksandr Valialkin
c4b10109d7
docs/CHANGELOG.md: add links to stream parsing mode in descriptions for 6e8611f301 and 6788704152 2023-08-29 10:47:40 +02:00
Aliaksandr Valialkin
66cb5930ae
docs/CHANGELOG.md: remove unneeded utm_source and utm_medium query args in the link to Google Lighthouse
Remove the line about consistent rounding of values in vmui, since it looks like it has been broken and needs to be returned back.
See https://github.com/VictoriaMetrics/VictoriaMetrics/pull/4872#issuecomment-1696981947 for details.

This is a follow-up for e865989fa9
2023-08-29 10:27:57 +02:00
Aliaksandr Valialkin
1159b31270
app/vmagent/remotewrite: do not retry request immediately on io.ErrUnexpectedEOF, since this error isn't returned on stale connection
Also, mention the https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4139 in comments to the code
in order to simplify further maintenance of this code.

This is a follow-up for 992a1c0a3a
2023-08-29 09:48:49 +02:00
Aliaksandr Valialkin
516627672e
docs/CHANGELOG.md: remove another blank line in order to reduce the difference with lts-1.93 branch 2023-08-29 09:48:48 +02:00
hagen1778
1464dda198
docs: fix typo for deduplication description in k8s guide
Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit a20234d2d0)
2023-08-28 21:32:42 +02:00
Aliaksandr Valialkin
13ed621e36
docs/CHANGELOG.md: remove superflouos blank lines 2023-08-28 10:00:40 +02:00
Aliaksandr Valialkin
35cd8836c9
docs/CHANGELOG.md: move the bugfix line into correct place after ddf87b32ed 2023-08-28 10:00:40 +02:00
Aliaksandr Valialkin
48d5940f2d
docs/CHANGELOG.md: explicitly mention that the bug in 1.93.0 may lead to data loss
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4873
2023-08-28 09:52:52 +02:00
Aliaksandr Valialkin
bba86475e8
docs/CHANGELOG.md: return back the line accidentally deleted at 6abd575cbe
The line has been originally added in 481a2c70fd
2023-08-28 09:48:11 +02:00
Aliaksandr Valialkin
baea9da66b
docs/stream-aggregation.md: use 5m instead of 300 in the example query for rate() calculation from "increase" results
This makes the query easier to read and understand

Follow-up for 0df506de54
2023-08-28 09:37:34 +02:00
Aliaksandr Valialkin
453077808d
docs/vmauth.md: consistently prepend command-line flags with a single - 2023-08-28 09:28:24 +02:00
Daria Karavaieva
c730e1ed0c
vmanomaly guide docker compose run fix according licensing (#4876)
* docker-compose run fix according to licensing

* public v1.5.0 of vmanomaly

* new file list image

* 1.93.1 version of VM
2023-08-28 09:23:54 +02:00
Github Actions
0ba4c1c65c
Automatic update operator docs from VictoriaMetrics/operator@d40882c (#4894)
Co-authored-by: Alexander Marshalov <_@marshalov.org>
2023-08-28 09:17:37 +02:00
Aliaksandr Valialkin
1e34318dca
docs/CHANGELOG.md: clarify the description of b7d07e5acf
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4817
2023-08-28 09:12:49 +02:00
crossoverJie
db0ae3fffb
lib/logstorage: add nil check for ptwHot.pt (#4896)
(cherry picked from commit cde5029bce)
2023-08-27 09:14:28 +02:00
Github Actions
40b1f9611c
Automatic update operator docs from VictoriaMetrics/operator@e00fe21 (#4891)
(cherry picked from commit ef6468584c)
2023-08-27 09:06:02 +02:00
Zakhar Bessarab
46e86add2f
lib/promscrape/client: sync timeout for HostClient and http.Client (#4889)
Initially, stream parse mode was reading data from response and parsing it on flight. This was causing longer delay to read the whole response and required increasing timeout value to allow data processing while reading. So that 908e35affd increased timeout value to fix this.

But after 74c00a8762 response in stream parse mode is saved into memory and then parsed eliminating necessity of having timeout value higher that for usual scrape.

Updates: https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4847
Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
(cherry picked from commit 6e8611f301)
2023-08-27 09:06:00 +02:00
hagen1778
dc837cc65f
docs: follow-up after 72167a697e
72167a697e
Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit e865989fa9)
2023-08-27 09:05:58 +02:00
Yury Molodov
2aff0ce327
vmui: ui improvements (#4872)
* vmui: chart refactoring to enhance code structure

* vmui: improve ui

(cherry picked from commit 72167a697e)
2023-08-27 09:05:55 +02:00
hagen1778
6c99e5445d
docs: make docs-sync
Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 65415b56af)
2023-08-27 09:05:51 +02:00
hagen1778
ff20d3e97c
docs: update remote-read api docs
Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit f4577005be)
2023-08-27 09:05:51 +02:00
Dmytro Kozlov
939952068b
docs: add migration guide from Promscale (#4865)
Signed-off-by: dmitryk-dk d.kozlov@victoriametrics.com
(cherry picked from commit 0b0e0bb50e)
2023-08-27 09:05:44 +02:00
hagen1778
b42187cf82
docs: typo fix for vmauth
Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit ffbebfdfe6)
2023-08-27 09:05:41 +02:00
hagen1778
49327b584d
docs: mention that vmauth doesn't follow redirects
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4868

Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 4d316a23ae)
2023-08-27 09:05:37 +02:00
hagen1778
b18e9b5bb0
app/vmagent: follow-up after 6788704152
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4884
Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 4ebe8bb1d5)
2023-08-27 09:05:22 +02:00
Zakhar Bessarab
1242460fa6
lib/promscrape/client: make User-Agent consistent between fasthttp and native client (#4886)
User agent was not set for native client which resulted in using one provided by Golang.

See: https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4884

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
(cherry picked from commit 6788704152)
2023-08-27 09:05:08 +02:00
hagen1778
33bf28e1bd
app/vmagent: fix comment typo after 992a1c0a3a
Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 757ae4275b)
2023-08-27 09:05:04 +02:00
Roman Khavronenko
b9a2512ac3
vmagent: retry failed write request on the closed connection (#4857)
* vmagent: retry failed write request on the closed connection

 Retry failed write request on the closed connection immediately,
 without waiting for backoff. This should improve data delivery speed
 and reduce amount of error logs emitted by vmagent when using idle connections.

 https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4139

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* vmagent: retry failed write request on the closed connection

Re-instantinate request before retry as body could have been already spoiled.

Signed-off-by: hagen1778 <roman@victoriametrics.com>

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: Nikolay <nik@victoriametrics.com>
(cherry picked from commit 992a1c0a3a)
2023-08-27 09:04:59 +02:00
Roman Khavronenko
68150655d2
vmalert: correctly re-instantinate HTTP req on retries (#4864)
* vmalert: correctly re-instantinate HTTP req on retries

Previosly, request retry to datasource re-used existing HTTP request.
But if request object was already partially processed (body was read),
then retry will be unsuccessful.

The change re-instantinates HTTP request object before retry.

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* vmalert: review fix

Signed-off-by: hagen1778 <roman@victoriametrics.com>

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit ddf87b32ed)
2023-08-27 09:04:42 +02:00
Github Actions
74967aa704
Automatic update operator docs from VictoriaMetrics/operator@8a25794 (#4883)
(cherry picked from commit 2122eb18fe)
2023-08-27 09:04:34 +02:00
Aliaksandr Valialkin
c813b5e4b1
lib/promscrape: add -promscrape.cluster.memberLabel command-line flag
This flag allows specifying an additional label to add to all the scraped metrics.
The flag must contain label name to add. The label value will be equal to -promscrape.cluster.memberNum.

This functionality can help when there is a need to differentiate metrics scraped
by distinct vmagent instances in the cluster according to https://docs.victoriametrics.com/vmagent.html#scraping-big-number-of-targets

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4247

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4247#issuecomment-1692279393
2023-08-24 22:04:34 +02:00
Aliaksandr Valialkin
18f8e90bd8
docs/stream-aggregation.md: typo fix after 54f522ac25 2023-08-24 22:04:24 +02:00
hagen1778
30bc1752dd
docs: mention 1.93.0 contains a bug
Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 59dee2e714)
2023-08-23 16:44:23 +02:00
hagen1778
e49c92fd1f
marketplace: bump DO to 1.93.1
Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit df6d27650e)
2023-08-23 16:44:23 +02:00