Commit graph

636 commits

Author SHA1 Message Date
Aliaksandr Valialkin
4e03f74273
deployment: update VictoriaLogs Docker image from v1.3.2-victorialogs to v1.4.0-victorialogs
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.4.0-victorialogs
2024-12-23 19:45:03 +01:00
f41gh7
aa3dbff79a
docs: update VM apps version to v1.108.1
Signed-off-by: f41gh7 <nik@victoriametrics.com>
2024-12-19 12:25:43 +01:00
Dmytro Kozlov
0622ec0b5d
deployment/docker: rename victorialogs-datasource to victoriametrics-logs-datasource (#7874)
### Describe Your Changes

Renamed victorialogs-datasource to victoriametrics-logs-datasource.

We prepared the victorialogs Grafana plugin for sign and updated the
plugin ID. This action require to update configs in our ops repository

Please check this
[release](https://github.com/VictoriaMetrics/victorialogs-datasource/releases/tag/v0.13.0)
and https://github.com/VictoriaMetrics/victorialogs-datasource/pull/161
with changes

### Checklist

The following checks are **mandatory**:

- [X] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-12-18 22:41:07 +01:00
Mathias Palmersheim
fb76aad365
fixed #7804 Added NoSelfMonitoringMetrics rule (#7805)
### Describe Your Changes

fixes #7804 by adding alert for missing uptime metric in vmanomaly

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-12-18 22:40:51 +01:00
hagen1778
c213b11234
deployment/docker: fix typo after d86788e9a2
Thanks to @Haleygo for pointing it out here https://github.com/VictoriaMetrics/VictoriaMetrics/pull/7843#issuecomment-2545949268

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-12-18 22:40:51 +01:00
Dima Shur
430007b47e
deployment/docker: set vmalert --remoteWrite.url to port 8429 (vmagent) (#7843)
### Describe Your Changes

Updated docker.compose.yml, set remotewrite.url to port 8429 so it would
correspond to documentation

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-12-18 22:40:50 +01:00
f41gh7
8e38e50380
docs: bump last VM versions
Signed-off-by: f41gh7 <nik@victoriametrics.com>
2024-12-16 12:20:16 +01:00
Roman Khavronenko
7489b960d6
deployment/docker: update base Alpine docker image from 3.20.3 to 3.21.0 (#7798)
See https://alpinelinux.org/posts/Alpine-3.21.0-released.html

### Describe Your Changes

Please provide a brief description of the changes you made. Be as
specific as possible to help others understand the purpose and impact of
your modifications.

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 87c1b2de6f)
2024-12-11 17:10:08 +01:00
hagen1778
7643e66173
deployment: bump victorialogs datasource version
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-12-10 22:15:47 +01:00
f41gh7
78ad858ff7
app/{vminsert,vmagent}: drop time series on exceeding labels limits.
Previously, time series with labels exceeding the configured limits were truncated and written to storage, potentially causing data inconsistency. This could lead to collisions between time series and make it difficult to identify the source due to truncated labels.

This commit changes the behavior:
*  Such time series are now rejected outright.
* Rejected time series are logged to stdout, and corresponding counters are incremented.
* removes `vm_too_long_label_values_total`, `vm_too_long_label_names_total`, `vm_metrics_with_dropped_labels_total` metrics.
* adds new values `[too_many_labels,too_long_label_name,too_long_label_value]`  to `reason` label of the `vm_rows_ignored_total` metric name

related issues:
- https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6928
- https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7661
2024-12-10 22:15:38 +01:00
Dmytro Kozlov
169a256834
deployment/docker: update victorialogs datasource versions to the latest releases (#7787)
### Describe Your Changes
Updated victorialogs-datasource to the latest
[release](https://github.com/VictoriaMetrics/victorialogs-datasource/tree/v0.11.0)
### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-12-10 21:43:38 +01:00
Aliaksandr Valialkin
f41a32609c
deployment: update VictoriaLogs Docker image from v1.3.1-victorialogs to v1.3.2-victorialogs
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.3.2-victorialogs

(cherry picked from commit 75a2e23b7e)
2024-12-09 12:23:38 +01:00
Aliaksandr Valialkin
c691e14908
deployment: update VictoriaLogs Docker image from v1.3.0-victorialogs to v1.3.1-victorialogs
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.3.1-victorialogs

(cherry picked from commit 30029f1e39)
2024-12-09 12:23:36 +01:00
Aliaksandr Valialkin
654bf0e2ab
deployment: update VictoriaLogs Docker image from v1.2.0-victorialogs to v1.3.0-victorialogs
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.3.0-victorialogs

(cherry picked from commit 08ce6ef825)
2024-12-09 12:23:34 +01:00
Andrii Chubatiuk
3be3705097
deployment/rules: updated sum expressions in alerts to be able to inject cluster labels in helm charts scripts (#7670)
### Describe Your Changes

Many users are running k8s-stack in multiple kubernetes clusters and to
configure a proper routing in alertmanager it's required to support
`cluster` label in alerting rules. It's now implemented in helm-chart
hack scripts, but it's tricky part to define if cluster label should be
added or not, when functions has no `by` expression. Updated existing
alerts to provide later an ability to inject cluster label later

Also take into an account `storage.minFreeDiskSpaceBytes` in
`DiskRunsOutOfSpace` alerts

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
(cherry picked from commit fefa3e7936)
2024-12-09 12:23:31 +01:00
Aliaksandr Valialkin
69e01c4801
deployment: update VictoriaLogs Docker image from v1.1.0-victorialogs to v1.2.0-victorialogs
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.2.0-victorialogs

(cherry picked from commit ceb081a018)
2024-12-09 12:23:29 +01:00
hagen1778
98cfbd783d
deployment: bump victorialogs plugin version to v0.10.0
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-12-05 15:16:55 +01:00
Aliaksandr Valialkin
59647f574d
deployment: update VictoriaLogs Docker image from v1.0.0-victorialogs to v1.1.0-victorialogs
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.1.0-victorialogs
2024-12-05 15:16:55 +01:00
Aliaksandr Valialkin
f8a294cc66
deployment/docker: update Go builder from Go1.23.3 to Go1.23.4
See https://github.com/golang/go/issues?q=milestone%3AGo1.23.4+label%3ACherryPickApproved
2024-12-05 15:16:54 +01:00
Fred Navruzov
b7e9a7d016
docs/vmanomaly: release v1.18.8 (#7734)
### Describe Your Changes

docs/vmanomaly: release v1.18.8

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-12-05 15:16:52 +01:00
Dmytro Kozlov
c5d8098f21
deployment/docker: update victorialogs datasource versions to the latest releases (#7726)
### Describe Your Changes

Update victorialogs datasource to the latest release
[version](https://github.com/VictoriaMetrics/victorialogs-datasource/releases/tag/v0.9.0)

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-12-05 15:16:51 +01:00
Fred Navruzov
198c69757d
docs/vmanomaly - release v1.18.7 (#7719)
- docs/vmanomaly - release v1.18.7
- modified table markdown for proper rendering on vmanomaly component
pages

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-12-05 15:16:50 +01:00
f41gh7
4bf12add66
{docs,deployment}: update vm apps to the latest v1.107.0 release version 2024-12-02 11:09:57 +01:00
Andrii Chubatiuk
8271094c98
deployment/docker: use vector for local setup (#7681) 2024-11-28 13:50:02 +01:00
Fred Navruzov
ae673e8b34
docs/vmanomaly - release 1.18.5 (#7684)
### Describe Your Changes

docs/vmanomaly - release 1.18.5 doc updates

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-11-28 13:50:02 +01:00
Hui Wang
5f9db9a61f
alerts-vmalert: reserve rule name for description (#7659) 2024-11-26 18:50:30 +01:00
Dmytro Kozlov
78175a45e0
deployment/docker: update victoriametrics datasource versions to the latest releases (#7623)
### Describe Your Changes

Updated victoriametrics-datasource to the newest
[release](https://github.com/VictoriaMetrics/victoriametrics-datasource/releases/tag/v0.10.3)
version

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-11-22 17:38:13 +01:00
Dmytro Kozlov
5e4e7648fb
deployment/docker: update victorialogs datasource versions to the latest releases (#7604)
### Describe Your Changes

Updated the victorialogs data source version to the v0.8.0 release

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-11-20 18:37:36 +01:00
Fred Navruzov
e53a69c27d
docs/vmanomaly: add self-monitoring section (#7558)
- Added self-monitoring guide for `vmanomaly`.
- Added cross-referencing on other pages.
- Slight improvements in wording on related pages
- Update references to v1.18.4
- [x] publish Grafana dashboard to
https://grafana.com/orgs/victoriametrics/dashboards:
https://grafana.com/grafana/dashboards/22337-victoriametrics-vmanomaly/

@AndrewChubatiuk , JFYI if it somehow impacts your work on supporting
`vmanomaly` in operator.

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-11-20 16:32:45 +01:00
f41gh7
99bc72df8c
{deployment/,docs/}: point app versions to the lastest v1.106.1 release
Signed-off-by: f41gh7 <nik@victoriametrics.com>
2024-11-20 16:31:56 +01:00
Hui Wang
aa817602fa
add vlogs type of rule in example (#7548) 2024-11-20 16:31:55 +01:00
Fred Navruzov
78114e2cd3
docs/vmanomaly - release 1.18.2 (#7533)
### Describe Your Changes

docs update for `vmanomaly` release 1.18.2

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-11-14 17:21:33 +01:00
Aliaksandr Valialkin
fa5ea09b2b
deployment/docker: update VictoriaLogs from v0.42.0-victorialogs to v1.0.0-victorialogs
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.0.0-victorialogs
2024-11-14 17:21:32 +01:00
Hui Wang
1bff6c1bbd
dashboards: add file label filter to vmalert dashboard panels (#7515)
Previously, metrics from groups with the same name but in different
files could be mixed in the results.

e.g. the evaluation time
[here](https://grafana.maas.victoriametrics.com/d/LzldHAVnz/victoriametrics-vmalert?orgId=1&var-ds=PE8D8DB4BEE4E4B22&var-job=All&var-instance=All&var-file=%2Fetc%2Fvmalert%2Fconfig%2Fvm-per-tenant-rulefiles-0%2Fmaas-tenant-1011-maas-1011-vm-health.yaml&var-group=All&var-topk=5&editPanel=23)
is the total for multiple groups from different tenants.
2024-11-14 17:21:31 +01:00
Fred Navruzov
b6071e07f4
docs: vmanomaly - release 1.18.1 (#7517)
### Describe Your Changes

doc updates for vmanomaly v1.18.1

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-11-14 17:21:30 +01:00
Aliaksandr Valialkin
2de234e15c
deployment/docker: update VictoriaLogs Docker image from v0.41.0-victorialogs to v0.42.0-victorialogs
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.42.0-victorialogs
2024-11-14 17:21:29 +01:00
Aliaksandr Valialkin
871bfc6ed8
deployment: update Go builder from Go1.23.1 to Go1.23.3
See https://github.com/golang/go/issues?q=milestone%3AGo1.23.2+label%3ACherryPickApproved
and https://github.com/golang/go/issues?q=milestone%3AGo1.23.3+label%3ACherryPickApproved
2024-11-14 17:21:28 +01:00
Phuong Le
9ad958212d
deployment/docker/base: fix the typo that causes InvalidDefaultArgInFrom
(cherry picked from commit 4d383fdb9a)
2024-11-07 13:00:22 +01:00
Aliaksandr Valialkin
722873e18b
deployment/docker: update VictoriaLogs from v0.40.0-victorialogs to v0.41.0-victorialogs
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.41.0-victorialogs

(cherry picked from commit a549240852)
2024-11-07 13:00:21 +01:00
Zakhar Bessarab
577a7a247d
{docker,docs}: update references to latest release
Update references to latest release - v1.106.0.

---------

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
(cherry picked from commit be677065bc)
2024-11-06 13:58:17 +01:00
Andrii Chubatiuk
23dcec3911
vlinsert: support datadog logs
This commit adds the following changes:

- Added support to push datadog logs with examples of how to ingest data
using Vector and Fluentbit
- Updated VictoriaLogs examples directory structure to have single
container image for victorialogs, agent (fluentbit, vector, etc) but
multiple configurations for different protocols

Related issue https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6632

(cherry picked from commit e0930687f1)
2024-11-06 13:58:16 +01:00
Arie Heinrich
7a819ae5ce
docs: spelling fixes (#7420)
### Describe Your Changes

Christmas is early and you get the first present in the shape of
spelling fixes.
Sorry for the big amount :)

### Checklist

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

(cherry picked from commit 2e8f420d84)
2024-11-06 13:58:16 +01:00
Roman Khavronenko
d71c745d19
deployment/alerts: add RemoteWriteDroppingData to vmalert rules (#7393)
### Describe Your Changes

Please provide a brief description of the changes you made. Be as
specific as possible to help others understand the purpose and impact of
your modifications.

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 3f0e2ab3b2)
2024-10-31 14:11:10 +01:00
Aliaksandr Valialkin
dd6781b179
deployment/docker: update VictoriaLogs from v0.39.0-victorialogs to v0.40.0-victorialogs
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.40.0-victorialogs

(cherry picked from commit 24b6e117dd)
2024-10-31 14:11:09 +01:00
Dmytro Kozlov
93b3010cb1
deployment/docker: update datasource versions to the latest releases (#7396)
### Describe Your Changes

Updated the versions of the data sources to the latest releases

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

(cherry picked from commit 6a6d08d03d)
2024-10-31 14:11:08 +01:00
hagen1778
0e6ed4171b
deployment/alerts: consistently update path to alerting rules
Follow-up after 68bad22fd2

Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 6494606924)
2024-10-30 16:44:51 +01:00
Aliaksandr Valialkin
50741d548e
deployment/docker: update VictoriaLogs from v0.38.0-victorialogs to v0.39.0-victorialogs
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.39.0-victorialogs

(cherry picked from commit 25bca94d04)
2024-10-30 15:19:52 +01:00
Aliaksandr Valialkin
16be99ccab
deployment/docker: update VictoriaLogs from v0.37.0-victorialogs to v0.38.0-victorialogs
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.38.0-victorialogs

(cherry picked from commit c43a6ce0eb)
2024-10-30 09:52:53 +01:00
Hui Wang
9616814728
vmalert: integrate with victorialogs (#7255)
address https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6706.
See
https://github.com/VictoriaMetrics/VictoriaMetrics/blob/vmalert-support-vlog-ds/docs/VictoriaLogs/vmalert.md.

Related fix
https://github.com/VictoriaMetrics/VictoriaMetrics/pull/7254.

Note: in this pull request, vmalert doesn't support
[backfilling](https://github.com/VictoriaMetrics/VictoriaMetrics/blob/vmalert-support-vlog-ds/docs/VictoriaLogs/vmalert.md#rules-backfilling)
for rules with a customized time filter. It might be added in the
future, see [this
issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7289)
for details.

Feature can be tested with image
`victoriametrics/vmalert:heads-vmalert-support-vlog-ds-0-g420629c-scratch`.

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 68bad22fd2)
2024-10-29 16:32:00 +01:00
Fred Navruzov
61657e56e8
docs/vmanomaly - release 1.18.0 (#7378)
### Describe Your Changes

docs/vmanomaly - release 1.18.0

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

(cherry picked from commit 5d73b8b866)
2024-10-29 16:31:59 +01:00
Andrii Chubatiuk
da14c7b26e
app/vlinsert: adds journald ingestion support
This commit allows to ingest logs with journald format. 

https://www.freedesktop.org/software/systemd/man/latest/systemd-journal-remote.service.html

related issue: https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4618
2024-10-27 20:42:41 +01:00
Fred Navruzov
63ba28fdc2
docs/vmanomaly-release-1.17.2 (#7322)
### Describe Your Changes

- release 1.17.2 updates
- added sections on logging and CLI args to docs

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-10-22 17:28:50 +02:00
f41gh7
6d2349af63
deployment: bump VM to v1.105.0
Signed-off-by: f41gh7 <nik@victoriametrics.com>
2024-10-22 17:28:49 +02:00
Fred Navruzov
8d9485dfdc
docs/vmanomaly: release 1.17.1 (#7302)
### Describe Your Changes

docs/vmanomaly: release 1.17.1

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

(cherry picked from commit 7a538bbe78)
2024-10-21 08:28:37 +02:00
Aliaksandr Valialkin
e4693506a3
deployment: update VictoriaLogs from v0.36.0-victorialogs to v0.37.0-victorialogs
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.37.0-victorialogs

(cherry picked from commit 025eec2cb0)
2024-10-18 11:42:18 +02:00
Fred Navruzov
f612a9db1e
docs/vmanomaly: release v1.17.0 (#7285)
### Describe Your Changes

docs/vmanomaly: release v1.17.0

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

(cherry picked from commit bc65c9f399)
2024-10-18 11:42:14 +02:00
Andrii Chubatiuk
7b49d4f5dc
vlogs: added basic alerts (#7252)
### Describe Your Changes

Added basic VLogs alerts

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
2024-10-17 11:11:45 -03:00
Aliaksandr Valialkin
255f684b2b
deployment: update VictoriaLogs Docker image from v0.35.0-victorialogs to v0.36.0-victorialogs 2024-10-17 11:09:17 -03:00
Lauri Tirkkonen
59c2073987
deployment/alerts: fix quoting on DiskRunsOutOfSpace (#7234)
### Describe Your Changes

there's an extra `"` at the end of the dashboard url for this alert;
remove it by making the quoting consistent with other alerts in this
file.

### Checklist

The following checks are **mandatory**:

- [X] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

Co-authored-by: Lauri Tirkkonen <lauri@hacktheplanet.fi>
(cherry picked from commit 8fe41b2b08)
2024-10-11 14:28:08 +02:00
Aliaksandr Valialkin
0fe533e0be
deployment: update VictoriaLogs from v0.34.0-victorialogs to v0.35.0-victorialogs
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.35.0-victorialogs

(cherry picked from commit baa87b5b36)
2024-10-11 14:27:46 +02:00
Fred Navruzov
2cdf807231
docs/vmanomaly: updates for v1.16.3 (#7203)
### Describe Your Changes

docs/vmanomaly: updates for v1.16.3

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

(cherry picked from commit f61d8c3ebb)
2024-10-11 14:25:19 +02:00
Aliaksandr Valialkin
00bd1c68a2
deployment: update VictoriaLogs image tag from v0.33.0-victorialogs to v0.34.0-victorialogs
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.34.0-victorialogs
2024-10-08 12:30:49 +02:00
Zakhar Bessarab
06947c2685
vmagent: add support of HTTP2 client for Kubernetes SD (#7114)
### Describe Your Changes

Currently, vmagent always uses a separate `http.Client` for every group
watcher in Kubernetes SD. With a high number of group watchers this
leads to large amount of opened connections.

This PR adds 2 changes to address this:
- re-use of existing `http.Client` - in case `http.Client` is connecting
to the same API server and uses the same parameters it will be re-used
between group watchers
- HTTP2 support - this allows to reuse connections more efficiently due
to ability of using streaming via existing connections.

See this issue for the details and test results -
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5971

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
(cherry picked from commit eefae85450)
2024-10-08 10:37:48 +02:00
hagen1778
ab39f88857
deployment: bump VM to v1.104.0
Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 07902baa8e)
2024-10-04 10:42:28 +02:00
Fred Navruzov
b57ac5ced9
docs/vmanomaly: patch release 1.16.1 (#7169)
### Describe Your Changes

`vmanomaly` patch release 1.16.1 updates

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

(cherry picked from commit 150ee902fd)
2024-10-04 10:40:05 +02:00
Fred Navruzov
d205998501
docs/vmanomaly - update versions in examples (#7163)
### Describe Your Changes

update `vnanomaly` versions in examples

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

(cherry picked from commit dde2a0cb25)
2024-10-02 12:41:14 +02:00
Fred Navruzov
42ddf7cfcc
docs/vmanomaly - release 1.16.0 docs (#7159)
### Describe Your Changes

doc updates for vmanomaly v1.16.0

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

(cherry picked from commit 0e54cfe350)
2024-10-02 12:41:14 +02:00
Aliaksandr Valialkin
ee784f9334
deployment: update VictoriaLogs docker image from v0.32.1-victorialogs to v0.33.0-victorialogs
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.33.0-victorialogs
2024-10-01 13:58:36 +02:00
Aliaksandr Valialkin
975f0f3610
deployment/docker: update VictoriaLogs from v0.32.0-victorialogs to v0.32.1-victorialogs
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.32.1-victorialogs
2024-09-30 14:38:27 +02:00
Aliaksandr Valialkin
bf746f9975
deployment: update VictoriaLogs docker image from v0.31.0-victorialogs to v0.32.0-victorialogs
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.32.0-victorialogs
2024-09-29 14:54:42 +02:00
Andrii Chubatiuk
a18d4f79d4
deployment/victorialogs: rename not executable compose files (#7124)
### Describe Your Changes

Renamed base compose files to prevent envs to be created from them

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-09-29 10:29:49 +02:00
Roman Khavronenko
3e5cba3e78
deployment: separate datasource between single and cluster versions (#7119)
Before, single and cluster deployments were provisioned with both
Grafana datasources: single and cluster. But this resulted into a
problem: single DS didn't work for cluster and vice versa. See
https://github.com/VictoriaMetrics/VictoriaMetrics/pull/7113

This PR splits datasource file into 2 files: single and cluster. Now,
these files are separately provisioned to single and cluster deployments
correspondingly.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-09-28 21:21:18 +02:00
Andrii Chubatiuk
3df920c60d
docs/victorialogs: change fluentd input type in examples (#7123)
### Describe Your Changes

Use fluentd logging driver in examples to have enriched data in
VictoriaLogs

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-09-28 21:21:17 +02:00
Andrii Chubatiuk
6adb3f0dc2
victorialogs: marked fluentd support in roadmap, added syslog example (#7098)
### Describe Your Changes

Marked fluentd in victorialogs roadmap
Added fluentd syslog example setup

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

(cherry picked from commit 05a64a8c14)
2024-09-27 14:43:04 +02:00
Aliaksandr Valialkin
7ef9600da9
deployment: update VictoriaLogs docker image from v0.30.1-victorialogs to v0.31.0-victorialogs
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.31.0-victorialogs
2024-09-27 13:59:49 +02:00
Aliaksandr Valialkin
fc6ddad117
deployment: update VictoriaLogs docker image tag from v0.30.0-victorialogs to v0.30.1-victorialogs
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.30.1-victorialogs
2024-09-27 11:26:31 +02:00
Aliaksandr Valialkin
f9102b9996
deployment: update VictoriaLogs docker image from v0.29.0-victorialogs to v0.30.0-victorialogs
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.30.0-victorialogs
2024-09-27 09:34:18 +02:00
Andrii Chubatiuk
73afeaf510
deployment: restructure victorialogs examples (#6971)
### Describe Your Changes

- Use common compose.yaml file for all victorialogs setups to set
version in a single place and override it on demand for each agent and
protocol
- Replaced multiple victorialogs instances in HA setup with single setup
with `deploy.replica` parameter set
- Added fluentd setup

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

(cherry picked from commit e75ae1b274)
2024-09-25 18:39:43 +02:00
Dmytro Kozlov
20fb79eb8d
deployment/docker: update grafana datasources to the latest version (#7083)
### Describe Your Changes

Updated grafana plugins to the latest releases

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

(cherry picked from commit 91b28d0527)
2024-09-24 16:58:15 +02:00
hagen1778
6bdd0489e7
deployment/alerts: fix copy&paste typo in TooHighGoroutineSchedulingLatency
Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit a5c002edef)
2024-09-24 16:58:14 +02:00
Roman Khavronenko
deb2f87074
deployment: add panel and alerts for displying go scheduler latency (#7078)
The panel and alerting rule should help to understand whether VM
component doesn't have enough CPU resources or gets throttled. The alert
is applicable for all VM components.
The panel was added to vmalert, vmagent, vmsingle, vm clusert and
victorialogs dashes.

-------------------

This alerting rule should have help us identify resource shortage for
sandbox vmagent - see [this
link](https://play.victoriametrics.com/select/accounting/1/6a716b0f-38bc-4856-90ce-448fd713e3fe/prometheus/graph/#/?g0.range_input=23d13h25m25s424ms&g0.end_input=2024-09-23T14%3A11%3A00&g0.relative_time=none&g0.tab=0&g0.expr=histogram_quantile%280.99%2C+sum%28rate%28go_sched_latencies_seconds_bucket%7Bjob%3D%22vmagent-monitoring-vmagent%22%7D%5B5m%5D%29%29+by+%28le%2C+job%2C+instance%29%29+%3E+0.1)
for example. We weren't aware of resource shortage, because VM metrics
assumed this vmagent had 1vCPU while in fact its limit was 0.2vCPU.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 4d0b41e63b)
2024-09-24 16:58:14 +02:00
Thomas Danielsson
4f93422064
docs: typo filebeat -> fluent-bit (#7000)
### Describe Your Changes

Fix a typo; `filebeat.yml` -> `fluent-bit.conf`

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

Signed-off-by: Thomas Danielsson <thomas@elajt.se>
(cherry picked from commit 69b976f08e)
2024-09-19 15:48:09 +02:00
Thomas Danielsson
066137feaa
typo: added missing '{' (#7044)
### Describe Your Changes

Added missing `{` in vmalert rule.

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

(cherry picked from commit 75cc7922c3)
2024-09-19 15:48:08 +02:00
Artem Navoiev
bcab5861ae
victorialogs: add HA example for logstash and fluentbit (#6968)
### Describe Your Changes

Please provide a brief description of the changes you made. Be as
specific as possible to help others understand the purpose and impact of
your modifications.

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
(cherry picked from commit 277fed9990)
2024-09-19 15:46:52 +02:00
Artem Navoiev
c073206f0c
docs: update roadmap, mark telegraf and otel as done and link the docs
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
(cherry picked from commit 47cd7bb4c1)
2024-09-19 15:46:51 +02:00
Aliaksandr Valialkin
c68c7c0d1a
deployment: update VictoriaLogs from v0.28.0-victorialogs to v0.29.0-victorialogs
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.29.0-victorialogs
2024-09-08 21:23:29 +02:00
Aliaksandr Valialkin
8eef397d29
deployment/docker: update base Alpine docker image from 3.20.2 to 3.20.3
See https://alpinelinux.org/posts/Alpine-3.17.10-3.18.9-3.19.4-3.20.3-released.html
2024-09-08 19:27:05 +02:00
Aliaksandr Valialkin
e90e809c00
deployment: update Go builder from Go1.23.0 to Go1.23.1
See https://github.com/golang/go/issues?q=milestone%3AGo1.23.1+label%3ACherryPickApproved
2024-09-06 22:57:56 +02:00
Dmytro Kozlov
a4b0542be7
deployment: upgrade datasource versions (#6935)
### Describe Your Changes

Upgraded victoriametrics and victorialogs data source versions.

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-09-05 16:57:46 +02:00
Andrii Chubatiuk
711f2cc4f2
vlinsert: added opentelemetry logs support
Commit adds the following changes:

* Adds support of OpenTelemetry logs for Victoria Logs with protobuf encoded messages

*  json encoding is not supported for the following reasons:
   - It brings a lot of fragile code, which works inefficiently.
   - json encoding is impossible to use with language SDK.

* splits metrics and logs structures at lib/protoparser/opentelemetry/pb package.

* adds docs with examples for opentelemetry logs.

---
Related issue: https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4839

Co-authored-by: AndrewChubatiuk <andrew.chubatiuk@gmail.com>
Co-authored-by: f41gh7 <nik@victoriametrics.com>
2024-09-03 20:24:01 +02:00
Andrii Chubatiuk
d5fe4566e5
app/vlinsert: support getting _msg_field, _time_field, _stream_fields and _ignore_fields from headers
*  Many collectors don't support forwarding url query params to the remote system. It makes impossible to define stream fields for it. Workaround with proxy between VictoriaLogs and log shipper is too complicated solution.

* This commit adds the following changes:
 * Adds fallback to to headers params, if query param is empty for:
     _msg_field -> VL-Msg-Field
    _stream_fields -> VL-Stream-Fields
    _ignore_fields -> VL-Ignore-Fields
    _time_field -> VL-Time-Field
 * removes deprecations from victorialogs compose files, added more
output format examples for logstash, telegraf, fluent-bit

 related issue: https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5310
2024-09-03 20:24:00 +02:00
Aliaksandr Valialkin
ac507466c3
all: suppress InvalidDefaultArgInFrom warning emitted by docker build when building Docker packages via make package-* command
Recent versions of `docker build` started generating the InvalidDefaultArgInFrom warning if Dockerfile contains
an ARG without default value. While this warning doesn't affect building Docker packages via `make package-*` commands,
it is better suppressing the warning, so it doesn't clutter `make package-*` output with the noise,
which can hide real issues in the future.
2024-09-03 14:05:43 +02:00
Marco Maurer (-Kilchhofer)
b367a36183
deployment/alerts: add missing description for VMagent alerts (#6921)
The 3 alerts for VMagent:
- `RejectedRemoteWriteDataBlocksAreDropped`
- `TooManyScrapeErrors`
- `TooManyWriteErrors`

missed the description annotation.
I moved the summary to description and added a generic summary to these
alerts.

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

Signed-off-by: Marco Maurer <marco.kilchhofer@gmail.com>
(cherry picked from commit f17fca718d)
2024-09-03 10:49:32 +02:00
hagen1778
c6d83a8a8d
deployment: bump VM version to 1.103.0
Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit b58a10c389)
2024-08-30 11:17:26 +02:00
Roman Khavronenko
f599f3ad86
deployment/docker: update Go builder from Go1.22.5 to Go1.23.0 (#6861)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-08-22 23:56:12 +02:00
f41gh7
6939cc9924
deployment: update compose images to v1.102.1 release
Signed-off-by: f41gh7 <nik@victoriametrics.com>
2024-08-02 11:29:48 +02:00
Hui Wang
e0c62e5c50
security: upgrade base docker image (Alpine) from 3.20.1 to 3.20.2 (#6684)
See https://www.alpinelinux.org/posts/Alpine-3.20.1-released.html

>including security fix for:
OpenSSL CVE-2024-5535
2024-07-25 11:02:23 +02:00
Aliaksandr Valialkin
c57c169dc4
deployment: update VictoriaMetrics Docker image from v1.101.0 to v1.102.0
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.102.0
2024-07-17 22:35:00 +02:00
Aliaksandr Valialkin
8e53212380
Revert "deployment: build image for vmagent streamaggr benchmark (#6515)"
This reverts commit 6b128da811.

Reason for revert: this complicates and slows down CI/CD without giving significant benefits in return.

The idea of automatic building, publishing and deploying Docker images to our playground on every pull request
and commit isn't very bright because of the following reasons:

- It slows down CI/CD pipeline
- It increases costs on CPU time spent at CI/CD pipeline
- It contradicts goal #7 at https://docs.victoriametrics.com/goals/#goals and non-goal #8 at https://docs.victoriametrics.com/goals/#non-goals

The previous workflow was much better - if we need to deploy some new Docker image at playground or staging environment,
then just __manually__ build and deploy the needed Docker image there. If the manual process requires making too many
steps, then think on how to automate these steps into a single Makefile command.

Updates https://github.com/VictoriaMetrics/ops/pull/1297
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6515
2024-07-16 13:34:16 +02:00
Aliaksandr Valialkin
6b4c510091
docs/VictoriaLogs/CHANGELOG.md: cut v0.28.0-victorialogs release 2024-07-10 03:08:26 +02:00