Commit graph

10237 commits

Author SHA1 Message Date
Nikolay
295d2b9721
app/vmauth: allow to serve internal API and different address
vmauth uses 'lib/httpserver' for serving HTTP requests. This server
unconditionally defines built-in routes (such as '/metrics',
'/health', etc). It makes impossible to proxy `HTTP` requests to  backends with the same routes.
Since vmauth's httpserver matches built-in route and return local
response.

 This commit adds new flag `httpInternalListenAddr` with
default empty value. Which removes internal API routes from public
router and exposes it at separate http server.

For example given configuration disables private routes at `0.0.0.0:8427` address and serves it at `0.0.0.0:8426`:

`./bin/vmauth --auth.config=config.yaml --httpListenAddr=:8427 --httpInternalListenAddr=127.0.0.1:8426`

Related issues:
- https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6468
- https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7345
2025-02-06 13:44:34 +01:00
Joost Buskermolen
824f531490
app/vmselect: expose /-/healthy and /-/ready endpoints on full Prometheus path
This commit improves integration with third-party solutions who rely on non-root
endpoints (i.e. MinIO) when the vmselect path has been specified in the
configured Prometheus URL like:
`http://vmselect.:8481/select/0/prometheus`

Comparable change has been done before
(b885a3b6e9), however only takes care of
the root path. This means endpoints `-/healthy` and `-/ready` are still
not available on full vmselect Prometheus paths, resulting in
unsupported path requests.

This change makes these endpoints available on the full paths like:
`/select/0/prometheus/-/healthy` and `/select/0/prometheus/-/ready`,
thus achieving full Prometheus compatibility for external dependencies.

Related issues:
- https://github.com/minio/console/issues/2829
- https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1833

---

Signed-off-by: Joost Buskermolen <j.buskermolen@cloudmeesters.com>
2025-02-05 17:00:11 +01:00
hagen1778
882db7080b
docs: fix copy&paste typo in change type
Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 553e95e293)
2025-02-05 15:30:06 +01:00
Github Actions
6d6318aa27
Automatic update operator docs from VictoriaMetrics/operator@8192989 (#8207)
Automated changes by
[create-pull-request](https://github.com/peter-evans/create-pull-request)
GitHub action

Signed-off-by: Github Actions <133988544+victoriametrics-bot@users.noreply.github.com>
Co-authored-by: f41gh7 <18450869+f41gh7@users.noreply.github.com>
(cherry picked from commit 46fbb95b16)
2025-02-05 15:29:16 +01:00
Github Actions
bc367f52d4
Automatic update helm docs from VictoriaMetrics/helm-charts@0155ba2 (#8217)
Automated changes by
[create-pull-request](https://github.com/peter-evans/create-pull-request)
GitHub action

Signed-off-by: Github Actions <133988544+victoriametrics-bot@users.noreply.github.com>
Co-authored-by: AndrewChubatiuk <3162380+AndrewChubatiuk@users.noreply.github.com>
(cherry picked from commit fbe592f495)
2025-02-05 15:29:16 +01:00
Artem Fetishev
fd0bd8a67f
lib/storage: fix cardinality limiting for cases when insertion takes fast path (#8218)
### Describe Your Changes

The cardinality limiter in this case does not receive the actual
metricID but some other value found in r.TSID.MetricID and is not
initialized. Depending on the system and/or go runtime implementation,
this value can be 0 or some garbage value (which shouldn't have too wide
a range). Thus, there basically no limit for inserted metricIDs.

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Signed-off-by: Artem Fetishev <rtm@victoriametrics.com>
Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>

(cherry picked from commit 631b736bc2)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2025-02-05 15:29:07 +01:00
Fred Navruzov
2810d83cdd
docs/vmanomaly: fix 404 links (#8219)
### Describe Your Changes

- fixed recently found 404 under `/anomaly-detection/` docs section
- get rid of remaining .html / .index.html endings in links under
`/anomaly-detection/` docs section

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

(cherry picked from commit ceefa406cc)
2025-02-05 15:26:19 +01:00
Zakhar Bessarab
323163d6b0
{docs,app/vmctl}: clarify how vm-rate-limit is applied
### Describe Your Changes

Explicitly mention that `--vm-rate-limit` is used for each individual
worker defined by `--vm-concurrency`.

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
(cherry picked from commit 2748681f40)
2025-02-04 18:24:09 +04:00
Zakhar Bessarab
9494ebe7fc
docs/streaming-aggregation: fix typo in metric name
### Describe Your Changes

Fix metric name labels set not being "closed" by `}`.

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
(cherry picked from commit e2cfd351c0)
2025-02-04 13:10:32 +04:00
f41gh7
030c1aed06
app/vmagent/kafka: properly close producer client
At v1.107.0 release with commit 21a7f06fc4beeb6ad32b9f7fd88704ed33674905 was introduced regression.

It prevents kafka producer client from graceful shutdown. This change also removed kafka message headers.

 This commit properly closes client.

 It also addresses the following issues:

* properly add headers to kafka message
* add url_id to the exported metrics, which prevents possible metrics clash if multiple remote write targets have
 the same url and topic name.

Signed-off-by: f41gh7 <nik@victoriametrics.com>
2025-02-03 14:22:07 +01:00
hagen1778
d472464b0e
docs: mention tlast_over_time in influx guide
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/8188

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2025-02-03 14:20:05 +01:00
hagen1778
5dcbdd8f16
docs: fix typo in image name
Follow-up after 35d77a3bed

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2025-02-03 14:20:05 +01:00
Artem Fetishev
1ea2586856
RemoteWriteConnectionIsSaturated alert: add another saturation cause to the alert description (#8195)
### Describe Your Changes

Currently the alert descrption considers only one end of the connection
(vmagent). While saturation can also be caused by slowness of the
receiving components (vminsert, vmstorage). Update the alert description
with a brief suggestion to also check the dashboards of these
components.

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Signed-off-by: Artem Fetishev <rtm@victoriametrics.com>
Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 30af662d84)
2025-02-01 22:31:56 +01:00
Roman Khavronenko
e803b9b68b
metricsql: bump to v0.83.0 (#8141)
metricsql: bump to v0.83.0

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7703

The update also returns an error if metric name is specified twice in
metrics selector.
For example, `foo{__name__="bar"}` is not allowed anymore. It would
successfully parse before
this change, but it won't satisfy the search filter any way. So it had
no sense in supporting this. This is why some test cases were removed.

Signed-off-by: hagen1778 <roman@victoriametrics.com>

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit fa2107bbec)
2025-02-01 22:31:56 +01:00
Jay
feaa8b9302
docs: fix typo in docker examples readme (#8194)
### Describe Your Changes

Fixed misspelling of the word "components" from "componetns"

### Checklist

The following checks are **mandatory**:

- [X] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

(cherry picked from commit 1427ea3ca9)
2025-02-01 22:31:56 +01:00
hagen1778
a6ef648687
app/vmalert: mention that remoteWrite.concurrency depends on CPU
Mnetion explicitly that `remoteWrite.concurrency` deopends on number
of available CPU cores. Updated docs to rm auto-printed default value.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/8151

Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 5561970db0)
2025-02-01 22:31:55 +01:00
Github Actions
cce974bf7d
Automatic update operator docs from VictoriaMetrics/operator@7930b3a (#8202)
Automated changes by
[create-pull-request](https://github.com/peter-evans/create-pull-request)
GitHub action

Signed-off-by: Github Actions <133988544+victoriametrics-bot@users.noreply.github.com>
Co-authored-by: f41gh7 <18450869+f41gh7@users.noreply.github.com>
(cherry picked from commit 4d42e29967)
2025-02-01 22:31:55 +01:00
Roman Khavronenko
13cd76347d
app/vmselect/promql: fix discrepancies when using or binary operator
The change covers various corner cases when using `or` binary operator.
See corresponding issues and pull request here to see the cases:
https://github.com/VictoriaMetrics/VictoriaMetrics/pull/7770

Related issues:
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7759
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7640

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 72837919ae)
2025-02-01 22:31:55 +01:00
Zakhar Bessarab
9c4e2e58bd
docs/release-guide: add a note for vmui build (#8183)
### Describe Your Changes

Please provide a brief description of the changes you made. Be as
specific as possible to help others understand the purpose and impact of
your modifications.

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
(cherry picked from commit 6166149474)
2025-01-31 17:00:48 +04:00
Github Actions
68daa6d0cd
Automatic update Grafana datasource docs from VictoriaMetrics/victorialogs-datasource@de3daa7 (#8196)
(cherry picked from commit f34e82eb9b)
2025-01-31 17:00:46 +04:00
Github Actions
146e28daf3
Automatic update helm docs from VictoriaMetrics/helm-charts@485594e (#8197)
Automated changes by
[create-pull-request](https://github.com/peter-evans/create-pull-request)
GitHub action

Signed-off-by: Github Actions <133988544+victoriametrics-bot@users.noreply.github.com>
Co-authored-by: AndrewChubatiuk <3162380+AndrewChubatiuk@users.noreply.github.com>
(cherry picked from commit 2545082c0f)
2025-01-31 17:00:44 +04:00
Github Actions
864c925dac
Automatic update Grafana datasource docs from VictoriaMetrics/victoriametrics-datasource@35e86ad (#8190)
(cherry picked from commit 676669020f)
2025-01-31 17:00:42 +04:00
Zakhar Bessarab
d25f431bba
docs/victoria-logs/quickstart: use link to docs for chart (#8186)
### Describe Your Changes

Please provide a brief description of the changes you made. Be as
specific as possible to help others understand the purpose and impact of
your modifications.

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
(cherry picked from commit 071db473a7)
2025-01-31 16:59:51 +04:00
Andrii Chubatiuk
5b5c21e377
fixed cloud docs links (#8198)
### Describe Your Changes

fixed some cloud docs links

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

(cherry picked from commit 94ac051647)
2025-01-31 16:59:50 +04:00
Zakhar Bessarab
9d70632d74
deployment/docker: install VL datasource from marketplace (#8199)
### Describe Your Changes

Please provide a brief description of the changes you made. Be as
specific as possible to help others understand the purpose and impact of
your modifications.

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
(cherry picked from commit 3419c56eef)
2025-01-31 16:59:50 +04:00
Github Actions
ff92c0e322
Automatic update operator docs from VictoriaMetrics/operator@946c30e (#8193)
Automated changes by
[create-pull-request](https://github.com/peter-evans/create-pull-request)
GitHub action

Signed-off-by: Github Actions <133988544+victoriametrics-bot@users.noreply.github.com>
Co-authored-by: f41gh7 <18450869+f41gh7@users.noreply.github.com>
(cherry picked from commit 3091133a36)
2025-01-31 16:59:49 +04:00
Roman Khavronenko
656e0f9f06
deployment: use signed metrics datasource (#8189)
* use signed datasource
* bump Grafana because on v10 pre-installing+provisioning didn't work
* consistently rename provisioning folder

### Describe Your Changes

Please provide a brief description of the changes you made. Be as
specific as possible to help others understand the purpose and impact of
your modifications.

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 63a1baf10f)
2025-01-30 16:02:13 +01:00
Zakhar Bessarab
5c4c20d0e0
app/vmui/make: add a step to enforce license type override
- update license file content
- add a build step to enforce setting correct license type

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2025-01-30 14:35:03 +01:00
hagen1778
8083f0c216
dashboards: improve wording for SlowInserts panel info
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2025-01-30 14:30:06 +01:00
f41gh7
6e5c13bd1b
docs/changelog: add recommendation for lts releases upgrade
Signed-off-by: f41gh7 <nik@victoriametrics.com>
2025-01-30 14:30:05 +01:00
hagen1778
77fe40c0e5
docs: fix broken vminsert link in changelog
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2025-01-30 14:30:05 +01:00
Yury Molodov
a412cbe66a
vmui/logs: add _msg to group view field list (#8177)
### Describe Your Changes

add the `_msg` field to the list of fields for the group view, allowing
users to select multiple fields, including `_msg`, for log display.

![image](https://github.com/user-attachments/assets/90336315-8708-424d-9ed8-aa8d9a0d07a9)

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2025-01-29 16:38:57 +01:00
Github Actions
d7b9a5c0b2
Automatic update Grafana datasource docs from VictoriaMetrics/victorialogs-datasource@b532af3 (#8181) 2025-01-29 16:38:57 +01:00
Yury Molodov
952f6ed87a
vmui: refactor clipboard handling with useCopyToClipboard hook (#8178)
### Describe Your Changes

Replaced `await navigator.clipboard.writeText` with `useCopyToClipboard`
hook.
See [this pull
request](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/7778)
for details.

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2025-01-29 16:38:57 +01:00
Yury Molodov
f79f092c9d
vmui/logs: fix dropdown menu in group view settings (#8176)
### Describe Your Changes

Fixed an issue where dropdown menus were not visible in the Group View
settings.
Ref issue: #8153 

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
2025-01-29 16:38:57 +01:00
f41gh7
39e69b103e
app/vmselect: properly cancel long running requests on client connection close
At this time `bufferedwriter` [silently ignores connection close
errors](78eaa056c0/lib/bufferedwriter/bufferedwriter.go (L67)).
It may be very convenient in some situations (to not log such
unimportant errors), but it's too implicit and unsafe for the others.
For example, if you close [export
API](https://docs.victoriametrics.com/#how-to-export-time-series) client
connection in the middle of communication, VictoriaMetrics won't notice
it and will start to hog CPU by exporting all the data into nowhere
until it process all of them. If you'll make a few retries, it will be
effectively a DoS on the server.

This commit replaces this implicit error suppressing with explicit error
handling which fixes the issue with export API.

Issue was introduced at e78f3ac8ac
2025-01-29 16:38:38 +01:00
Yury Molodov
a67d6a578d
vmui/logs: fix transparency for bars in hits chart (#8174)
### Describe Your Changes

Restored transparency for bars in the hits chart to improve visibility.
Ref issue: #8152

| State  | Preview |
|--------|---------|
| Before |
![image](https://github.com/user-attachments/assets/2f95cdd9-0b39-4420-9bf7-011274eadc86)
|
| After |
![image](https://github.com/user-attachments/assets/9123d8e7-e615-41ab-9193-4c50ce262e7f)
|

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2025-01-29 16:34:16 +01:00
Yury Molodov
3419b0f9c8
vmui: add heatmap-to-line chart switch (#8169)
### Describe Your Changes

- Added support for switching from heatmap to line chart (the switch is
displayed only for heatmap data).
- Added a warning for heatmap rendering issues with a button to switch
to a line chart.
- Fixed a chart rendering issue when switching from heatmap to line
chart.

Ref issue: #8057 

<img
src="https://github.com/user-attachments/assets/2efc901a-3dcd-4b5b-aeaa-8f22b2dcedaa"
width="600"/>
<hr/>
<img
src="https://github.com/user-attachments/assets/2594cbb6-8356-4448-9576-0d7da4749256"
width="600"/>
<hr/>
<img
src="https://github.com/user-attachments/assets/ccfeb8b4-7e98-41a7-90f4-ee9dcf971141"
width="600"/>

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
2025-01-29 16:34:15 +01:00
Github Actions
fcae3964a1
Automatic update Grafana datasource docs from VictoriaMetrics/victoriametrics-datasource@0d6e500 (#8180)
(cherry picked from commit 0e08a7a125)
2025-01-29 13:29:23 +01:00
Roman Khavronenko
6c596d2f4d
lib/cgroup: warn users about using fractional CPU quotas (#8175)
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7988

### Describe Your Changes

Please provide a brief description of the changes you made. Be as
specific as possible to help others understand the purpose and impact of
your modifications.

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 13c4324bb5)
2025-01-29 13:29:23 +01:00
Aliaksandr Valialkin
bacf58de76
lib/logstorage: remove unnecesary abstraction - RowsFormatter
It is better to use the AppendFieldsToJSON function directly
instead of hiding it under RowsFormatter abstraction.

(cherry picked from commit 95f182053b)
2025-01-29 13:29:23 +01:00
Aliaksandr Valialkin
d63a2aa4ab
docs/Articles.md: add a link to https://tanmay-bhat.medium.com/reducing-inter-az-traffic-in-victoriametrics-with-zonekeeper-3bd7e1526796
Thanks to Tanmay Bhat ( https://tanmay-bhat.medium.com/ ) for the great article about VictoriaMetrics and vmagent!

(cherry picked from commit ec64a1fd7c)
2025-01-29 13:29:22 +01:00
Aliaksandr Valialkin
0aabf19962
go.mod: update Go from v1.23.5 to 1.23.5
This is a follow-up for 489631b227

(cherry picked from commit f61b5da617)
2025-01-29 13:29:22 +01:00
Aliaksandr Valialkin
40646a125e
lib/logstorage: ignore logs with too long field names during data ingestion
Previously too long field names were silently truncated. This is not what most users expect.
It is better ignoring the whole log entry in this case and logging it with the WARNING message,
so human operator could notice and fix the ingestion of incorrect logs ASAP.

The commit also adds and updates the following entries to VictoriaLogs faq:

- https://docs.victoriametrics.com/victorialogs/faq/#how-many-fields-a-single-log-entry-may-contain
- https://docs.victoriametrics.com/victorialogs/faq/#what-is-the-maximum-supported-field-name-length
- https://docs.victoriametrics.com/victorialogs/faq/#what-length-a-log-record-is-expected-to-have

These entries are referred at `-insert.maxLineSizeBytes` and `-insert.maxFieldsPerLine` command-line descriptions
and at the WARNING messages, which are emitted when log entries are ignored because of some of these limits
are exceeded.

(cherry picked from commit 3c036e0d31)
2025-01-29 13:29:22 +01:00
Aliaksandr Valialkin
639ea71a95
app/vlogscli: show compact output mode line in the help output to be consistent with the naming for other output modes
(cherry picked from commit aacb9b2726)
2025-01-29 13:29:22 +01:00
Alexander Marshalov
34efc65338
vmcloud docs: actualize parameters description due to changes in v1.108.0 (#6928) (#8172)
vmcloud docs: actualize parameters description due to changes in
v1.108.0 (#6928)

(cherry picked from commit c753f75e18)
2025-01-29 13:29:21 +01:00
f41gh7
9db5ca0147
docs/changelog: mention v1.102.12 and v1.97.17 releases
Signed-off-by: f41gh7 <nik@victoriametrics.com>
(cherry picked from commit 330461c635)
2025-01-29 13:29:21 +01:00
Roman Khavronenko
3fea6140ef
dashboards: consistently use process_cpu_cores_available for CPU usage
Related issue:
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7988

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 5a1a28ba87)
2025-01-29 13:29:21 +01:00
hagen1778
12e44a66b7
docs: re-order changes for better readability
Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 5b18dc0214)
2025-01-29 13:29:21 +01:00
Fred Navruzov
d8ad3310a5
docs/vmanomaly: release v1.19.2 (#8165)
### Describe Your Changes

docs/vmanomaly: release v1.19.2 (patch that addresses some of the bugs
found in 1.19.0)

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Co-authored-by: Mathias Palmersheim <mathias@victoriametrics.com>
2025-01-27 23:10:58 +01:00