Commit graph

10226 commits

Author SHA1 Message Date
dmitryk-dk
629a943683
Merge branch 'master' into get-vllogs-tenants
# Conflicts:
#	app/vlselect/main.go
2025-03-04 09:48:10 +01:00
Fred Navruzov
6828cca5a6
docs/vmanomaly: release v1.20.0 (#8422)
### Describe Your Changes

> ⚠️ Even if approved, please don't merge it on my behalf, I
still may apply a couple of re-phrasings before merging it on Monday
03.03.2025

- Aligned `vmanomaly` docs with release v1.20.0
- Re-structured root page of anomaly detection docs for clarity
- Added several sections to FAQ, e.g. on how to incorporate domain
knowledge into anomaly detection configs

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2025-03-04 00:24:34 +02:00
hagen1778
fc5d495900
app/vmui: update error message for no matched rules
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2025-03-03 16:56:15 +01:00
Yury Molodov
974c094a52
vmui: fix infinite loader on downsampling page (#8428)
### Describe Your Changes

This PR fixes an issue where the Downsampling filters debug page would
get stuck in an infinite loading state when labels had no matches. Now,
the case is properly handled.

Related issue: #8339

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
2025-03-03 16:48:11 +01:00
Max Kotliar
649f4d5e00
docs/downsampling: fix terminology. replace "deduplication" with "downsample". (#8421)
### Describe Your Changes

I guess a "downsampling" term is more appropriate than "deduplication"
in the downsampling paragraph.

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2025-03-03 14:12:36 +01:00
Nikolay
a55ac4495f
docs: update release guide (#8423)
* move testing step after assets upload
* add LTS build step before testing step

Signed-off-by: f41gh7 <nik@victoriametrics.com>
2025-03-03 14:10:06 +01:00
Hui Wang
52988ebdc8
vmalert: add time buckets stats pipe check for vlogs expression (#8400)
VictoriaLogs inserts `_time` field as a label in result when query with
[time buckets stats
pipe](https://docs.victoriametrics.com/victorialogs/logsql/#stats-by-time-buckets),
making the result meaningless and may lead to cardinality issues.
>curl --location --request POST
'https://play-vmlogs.victoriametrics.com/select/logsql/stats_query?query=_time%3A1m%20%7C%20stats%20by%20(_time%3A10s)%20count%20()%20as%20total'

>{"status":"success","data":{"resultType":"vector","result":[{"metric":{"__name__":"total","_time":"2025-01-24T12:31:30Z"},"value":[1737721904.4476516,"12"]},{"metric":{"__name__":"total","_time":"2025-01-24T12:31:10Z"},"value":[1737721904.4476516,"10"]},{"metric":{"__name__":"total","_time":"2025-01-24T12:31:00Z"},"value":[1737721904.4476516,"10"]},{"metric":{"__name__":"total","_time":"2025-01-24T12:31:20Z"},"value":[1737721904.4476516,"12"]},{"metric":{"__name__":"total","_time":"2025-01-24T12:30:50Z"},"value":[1737721904.4476516,"10"]},{"metric":{"__name__":"total","_time":"2025-01-24T12:30:40Z"},"value":[1737721904.4476516,"9"]}]}}%

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
2025-03-03 14:09:34 +01:00
hagen1778
065a3d068c
docs/victorialogs/vmalert: apply consistent formatting and fix typos
* use unified numeric list;
* apply line width limits;
* remove time filter from quantile examples, as we suggest to
not use time filters.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2025-03-03 13:57:30 +01:00
Artem Fetishev
43a742ba0c
docs: bump last LTS versions
Signed-off-by: Artem Fetishev <rtm@victoriametrics.com>
2025-03-03 11:44:37 +01:00
Artem Fetishev
e69c744dd4
docs/CHANGELOG.md: update changelog with LTS release notes
Signed-off-by: Artem Fetishev <rtm@victoriametrics.com>
2025-03-03 11:43:00 +01:00
hagen1778
6db97d6f79
lib/timeutil: add test for ParseDuration
See https://github.com/VictoriaMetrics/VictoriaMetrics/pull/8403#discussion_r1976110052

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2025-03-03 10:46:01 +01:00
Max Kotliar
7451a3631a
docs: fix typo in docs (#8420)
### Describe Your Changes

Just a small typo in the docs

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2025-03-03 10:26:41 +01:00
Roman Khavronenko
63f6ac3ff8
lib/promutils: move time-related funcs from promutils to timeutil (#8403)
Since funcs `ParseDuration` and `ParseTimeMsec` are used in vlogs,
vmalert, victoriametrics and other components, importing promutils only
for this reason makes them to export irrelevant
`vm_rows_invalid_total{type="prometheus"}` metric.

This change removes `vm_rows_invalid_total{type="prometheus"}` metric
from /metrics page for these components.

### Describe Your Changes

Please provide a brief description of the changes you made. Be as
specific as possible to help others understand the purpose and impact of
your modifications.

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2025-03-03 10:25:42 +01:00
Hui Wang
281f1a94e4
vmalert-tool: clean up the temporary storage path when process is ter… (#8407)
…minated by SIGTERM or SIGINT
2025-02-28 15:17:05 +01:00
Zakhar Bessarab
7f6156e46d
docs: fix link to quay docker image for single-node (#8414)
### Describe Your Changes

Previous link was incorrectly created by adding repository name to
organization link.
All other links are using proper schema.

This is a follow-up for 6ff61a1c.

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2025-02-28 15:13:52 +01:00
Hui Wang
4015db18bc
vmalert-tool: add flag -httpListenPort to specify the port used dur… (#8408)
…ing testing and make the temporary storage path more unique

address https://github.com/VictoriaMetrics/VictoriaMetrics/issues/8393

---------

Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
2025-02-28 11:25:35 +01:00
Zakhar Bessarab
99de272b72
lib/promrelabel/scrape_url: properly parse IPv6 address from __address__ label
Fix parsing of IPv6 addresses after discovery. Previously, it could lead
to target being discovered and discarded afterwards.

See: https://github.com/VictoriaMetrics/VictoriaMetrics/issues/8374

---------

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2025-02-28 14:19:10 +04:00
Zakhar Bessarab
6ff61a1c09
docs/all: add refs to Quay mirror for docker images (#8406)
### Describe Your Changes

See: https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4116

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
2025-02-28 11:14:10 +01:00
Zakhar Bessarab
ed0db37d1d
app/vmbackupmanager: fix error propagation for policy apply
Previously error text was silently discarded.

This commit properly formats error message and add actual error cause to it.
2025-02-28 10:40:50 +01:00
Hui Wang
a36c8a9679
app/vmgateway: fix data query in rate limiter
Properly format datasource query request in rate limiter
2025-02-28 10:40:46 +01:00
hagen1778
c80e2d16a5
deployment/docker: fix typo in markdown
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2025-02-28 09:57:12 +01:00
hagen1778
e10b963a4d
docs: link to internal docs for alerting and recording rules
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2025-02-28 09:46:57 +01:00
hagen1778
3a04d9aca8
docs: fix markdown in stream aggregation docs
Availability mark was breaking the line.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2025-02-28 09:07:51 +01:00
Aliaksandr Valialkin
bfa9fee424
docs/VictoriaLogs/CHANGELOG.md: typo fix: filter -> filters 2025-02-28 00:09:00 +01:00
Aliaksandr Valialkin
d7431e69fd
deployment/docker: update VictoriaLogs Docker image tag from v1.14.0-victorialogs to v1.15.0-victorialogs
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.15.0-victorialogs
2025-02-28 00:02:55 +01:00
Aliaksandr Valialkin
0e5e90fe74
docs/VictoriaLogs/CHANGELOG.md: cut v1.15.0-victorialogs release 2025-02-27 22:58:18 +01:00
Aliaksandr Valialkin
5790c5c75d
Makefile: update golangc-lint from v1.64.4 to v1.64.5
See https://github.com/golangci/golangci-lint/releases/tag/v1.64.5
2025-02-27 22:54:19 +01:00
Aliaksandr Valialkin
744ac496bd
lib/logstorage: add ability to specify field name prefixes inside fields (...) lists passed to pack_json and pack_logfmt pipes 2025-02-27 22:54:18 +01:00
Aliaksandr Valialkin
03a04b4408
docs/changelog/CHANGELOG.md: upgrading from Go1.23.6 to Go1.24.0 doesn't fix any know security issues
The upgrade from Go1.23.6 to Go1.24.0 is a feature, not a security fix.

This is a follow-up for ea6ed4232a
2025-02-27 22:54:18 +01:00
Zhu Jiekun
117b7ce6b7
bugfix: negative rate result when lookbehind window longer than search.maxLookback (#8378)
### Describe Your Changes

#8342 

fix negative rate result when the lookbehind window is longer than
`-search.maxLookback` or `-search.maxStalenessInterval` and data
contains gap.

This issue was introduced since
[v1.110.0](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/8072).

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2025-02-27 22:52:39 +01:00
Roman Khavronenko
38d46d149f
lib/prompbmarshal: move MustParsePromMetrics to protoparser/prometheus (#8405)
`MustParsePromMetrics` imports `lib/protoparser/prometheus`, and this
package exposes the following metrics:
```
vm_protoparser_rows_read_total{type="promscrape"}
vm_rows_invalid_total{type="prometheus"}
```

It means every package that uses `lib/prompbmarshal` will start exposing
these metrics. For example, vlogs imports `lib/protoparser/common` which
uses `lib/prompbmarshal.Label`. And only because of this vlogs starts
exposing unrelated prometheus metrics on /metrics page.

Moving `MustParsePromMetrics` to `lib/protoparser/prometheus` seems like
the leas intrusive change.


-----------

Depends on another change
https://github.com/VictoriaMetrics/VictoriaMetrics/pull/8403

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2025-02-27 22:50:27 +01:00
Aliaksandr Valialkin
84d5771b41
lib/logstorage: allow passing * at in(*), contains_any(*) and contains_all(*)
Such filters are equivalent to `match all` filter aka `*`. These filters are needed for VictoriaLogs plugin for Grafana.

See https://github.com/VictoriaMetrics/victorialogs-datasource/issues/238#issuecomment-2685447673
2025-02-27 11:37:43 +01:00
Roman Khavronenko
f39ce2aeef
dashboards: update victorialogs dashboard (#8398)
* stats panels:
* * add ingested logs/bytes over 24h
* * fix correctness compression ratio and disk space usage panels
* overview panels:
* * add requests errors rate, logs rate, query duration panels
* add troubleshooting section:
* * add non-default falgs, dropped logs panels
* update descriptions of the panels and version requirements

Depends on https://github.com/VictoriaMetrics/VictoriaMetrics/pull/8396

--------------

<img width="1489" alt="image"
src="https://github.com/user-attachments/assets/5257b8e1-a2e7-4143-94c7-bb42b19e4206"
/>

<img width="1499" alt="image"
src="https://github.com/user-attachments/assets/1b521c58-6e12-4d38-b052-93abdac3b4fc"
/>

<img width="1468" alt="image"
src="https://github.com/user-attachments/assets/3010cf0a-8ca7-48d4-9967-328b37fdbc28"
/>

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2025-02-27 11:15:52 +01:00
Roman Khavronenko
ba2bf9e73a
vlogs: add metrics to track select queries duration (#8396)
This change would help to track slow queries via alerts or dashboards.

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2025-02-27 11:11:26 +01:00
hagen1778
47d0bba8f0
docs: update never-firing alerts chapter
* add availability marker
* mention article https://victoriametrics.com/blog/never-firing-alerts/

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2025-02-27 09:57:36 +01:00
Zakhar Bessarab
da059d1ea1
deployment/docker/make: publish images to multiple repos (#8391)
### Describe Your Changes

Updated `publish-via-docker` task to push images to multiple registries
as defined by `DOCKER_REGISTRIES`. By default, publish pushes images to
both docker.io and quay.io. It is possible to choose a single repo by
overriding `DOCKER_REGISTRIES`, for example: `DOCKER_REGISTRIES=quay.io
make publish-victoria-logs`

Note that `package-via-docker` task is not using multiple registries as
there is usually little sense in building same image with different tags
for local use.

See: https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4116
### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2025-02-27 11:49:50 +04:00
f41gh7
ea6ed4232a
deployment/docker: update Go builder from 1.23.6 to 1.24.0
Changes: https://github.com/golang/go/issues?q=milestone%3AGo1.24
Release notes: https://tip.golang.org/doc/go1.24

Signed-off-by: f41gh7 <nik@victoriametrics.com>
2025-02-26 19:14:49 +01:00
Andrii Chubatiuk
b27e9437a3
app/vminsert: properly apply relabelingConfig for scrapped metrics
Due to regression introduced at v1.108.0 release with the following commit:
https://github.com/VictoriaMetrics/VictoriaMetrics/pull/7399/files#diff-0f7ec2fdf4bb6e94f786c37f19ec837995eca8c61529c3f3436ae332a4a93a5bR56

VictoriaMetrics single-node started to ignore global relabeling configuration.

 This commit properly checks if relabeling is needed.

Related issue:
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/8389
2025-02-26 16:05:26 +01:00
Zhu Jiekun
3d3480140c
lib/storage: properly cache extDB metricsID on search error
Previously, if indexDB search failed for some reason during search at previous indexDB (aka extDB), VictoriaMetrics stored empty search result at cache. It could cause incorrect search results at subsequent requests.

 This commit checks search error and stores request results only on success. 

Related issue:
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/8345
2025-02-26 15:48:25 +01:00
Aliaksandr Valialkin
ae5e28524e
lib/logstorage: do not treat a string with leading zeros as a number at tryParseUint64
The "00123" string shouldn't be treated as 123 number.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/8361
2025-02-25 22:10:36 +01:00
Aliaksandr Valialkin
b484112f22
docs/VictoriaLogs/CHANGELOG.md: fix link to elasticsearch data ingestion docs 2025-02-25 21:31:16 +01:00
Aliaksandr Valialkin
1da04db33a
deployment/docker: update VictoriaLogs Docker image tag from v1.13.0-victorialogs to v1.14.0-victorialogs
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.14.0-victorialogs
2025-02-25 21:24:42 +01:00
Aliaksandr Valialkin
ef2415e0c5
docs/VictoriaLogs/CHANGELOG.md: cut v1.14.0-victorialogs release 2025-02-25 21:19:58 +01:00
Aliaksandr Valialkin
0de7c8987c
app/vlselect/logsql: follow-up for 521af4d7a3
The ts variable must be initialized to the q.GetTimestamp() for every processed row inside the `for range timestamps` loop.

The original bug has been introduced in the commit 42c21ff671

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/8312
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/8385
2025-02-25 21:07:54 +01:00
Zhu Jiekun
521af4d7a3
fix: [stats_query_range] fix inconsistent result caused by shared variable (#8385)
### Describe Your Changes

Previously, `timestamp` variable was shared by goroutines and modified
during `writeBlock`, verified in
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/8312.

This pull request create new variable within each goroutine to avoid
race condition.

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
- [ ]  Test on sandbox.

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2025-02-25 20:56:03 +01:00
Aliaksandr Valialkin
1f75e5bb59
lib/logstorage: optimize common regex filters generated by Grafana
For example, `field:~".+"`, `field:~".*"` or `field:""`

Replace such filters to faster ones. For example, `field:~".*"` is replaced with `*`,
while `field:~".+"` is replaced with `field:*`.
2025-02-25 20:34:28 +01:00
Aliaksandr Valialkin
fef2cb9dc7
lib/regexutil: speed up Regex.MatchString for ".*" 2025-02-25 20:34:28 +01:00
Roman Khavronenko
72bee659f5
docs: point github badge to the main action for master branch (#8388)
Before, badge could have shown the latest build result. Even for open
PRs. Now, it shows the latest build result for `master` branch.

See
https://docs.github.com/en/actions/monitoring-and-troubleshooting-workflows/monitoring-workflows/adding-a-workflow-status-badge

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2025-02-25 18:59:05 +01:00
hagen1778
49fff7989b
docs: fix link to release for search.disableImplicitConversion feature
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2025-02-25 18:54:44 +01:00
Aliaksandr Valialkin
82cdcec6c6
lib/logstorage: run make fmt after 30974e7f3f 2025-02-25 18:37:40 +01:00