Commit graph

10249 commits

Author SHA1 Message Date
Zakhar Bessarab
dc43c73ae0
lib/storage/search_options: do not use ref to atomic as it is not needed
`searchOptions` are always passed by ref, so there is no need to pass atomic.Uint64 by ref as well.

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2025-03-12 19:50:44 +04:00
Zakhar Bessarab
3caecce812
lib/storage/search_options: address review feedback
- remove pooling for `searchOptions`
- update metric to use `path` as label

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2025-03-12 19:50:44 +04:00
Zakhar Bessarab
62c2896c9c
lib/storage/search_options: use summaries for tracking of read metricIDs
Use summaries in order to reduce cardinality generated by these metrics.
Also update label to include source of metric. Include request path for cases when it is guaranteed to be used by a single endpoint, use generic name in other cases ("search" - used by both "/query" and "/query_range", "search_metric_names" used by "/series" and graphite tag handlers).

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2025-03-12 19:50:44 +04:00
Zakhar Bessarab
e6e106b02c
{lib,app}/vmstorage: move metrics implementation to storage
This will reduce difference between single-node and cluster versions. Also, this will be a good starting point for https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7154

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2025-03-12 19:50:44 +04:00
Zakhar Bessarab
deb5836c92
{lib,app}/vmstorage: added metrics to show number of series read per query
Added metrics to track number of series read per individual query. Based on checking number of metricIDs fetched when performing search.

See: https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7029

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2025-03-12 19:50:39 +04:00
Aliaksandr Valialkin
0c4b1143a3
Makefile: update golangci-lint from v1.64.5 to v1.64.7
See https://github.com/golangci/golangci-lint/releases/tag/v1.64.7
2025-03-12 16:35:38 +01:00
Aliaksandr Valialkin
034a456a41
app/vlinsert: follow-up for 67f8fa66ed
- Properly handle negative timestamps (e.g. timestamps before 1970-01-01)

- Optimize parsing floating-point timestamps by eliminating the memory allocation
  needed for returning an error from strconv.ParseInt. Instead, check whether the string contains a dot,
  and then parse it as a floating-point number.

- Add tests for ParseUnixTimestamp function.

- Make the code easier to understand and maintain by removing unneeded generic function toNano().

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/8470
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/8472
2025-03-12 16:35:37 +01:00
Aliaksandr Valialkin
42be7236d5
docs/VictoriaLogs/CHANGELOG.md: move the description of the fix for the proper OpenTelemetry attributes conversion into JSON into the correct place
The bugfix isn't released yet, so move it from v1.15.0-victorialogs release to the tip.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/8384
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/8387

This is a follow-up for 26fba57cfa
2025-03-12 15:57:39 +01:00
Aliaksandr Valialkin
ca65aa1cce
lib/logstorage: properly parse floating-point numbers with leading zeroes in fractional part
Parsing for floating-point numbers with leading zeroes such as 1.023, 1.00234 has been broken
in the commit ae5e28524e .

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/8464
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/8361
2025-03-12 15:29:21 +01:00
Emre Yazıcı
0fe01941c3
app/vmalert: add vmalert_alerts_send_duration_seconds metric (#8468)
### Describe Your Changes

Add `vmalert_alerts_send_latency_seconds` metric for
alertmanager.notifier.

To measure the time for alertmanager calls to send alerts per notifier.
This is needed to see the latency for each notifier from vmalert calls.

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Signed-off-by: emreya <e.yazici1990@gmail.com>
Co-authored-by: Hui Wang <haley@victoriametrics.com>
Co-authored-by: Roman Khavronenko <hagen1778@gmail.com>
(cherry picked from commit cfd2c6e5e7)
2025-03-12 14:32:01 +01:00
alicja-karasiewicz
5467d68954
feat: make topN limit configurable from CLI
### Describe Your Changes

Implement changes mentioned in #6898 

Allow the administrator to specify the limit of returned TSDB series in
`/api/v1/status/tsdb` by making a TopN limit configurable from CLI.

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Signed-off-by: alicja-karasiewicz <alicja.karasiewicz@allegro.com>
Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
Co-authored-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2025-03-12 11:30:17 +04:00
Nikolay
3a3d948a82
docs: update metric names stats description (#8483)
* add version since feature is available
* add cluster endpoint paths

### Describe Your Changes

Please provide a brief description of the changes you made. Be as
specific as possible to help others understand the purpose and impact of
your modifications.

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

Signed-off-by: f41gh7 <nik@victoriametrics.com>
(cherry picked from commit 11436d5f00)
2025-03-12 11:02:11 +04:00
Evgeny
c223aade17
lib/promscrape: use original job name as scrapePool value in targets api (#8457)
### Fix scrapePool name

If in the scrape file, I do some magic and manipulate the job name then
Prometheus will show scrapePool as the original job name in the targets
API, but vmagent will set it to the final value which is wrong.
example
```
job: consul-targets
...

- source_labels: [ __meta_consul_service ]
      regex: (\w+)[_-]exporter
      target_label: job
      replacement: $1
```

curl to prom API will show
`"scrapePool": "consul-targets",`
vmagent:
`""scrapePool": "node",`

before changes:
```
curl -s 'http://localhost:8429/api/v1/targets' | jq -r '.data.activeTargets[].scrapePool'| sort|uniq
blackbox
pgbackrest
postgres
```
after changes
```
curl -s 'http://localhost:8429/api/v1/targets' | jq -r '.data.activeTargets[].scrapePool'| sort|uniq
blackbox
consul-targets
```

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Co-authored-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 486b9e1c64)
2025-03-11 13:13:41 +01:00
Roman Khavronenko
a7577f1f49
vendor: bump go-control-plane/envoy to v1.32.4
Solves the following error:
verifying github.com/envoyproxy/go-control-plane/envoy@v1.32.3/go.mod:
checksum mismatch

See https://github.com/envoyproxy/go-control-plane/issues/1083

(cherry picked from commit 18d6c715ac)
2025-03-11 13:13:23 +01:00
hagen1778
afbdbc6a3a
docs: order releases in changelog by their version
Ordering changes by release versions enhances the searchability
of the documentation. For example, tracking which release got
the bugfix becomes easier if releases are already sorted.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 6a1c70115a)
2025-03-11 10:13:25 +01:00
Alexander Marshalov
473d346be4
vmcloud docs: information about new APIs in Cloud Public API: cloud providers, regions, tiers, deployments and access tokens. (#8442)
…

(cherry picked from commit 9007a4803c)
2025-03-11 10:13:25 +01:00
Jose Gómez-Sellés
f218cacfea
docs/cloud: add account management section (#8467)
### Describe Your Changes

This PR updates the documentation by removing old assets and adding the
user management chapter, divided in different sections.

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

(cherry picked from commit 0873d1d8ab)
2025-03-11 10:13:24 +01:00
Aliaksandr Valialkin
57eb87947e
deployment: update Go builder from Go1.24.0 to Go1.24.1
See https://github.com/golang/go/issues?q=milestone%3AGo1.24.1+label%3ACherryPickApproved

(cherry picked from commit edecc433ff)
2025-03-11 10:13:24 +01:00
Artem Fetishev
6a48fd49de
docs: update vm apps versions to the v1.113.0 release
Signed-off-by: Artem Fetishev <rtm@victoriametrics.com>
2025-03-10 15:45:17 +01:00
Artem Fetishev
e0a16e9a87
docs/changelog: mention LTS releases
Signed-off-by: Artem Fetishev <rtm@victoriametrics.com>
2025-03-10 14:47:03 +01:00
Naveen
63ba8fc149
docs: update vlogs README.md (#8460)
### Describe Your Changes

Fixed the typo in the documentation. Updated `Ir provides` to `It
provides`

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

(cherry picked from commit 179c530095)
2025-03-10 13:54:23 +01:00
Andrii Chubatiuk
394654c127
lib/streamaggr: fixed streamaggr panic (#8471)
### Describe Your Changes

fixes #8469

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

(cherry picked from commit c174a046e2)
2025-03-10 13:54:23 +01:00
Andrii Chubatiuk
5a05788719
app/vlinsert: support floats for elasticseach timestamps (#8472)
### Describe Your Changes

fixes https://github.com/VictoriaMetrics/VictoriaMetrics/issues/8470

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 67f8fa66ed)
2025-03-10 13:54:23 +01:00
Artem Fetishev
15ba3ae447
docs/CHANGELOG.md: cut v1.113.0
Signed-off-by: Artem Fetishev <rtm@victoriametrics.com>
2025-03-07 15:02:30 +01:00
Artem Fetishev
b299cf8767
make docs-update-version 2025-03-07 14:52:54 +01:00
Artem Fetishev
cb94d05ae1
app/{vmselect,vlselect}: run make vmui-update vmui-logs-update
Signed-off-by: Artem Fetishev <rtm@victoriametrics.com>
2025-03-07 14:20:19 +01:00
hagen1778
c3e0bbaa88
docs: add available release mark to vmalert chaining groups
Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit b131c3bc22)
2025-03-07 14:06:26 +01:00
f41gh7
d50b963add
lib/storage: reject downsampling rules with zero interval configuration
Using zero interval for downsampling rules is not useful and caused a panic when performing validation of intervals.

Reject such rules during parsing in order to highlight incorrect usage and prevent panics.

Related issue:
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/8454
---------
Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2025-03-07 13:27:52 +01:00
Artem Fetishev
dbde5b47df
lib/metricnamestats: follow-up after b85b28d30a: Fix flaky integration tests
Signed-off-by: Artem Fetishev <rtm@victoriametrics.com>
2025-03-07 12:17:04 +01:00
hagen1778
036a09d71b
docs: change #tip changes order to reflect importance
Put more important features first in the list.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 021c2552dd)
2025-03-07 10:49:37 +01:00
hagen1778
c0ae0d8165
docs: restore accidentally dropped changelog line
Line about `$__interval` was accidentally dropped in
b85b28d30a (diff-6564e3f60c3a7942189fe87a0c8f02e0f9841a71f914d64cd5487eb8b23ad66a)

The order was changed intentionally, so this commit could be cherry-picked
to cluster branch.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit f7e1c430bb)
2025-03-07 10:00:29 +01:00
Hui Wang
9435d02102
vmalert: allow chaining groups with eval_offset (#8402)
address https://github.com/VictoriaMetrics/VictoriaMetrics/issues/860,
see
https://github.com/VictoriaMetrics/VictoriaMetrics/blob/change-evaloffset-behavior/docs/vmalert.md#chaining-groups

Also related to
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/8154

(cherry picked from commit e8e2ef54a0)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2025-03-07 10:00:26 +01:00
Zakhar Bessarab
dc7a953dd3
docs/victoria-logs/data-ingestion/promtail: fix typo (#8451)
### Describe Your Changes

Please provide a brief description of the changes you made. Be as
specific as possible to help others understand the purpose and impact of
your modifications.

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
(cherry picked from commit fd7b016c5b)
2025-03-07 11:03:52 +04:00
f41gh7
e24a9d3053
lib/metricnamestats: follow-up after b85b28d30a
* properly save state for cross-device mount points
* properly check empty state for tracker

Signed-off-by: f41gh7 <nik@victoriametrics.com>
2025-03-06 23:18:42 +01:00
Nikolay
773b8b0b28
lib/storage: add tracker for time series metric names statistics
This feature allows to track query requests by metric names. Tracker
state is stored in-memory, capped by 1/100 of allocated memory to the
storage. If cap exceeds, tracker rejects any new items add and instead
registers query requests for already observed metric names.

This feature is disable by default and new flag:
`-storage.trackMetricNamesStats` enables it.

  New API added to the select component:

* /api/v1/status/metric_names_stats - which returns a JSON
object
    with usage statistics.
* /admin/api/v1/status/metric_names_stats/reset - which resets internal
    state of the tracker and reset tsid/cache.

   New metrics were added for this feature:

  * vm_cache_size_bytes{type="storage/metricNamesUsageTracker"}
  * vm_cache_size{type="storage/metricNamesUsageTracker"}
  * vm_cache_size_max_bytes{type="storage/metricNamesUsageTracker"}

  Related issue:
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4458
---------

Signed-off-by: f41gh7 <nik@victoriametrics.com>
Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
2025-03-06 22:10:41 +01:00
Zakhar Bessarab
dea3eb20cb
app/vmselect/promql: fix panic with using @ with series which is not present at the start of the query (#8445)
### Describe Your Changes

Previously, "selector @ another_selector" assumed that
"another_selector" metric is supposed to exist since "start" used in the
query.

If the query was evaluated in the following case (timestamps):
- start - 2, end - 10
- "another_selector" 5,6,7,8,9,10
- "selector" The resulting "at" timestamp would be taken from NaN (as
`int64(NaN * 1000)`), causing a panic or invalid behavior later.

Note that type cast of `NaN` to int64 is also platform-dependent, so
value of `int64(math.NaN() * 1000)` can produce `0` or max int64 on
different platforms and versions of Go.

This commit changes this and checks for the first non-NaN value. This
makes it easier to use for users as series are not always aligned and
returning an error in this case would disallow using this for some time
ranges.

See: https://github.com/VictoriaMetrics/VictoriaMetrics/issues/8444

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 7dfaef9088)
2025-03-06 16:42:51 +01:00
Dmytro Kozlov
e2ce76ecf7
vendore: bump metricsql ot v0.84.1 (#8450)
### Describe Your Changes

Updated MetricsQL dependency to v0.84.1

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/8435

### Checklist

The following checks are **mandatory**:

- [X] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: Zhu Jiekun <jiekun@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 75601c2d9a)
2025-03-06 15:18:45 +01:00
hagen1778
fbec45af0c
docs/changelog: fix metric name in changelog for vlogs request duration
Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit bcbe5e80b3)
2025-03-06 15:18:44 +01:00
f41gh7
dafe7c35a8
app/vmgateway properly handle trailing slash when applying rate limiter
Previously, the trailing slash was removed and caused an incorrect redirect path when visiting VMUI.

 This commit leaves it as is. Also it applies minor refactoring to url formatting.

Related issue:
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/8439
2025-03-05 18:43:19 +01:00
Andrii Chubatiuk
c72d5690cc
lib/protoparser/opentelemetry: properly marshal nested attributes into JSON
Previously, opentelemetry attribute parsed added extra field names according to 
golang JSON parser spec for structs:

```
struct AnyValue{
 StringValue string
}
```
 Was serialized into:
```
{"StringValue": "some-string"}
```
 While opentelemetry-collector serializes it as
```
"some-string"
```

 This commit changes this behaviour it makes parses compatible with opentelemetry-collector format. See test cases for examples.

Related issue:
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/8384
2025-03-05 18:38:25 +01:00
hagen1778
8eaeeb9e85
docs/changelog: fix formatting of update notes
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2025-03-05 18:38:24 +01:00
Max Kotliar
235b453928
docs/vmalert: fix available from version (#8433)
### Describe Your Changes

Fix available from version hint. The feature was introduced in
[v1.91.0](https://docs.victoriametrics.com/changelog/changelog_2023/#v1910).

I noticed that the sentence uses both `{{% available_from "v1.91.0" %}}`
and a manual reference like `starting from
[v1.91](https://docs.victoriametrics.com/changelog/#v1910)`. Does {{%
available_from %}} fully supersede the manual changelog reference, and
should the later be removed? Or should\could both be used together?

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

(cherry picked from commit e11f5eda1c)
2025-03-05 14:42:35 +01:00
Andrii Chubatiuk
2807191dae
docs: remove VictoriaMetrics prefix from anomaly detection menu items titles (#8427)
### Describe Your Changes

Removed VictoriaMetrics prefix from anomaly detection menu items

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

(cherry picked from commit bbdb650f2f)
2025-03-05 14:42:35 +01:00
Jose Gómez-Sellés
e7136c6ac4
Add features and guides to VMCloud docs (#8373)
### Describe Your Changes

This PR adds the remaining subsections for the get started part. Some
content is taken from the product page.
Ideally, the get-started page would have some cards instead of raw
links. We should explore that.

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

(cherry picked from commit 400101c674)
2025-03-05 14:42:35 +01:00
Zhu Jiekun
c6caf7e7bf
app/vmgateway: properly ratelimit for ingestion path
Commit cd39df1 introduced regression, which caused any write path related limits to be ignored.

This commit fixes match typo and adds check to prevent such kind of regression in future.
2025-03-04 18:42:48 +01:00
Aliaksandr Valialkin
be8ed94b42
docs/VictoriaLogs/README.md: mention about JSONBench benchmark results in the benchmarks section
See https://docs.victoriametrics.com/victorialogs/#benchmarks
2025-03-04 18:35:58 +01:00
Fred Navruzov
598a2a9b0a
docs/vmanomaly: release v1.20.0 (#8422)
### Describe Your Changes

> ⚠️ Even if approved, please don't merge it on my behalf, I
still may apply a couple of re-phrasings before merging it on Monday
03.03.2025

- Aligned `vmanomaly` docs with release v1.20.0
- Re-structured root page of anomaly detection docs for clarity
- Added several sections to FAQ, e.g. on how to incorporate domain
knowledge into anomaly detection configs

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2025-03-04 18:35:57 +01:00
hagen1778
25b5528f67
app/vmui: update error message for no matched rules
Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit fc5d495900)
2025-03-03 17:02:25 +01:00
Yury Molodov
c574a11ca4
vmui: fix infinite loader on downsampling page (#8428)
### Describe Your Changes

This PR fixes an issue where the Downsampling filters debug page would
get stuck in an infinite loading state when labels had no matches. Now,
the case is properly handled.

Related issue: #8339

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 974c094a52)
2025-03-03 17:02:25 +01:00
Max Kotliar
31685c919f
docs/downsampling: fix terminology. replace "deduplication" with "downsample". (#8421)
### Describe Your Changes

I guess a "downsampling" term is more appropriate than "deduplication"
in the downsampling paragraph.

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

(cherry picked from commit 649f4d5e00)
2025-03-03 14:14:58 +01:00