Commit graph

5469 commits

Author SHA1 Message Date
Aliaksandr Valialkin
1ad776f150
docs/VictoriaLogs/CHANGELOG.md: cut v1.10.0-victorialogs release 2025-02-12 22:15:41 +01:00
Aliaksandr Valialkin
66b5c619fa
app/vlinsert: add a link to the pull request at systemd repository, which enables compression support
This should simplify maintenance of this code in the future.
While at it, clarify the change at the docs/VictoriaLogs/CHANGELOG.md.

This is a follow-up commit for 3c9f9f49b0.
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/8264
Updates https://github.com/systemd/systemd/pull/34822
2025-02-12 22:10:05 +01:00
Aliaksandr Valialkin
7b38f7b5ef
lib/logstorage: refactor bucketing code
1. Use distinct code paths for blockResult.getValues() and blockResult.getValuesBucketed().
   This should simplify debugging and maintenance of the resulting code.

2. Do not load column values if all the values in the block fit the same bucket.
   Use blockResultColumn.minValue and blockResultColumn.maxValue for determining whether
   column values must be loaded via blockResultColumn.getValuesEncoded().
   This signiciantly improves performance for big buckets, which cover all the column
   values in a block.

3. Properly calculate buckets for negative values.

4. Properly adjust weekly buckets by Monday.
2025-02-12 21:47:46 +01:00
Yury Molodov
537ea61271
vmui/logs: fix autocomplete ignoring selected tenant (#8265)
### Describe Your Changes

Fix autocomplete not passing `AccountID` and `ProjectID` headers when
fetching suggestions in VictoriaLogs UI.

Related: #8042

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 778bcea30d)
2025-02-12 12:47:43 +01:00
Github Actions
2ca33b1cec
Automatic update helm docs from VictoriaMetrics/helm-charts@ab4b3d4 (#8278)
Automated changes by
[create-pull-request](https://github.com/peter-evans/create-pull-request)
GitHub action

Signed-off-by: Github Actions <133988544+victoriametrics-bot@users.noreply.github.com>
Co-authored-by: AndrewChubatiuk <3162380+AndrewChubatiuk@users.noreply.github.com>
(cherry picked from commit d4632aceac)
2025-02-12 12:47:42 +01:00
Andrii Chubatiuk
e9ca6eaaf0
app/vlinsert: add OpenTelemetry ingested logs trace_id and span_id
This commit parses additional optional fields from OpenTelemetry logs protocol.

Related issue:

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/8255
(cherry picked from commit 3a27073634)
2025-02-12 12:47:42 +01:00
Andrii Chubatiuk
6254810e12
app/vlinsert: add journald content negotiation, which enables compression on a client
Adding Accept-Encoding response header to support content negotiation,
which was introduced in [this
PR](https://github.com/systemd/systemd/pull/34822) and enables
compression on journald

(cherry picked from commit 3c9f9f49b0)
2025-02-12 12:47:42 +01:00
Phuong Le
c2f3aa291a
lib/protoparser/clusternative: properly return error from parsing callback
Previously the `callbackErr` is silently ignored in clusternative parser, which is used at vminsert for parsing clusterNative requests and at vmstorage for parsing vminsert requests.

 This commit  fixes that by properly return callbackError after reading all block metrics. This aligns
with other parsers in `lib/protoparser`.
2025-02-12 10:32:28 +01:00
Yury Molodov
89bf146165
vmui: update default step calculation for instant queries (#8266)
### Describe Your Changes

Updated default `step` calculation for `Table` and `JSON` views.

- When switching to the `Table` or `JSON` view, the default step is now
automatically set to `end - start`, where `end` and `start` correspond
to the selected time range.
- If the user has manually set a step value, it will remain unchanged
when switching views or changing the time range.
- The UI now explicitly indicates when the step is automatically
calculated.

<details> <summary>Demo</summary>

https://github.com/user-attachments/assets/2540de24-36ed-4764-a047-1c6b48a80ed4

</details>

**Note:** These views use the
[`/api/v1/query`](https://docs.victoriametrics.com/keyconcepts/#instant-query)
endpoint for instant queries.

Related issue: #8240

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 71160e64b1)
2025-02-11 23:02:22 +01:00
hagen1778
c49cc43761
docs: add year 2025 changelog
The 2025 changelog was in the parent directory - a default page
that opens for /changelog.
But it seems like it was confusing for users, so add 2025 that mirrors
/changelog page.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 5f9257a6d2)
2025-02-11 23:02:21 +01:00
hagen1778
afc6636e0f
docs: fix broken changelog link
Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 7f5d41dda7)
2025-02-11 23:02:21 +01:00
hagen1778
179f20d7bc
docs: separate security and feature changes
Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 73da21c1ea)
2025-02-11 23:02:21 +01:00
Phuong Le
52fd89e426
docs: search.lookback-delta -> query.lookback-delta (#8280)
(cherry picked from commit 23147c8339)
2025-02-11 23:02:20 +01:00
Evgeny
02d0da6281
fix race where the same list is used from 2 goroutines
### Describe Your Changes

There is an issue described in #8040 this should fix it
- The alerts slice is shared across multiple goroutines (since send() is
called concurrently).
- `alerts[:0]` creates a new slice header, but it still references the
same underlying array.
- Appending (append(alertsToSend, a)) modifies the underlying array,
which may also be used by another goroutine.
Solution: Use a separate slice copy for each goroutine.

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Co-authored-by: Evgeny Kuzin <evgeny@hudson-trading.com>
Co-authored-by: Hui Wang <haley@victoriametrics.com>
(cherry picked from commit 4e9fb93acc)
2025-02-10 21:41:37 +04:00
Zakhar Bessarab
638b9b1a6b
{docs,deployment}: update references to the latest release (#8273)
### Describe Your Changes

- Update references to the latest release - v1.110.1
- Update links to the latest LTS releases

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
(cherry picked from commit 04943907e3)
2025-02-10 21:33:06 +04:00
Aliaksandr Valialkin
1c2bc3453d
deployment/docker: update VictoriaLogs image from v1.9.0-victorialogs to v1.9.1-victorialogs
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.9.1-victorialogs

(cherry picked from commit d142d43ae9)
2025-02-10 21:33:04 +04:00
Aliaksandr Valialkin
b4a649903a
app/vlselect/vmui: run make vmui-logs-update after 6a8fa799c6
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/8177
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/7778
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/8178
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/8176
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/8174

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/8153
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/8152

(cherry picked from commit 4600724985)
2025-02-10 21:32:54 +04:00
Zakhar Bessarab
aa6303b84d
docs/changelog: mention LTS releases, update release date
Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
(cherry picked from commit 6ed31b9180)
2025-02-10 21:32:52 +04:00
Aliaksandr Valialkin
c4b5cbf1be
deployment/docker: update VictoriaLogs docker image from v1.8.0-victorialogs to v1.9.0-victorialogs
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.9.0-victorialogs

(cherry picked from commit 00fb218269)
2025-02-10 21:32:50 +04:00
Aliaksandr Valialkin
ad5f865b2b
docs/VictoriaLogs/CHANGELOG.md: cut v1.9.0-victorialogs release
(cherry picked from commit aa0ac1d0ed)
2025-02-10 21:32:48 +04:00
Aliaksandr Valialkin
2d7263dd5f
lib/logstorage: properly compare RFC3339 timestamps with sub-second precision in lessString()
Previously RFC3339 timestamps with sub-second precision could be incorrectly compared by lessString().
For example, 2025-01-20T10:20:30.1Z was incorrectly treated as smaller than 2025-01-20T10:20:30.09Z,
because the first timestamp has smaller decimal number after the last dot than the second timestamp.

(cherry picked from commit 81d359507d)
2025-02-10 21:32:45 +04:00
Aliaksandr Valialkin
237a3d60b0
app/vlinsert: continue parsing JSON lines in the input stream after parse errors
Previosly the parsing of the input stream was stopped after the first parse error.
This isn't what most users expect when ingesting JSON lines in a stream where some JSON lines may be invalid.

(cherry picked from commit ebac07bcf6)
2025-02-10 21:32:31 +04:00
Github Actions
ab0511ee8b
Automatic update helm docs from VictoriaMetrics/helm-charts@0fe31fd (#8256)
Automated changes by
[create-pull-request](https://github.com/peter-evans/create-pull-request)
GitHub action

Signed-off-by: Github Actions <133988544+victoriametrics-bot@users.noreply.github.com>
Co-authored-by: zekker6 <1367798+zekker6@users.noreply.github.com>
(cherry picked from commit 92bfb7ba15)
2025-02-10 21:32:09 +04:00
Aliaksandr Valialkin
60a5ccf280
deployment/docker: update Go builder from Go1.23.5 to Go1.23.6
See https://github.com/golang/go/issues?q=milestone%3AGo1.23.6+label%3ACherryPickApproved
2025-02-09 22:51:00 +01:00
Aliaksandr Valialkin
7690873787
app/vlinsert: accept timestamps with microsecond and nanosecond precision at _time field 2025-02-09 22:50:59 +01:00
Zakhar Bessarab
f01f63a6bb
docs: cut v1.111.0
Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2025-02-07 19:42:28 +04:00
Zakhar Bessarab
493654ecca
docs: update refs to new release
Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
(cherry picked from commit 80a5f6863b)
2025-02-07 18:53:24 +04:00
Aliaksandr Valialkin
a6171ca7e2
lib/logstorage: optimize performance for stats, top and uniq pipes a bit
Split unique values (groups) into shards according to the configured concurrency
during processing of the matching rows if the number of unique values exceeds the hardcoded threshold.
Previously this splitting was performed unconditionally at the merge stage when merging independently
calculated per-CPU states into a single state. It is faster to perform the split during rows processing
if the number of unique values is big.

This gives up to 30% perfromance improvements when these pipes are applied to big number of unique values (groups).

(cherry picked from commit 48602a1ae8)
2025-02-07 18:32:30 +04:00
f41gh7
a98163a9e0
app/vmselect/netstorage: stop exposing `vm_index_search_duration_seconds metric
This metric records time spent on search operations in the index.
It was introduced in
[v1.56.0](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.56.0).
However, this metric was used neither in dashboards nor in alerting
rules.
It also has high cardinality because index search operations latency can
differ by 3 orders of magnitude.

See
[example](https://play.victoriametrics.com/select/accounting/1/6a716b0f-38bc-4856-90ce-448fd713e3fe/prometheus/graph/#/cardinality?date=2025-02-05&match=vm_index_search_duration_seconds_bucket&topN=10&focusLabel=).

 Hence, dropping it as unused.

---------
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2025-02-06 13:48:32 +01:00
Aliaksandr Valialkin
851a5636aa
lib/logstorage: properly limit the number of concurrent workers at stats, top and uniq pipes according to the provided options(concurrency=N)
The number of worker shards per each pipe processor is created during query initialization.
This number equals to the `options(concurrency=N)` if this option is set or to the number of available CPU cores.
This means that all the pipes must adhere the given concurrency when passing data blocks
to the next pipe.

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/8201

The bug has been introduced in 0214aa328e
2025-02-06 13:44:35 +01:00
Github Actions
2874ee09ce
Automatic update operator docs from VictoriaMetrics/operator@96519dc (#8235)
Automated changes by
[create-pull-request](https://github.com/peter-evans/create-pull-request)
GitHub action

Signed-off-by: Github Actions <133988544+victoriametrics-bot@users.noreply.github.com>
Co-authored-by: f41gh7 <18450869+f41gh7@users.noreply.github.com>
2025-02-06 13:44:34 +01:00
Nikolay
295d2b9721
app/vmauth: allow to serve internal API and different address
vmauth uses 'lib/httpserver' for serving HTTP requests. This server
unconditionally defines built-in routes (such as '/metrics',
'/health', etc). It makes impossible to proxy `HTTP` requests to  backends with the same routes.
Since vmauth's httpserver matches built-in route and return local
response.

 This commit adds new flag `httpInternalListenAddr` with
default empty value. Which removes internal API routes from public
router and exposes it at separate http server.

For example given configuration disables private routes at `0.0.0.0:8427` address and serves it at `0.0.0.0:8426`:

`./bin/vmauth --auth.config=config.yaml --httpListenAddr=:8427 --httpInternalListenAddr=127.0.0.1:8426`

Related issues:
- https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6468
- https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7345
2025-02-06 13:44:34 +01:00
Joost Buskermolen
824f531490
app/vmselect: expose /-/healthy and /-/ready endpoints on full Prometheus path
This commit improves integration with third-party solutions who rely on non-root
endpoints (i.e. MinIO) when the vmselect path has been specified in the
configured Prometheus URL like:
`http://vmselect.:8481/select/0/prometheus`

Comparable change has been done before
(b885a3b6e9), however only takes care of
the root path. This means endpoints `-/healthy` and `-/ready` are still
not available on full vmselect Prometheus paths, resulting in
unsupported path requests.

This change makes these endpoints available on the full paths like:
`/select/0/prometheus/-/healthy` and `/select/0/prometheus/-/ready`,
thus achieving full Prometheus compatibility for external dependencies.

Related issues:
- https://github.com/minio/console/issues/2829
- https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1833

---

Signed-off-by: Joost Buskermolen <j.buskermolen@cloudmeesters.com>
2025-02-05 17:00:11 +01:00
hagen1778
882db7080b
docs: fix copy&paste typo in change type
Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 553e95e293)
2025-02-05 15:30:06 +01:00
Github Actions
6d6318aa27
Automatic update operator docs from VictoriaMetrics/operator@8192989 (#8207)
Automated changes by
[create-pull-request](https://github.com/peter-evans/create-pull-request)
GitHub action

Signed-off-by: Github Actions <133988544+victoriametrics-bot@users.noreply.github.com>
Co-authored-by: f41gh7 <18450869+f41gh7@users.noreply.github.com>
(cherry picked from commit 46fbb95b16)
2025-02-05 15:29:16 +01:00
Github Actions
bc367f52d4
Automatic update helm docs from VictoriaMetrics/helm-charts@0155ba2 (#8217)
Automated changes by
[create-pull-request](https://github.com/peter-evans/create-pull-request)
GitHub action

Signed-off-by: Github Actions <133988544+victoriametrics-bot@users.noreply.github.com>
Co-authored-by: AndrewChubatiuk <3162380+AndrewChubatiuk@users.noreply.github.com>
(cherry picked from commit fbe592f495)
2025-02-05 15:29:16 +01:00
Artem Fetishev
fd0bd8a67f
lib/storage: fix cardinality limiting for cases when insertion takes fast path (#8218)
### Describe Your Changes

The cardinality limiter in this case does not receive the actual
metricID but some other value found in r.TSID.MetricID and is not
initialized. Depending on the system and/or go runtime implementation,
this value can be 0 or some garbage value (which shouldn't have too wide
a range). Thus, there basically no limit for inserted metricIDs.

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Signed-off-by: Artem Fetishev <rtm@victoriametrics.com>
Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>

(cherry picked from commit 631b736bc2)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2025-02-05 15:29:07 +01:00
Fred Navruzov
2810d83cdd
docs/vmanomaly: fix 404 links (#8219)
### Describe Your Changes

- fixed recently found 404 under `/anomaly-detection/` docs section
- get rid of remaining .html / .index.html endings in links under
`/anomaly-detection/` docs section

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

(cherry picked from commit ceefa406cc)
2025-02-05 15:26:19 +01:00
Zakhar Bessarab
323163d6b0
{docs,app/vmctl}: clarify how vm-rate-limit is applied
### Describe Your Changes

Explicitly mention that `--vm-rate-limit` is used for each individual
worker defined by `--vm-concurrency`.

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
(cherry picked from commit 2748681f40)
2025-02-04 18:24:09 +04:00
Zakhar Bessarab
9494ebe7fc
docs/streaming-aggregation: fix typo in metric name
### Describe Your Changes

Fix metric name labels set not being "closed" by `}`.

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
(cherry picked from commit e2cfd351c0)
2025-02-04 13:10:32 +04:00
f41gh7
030c1aed06
app/vmagent/kafka: properly close producer client
At v1.107.0 release with commit 21a7f06fc4beeb6ad32b9f7fd88704ed33674905 was introduced regression.

It prevents kafka producer client from graceful shutdown. This change also removed kafka message headers.

 This commit properly closes client.

 It also addresses the following issues:

* properly add headers to kafka message
* add url_id to the exported metrics, which prevents possible metrics clash if multiple remote write targets have
 the same url and topic name.

Signed-off-by: f41gh7 <nik@victoriametrics.com>
2025-02-03 14:22:07 +01:00
hagen1778
d472464b0e
docs: mention tlast_over_time in influx guide
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/8188

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2025-02-03 14:20:05 +01:00
hagen1778
5dcbdd8f16
docs: fix typo in image name
Follow-up after 35d77a3bed

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2025-02-03 14:20:05 +01:00
Roman Khavronenko
e803b9b68b
metricsql: bump to v0.83.0 (#8141)
metricsql: bump to v0.83.0

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7703

The update also returns an error if metric name is specified twice in
metrics selector.
For example, `foo{__name__="bar"}` is not allowed anymore. It would
successfully parse before
this change, but it won't satisfy the search filter any way. So it had
no sense in supporting this. This is why some test cases were removed.

Signed-off-by: hagen1778 <roman@victoriametrics.com>

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit fa2107bbec)
2025-02-01 22:31:56 +01:00
hagen1778
a6ef648687
app/vmalert: mention that remoteWrite.concurrency depends on CPU
Mnetion explicitly that `remoteWrite.concurrency` deopends on number
of available CPU cores. Updated docs to rm auto-printed default value.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/8151

Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 5561970db0)
2025-02-01 22:31:55 +01:00
Github Actions
cce974bf7d
Automatic update operator docs from VictoriaMetrics/operator@7930b3a (#8202)
Automated changes by
[create-pull-request](https://github.com/peter-evans/create-pull-request)
GitHub action

Signed-off-by: Github Actions <133988544+victoriametrics-bot@users.noreply.github.com>
Co-authored-by: f41gh7 <18450869+f41gh7@users.noreply.github.com>
(cherry picked from commit 4d42e29967)
2025-02-01 22:31:55 +01:00
Roman Khavronenko
13cd76347d
app/vmselect/promql: fix discrepancies when using or binary operator
The change covers various corner cases when using `or` binary operator.
See corresponding issues and pull request here to see the cases:
https://github.com/VictoriaMetrics/VictoriaMetrics/pull/7770

Related issues:
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7759
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7640

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 72837919ae)
2025-02-01 22:31:55 +01:00
Zakhar Bessarab
9c4e2e58bd
docs/release-guide: add a note for vmui build (#8183)
### Describe Your Changes

Please provide a brief description of the changes you made. Be as
specific as possible to help others understand the purpose and impact of
your modifications.

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
(cherry picked from commit 6166149474)
2025-01-31 17:00:48 +04:00
Github Actions
68daa6d0cd
Automatic update Grafana datasource docs from VictoriaMetrics/victorialogs-datasource@de3daa7 (#8196)
(cherry picked from commit f34e82eb9b)
2025-01-31 17:00:46 +04:00
Github Actions
146e28daf3
Automatic update helm docs from VictoriaMetrics/helm-charts@485594e (#8197)
Automated changes by
[create-pull-request](https://github.com/peter-evans/create-pull-request)
GitHub action

Signed-off-by: Github Actions <133988544+victoriametrics-bot@users.noreply.github.com>
Co-authored-by: AndrewChubatiuk <3162380+AndrewChubatiuk@users.noreply.github.com>
(cherry picked from commit 2545082c0f)
2025-01-31 17:00:44 +04:00
Github Actions
864c925dac
Automatic update Grafana datasource docs from VictoriaMetrics/victoriametrics-datasource@35e86ad (#8190)
(cherry picked from commit 676669020f)
2025-01-31 17:00:42 +04:00
Zakhar Bessarab
d25f431bba
docs/victoria-logs/quickstart: use link to docs for chart (#8186)
### Describe Your Changes

Please provide a brief description of the changes you made. Be as
specific as possible to help others understand the purpose and impact of
your modifications.

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
(cherry picked from commit 071db473a7)
2025-01-31 16:59:51 +04:00
Andrii Chubatiuk
5b5c21e377
fixed cloud docs links (#8198)
### Describe Your Changes

fixed some cloud docs links

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

(cherry picked from commit 94ac051647)
2025-01-31 16:59:50 +04:00
Github Actions
ff92c0e322
Automatic update operator docs from VictoriaMetrics/operator@946c30e (#8193)
Automated changes by
[create-pull-request](https://github.com/peter-evans/create-pull-request)
GitHub action

Signed-off-by: Github Actions <133988544+victoriametrics-bot@users.noreply.github.com>
Co-authored-by: f41gh7 <18450869+f41gh7@users.noreply.github.com>
(cherry picked from commit 3091133a36)
2025-01-31 16:59:49 +04:00
Zakhar Bessarab
5c4c20d0e0
app/vmui/make: add a step to enforce license type override
- update license file content
- add a build step to enforce setting correct license type

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2025-01-30 14:35:03 +01:00
f41gh7
6e5c13bd1b
docs/changelog: add recommendation for lts releases upgrade
Signed-off-by: f41gh7 <nik@victoriametrics.com>
2025-01-30 14:30:05 +01:00
hagen1778
77fe40c0e5
docs: fix broken vminsert link in changelog
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2025-01-30 14:30:05 +01:00
Yury Molodov
a412cbe66a
vmui/logs: add _msg to group view field list (#8177)
### Describe Your Changes

add the `_msg` field to the list of fields for the group view, allowing
users to select multiple fields, including `_msg`, for log display.

![image](https://github.com/user-attachments/assets/90336315-8708-424d-9ed8-aa8d9a0d07a9)

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2025-01-29 16:38:57 +01:00
Github Actions
d7b9a5c0b2
Automatic update Grafana datasource docs from VictoriaMetrics/victorialogs-datasource@b532af3 (#8181) 2025-01-29 16:38:57 +01:00
Yury Molodov
952f6ed87a
vmui: refactor clipboard handling with useCopyToClipboard hook (#8178)
### Describe Your Changes

Replaced `await navigator.clipboard.writeText` with `useCopyToClipboard`
hook.
See [this pull
request](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/7778)
for details.

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2025-01-29 16:38:57 +01:00
Yury Molodov
f79f092c9d
vmui/logs: fix dropdown menu in group view settings (#8176)
### Describe Your Changes

Fixed an issue where dropdown menus were not visible in the Group View
settings.
Ref issue: #8153 

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
2025-01-29 16:38:57 +01:00
f41gh7
39e69b103e
app/vmselect: properly cancel long running requests on client connection close
At this time `bufferedwriter` [silently ignores connection close
errors](78eaa056c0/lib/bufferedwriter/bufferedwriter.go (L67)).
It may be very convenient in some situations (to not log such
unimportant errors), but it's too implicit and unsafe for the others.
For example, if you close [export
API](https://docs.victoriametrics.com/#how-to-export-time-series) client
connection in the middle of communication, VictoriaMetrics won't notice
it and will start to hog CPU by exporting all the data into nowhere
until it process all of them. If you'll make a few retries, it will be
effectively a DoS on the server.

This commit replaces this implicit error suppressing with explicit error
handling which fixes the issue with export API.

Issue was introduced at e78f3ac8ac
2025-01-29 16:38:38 +01:00
Yury Molodov
a67d6a578d
vmui/logs: fix transparency for bars in hits chart (#8174)
### Describe Your Changes

Restored transparency for bars in the hits chart to improve visibility.
Ref issue: #8152

| State  | Preview |
|--------|---------|
| Before |
![image](https://github.com/user-attachments/assets/2f95cdd9-0b39-4420-9bf7-011274eadc86)
|
| After |
![image](https://github.com/user-attachments/assets/9123d8e7-e615-41ab-9193-4c50ce262e7f)
|

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2025-01-29 16:34:16 +01:00
Yury Molodov
3419b0f9c8
vmui: add heatmap-to-line chart switch (#8169)
### Describe Your Changes

- Added support for switching from heatmap to line chart (the switch is
displayed only for heatmap data).
- Added a warning for heatmap rendering issues with a button to switch
to a line chart.
- Fixed a chart rendering issue when switching from heatmap to line
chart.

Ref issue: #8057 

<img
src="https://github.com/user-attachments/assets/2efc901a-3dcd-4b5b-aeaa-8f22b2dcedaa"
width="600"/>
<hr/>
<img
src="https://github.com/user-attachments/assets/2594cbb6-8356-4448-9576-0d7da4749256"
width="600"/>
<hr/>
<img
src="https://github.com/user-attachments/assets/ccfeb8b4-7e98-41a7-90f4-ee9dcf971141"
width="600"/>

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
2025-01-29 16:34:15 +01:00
Github Actions
fcae3964a1
Automatic update Grafana datasource docs from VictoriaMetrics/victoriametrics-datasource@0d6e500 (#8180)
(cherry picked from commit 0e08a7a125)
2025-01-29 13:29:23 +01:00
Roman Khavronenko
6c596d2f4d
lib/cgroup: warn users about using fractional CPU quotas (#8175)
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7988

### Describe Your Changes

Please provide a brief description of the changes you made. Be as
specific as possible to help others understand the purpose and impact of
your modifications.

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 13c4324bb5)
2025-01-29 13:29:23 +01:00
Aliaksandr Valialkin
d63a2aa4ab
docs/Articles.md: add a link to https://tanmay-bhat.medium.com/reducing-inter-az-traffic-in-victoriametrics-with-zonekeeper-3bd7e1526796
Thanks to Tanmay Bhat ( https://tanmay-bhat.medium.com/ ) for the great article about VictoriaMetrics and vmagent!

(cherry picked from commit ec64a1fd7c)
2025-01-29 13:29:22 +01:00
Aliaksandr Valialkin
40646a125e
lib/logstorage: ignore logs with too long field names during data ingestion
Previously too long field names were silently truncated. This is not what most users expect.
It is better ignoring the whole log entry in this case and logging it with the WARNING message,
so human operator could notice and fix the ingestion of incorrect logs ASAP.

The commit also adds and updates the following entries to VictoriaLogs faq:

- https://docs.victoriametrics.com/victorialogs/faq/#how-many-fields-a-single-log-entry-may-contain
- https://docs.victoriametrics.com/victorialogs/faq/#what-is-the-maximum-supported-field-name-length
- https://docs.victoriametrics.com/victorialogs/faq/#what-length-a-log-record-is-expected-to-have

These entries are referred at `-insert.maxLineSizeBytes` and `-insert.maxFieldsPerLine` command-line descriptions
and at the WARNING messages, which are emitted when log entries are ignored because of some of these limits
are exceeded.

(cherry picked from commit 3c036e0d31)
2025-01-29 13:29:22 +01:00
Alexander Marshalov
34efc65338
vmcloud docs: actualize parameters description due to changes in v1.108.0 (#6928) (#8172)
vmcloud docs: actualize parameters description due to changes in
v1.108.0 (#6928)

(cherry picked from commit c753f75e18)
2025-01-29 13:29:21 +01:00
f41gh7
9db5ca0147
docs/changelog: mention v1.102.12 and v1.97.17 releases
Signed-off-by: f41gh7 <nik@victoriametrics.com>
(cherry picked from commit 330461c635)
2025-01-29 13:29:21 +01:00
hagen1778
12e44a66b7
docs: re-order changes for better readability
Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 5b18dc0214)
2025-01-29 13:29:21 +01:00
Fred Navruzov
d8ad3310a5
docs/vmanomaly: release v1.19.2 (#8165)
### Describe Your Changes

docs/vmanomaly: release v1.19.2 (patch that addresses some of the bugs
found in 1.19.0)

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Co-authored-by: Mathias Palmersheim <mathias@victoriametrics.com>
2025-01-27 23:10:58 +01:00
Zakhar Bessarab
35ddfc5d62
docs/{vmauth,VictoriaLogs}: add examples of using headers per-user
### Describe Your Changes
- add example a generic example to vmauth docs
- add an multi-tenancy usage example to VictoriaLogs docs
### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
(cherry picked from commit de144899b2)
2025-01-27 20:16:03 +04:00
Aliaksandr Valialkin
d60238d917
lib/storage: open per-month partitions in parallel
This should reduce the time needed for opening the storage with retentions exceeding a few months.

While at at, limit the concurrency of opening partitions in parallel to the number of available CPU cores,
since higher concurrency may increase RAM usage and CPU usage without performance improvements
if opening a single partition is CPU-bound task.

This is a follow-up for 17988942ab
2025-01-27 16:12:25 +01:00
f41gh7
6168ab3b3d
docs/changelog: mention lts releases
Signed-off-by: f41gh7 <nik@victoriametrics.com>
2025-01-27 11:06:12 +01:00
f41gh7
3fc6de40cf
docs: point apps versions to the latest releases
v1.110.0
v1.102.11
v1.97.16

Signed-off-by: f41gh7 <nik@victoriametrics.com>
2025-01-27 11:06:11 +01:00
f41gh7
08404f2169
CHANGELOG.md: cut v1.110.0 release 2025-01-27 11:06:11 +01:00
Aliaksandr Valialkin
c51282d698
lib/logstorage: open per-day partitions in parallel during startup
This significantly reduces startup times when the storage contains large partitions over many days.
2025-01-27 00:35:35 +01:00
Aliaksandr Valialkin
4f9e51e106
lib/logstorage: block_stat pipe: return the path to the part where the block is stored 2025-01-26 22:53:15 +01:00
Aliaksandr Valialkin
a0ca7e9eea
docs/VictoriaLogs: small updates 2025-01-26 22:07:45 +01:00
Aliaksandr Valialkin
c252d2651b
docs/VictoriaLogs/sql-to-logsql.md: pay attention to the fact that stats() pipe at LogsQL has better usability than GROUP BY at SQL 2025-01-26 22:07:44 +01:00
Aliaksandr Valialkin
6a92e3f348
deployment/docker: update VictoriaLogs from v1.7.0-victorialogs to v1.8.0-victorialogs
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.8.0-victorialogs
2025-01-26 22:07:43 +01:00
Aliaksandr Valialkin
96246c0e04
docs/VictoriaLogs/CHANGELOG.md: cut v1.8.0-victorialogs release 2025-01-26 22:06:47 +01:00
Aliaksandr Valialkin
9754f4b298
docs/VictoriaLogs/CHANGELOG.md: move the changes from 87739bbbef to the correct place
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/7750
2025-01-26 22:06:46 +01:00
Yury Molodov
70d7f4aba9
vmui/logs: update hits chart legend (#7750)
### Describe Your Changes

- Added the `fields_limit` parameter for the `hits` query to limit the
number of returned fields. This reduces the response size and decreases
memory usage.

#### Legend Menu:
A context menu has been added for legend items, which includes:  
1. Metric name
2. **Copy _stream name** - copies the full stream name in the format
`{field1="value1", ..., fieldN="valueN"}`.
3. **Add _stream to filter** - adds the full stream value to the current
filter:
   `_stream: {field1="value1", ..., fieldN="valueN"} AND (old_expr)`.  
4. **Exclude _stream from filter** - excludes the stream from the
current filter:
`(NOT _stream: {field1="value1", ..., fieldN="valueN"}) AND (old_expr)`.
5. List of fields with options:  
   - Copy as `field: "value"`.  
   - Add to filter: `field: "value" AND (old_expr)`.  
   - Exclude from filter: `-field: "value" AND (old_expr)`.  
6. Total number of hits for the stream.  

Related issue: #7552

<details>
  <summary>UI Demo - Legend Menu</summary>
  
<img width="400"
src="https://github.com/user-attachments/assets/ee1954b2-fdce-44b4-a2dc-aa73096a5414"/>
<img width="400"
src="https://github.com/user-attachments/assets/19d71f04-c207-4143-a176-c5f221592e3d"/>

</details>

---

#### Legend:
1. Displays the total number of hits for the stream.  
2. Added hints below the legend for total hits and graph interactions.
3. Click behavior is now the same as in other vmui charts:  
   - `click` - shows only the selected series.  
   - `click + ctrl/cmd` - hides the selected series.  
   
<details>
  <summary>UI Demo - Legend</summary>

before: 
<img
src="https://github.com/user-attachments/assets/18270842-0c39-4f63-bcda-da62e15c3c73"/>

after:
<img
src="https://github.com/user-attachments/assets/351cad3a-f763-4b1d-b3be-b569b5472a7c"/>
  
</details>

---

#### Tooltip:  
1. The `other` label is moved to the end, and others are sorted by
value.
2. Values are aligned to the right.  
3. Labels are truncated and always shown in a single line for better
readability; the full name is available in the legend.

<details>
  <summary>UI Demo - Tooltip</summary>
  
| before     | after        |
|----------|----------|
| <img
src="https://github.com/user-attachments/assets/adccff38-e2e6-46e4-a69e-21381982489c"/>
| <img
src="https://github.com/user-attachments/assets/81008897-d816-4aed-92cb-749ea7e0ff1e"/>
|
  
</details>

---

#### Group View (tab Group):
Groups are now sorted by the number of records in descending order.

<details>
  <summary>UI Demo - Group View</summary>

before: 
<img width="800"
src="https://github.com/user-attachments/assets/15b4ca72-7e5d-421f-913b-c5ff22c340cb"/>

after:
<img width="800"
src="https://github.com/user-attachments/assets/32ff627b-6f30-4195-bfe7-8c9b4aa11f6b"/>
  
</details>
2025-01-26 22:06:13 +01:00
Aliaksandr Valialkin
93ea7c5cf6
docs/VictoriaLogs/CHANGELOG.md: typo fix after ad6c587494: ignore_global_time_range -> ignore_global_time_filter 2025-01-26 22:05:37 +01:00
Aliaksandr Valialkin
fea934936b
lib/logstorage: properly propagate extra filters to all the subqueries
The purpose of extra filters ( https://docs.victoriametrics.com/victorialogs/querying/#extra-filters )
is to limit the subset of logs, which can be queried. For example, it is expected that all the queries
with `extra_filters={tenant=123}` can access only logs, which contain `123` value for the `tenant` field.

Previously this wasn't the case, since the provided extra filters weren't applied to subqueries.
For example, the following query could be used to select all the logs outside `tenant=123`, for any `extra_filters` arg:

    * | union({tenant!=123})

This commit fixes this by propagating extra filters to all the subqueries.

While at it, this commit also properly propagates [start, end] time range filter from HTTP querying APIs
into all the subqueries, since this is what most users expect. This behaviour can be overriden on per-subquery
basis with the `options(ignore_global_time_filter=true)` option - see https://docs.victoriametrics.com/victorialogs/logsql/#query-options

Also properly apply apply optimizations across all the subqueries. Previously the optimizations at Query.optimize()
function were applied only to the top-level query.
2025-01-26 22:05:05 +01:00
Aliaksandr Valialkin
9c23c61a2e
docs/VictoriaLogs/LogsQL.md: show how to unroll the returned histogram buckets into separate rows at histogram pipe docs 2025-01-24 16:39:47 +01:00
Aliaksandr Valialkin
fb68ba3745
docs/VictoriaLogs/sql-to-logsql.md: show how to substitute complex SQL query with top pipe 2025-01-24 16:39:46 +01:00
Nikolay
cab5cf3c4c
app/vmselect: fixes panic data race at query tracing
Previously, NewChild elements of querytracer could be referenced by concurrent
storageNode goroutines. After earlier return ( if search.skipSlowReplicas is set), it is
possible, that tracer objects could be still in-use by concurrent workers.
  It may cause panics and data races. Most probable case is when parent tracer is finished, but children
still could write data to itself via Donef() method. It triggers read-write data race at trace
formatting.

This commit adds a new methods to the querytracer package, that allows to
create children not referenced by parent and add it to the parent later.

 Orphaned child must be registered at the parent, when goroutine returns. It's done synchronously by the single caller  via finishQueryTracer call. 
If child didn't finished work and reference for it is used by concurrent goroutine, new child must be created instead with
context message.
 It prevents panics and possible data races.

Related issue:
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/8114

---------

Signed-off-by: f41gh7 <nik@victoriametrics.com>
Co-authored-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
2025-01-24 13:55:32 +01:00
Roman Khavronenko
f52736eb98
docs: add avaialbe_version notion to new -search.maxDeleteDuration (#8142)
Follow-up for
4574958e2e

### Describe Your Changes

Please provide a brief description of the changes you made. Be as
specific as possible to help others understand the purpose and impact of
your modifications.

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2025-01-24 13:46:03 +01:00
Yury Molodov
50a37ee07d
vmui: save column settings in URL #7662 (#7979)
### Describe Your Changes

Added saving column settings in the URL for the table view. See #7662

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit f0d55a1c25)
2025-01-24 09:51:50 +01:00
Yury Molodov
7a13358cd2
vmui: fix issue with query execution and line breaks in query editor
This commit fixes incorrect behaviour when pressing `Enter` did not execute the query, and
`Shift+Enter` did not insert a new line.

- The issue occurred when autocomplete was disabled.
- This problem affected the query editor in both the VictoriaMetrics UI
and VictoriaLogs UI.

Related issue:
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/8058

(cherry picked from commit f31dece58d)
2025-01-24 09:51:50 +01:00
Zakhar Bessarab
aef93b1889
app/vmselect/prometheus: fix panic when performing delete with "multitenant" auth token
Initially delete_series API wasn't implemented for mulitenant auth token.

 This commit fixes it and properly handle delete series requests for mulitenant auth token.
It also adds integration tests for this case.

Related issue:
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/8126

Introduced at v1.104.0 release:
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1434
---------

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
Co-authored-by: f41gh7 <nik@victoriametrics.com>
2025-01-24 08:32:50 +01:00
Zakhar Bessarab
5f56375564
app/vmselect/prometheus: prevent panic when using "multitenant" at /api/v1/series/count requests
Adding support of multi-tenant reads to /api/v1/series/count would
require introducing a breaking change to a `netstorage` RPC, so
currently vmselect will explicitly deny these requests.

Related issues:
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/8126

Introduced at v1.104.0 release:
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1434

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2025-01-24 07:58:54 +01:00
Aliaksandr Valialkin
2aeba2b2cc
docs/VictoriaLogs/sql-to-logsql.md: add a guide on how to convert SQL to LogsQL 2025-01-24 07:52:52 +01:00
Aliaksandr Valialkin
755a0bb3d9
docs/VictoriaLogs/README.md: add a link to VictoriaLogs playground 2025-01-24 07:52:52 +01:00
Aliaksandr Valialkin
9c8fc93553
docs/VictoriaLogs/LogsQL.md: mention that field pipe can be used for improving query performance 2025-01-24 07:52:52 +01:00
hagen1778
ea3a7c3528
deployment: reflect metrics datasource ID change
See https://github.com/VictoriaMetrics/victoriametrics-datasource

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2025-01-24 07:52:51 +01:00
Roman Khavronenko
9261da53a0
app/vmselect/promql: respect staleness in removeCounterResets (#8073)
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/8072

### Describe Your Changes

Please provide a brief description of the changes you made. Be as
specific as possible to help others understand the purpose and impact of
your modifications.

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2025-01-24 07:52:51 +01:00
Zhu Jiekun
3ca0ca1f45
docs/vmagent: estimating the message size and rate when using Kafka
When using Kafka, it's important to estimate the message rate and size
before applying for resources.

This commit  explains why and how to use remote write metrics to
do the evaluation.
2025-01-24 07:52:51 +01:00
Jose Gómez-Sellés
e1852e8aa1
doc: fix typo in vmsingle k8s monitoring guide (#8120)
As reported by a user, the value in [VictoriaMetrics Single helm
chart](https://github.com/VictoriaMetrics/helm-charts/blob/master/charts/victoria-metrics-single/values.yaml#L420)
and actual example, should be "enabled", not "enable". This commit fixes
it.


### Describe Your Changes

Please provide a brief description of the changes you made. Be as
specific as possible to help others understand the purpose and impact of
your modifications.

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2025-01-24 07:52:51 +01:00
Aliaksandr Valialkin
bfd198a4d5
lib/logstorage: add hash pipe for calculating hash over the given log field
This pipe may be useful for sharding log entries among hash buckets.
2025-01-24 07:52:51 +01:00
Aliaksandr Valialkin
0a586ecfd8
lib/logstorage: add an ability to set query concurrency on a per-query basis
This is done via 'options(concurrency=N)' prefix for the query.
For example, the following query is executed on at most 4 CPU cores:

    options(concurrency=4) _time:1d | count_uniq(user_id)

This allows reducing RAM and CPU usage at the cost of longer query execution times,
since by default every query is executed in parallel on all the available CPU cores.

See https://docs.victoriametrics.com/victorialogs/logsql/#query-options
2025-01-24 07:52:50 +01:00
Daria Karavaieva
e7c806e5c3
docs/vmanomaly: format & fix docs (#8122)
### Describe Your Changes

- fix formatting in Presets
- change integration guide config according to new format + optimisation
of prophet
- minor fixes

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2025-01-24 07:52:49 +01:00
Daria Karavaieva
06d0afd79d
docs/vmanomaly: prod ready config example (#8107)
### Describe Your Changes

- provide prod ready config examples for vmanomaly
- minor formating fix

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

(cherry picked from commit f932deb47a)
2025-01-22 13:50:59 +01:00
Zhu Jiekun
98acb06a2e
docs: add curl file upload example to csv import doc (#8113)
### Describe Your Changes

Provide `curl` file upload example to csv import related section.

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/8080

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

(cherry picked from commit 77f446d095)
2025-01-22 13:50:59 +01:00
Github Actions
2676d9baa8
Automatic update helm docs from VictoriaMetrics/helm-charts@b26141b (#8116)
Automated changes by
[create-pull-request](https://github.com/peter-evans/create-pull-request)
GitHub action

Signed-off-by: Github Actions <133988544+victoriametrics-bot@users.noreply.github.com>
Co-authored-by: f41gh7 <18450869+f41gh7@users.noreply.github.com>
(cherry picked from commit 5f8810fc8d)
2025-01-22 13:50:58 +01:00
Nikolay
af4446725d
app/vmauth: remove readTrackingBody pool (#8104)
Sync.Pool for readTrackingBody was added in order to reduce potential
load on garbage collector. But golang net/http standard library does not
allow to reuse request body, since it closes body asynchronously after
return. Related issue: https://github.com/golang/go/issues/51907

This commit removes sync.Pool in order to fix potential panic and data
race at requests processing.

 Affected releases:
- all releases after v1.97.7

 Related issue:
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/8051

Signed-off-by: f41gh7 <nik@victoriametrics.com>
Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
(cherry picked from commit 80ead7cfa4)
2025-01-22 13:33:38 +01:00
Zakhar Bessarab
e77493b5b9
docs/guides/guide-vmcluster-multiple-retention-setup: fix some typos (#8112)
### Describe Your Changes

- fix typo in text
- fix typos in schema, re-create schema and upload source

Current doc:
https://docs.victoriametrics.com/guides/guide-vmcluster-multiple-retention-setup/

Here is an updated schema preview:

![image](https://github.com/user-attachments/assets/a65613c0-f3f6-4df5-bc70-fd70f1677386)

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
(cherry picked from commit 8772288bd6)
2025-01-22 13:33:37 +01:00
Fred Navruzov
2a259398e0
docs/vmanomaly: release v1.19.1 (#8108)
### Describe Your Changes

docs/vmanomaly: release v1.19.1

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

(cherry picked from commit 299d66fd98)
2025-01-22 13:33:34 +01:00
Zakhar Bessarab
e3054bbcf8
lib/license: trim whitespaces before license key validation
Validation expects license key to be present without any addition whitespaces, so having a trailing whitespaces would lead to verification failure.

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2025-01-22 12:07:28 +04:00
hagen1778
d28a245a20
docs: move changelog line to the corresponding section
Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 9e6fc9269d)
2025-01-21 20:32:31 +04:00
hagen1778
dfec66a0fd
docs: rm misleading default value for -maxConcurrentInserts
See https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5494
and https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5494#issuecomment-2603067137

Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 661f9fc3e2)
2025-01-21 12:51:36 +01:00
Andrii Chubatiuk
49694efc02
lib/protoparser/opentelemetry: do not drop histogram buckets, when sum is absent (#8054)
Despite requirement in OpenTelemetry spec that histograms should contain
sum, [OpenTelemetry collector promremotewrite
translator](37c8044abf/pkg/translator/prometheusremotewrite/helper.go (L222))
and [Prometheus OpenTelemetry
parsing](d52e689a20/storage/remote/otlptranslator/prometheusremotewrite/helper.go (L264))
skip only sum if it's absent. Our current implementation drops buckets
if sum is absent, which causes issues for users, that are expecting a
similar to Prometheus behaviour

### Describe Your Changes

Please provide a brief description of the changes you made. Be as
specific as possible to help others understand the purpose and impact of
your modifications.

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
(cherry picked from commit 2adb5fe014)
2025-01-21 12:51:36 +01:00
Github Actions
2ab56f1615
Automatic update operator docs from VictoriaMetrics/operator@5c7959f (#8102)
Automated changes by
[create-pull-request](https://github.com/peter-evans/create-pull-request)
GitHub action

Signed-off-by: Github Actions <133988544+victoriametrics-bot@users.noreply.github.com>
Co-authored-by: f41gh7 <18450869+f41gh7@users.noreply.github.com>
(cherry picked from commit ce917a4cc3)
2025-01-21 12:51:35 +01:00
Github Actions
7d69cb57e2
Automatic update helm docs from VictoriaMetrics/helm-charts@21bf09b (#8098)
Automated changes by
[create-pull-request](https://github.com/peter-evans/create-pull-request)
GitHub action

Signed-off-by: Github Actions <133988544+victoriametrics-bot@users.noreply.github.com>
Co-authored-by: zekker6 <1367798+zekker6@users.noreply.github.com>
(cherry picked from commit b3de1c029c)
2025-01-21 12:51:35 +01:00
Fred Navruzov
9c982e60db
docs/vmanomaly: release v1.19.0 (#8097)
### Describe Your Changes

Updated `vmanomaly` docs to release v1.19.0. Also, slight improvements
on other pages, like FAQ

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

(cherry picked from commit 461c7a5ad7)
2025-01-21 12:51:35 +01:00
Aliaksandr Valialkin
3de20efbfd
deployment/docker: update Go builder from Go1.23.4 to Go1.23.5
See https://github.com/golang/go/issues?q=milestone%3AGo1.23.5+label%3ACherryPickApproved

(cherry picked from commit 489631b227)
2025-01-21 12:51:35 +01:00
Aliaksandr Valialkin
e7e4aa4ce3
deployment/docker: update VictoriaLogs from v1.6.1-victorialogs to v1.7.0-victorialogs
(cherry picked from commit e78ff0dc2a)
2025-01-21 12:51:34 +01:00
Aliaksandr Valialkin
2f12e8b692
docs/VictoriaLogs/CHANGELOG.md: cut v1.7.0-victorialogs
(cherry picked from commit 81c313fd89)
2025-01-21 12:51:34 +01:00
Aliaksandr Valialkin
e0a6011f51
vendor/github.com/VictoriaMetrics/metricsql: update from v0.81.1 to v0.82.0
This introduces the ability to copy-n-paste queries with $__interval and $__rate_interval
placeholders into VictoriaMetrics - these placeholders are automatically replaced with 1i,
which equals to the `step` query arg value passed to /api/v1/query_range and /api/v1/query.

(cherry picked from commit e9de665289)
2025-01-21 12:51:34 +01:00
Aliaksandr Valialkin
5095833357
lib/logstorage: add ability to execute INNER JOIN with join pipe
(cherry picked from commit bfbe06e912)
2025-01-21 12:51:33 +01:00
Aliaksandr Valialkin
a85bce0d68
docs/VictoriaLogs/LogsQL.md: clarify docs about LogsQL pipes a bit
(cherry picked from commit 71a7d0db4a)
2025-01-21 12:51:33 +01:00
Aliaksandr Valialkin
df6a371615
docs/VictoriaLogs/FAQ.md: add questions on how to determine the number of unique log streams and unique field values
(cherry picked from commit e8748e4747)
2025-01-21 12:51:33 +01:00
Yury Molodov
72a78d89a6
vmui/logs: improvements to grouped view in logs (#7815)
### Describe Your Changes

This PR introduces improvements for the `Group view` tab:

1. **Reduced text size and improved styles**
   Related issue: #7479
   <details>
   <summary>Demo UI</summary>

<img
src="https://github.com/user-attachments/assets/c96aed95-e668-49a5-af20-778580e8a884"/>

   </details>

2. **Added the ability to select a field to display instead of `_msg`**
   You can select fields to display in the field list or in the settings
   Related issue: #7419
   <details>
   <summary>Demo UI</summary>

<img width="600"
src="https://github.com/user-attachments/assets/b5ae1433-bf40-4151-986b-ba057791dcee"/>
<img width="600"
src="https://github.com/user-attachments/assets/2c3d8c5c-1543-43ee-8241-8fd4d4b99b1d"/>

   </details>

3. **Added date format customization**
   <details>
   <summary>Demo UI</summary>

<img width="400"
src="https://github.com/user-attachments/assets/b095adfd-cbe3-45a6-bd7d-dc574d3edfd1"/>
<img width="400"
src="https://github.com/user-attachments/assets/0e8b0fbc-2c01-40b7-b9b8-7ef04d21ef9b"/>

   </details>

4. **Added an option to display log messages in a single line**
   <details>
   <summary>Demo UI</summary>

<img width="400"
src="https://github.com/user-attachments/assets/43d64a57-19c9-4b15-9009-13b8545d906c"/>
<img
src="https://github.com/user-attachments/assets/eaada2b1-9474-4496-ac39-b38332e637c1"/>

   </details>

5. **Added an option to display group labels in a single line**
   <details>
   <summary>Demo UI</summary>

<img width="400"
src="https://github.com/user-attachments/assets/05be097a-7b19-4e7b-9cf4-181fd802728b"/>
<img
src="https://github.com/user-attachments/assets/d4405fa6-3829-4713-8537-11edc6da5c4f"/>
<img
src="https://github.com/user-attachments/assets/90d2c48d-ba3b-4051-a645-22e7c8945dcb"/>

   </details>

7. **Settings indicator for modified defaults**
A red marker is displayed on the settings icon if the default settings
are changed. Hovering over it shows all the modified parameters.
   <details>
   <summary>Demo UI</summary>

<img
src="https://github.com/user-attachments/assets/19fbaf9b-684b-4cac-ad80-2db5e63804e9"/>

   </details>

(cherry picked from commit 17b3f24a37)
2025-01-20 13:49:13 +01:00
Zhu Jiekun
21f6d84b4b
docs: update docs for *authKey, add authKey to HTTP 401 resp body (#7971)
### Describe Your Changes

optimize for
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6226

for user who set `*AuthKey` flag, they will receive new response in
body:
```go
// query arg not set
The provided authKey '' doesn't match -search.resetCacheAuthKey

// incorrect query arg
The provided authKey '5dxd71hsz==' doesn't match -search.resetCacheAuthKey
```

previously, they receive:
```
The provided authKey doesn't match -search.resetCacheAuthKey
```

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

(cherry picked from commit 1f0b03aebe)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2025-01-20 13:49:12 +01:00
Github Actions
9cf08f500a
Automatic update operator docs from VictoriaMetrics/operator@7da9289 (#8093)
Automated changes by
[create-pull-request](https://github.com/peter-evans/create-pull-request)
GitHub action

Signed-off-by: Github Actions <133988544+victoriametrics-bot@users.noreply.github.com>
Co-authored-by: f41gh7 <18450869+f41gh7@users.noreply.github.com>
(cherry picked from commit fc8710c071)
2025-01-20 13:42:24 +01:00
Yury Molodov
e72ecc10c6
vmui/logs: update default graph to bars with color fill #7101 (#8060)
### Describe Your Changes

Updated the default graph type in the hits panel to bars with color fill
for better readability. Removed options for lines, stepped lines, and
points.

Ref issue: #7101

<img
src="https://github.com/user-attachments/assets/62522c97-7e5e-426a-b597-8457b2360f7e"
width="400"/>

(cherry picked from commit a7f36eef0e)
2025-01-20 13:42:24 +01:00
Zakhar Bessarab
46f9076ade
docs/release-guide: update helm release guide (#8090)
### Describe Your Changes

Release procedure was updated on helm charts side, update documented
procedure with up-to-date steps for the release.

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
(cherry picked from commit 54ab08d839)
2025-01-20 13:42:24 +01:00
Zakhar Bessarab
b80a93146f
{docs,deployment}: use explicit version tag instead of "latest" (#8089)
### Describe Your Changes

Using latest leads to inconsistent results between runs and might also
lead to data corruption since "latest" can be updated by any development
build.

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
(cherry picked from commit a3ea6d9e61)
2025-01-20 13:42:23 +01:00
Github Actions
35606eefe3
Automatic update helm docs from VictoriaMetrics/helm-charts@ba82ea6 (#8087)
Automated changes by
[create-pull-request](https://github.com/peter-evans/create-pull-request)
GitHub action

Signed-off-by: Github Actions <133988544+victoriametrics-bot@users.noreply.github.com>
Co-authored-by: AndrewChubatiuk <3162380+AndrewChubatiuk@users.noreply.github.com>
(cherry picked from commit f19c760f4f)
2025-01-20 13:14:12 +04:00
Aliaksandr Valialkin
9798ce0276
docs/VictoriaLogs/FAQ.md: add answers on how to detect the most frequently seen logs and how to get field names in the selected logs
(cherry picked from commit 86e74de9db)
2025-01-20 13:14:10 +04:00
Aliaksandr Valialkin
aa37295358
docs/VictoriaLogs/FAQ.md: add an answer on how to determine log fields, which occupy the most of the disk space and how to deal with such log fields
(cherry picked from commit 4d4253ee17)
2025-01-20 13:14:08 +04:00
Github Actions
d2e5751515
Automatic update helm docs from VictoriaMetrics/helm-charts@139a32b (#8077)
Automated changes by
[create-pull-request](https://github.com/peter-evans/create-pull-request)
GitHub action

Signed-off-by: Github Actions <133988544+victoriametrics-bot@users.noreply.github.com>
Co-authored-by: zekker6 <1367798+zekker6@users.noreply.github.com>
(cherry picked from commit 8c7b5d22c9)
2025-01-17 19:46:55 +04:00
Zakhar Bessarab
0eef0b860a
docs: update reference to the latest release - v1.109.1
### Describe Your Changes

Please provide a brief description of the changes you made. Be as
specific as possible to help others understand the purpose and impact of
your modifications.

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
(cherry picked from commit 513f5da5de)
2025-01-17 19:46:53 +04:00
hagen1778
522c912f35
docs: mention v1.109.1 as recommendation to update
Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit abaf8574a8)
2025-01-17 15:30:27 +01:00
hagen1778
691303ee36
docs: re-purpose b26a68641c as bugfix
Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit f346b5aaaa)
2025-01-17 15:30:27 +01:00
hagen1778
374232ee49
docs: move change 4574958e2e to #tip section
Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 31398cc739)
2025-01-17 15:30:27 +01:00
Hui Wang
cc24696ade
vmselect: add -search.maxDeleteDuration to limit the duration of th… (#8039)
…e `/api/v1/admin/tsdb/delete_series` call

Previously, it is limited by `-search.maxQueryDuration`, and can be
small for delete calls.

part of https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7857.

(cherry picked from commit 4574958e2e)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2025-01-17 15:30:26 +01:00
Daria Karavaieva
57cb3dd537
docs/vmanomaly: fix column width (#8059)
### Describe Your Changes

Fixing column widht in tables for components`sections: reader, writer,
monitoring and scheduler

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

(cherry picked from commit aac5cd8574)
2025-01-17 15:29:17 +01:00
Github Actions
fe57d0ce2f
Automatic update operator docs from VictoriaMetrics/operator@e9d1493 (#8074)
Automated changes by
[create-pull-request](https://github.com/peter-evans/create-pull-request)
GitHub action

Signed-off-by: Github Actions <133988544+victoriametrics-bot@users.noreply.github.com>
Co-authored-by: f41gh7 <18450869+f41gh7@users.noreply.github.com>
(cherry picked from commit d3c02b8f5d)
2025-01-17 15:29:17 +01:00
Zakhar Bessarab
6b551c82fb
docs/changelog/CHANGELOG.md: cut v1.109.1
Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2025-01-17 14:37:13 +04:00
Aliaksandr Valialkin
73eddc4da9
deployment/docker: update VictoriaLogs Docker image from v1.6.0-victorialogs to v1.6.1-victorialogs
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.6.1-victorialogs

(cherry picked from commit 0f7b853a88)
2025-01-17 13:26:55 +04:00
Aliaksandr Valialkin
7562348c1e
docs/VictoriaLogs/CHANGELOG.md: cut v1.6.1-victorialogs
(cherry picked from commit 70f0a974b8)
2025-01-17 13:26:54 +04:00
Aliaksandr Valialkin
403dbd6047
lib/logstorage: properly pass tenantIDs list to initStreamFilters
Previously an empty tenantIDs list was mistakenly passed to initStreamFilters
when the query already contained top-level stream filter.

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/8037

(cherry picked from commit 43d615ae87)
2025-01-17 13:26:48 +04:00
Zakhar Bessarab
140f0f302e
lib/license/online: remove delay before failing if all online verification attempts have failed (#816)
Enable components to fail faster in case all verification attempts have failed. Currently, there will be a final sleep before returning an error.

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2025-01-16 20:40:28 +04:00
Mathias Palmersheim
35f102c6ea
added retention filters to Victorialogs roadmap (#8020)
### Describe Your Changes

Fixes #8019 and adds retention filters to victorialogs roadmap as an
enterprise feature

### Checklist

The following checks are **mandatory**:

- [X] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

(cherry picked from commit 45bfe1f44c)
2025-01-16 17:07:39 +01:00
hagen1778
5a35ae4565
docs: add changelog line for 7d2a6764e7
Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit feeda42560)
2025-01-16 17:07:39 +01:00
Aliaksandr Valialkin
7af44f5e11
docs/VictoriaLogs/LogsQL.md: use top pipe in examples instead of stats by (...) count() | sort (...) limit N
`top` pipe is shorter and easier to understand

(cherry picked from commit 1645542a8a)
2025-01-16 17:07:38 +01:00
Aliaksandr Valialkin
6010daa536
docs/VictoriaLogs/logsql-examples.md: replace last pipe with first pipe, since it is easier to understand
(cherry picked from commit 151eb1e4b6)
2025-01-16 17:07:38 +01:00
Aliaksandr Valialkin
a48dccdde5
docs/VictoriaLogs/LogsQL.md: use proper backticks around hexnumencode:
(cherry picked from commit 5e4de8e860)
2025-01-16 17:07:37 +01:00
Aliaksandr Valialkin
c66520253e
docs/VictoriaLogs: typo fixes for block_stats pipe docs
(cherry picked from commit 6312d3bbba)
2025-01-16 17:07:37 +01:00
Aliaksandr Valialkin
506ca64514
docs/VictoriaLogs/LogsQL.md: add missing they word
(cherry picked from commit d2bede6b51)
2025-01-16 17:07:37 +01:00
Aliaksandr Valialkin
37a9d38877
docs/VictoriaLogs/LogsQL.md: fix incorrect url to VictoriaMetrics histogram buckets
This is a follow-up for d2a791bef3

(cherry picked from commit 5ca5069fc4)
2025-01-16 17:07:36 +01:00
Aliaksandr Valialkin
d9da20769c
deployment/docker: update VictoriaLogs Docker images from v1.5.0-victorialogs to v1.6.0-victorialogs
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.6.0-victorialogs

(cherry picked from commit 8a3c460f63)
2025-01-16 17:07:36 +01:00
Aliaksandr Valialkin
d8102b98fb
docs/VictoriaLogs/CHANGELOG.md: cut v1.6.0-victorialogs release
(cherry picked from commit ca653a515c)
2025-01-16 17:07:35 +01:00
Aliaksandr Valialkin
a9c47ecb81
deployment/docker: update Alpine Docker image from 3.21.0 to 3.21.2
See https://alpinelinux.org/posts/Alpine-3.21.1-released.html and https://alpinelinux.org/posts/Alpine-3.18.11-3.19.6-3.20.5-3.21.2-released.html

(cherry picked from commit e24a8f2088)
2025-01-16 17:07:34 +01:00
Aliaksandr Valialkin
7cbfe32d7e
lib/logstorage: add union pipe, which allows uniting results from multiple queries
(cherry picked from commit f27e120aeb)
2025-01-16 17:07:34 +01:00
Aliaksandr Valialkin
0deab437e9
lib/logstorage: math pipe: add rand() function
(cherry picked from commit 47fe8cf3be)
2025-01-16 17:07:33 +01:00
Daria Karavaieva
32cc73dc13
docs/vmanomaly: fix parameters field width (#8025)
### Describe Your Changes

- fix parameters width in Components section - reader, wtiter,
monitoring, scheduler

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

(cherry picked from commit 5813aa6602)
2025-01-16 17:07:33 +01:00
Roman Khavronenko
8426f984ae
docs: mention regression in v1.109.0 (#8046)
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/8045

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit ad3bd11334)
2025-01-16 17:07:32 +01:00
Github Actions
6db3646460
Automatic update helm docs from VictoriaMetrics/helm-charts@d8e5f03 (#8035)
Automated changes by
[create-pull-request](https://github.com/peter-evans/create-pull-request)
GitHub action

Signed-off-by: Github Actions <133988544+victoriametrics-bot@users.noreply.github.com>
Co-authored-by: AndrewChubatiuk <3162380+AndrewChubatiuk@users.noreply.github.com>
(cherry picked from commit 875c6663ef)
2025-01-16 17:07:31 +01:00
Zakhar Bessarab
ba5f272ee4
docs: update references to v1.109.0 (#8034)
### Describe Your Changes

Please provide a brief description of the changes you made. Be as
specific as possible to help others understand the purpose and impact of
your modifications.

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
(cherry picked from commit b48b7c454a)
2025-01-16 17:07:31 +01:00
Zakhar Bessarab
4475c56ead
docs/changelog: add missing entry for #7182 (#8030)
### Describe Your Changes

Please provide a brief description of the changes you made. Be as
specific as possible to help others understand the purpose and impact of
your modifications.

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
(cherry picked from commit 63bf1e008f)
2025-01-16 17:07:31 +01:00
Zakhar Bessarab
f8937d0338
docs/lts-releases: update links to latest releases (#8028)
### Describe Your Changes

Please provide a brief description of the changes you made. Be as
specific as possible to help others understand the purpose and impact of
your modifications.

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
(cherry picked from commit 419ac10c60)
2025-01-16 17:07:30 +01:00
Dmytro Kozlov
b1eecb4dac
cloud: update email images for cloud (#8024)
### Describe Your Changes

Updated email images with new support email

### Checklist

The following checks are **mandatory**:

- [X] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

(cherry picked from commit d631d2c100)
2025-01-14 14:29:52 +01:00
Zakhar Bessarab
37be3848b2
docs/release-guide: move testing on sandbox step before pushing tags (#8026)
### Describe Your Changes

Please provide a brief description of the changes you made. Be as
specific as possible to help others understand the purpose and impact of
your modifications.

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
(cherry picked from commit 89431458bf)
2025-01-14 14:29:52 +01:00
Zakhar Bessarab
638848f450
docs/changelog: port LTS changes, update release date (#8027)
### Describe Your Changes

Please provide a brief description of the changes you made. Be as
specific as possible to help others understand the purpose and impact of
your modifications.

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
(cherry picked from commit d8d0c0ac01)
2025-01-14 14:29:51 +01:00
Roman Khavronenko
e1abd66b9b
docs: cut new changelog doc for 2024 (#8023)
Cutting new changelod doc reduces the size of the current's year
changelog and improves navigation for users.

### Describe Your Changes

Please provide a brief description of the changes you made. Be as
specific as possible to help others understand the purpose and impact of
your modifications.

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit c0f5699bad)
2025-01-14 14:29:50 +01:00
chenlujjj
dc10b26306
lib/opentelemetry: log the metric name of unsupported metrics (#8018)
To resolve:
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/8009
Log the name of unsupported metrics.

(cherry picked from commit b26a68641c)
2025-01-14 14:29:49 +01:00
Aliaksandr Valialkin
e1c044f326
lib/logstorage: add histogram stats function for calculating histogram buckets over numeric fields
(cherry picked from commit d2a791bef3)
2025-01-14 14:29:48 +01:00
Daria Karavaieva
5923a5d0e7
docs/vmanomaly: fix table rendering (#8005)
### Describe Your Changes

- fix table rendering on writer and scheduler pages

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

(cherry picked from commit 92b6475fa6)
2025-01-14 14:29:46 +01:00
Andrii Chubatiuk
284a5f3c5f
docs: added search resupts page for docs.victoriametrics.com (#8017)
### Describe Your Changes

added search page required for docs site

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

(cherry picked from commit bda3546cfd)
2025-01-14 14:29:46 +01:00
Artem Navoiev
d0e435dd20
docs: cloud use separate support email for cloud (#8013)
### Describe Your Changes

Please provide a brief description of the changes you made. Be as
specific as possible to help others understand the purpose and impact of
your modifications.

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
(cherry picked from commit 2691cdefe3)
2025-01-14 14:29:46 +01:00
Github Actions
ebe0d36fa0
Automatic update helm docs from VictoriaMetrics/helm-charts@457951a (#8014)
Automated changes by
[create-pull-request](https://github.com/peter-evans/create-pull-request)
GitHub action

Signed-off-by: Github Actions <133988544+victoriametrics-bot@users.noreply.github.com>
Co-authored-by: f41gh7 <18450869+f41gh7@users.noreply.github.com>
(cherry picked from commit 93b8aa5c9d)
2025-01-14 14:29:46 +01:00
Aliaksandr Valialkin
fab8260899
docs/VictoriaLogs/README.md: add "Profiling" chapter
This chapter is needed for referring from Github issues when CPU or memory profiles are needed to be collected
in order to investigate issues with high CPU and/or RAM usage at VictoriaLogs.
2025-01-13 09:44:35 +01:00
Aliaksandr Valialkin
12f6df291c
deployment/docker: update VictoriaLogs Docker image tag from v1.4.0-victorialogs to v1.5.0-victorialogs
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.5.0-victorialogs
2025-01-13 07:49:14 +01:00
Aliaksandr Valialkin
78df661c9f
docs/VictoriaLogs/CHANGELOG.md: cut v1.5.0-victorialogs release 2025-01-13 07:49:13 +01:00
Aliaksandr Valialkin
c4b3def5b0
lib/logstorage: format pipe: add frequently used formatters
- url encoding / decoding with <urlencode:field> and <urldecode:field>
- base64 encoding / decoding with <base64encode:field> and <base64decode:field>
- hex encoding / decoding with <hexencode:field> and <hexdecode:field>
- hex encoding for integers with <hexnumencode:field> and <hexnumdecode:field>
2025-01-13 07:23:20 +01:00
Aliaksandr Valialkin
1ac4516919
lib/logstorage: stats pipe: use integer group keys if stats by(...) contains a single field with integer values
This reduces memory usage and improves performance, since access to a map with integer keys
is faster than access to a map with string keys.
2025-01-13 07:23:20 +01:00
Aliaksandr Valialkin
c17149cb0c
lib/logstorage: count_uniq and count_uniq_hash stats functions: avoid converting integer values to strings
Prevsiously integer values were converted to strings before being passed to `updateState()` function at `count_uniq`
and `count_uniq_hash`. Later such values are converted back to integers in order to track them via integer map of unique values.

This commit avoids the int -> string -> int conversion. Instead, it passes integers directly to the integer map of unique values.
This improves performance of `count_uniq` and `count_uniq_hash` functions even further.
2025-01-13 07:23:20 +01:00
Aliaksandr Valialkin
9ff6128102
lib/logstorage: add value_type filter to LogsQL
This filter can be used when debugging and exploring logs in order to understand better
which value types are used for storing the particular log fields.

The `value_type` filter complements `block_stats` pipe.
2025-01-13 07:23:19 +01:00
Aliaksandr Valialkin
fcc692d00e
lib/logstorage: improve performance for math pipe
- Pass the calculated results to the next pipe in float64 columns.
  Previously the results were converted to string columns. This could slow down further calculations.

- Use custom optimized logic for processing numeric columns, which are passed to math pipe.
  Previously all the input columns were converted to string and then converted to float64
  before math pipe calculations.

- Initialize the newly added columns at blockResult as soon as they are added.
  This improves performance when big number of columns are calculated by math pipe.
2025-01-13 07:23:19 +01:00
Aliaksandr Valialkin
6b59561f96
lib/logstorage: track integer values in integer maps when counting the number of unique values at count_uniq stats function
Previously integer values were tracked in string maps. Now every input value is parsed as integer.
On success the parsed integer is tracked via specialized maps, which hold only integers.
This reduces CPU usage and memory usage in general case.
2025-01-13 07:23:19 +01:00
Roman Khavronenko
0614eb97a5
app/vmselect/promql: account for staleness when populating realPrevValue (#8002)
When vmselect process a rollup function it fetches all the raw samples
on requested `start-end` interval of the query. It then loops through
the raw samples, picks the range of the samples based on provided `step`
interval and invokes a rollup function for each of the picked ranges of
samples.

During this processing, vmselect always populates the `realPrevValue`
field with the closest previous raw sample value before the picked range
of samples. This `realPrevValue` is used by rollup functions like
increase_pure or delta to decide whether the counter change happened or
not. For example, we get the counter value == 1. If we've seen this
counter before and its value was also 1 - then no change happened. If we
didn't see it before, then this counter should have started with value=0
and we need to account for `1-0=1` change. All this is required to deal
with situations when scrapes are missing or `step` is too small.

However, vmselect doesn't check how "old" is the `realPrevValue`. In
other words, it doesn't respect the staleness interval when picking it.
In result, depending on the `start` and `end` params, vmselect can use
`realPrevValue` which is a couple of hours old and is unlikely to be a
temporary scrape fail. In result, some increases can be incorrectly
ingnored by vmselect.

This change makes sure that vmselect doesn't populate `realPrevValue`
with samples that are older than staleness interval.

### Describe Your Changes

Please provide a brief description of the changes you made. Be as
specific as possible to help others understand the purpose and impact of
your modifications.

### Checklist

The following checks are **mandatory**:

- [ x ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).


-------------------

To reproduce, create a dataset with one metric `foo` which has samples
with value=1 on interval of couple of hours and resolution 15s, and a
gap for an hour in the middle:
<img width="769" alt="image"
src="https://github.com/user-attachments/assets/a39b2740-b741-45f8-ad18-093b7c57c3b3"
/>

Then run `increase(foo[1m])` expression on this time range (disable
cache):
<img width="1472" alt="image"
src="https://github.com/user-attachments/assets/463cece1-f359-4c75-a96c-60092a31cab2"
/>

In result, there will be one increase on the beginning of the series.
And no increase after the gap. Then change the time range so it starts
in the middle of the gap:
<img width="1505" alt="image"
src="https://github.com/user-attachments/assets/f4a460c3-9fd1-4ec7-ab47-15e716ec1019"
/>

Now, there is an increase>0 because the `realPrevValue` wasn't
populated. This is wrong, because it hides the increase of the series.

With the fix, the original increase query on full time range should show
2 increases:
<img width="1492" alt="image"
src="https://github.com/user-attachments/assets/aa9d8a6b-7b22-41f6-9eb9-83b3113a6982"
/>

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2025-01-13 07:23:17 +01:00
Dima Shur
07bb605682
docs: changed typo in label (enterpriSe instead of enterpriZe) (#7925)
### Describe Your Changes

Fixed typo in contributing.md (enterpriZe -> enterpriSe in the label
name)

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

(cherry picked from commit 7941877233)
2025-01-10 19:13:51 +04:00
Zakhar Bessarab
dd933fe52c
docs/CHANGELOG.md: cut v1.109.0
Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2025-01-10 18:57:58 +04:00
Nikolay
ca5e105ada
lib/storage: add a hint for merge about type of parts in merge (#7998)
Hint allows to choose type of cache to be used for index search:
- in-memory parts are storing recently ingested samples and should use
main cache. This improves ingestion speed and cache hit ration for
queries accessing recently ingested samples.
- merges of file parts is performed in background, using a separate
cache allows avoiding pollution of the main cache with irrelevant
entries.

Related issue:
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7182

---------

Signed-off-by: f41gh7 <nik@victoriametrics.com>
(cherry picked from commit e9f86af7f5)
2025-01-10 17:45:20 +04:00
Nikolay
688f667919
lib/storage: make finalDedup schedule interval configurable
This commit makes configurable interval for checking if final dedup
process for the historical data should be started. It allows to spread
resource utilisation for multiple vmstorage/vmsingle instances in time.
Since final dedup may add additional preasure on disk, backup systems
and make cluster less stable. Storage unconditionally adds 25% jitter to
the provided value, it should simplify configuration management at
Kubernetes ecosystem. Because Kubernetes application pods must have the
same configuration.

Related issue:
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7880

---------

Signed-off-by: f41gh7 <nik@victoriametrics.com>
Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
(cherry picked from commit 9ada784983)
2025-01-10 16:56:08 +04:00
Github Actions
c7484ec9bb
Automatic update helm docs from VictoriaMetrics/helm-charts@117506e (#7996)
Automated changes by
[create-pull-request](https://github.com/peter-evans/create-pull-request)
GitHub action

Signed-off-by: Github Actions <133988544+victoriametrics-bot@users.noreply.github.com>
Co-authored-by: AndrewChubatiuk <3162380+AndrewChubatiuk@users.noreply.github.com>

(cherry picked from commit 2564f10d98)
Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2025-01-10 16:51:19 +04:00
Zakhar Bessarab
441f9efa5d
app/vmalert/notifier: fix rendering of Alertmanager notification body
commitL  c7fc0d0d2f  enabled skipping alerts
in case there is no labels present for an alert. This made clause which
was adding a comma for the JSON list incorrect as it is not possible to
determine if the next alert will be skipped or not.

This fix renders all alert labels in advance allowing properly format
JSON payload for Alertmanager notification.

Related issue:
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7985

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

(cherry picked from commit 51b21dfd57)
Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2025-01-10 16:51:06 +04:00
Zhu Jiekun
3c1282ae7c
lib/promscrape: add Marathon service discovery
This commit adds support for [Marathon](https://mesosphere.github.io/marathon/)
service discovery to the scrape configuration.

The following flag is introduced:
```
  -promscrape.marathonSDCheckInterval duration
          Interval for checking for changes in Marathon service discovery. This works only if marathon_sd_configs is configured in '-promscrape.config' file. See https://docs.victoriametrics.com/sd_configs.html#marathon_sd_configs for details  (default 30s)
```

The service discovery could be config like:
```yaml
scrape_configs:
- job_name: marathon_job
  marathon_sd_configs:
      servers:
        - "..."
        - "..."
```
See:
[b555d94d1a/docs/sd_configs.md (marathon_sd_configs))

related issue:
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6642

---------

Co-authored-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
(cherry picked from commit 276989716f)
2025-01-10 16:50:39 +04:00
Artem Fetishev
7620da0355
Update VictoriaLogs FAQ: add a section about max log record length (#7984)
### Describe Your Changes

There has been a question in our public Slack on whether the length
limit of a log record is going to be changed. See:
https://victoriametrics.slack.com/archives/C05UNTPAEDN/p1736156255119689

This PR documents the max length and explains why it has been chosen.
This FAQ section could serve as an answer to more questions like this.

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Signed-off-by: Artem Fetishev <rtm@victoriametrics.com>
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
(cherry picked from commit 6cb3c0cac8)
2025-01-10 16:50:35 +04:00
Afolabi Badmos
355397be48
vmauth: fix bug in discovering ipv6 addresses (#7955)
### Describe Your Changes

Fixes error in `vmauth` when discovering ipv6 addresses.

`vmauth` attempts to [slice till
`:`](https://github.com/VictoriaMetrics/VictoriaMetrics/blob/master/app/vmauth/auth_config.go#L397)
in the discovered addresses without accounting for ipv6. This causes it
to fail in ipv6 only environments.

```sh
$ nslookup vmselect.ns.svc.cluster.local

...
Name: vmselect.ns.svc.cluster.local
Address: 2600:dead:beef:dead:beef::8
```

```sh
$ kubectl logs -f vmauth

...
error: dial tcp: lookup 2600: no such host
```

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Co-authored-by: f41gh7 <nik@victoriametrics.com>
(cherry picked from commit 77b0fcfdd9)
2025-01-10 16:50:29 +04:00
Yury Molodov
803a264a5b
vmui/logs: add autocomplete support for LogsQL (#6949)
### Describe Your Changes

This pull request adds support for autocomplete in LogsQL queries. The
new feature provides suggestions for field names, field values, and pipe
names as you type.

---------

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
(cherry picked from commit ee7fe11fd2)
2025-01-10 16:50:24 +04:00
Andrii Chubatiuk
9e4c4c57a5
docs: make badges in docs clickable (#7960)
### Describe Your Changes

added links to badges and made them clickable at
docs.victoriametrics.com

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

(cherry picked from commit 4c26fb6fe5)
2025-01-10 16:50:17 +04:00
Github Actions
2a0aec2079
Automatic update operator docs from VictoriaMetrics/operator@028ab8b (#7981)
Automated changes by
[create-pull-request](https://github.com/peter-evans/create-pull-request)
GitHub action

Signed-off-by: Github Actions <133988544+victoriametrics-bot@users.noreply.github.com>
Co-authored-by: f41gh7 <18450869+f41gh7@users.noreply.github.com>
(cherry picked from commit fc135094b3)
2025-01-10 16:50:15 +04:00
Roman Khavronenko
3582e13d22
docs: mention publicly available playgrounds (#7977)
The point of the new section is to highlight publicly available
playgrounds for users. All of them were mentioned in other parts of the
documentation, but we didn't have all of them in one place before.

### Describe Your Changes

Please provide a brief description of the changes you made. Be as
specific as possible to help others understand the purpose and impact of
your modifications.

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 5d42f21abd)
2025-01-10 16:50:12 +04:00
hagen1778
0db40bf8cc
docs: rm extra new lines as they bring no value
Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 28eeabded1)
2025-01-10 16:50:09 +04:00