Commit graph

9526 commits

Author SHA1 Message Date
Yury Molodov
5e64e9682f
vmui: add retention and downsampling filters debug pages (#7238)
### Describe Your Changes

- add VMUI pages for filters debug
- add `config.json` file to the root of the application. The file
structure is as follows:
  ```
  {
    "license": {
      "type": "enterprise" or "opensource"
    }
  }
  ```
- refactor navigation configuration files. This refactor enables more
flexible customization of menu elements.

UI:

<details>
<summary>Renention filters debug</summary>

Empty page:

![1723474670](https://github.com/user-attachments/assets/3824bf64-dd22-410a-beb5-9599b8769acd)

Results:

![1723474597](https://github.com/user-attachments/assets/1bc074ba-b6a7-4127-8638-65cb32e04db8)

Example config:

![1723541836](https://github.com/user-attachments/assets/ccdb7f75-4e77-42c4-98be-4bfa7809a3b0)

</details>

<details>
<summary>Downsampling filters debug</summary>

Empty page:

![1723474663](https://github.com/user-attachments/assets/7bbd07bd-adce-440f-ba43-f4218e237280)

Results:

![1723474589](https://github.com/user-attachments/assets/b793ae08-b685-427d-81f1-1c7c532a244a)

Example config:

![1723541828](https://github.com/user-attachments/assets/d2ee4e37-8945-4c0f-a4ca-cff5fe3cfcd2)

</details>

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

(cherry picked from commit 0ff17c3ec4)
2024-10-15 14:53:27 +02:00
Yury Molodov
7bc20086ec
vmui: add the ability to cancel running queries (#7204)
### Describe Your Changes

- Added functionality to cancel running queries on the Explore Logs and
Query pages.
- The loader was changed from a spinner to a top bar within the block.
This still indicates loading, but solves the issue of the spinner
"flickering," especially during graph dragging.

Related issue: #7097

https://github.com/user-attachments/assets/98e59aeb-905b-4b9d-bbb2-688223b22a82

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

(cherry picked from commit 6c9772b101)
2024-10-15 14:53:27 +02:00
Zakhar Bessarab
5609d7b53a
lib/jwt: accept scope encoded as a slice (#790)
Some IDPs encode scope as a slice of strings. Handle this gracefully by encoding a slice back to string.

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

(cherry picked from commit f61d8c3ebb)

---------

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit a8d8987825)
2024-10-15 14:41:05 +02:00
Andrii Chubatiuk
019171fdfc
lib/protoparser/influx: enable batch processing by default (#7165)
### Describe Your Changes

Fixes https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7090

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>

(cherry picked from commit daa7183749)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-10-15 11:51:48 +02:00
Aliaksandr Valialkin
0881e5fd5c
app/vlselect: do not show empty fields in query results
Empty fields are treated as non-existing fields by VictoriaLogs data model.
So there is no sense in returning empty fields in query results, since they may mislead and confuse users.

(cherry picked from commit bac193e50b)
2024-10-15 11:49:32 +02:00
Zhu Jiekun
ac382d8b79
docs: make all statement of active time series consistent (#7242)
### Describe Your Changes

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7212

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

(cherry picked from commit 343463fc0f)
2024-10-15 11:49:31 +02:00
Andrii Chubatiuk
822d6bbdbc
docs/victorialogs: fixed HA examples links (#7249)
### Describe Your Changes

Fixed VictoriaLogs HA examples references in docs

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

(cherry picked from commit 41e0b62099)
2024-10-15 11:49:31 +02:00
Aliaksandr Valialkin
f627d7f686
app/vlstorage: add support for forced merge via /internal/force_merge HTTP endpoint
(cherry picked from commit 3c73dbbacc)
2024-10-15 11:49:31 +02:00
Aliaksandr Valialkin
ac2b6e8704
lib/logstorage: make a copy of s.partitions slice when performing queries over the selected partitions
s.partitions can be changed when new partition is registered or when old partition is dropped.
This could lead to data races and panics when s.partitions slice is accessed by concurrently executed queries.

The fix is to make a copy of the selected partitions under s.partitionsLock before performing the query.

(cherry picked from commit b4b79a4961)
2024-10-15 11:49:31 +02:00
Aliaksandr Valialkin
b694ca4952
lib/logstorage: move getConstColumnValue() and getColumnHeader() methods from columnsHeader to blockSearch
This localizes blockSearch.getColumnsHeader() call at block_search.go .
This call is going to be optimized in the next commits in order to avoid
unmarshaling of header data for unneeded columns, which weren't requested
by getConstColumnValue() / getColumnHeader().

(cherry picked from commit 507b206a7d)
2024-10-15 11:49:30 +02:00
Aliaksandr Valialkin
beeb80e4f8
lib/logstorage: avoid redundant copying of column names and column values for dictionary-encoded columns during querying
Refer the original byte slice with the marshaled columnsHeader for columns names and dictionary-encoded column values.
This improves query performance a bit when big number of blocks with big number of columns are scanned during the query.

(cherry picked from commit 279e25e7c8)
2024-10-15 11:49:30 +02:00
Aliaksandr Valialkin
fe9e6b7495
docs/VictoriaLogs: replace incorrect usafe of VictoriaMetrics with VictoriaLogs
(cherry picked from commit 91f5417572)
2024-10-15 11:49:30 +02:00
Aliaksandr Valialkin
afe5158443
lib/logstorage: avoid calling columnsHeader.initFromBlockHeader() multiple times for the same blockSearch
This should improve performance when blockSearch.getColumnsHeader() is called multiple times
from different places of the code.

(cherry picked from commit 9e48074b59)
2024-10-15 11:49:30 +02:00
Aliaksandr Valialkin
f7496c17e4
docs/VictoriaLogs/Roadmap.md: remove OpenTelemetry, Fluentd and Telegraf from the list of data ingestion protocols, which are going to be supported by VictoriaLogs
These protocols are already supported:

- OpenTelemetry protocol - https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4839 ,
  https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6218 and 01430a155c

- Fluentd protocol - https://github.com/VictoriaMetrics/VictoriaMetrics/pull/7098
  and 05a64a8c14

- Telegraf protocol - https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5310 ,
  https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6443 and 1731c0eabf

(cherry picked from commit 200d723b9a)
2024-10-15 11:49:29 +02:00
Aliaksandr Valialkin
e581338b84
lib/logstorage: make sure that bs.br is non-nil before checking br.bs.bsw.bh.rowsCount there
br.bs may be nil when br contains the block with additional filters applied during pipe calculations.
For example, `* | count() if (error) errors`.

(cherry picked from commit 867f671cc4)
2024-10-15 11:49:29 +02:00
Zakhar Bessarab
4b1fe0d656
docs/victoria-logs: fix typo (#7245)
### Describe Your Changes

Please provide a brief description of the changes you made. Be as
specific as possible to help others understand the purpose and impact of
your modifications.

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
(cherry picked from commit 2239f5829f)
2024-10-15 11:49:29 +02:00
hagen1778
9d27db4fc9
docs: fix typos in change line
Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 22d3f67908)
2024-10-15 11:49:29 +02:00
Hui Wang
1908d44cf8
dashboards: fix description about pending datapoints (#7235)
See [our
playground](https://play-grafana.victoriametrics.com/d/oS7Bi_0Wz_vm/victoriametrics-cluster-vm?orgId=1&var-ds=P996FABE17B5F6D1E&var-job=All&var-job_insert=All&var-job_select=All&var-job_storage=All&var-instance=All)
for reference.

(cherry picked from commit d3f110373c)
2024-10-11 14:28:23 +02:00
Yury Molodov
25a87f6ed3
vmui: update dependencies npm (#7209)
### Describe Your Changes

Related issue: #7142

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

(cherry picked from commit c7771b1866)
2024-10-11 14:28:23 +02:00
Github Actions
42a67616ba
Automatic update helm docs from VictoriaMetrics/helm-charts@c584b99 (#7237)
Automated changes by
[create-pull-request](https://github.com/peter-evans/create-pull-request)
GitHub action

Signed-off-by: Github Actions <133988544+victoriametrics-bot@users.noreply.github.com>
Co-authored-by: AndrewChubatiuk <3162380+AndrewChubatiuk@users.noreply.github.com>
(cherry picked from commit 6a738e0b41)
2024-10-11 14:28:23 +02:00
Andrii Chubatiuk
68b6834542
lib/protoparser/opentelemetry: added exponential histograms support (#6354)
### Describe Your Changes

added opentelemetry exponential histograms support. Such histograms are automatically converted into
VictoriaMetrics histogram with `vmrange` buckets.

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 9eb0c1fd86)
2024-10-11 14:28:19 +02:00
Lauri Tirkkonen
59c2073987
deployment/alerts: fix quoting on DiskRunsOutOfSpace (#7234)
### Describe Your Changes

there's an extra `"` at the end of the dashboard url for this alert;
remove it by making the quoting consistent with other alerts in this
file.

### Checklist

The following checks are **mandatory**:

- [X] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

Co-authored-by: Lauri Tirkkonen <lauri@hacktheplanet.fi>
(cherry picked from commit 8fe41b2b08)
2024-10-11 14:28:08 +02:00
Dmytro Kozlov
9110fdea2c
docs/victoriametrics-cloud: fix button description (#7221)
### Describe Your Changes

Fixed button name in the cloud docs

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

(cherry picked from commit 95de37de2c)
2024-10-11 14:28:07 +02:00
Aliaksandr Valialkin
d2352fd84f
docs/VictoriaLogs/querying/vlogscli.md: add the list of features of vlogscli
(cherry picked from commit b9a8c1ff3a)
2024-10-11 14:28:07 +02:00
Github Actions
22c90cef71
Automatic update helm docs from VictoriaMetrics/helm-charts@c4a0442 (#7223)
Automated changes by
[create-pull-request](https://github.com/peter-evans/create-pull-request)
GitHub action

Signed-off-by: Github Actions <133988544+victoriametrics-bot@users.noreply.github.com>
Co-authored-by: AndrewChubatiuk <3162380+AndrewChubatiuk@users.noreply.github.com>
(cherry picked from commit 595298ac98)
2024-10-11 14:28:07 +02:00
n4mine
e2f0fe8315
docs: fix typo in docs/VictoriaLogs/data-ingestion/Vector.md (#7222)
### Describe Your Changes

Please provide a brief description of the changes you made. Be as
specific as possible to help others understand the purpose and impact of
your modifications.

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

(cherry picked from commit f060b67da5)
2024-10-11 14:28:07 +02:00
Github Actions
573c516804
Automatic update helm docs from VictoriaMetrics/helm-charts@0b610bd (#7220)
Automated changes by
[create-pull-request](https://github.com/peter-evans/create-pull-request)
GitHub action

Signed-off-by: Github Actions <133988544+victoriametrics-bot@users.noreply.github.com>
Co-authored-by: AndrewChubatiuk <3162380+AndrewChubatiuk@users.noreply.github.com>
(cherry picked from commit d3f4b01001)
2024-10-11 14:28:06 +02:00
Dmytro Kozlov
c010325acf
docs/victoriametrics-cloud: update images in the cloud documentation (#7210)
### Describe Your Changes

Update images with updated interface of the cloud solution

### Checklist

The following checks are **mandatory**:

- [ x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

(cherry picked from commit c910c1c6b8)
2024-10-11 14:28:06 +02:00
Roman Khavronenko
2c0892263f
docs: update stream aggregation docs (#7202)
This PR is based on
https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6777. The
differences are the following:
* it keeps backward compatibility for links
* it re-structures only original document file
* it adds #common-mistakes section, re-phrased

### Describe Your Changes

Please provide a brief description of the changes you made. Be as
specific as possible to help others understand the purpose and impact of
your modifications.

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
(cherry picked from commit ca2a08eabe)
2024-10-11 14:28:06 +02:00
Aliaksandr Valialkin
42eefebb06
app/vlselect/logsql: add missing return from ProcessLiveTailRequest() when the query cannot be live tailed
(cherry picked from commit fe022ed795)
2024-10-11 14:27:47 +02:00
Aliaksandr Valialkin
0fe533e0be
deployment: update VictoriaLogs from v0.34.0-victorialogs to v0.35.0-victorialogs
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.35.0-victorialogs

(cherry picked from commit baa87b5b36)
2024-10-11 14:27:46 +02:00
Aliaksandr Valialkin
b3bbf94310
lib/logstorage: disallow using pipe names as the first unquoted words in filter pipe
Improperly written pipes could be silently parsed as filter pipe.
For example, the following query:

   * | by (x)

was silently parsed to:

   * | filter "by" x

It is better to return error, so the user could identify and fix invalid pipe
instead of silently executing invalid query with `filter` pipe.

(cherry picked from commit 7b475ed95d)
2024-10-11 14:27:46 +02:00
Aliaksandr Valialkin
ff63816b06
docs/VictoriaLogs: cut v0.35.0 release
(cherry picked from commit 252aa792f7)
2024-10-11 14:27:46 +02:00
Aliaksandr Valialkin
8d4c1cb1d8
docs/VictoriaLogs: make vlogscli more visible
(cherry picked from commit 9413b2de91)
2024-10-11 14:27:46 +02:00
Aliaksandr Valialkin
834e2ad855
lib/logstorage: disallow using by as the first word in log filters, since it frequently clashes with stats by(...) pipe where stats word is omitted
(cherry picked from commit 6acf543b90)
2024-10-11 14:27:46 +02:00
Aliaksandr Valialkin
4bb5f588bc
app/vlogscli: add -accountID and -projectID command-line flags for querying the given tenants
(cherry picked from commit ad5d8097da)
2024-10-11 14:27:45 +02:00
Aliaksandr Valialkin
d07e09b1e4
app/vlogscli: add support for live tailing
(cherry picked from commit e31625e0b2)
Signed-off-by: hagen1778 <roman@victoriametrics.com>

# Conflicts:
#	Makefile
2024-10-11 14:27:26 +02:00
n4mine
fb47859c59
docs: fix typo in docs/VictoriaLogs/data-ingestion/Vector.md (#7208)
### Describe Your Changes

Please provide a brief description of the changes you made. Be as
specific as possible to help others understand the purpose and impact of
your modifications.

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

(cherry picked from commit 344d61da79)
2024-10-11 14:25:22 +02:00
Fred Navruzov
2cdf807231
docs/vmanomaly: updates for v1.16.3 (#7203)
### Describe Your Changes

docs/vmanomaly: updates for v1.16.3

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

(cherry picked from commit f61d8c3ebb)
2024-10-11 14:25:19 +02:00
kirti purohit
4c0e79254c
vmalert: parse multi doc yaml (#6995)
### Describe Your Changes

This PR adds the feature to parse a multi yaml doc following the
`\n---\n`
The issue is
[6753](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6753)

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Signed-off-by: kirti purohit <kirti.purohit@hpe.com>
Co-authored-by: kirti purohit <kirti.purohit@hpe.com>
Co-authored-by: Jiekun <jiekun@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 008b649658)
2024-10-11 14:25:17 +02:00
Artem Fetishev
be7dfd5ab3
app/(vmagent,vmalert)/remotewrite/client: Fix flag docs (#7198)
### Describe Your Changes

The flags docs mention the flag that does not exist (and never existed).
Perhaps that was a typo.

`s/retryMaxInterval/retryMaxTime/g`

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

Signed-off-by: Artem Fetishev <rtm@victoriametrics.com>
(cherry picked from commit e2c73dc89f)
2024-10-11 14:25:15 +02:00
Aliaksandr Valialkin
00bd1c68a2
deployment: update VictoriaLogs image tag from v0.33.0-victorialogs to v0.34.0-victorialogs
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.34.0-victorialogs
2024-10-08 12:30:49 +02:00
Aliaksandr Valialkin
db75455fbd
docs/VictoriaLogs/CHANGELOG.md: cut v0.34.0 release 2024-10-08 12:21:50 +02:00
Zakhar Bessarab
06947c2685
vmagent: add support of HTTP2 client for Kubernetes SD (#7114)
### Describe Your Changes

Currently, vmagent always uses a separate `http.Client` for every group
watcher in Kubernetes SD. With a high number of group watchers this
leads to large amount of opened connections.

This PR adds 2 changes to address this:
- re-use of existing `http.Client` - in case `http.Client` is connecting
to the same API server and uses the same parameters it will be re-used
between group watchers
- HTTP2 support - this allows to reuse connections more efficiently due
to ability of using streaming via existing connections.

See this issue for the details and test results -
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5971

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
(cherry picked from commit eefae85450)
2024-10-08 10:37:48 +02:00
Zakhar Bessarab
b9115d6882
make: add darwin builds for cluster (#7195)
### Describe Your Changes

Add darwin `amd64` and `arm64` builds for cluster binaries build.

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
2024-10-08 10:19:04 +02:00
Aliaksandr Valialkin
0a61222627
lib/logstorage: quote logfmt strings only if they contain special chars, which could break logfmt parsing and/or reading
(cherry picked from commit 462b7cd597)
2024-10-07 14:47:22 +02:00
Roman Khavronenko
f825a9de80
app/vmselect/promql: fix seriesFetched update logic (#7181)
### Describe Your Changes

evalInstantRollup could have overreport the number of fetched series if
`offset` checks will result into retry. This change updates fetched
series only if these checks were successful.

It also adds a comment to another potential place of over-reporting
series fetched. It doesn't fix it, because it would require spending
extra resources on such a check, while discrepancy in seriesFetched
doesn't affect calculations in any way.

Probably related to
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7170

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

Signed-off-by: hagen1778 <roman@victoriametrics.com>

(cherry picked from commit ebd393d8b3)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-10-07 14:47:22 +02:00
Github Actions
2d4cde5506
Automatic update operator docs from VictoriaMetrics/operator@80f95d2 (#7191)
Automated changes by
[create-pull-request](https://github.com/peter-evans/create-pull-request)
GitHub action

Signed-off-by: Github Actions <133988544+victoriametrics-bot@users.noreply.github.com>
Co-authored-by: Haleygo <39937150+Haleygo@users.noreply.github.com>
(cherry picked from commit 5481fa669c)
2024-10-07 14:46:22 +02:00
Meysam
42dfb2ffa0
docs: fix typo in vmagent docs (#7192)
(cherry picked from commit 5e7b3e035b)
2024-10-07 14:46:22 +02:00
Aliaksandr Valialkin
efe5935497
app/vlogscli: add ability to display query results in logfmt, single-line and multi-line json modes
(cherry picked from commit 492190885d)
2024-10-07 14:46:21 +02:00