Commit graph

10668 commits

Author SHA1 Message Date
Aliaksandr Valialkin
151eb1e4b6
docs/VictoriaLogs/logsql-examples.md: replace last pipe with first pipe, since it is easier to understand 2025-01-16 04:21:09 +01:00
Aliaksandr Valialkin
5e4de8e860
docs/VictoriaLogs/LogsQL.md: use proper backticks around hexnumencode: 2025-01-16 04:12:37 +01:00
Aliaksandr Valialkin
6312d3bbba
docs/VictoriaLogs: typo fixes for block_stats pipe docs 2025-01-16 03:53:50 +01:00
Aliaksandr Valialkin
d2bede6b51
docs/VictoriaLogs/LogsQL.md: add missing they word 2025-01-16 03:40:42 +01:00
Aliaksandr Valialkin
5ca5069fc4
docs/VictoriaLogs/LogsQL.md: fix incorrect url to VictoriaMetrics histogram buckets
This is a follow-up for d2a791bef3
2025-01-16 00:02:51 +01:00
Aliaksandr Valialkin
8a3c460f63
deployment/docker: update VictoriaLogs Docker images from v1.5.0-victorialogs to v1.6.0-victorialogs
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.6.0-victorialogs
2025-01-15 22:37:24 +01:00
Aliaksandr Valialkin
ca653a515c
docs/VictoriaLogs/CHANGELOG.md: cut v1.6.0-victorialogs release 2025-01-15 22:31:13 +01:00
Aliaksandr Valialkin
e5b4cf33bf
lib/logstorage: make golangci-lint happy after f27e120aeb 2025-01-15 22:28:13 +01:00
Aliaksandr Valialkin
e24a8f2088
deployment/docker: update Alpine Docker image from 3.21.0 to 3.21.2
See https://alpinelinux.org/posts/Alpine-3.21.1-released.html and https://alpinelinux.org/posts/Alpine-3.18.11-3.19.6-3.20.5-3.21.2-released.html
2025-01-15 22:26:26 +01:00
Aliaksandr Valialkin
f27e120aeb
lib/logstorage: add union pipe, which allows uniting results from multiple queries 2025-01-15 22:22:07 +01:00
Aliaksandr Valialkin
ee1ce90501
lib/logstorage: properly drop temporary directories created by filter* tests 2025-01-15 22:22:07 +01:00
Aliaksandr Valialkin
47fe8cf3be
lib/logstorage: math pipe: add rand() function 2025-01-15 22:22:06 +01:00
Daria Karavaieva
5813aa6602
docs/vmanomaly: fix parameters field width (#8025)
### Describe Your Changes

- fix parameters width in Components section - reader, wtiter,
monitoring, scheduler

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2025-01-15 20:17:53 +01:00
Aliaksandr Valialkin
b4f4ece162
lib/logstorage: improve performance of unique pipe for integer columns with big number of unique values 2025-01-15 19:53:10 +01:00
Aliaksandr Valialkin
bb00f7529f
lib/logstorage: improve performance when applying math calculations for _time, const and dict values 2025-01-15 19:53:10 +01:00
Roman Khavronenko
ad3bd11334
docs: mention regression in v1.109.0 (#8046)
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/8045

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2025-01-15 16:26:24 +01:00
Github Actions
875c6663ef
Automatic update helm docs from VictoriaMetrics/helm-charts@d8e5f03 (#8035)
Automated changes by
[create-pull-request](https://github.com/peter-evans/create-pull-request)
GitHub action

Signed-off-by: Github Actions <133988544+victoriametrics-bot@users.noreply.github.com>
Co-authored-by: AndrewChubatiuk <3162380+AndrewChubatiuk@users.noreply.github.com>
2025-01-15 15:16:02 +08:00
Zakhar Bessarab
b48b7c454a
docs: update references to v1.109.0 (#8034)
### Describe Your Changes

Please provide a brief description of the changes you made. Be as
specific as possible to help others understand the purpose and impact of
your modifications.

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2025-01-14 16:42:11 +01:00
Zakhar Bessarab
f523348b3f
deployment/docker: update to v1.109.0 (#8032)
### Describe Your Changes

Please provide a brief description of the changes you made. Be as
specific as possible to help others understand the purpose and impact of
your modifications.

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2025-01-14 16:39:06 +01:00
Zakhar Bessarab
63bf1e008f
docs/changelog: add missing entry for #7182 (#8030)
### Describe Your Changes

Please provide a brief description of the changes you made. Be as
specific as possible to help others understand the purpose and impact of
your modifications.

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2025-01-14 16:30:26 +01:00
Zakhar Bessarab
419ac10c60
docs/lts-releases: update links to latest releases (#8028)
### Describe Your Changes

Please provide a brief description of the changes you made. Be as
specific as possible to help others understand the purpose and impact of
your modifications.

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2025-01-14 16:22:29 +01:00
Zakhar Bessarab
1c599d9661
Merge tag 'v1.109.0' into oss/pmm-6401-read-prometheus-data-files 2025-01-14 18:51:09 +04:00
Dmytro Kozlov
d631d2c100
cloud: update email images for cloud (#8024)
### Describe Your Changes

Updated email images with new support email

### Checklist

The following checks are **mandatory**:

- [X] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2025-01-14 14:24:33 +01:00
Zakhar Bessarab
89431458bf
docs/release-guide: move testing on sandbox step before pushing tags (#8026)
### Describe Your Changes

Please provide a brief description of the changes you made. Be as
specific as possible to help others understand the purpose and impact of
your modifications.

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2025-01-14 14:23:32 +01:00
Zakhar Bessarab
d8d0c0ac01
docs/changelog: port LTS changes, update release date (#8027)
### Describe Your Changes

Please provide a brief description of the changes you made. Be as
specific as possible to help others understand the purpose and impact of
your modifications.

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
2025-01-14 14:23:01 +01:00
Roman Khavronenko
c0f5699bad
docs: cut new changelog doc for 2024 (#8023)
Cutting new changelod doc reduces the size of the current's year
changelog and improves navigation for users.

### Describe Your Changes

Please provide a brief description of the changes you made. Be as
specific as possible to help others understand the purpose and impact of
your modifications.

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2025-01-14 11:31:16 +01:00
Nikolay
277fdd1070
lib/storage: reduce test suite batch size (#8022)
Commit eef6943084 added new test
functions. Which checks various cases for metricName registration at
data ingestion.
Initial dataset size had 4 batches with 100 rows each. It works fine at
machines with 5GB+ memory.
But i386 architecture supports only 4GB of memory per process.

Due to given limitations, batch size should be reduced to 3 batches and
30 rows. It keeps the same
test funtionality, but reduces overall memory usage to ~3GB.

Signed-off-by: f41gh7 <nik@victoriametrics.com>
2025-01-14 11:27:50 +01:00
Roman Khavronenko
d290efb849
lib/opentlemetry: throttle log messages during parsing (#8021)
Samples parsing is a hot path. Bad client could easily overwhelm
receiver with bad or unsupported data. So it is better to throttle such
messages.

Follow-up after
b26a68641c

### Describe Your Changes

Please provide a brief description of the changes you made. Be as
specific as possible to help others understand the purpose and impact of
your modifications.

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2025-01-14 11:03:11 +01:00
chenlujjj
b26a68641c
lib/opentelemetry: log the metric name of unsupported metrics (#8018)
To resolve:
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/8009
Log the name of unsupported metrics.
2025-01-14 10:49:30 +01:00
Aliaksandr Valialkin
b88cda5c41
lib/logstorage: make golangci-lint happy after the commit d2a791bef3 2025-01-13 22:31:33 +01:00
Aliaksandr Valialkin
d2a791bef3
lib/logstorage: add histogram stats function for calculating histogram buckets over numeric fields 2025-01-13 22:30:19 +01:00
Aliaksandr Valialkin
99516a5730
lib/logstorage: top pipe: allow mixing the order of hits and rank suffixes 2025-01-13 22:30:19 +01:00
Aliaksandr Valialkin
aecc86c390
lib/logstorage: do not copy pipeTopkProcessorShard when obtaining parition keys 2025-01-13 22:30:19 +01:00
Aliaksandr Valialkin
500b54f5aa
app/vlogscli: typo fix, which could result in incomplete results in compact mode 2025-01-13 22:30:18 +01:00
Aliaksandr Valialkin
cc29692e27
lib/logstorage: track integer field values in integer map for top N (int_field)
This reduces memory usage by up to 2x for the map used for tracking hits.
This also reduces CPU usage for tracking integer fields.
2025-01-13 22:30:18 +01:00
Aliaksandr Valialkin
f018aa33cb
lib/logstorage: avoid callback overhead at visitValuesReadonly
Process values in batches instead of passing every value in the callback.
This improves performance of reading the encoded values from storage by up to 50%.
2025-01-13 22:30:17 +01:00
Daria Karavaieva
92b6475fa6
docs/vmanomaly: fix table rendering (#8005)
### Describe Your Changes

- fix table rendering on writer and scheduler pages

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2025-01-13 14:19:16 +01:00
Andrii Chubatiuk
bda3546cfd
docs: added search resupts page for docs.victoriametrics.com (#8017)
### Describe Your Changes

added search page required for docs site

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2025-01-13 14:18:25 +01:00
Artem Navoiev
2691cdefe3
docs: cloud use separate support email for cloud (#8013)
### Describe Your Changes

Please provide a brief description of the changes you made. Be as
specific as possible to help others understand the purpose and impact of
your modifications.

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2025-01-13 14:17:38 +01:00
Github Actions
93b8aa5c9d
Automatic update helm docs from VictoriaMetrics/helm-charts@457951a (#8014)
Automated changes by
[create-pull-request](https://github.com/peter-evans/create-pull-request)
GitHub action

Signed-off-by: Github Actions <133988544+victoriametrics-bot@users.noreply.github.com>
Co-authored-by: f41gh7 <18450869+f41gh7@users.noreply.github.com>
2025-01-13 12:12:48 +01:00
Aliaksandr Valialkin
7a7f188133
deployment/docker: update VictoriaLogs Docker image tag from v1.4.0-victorialogs to v1.5.0-victorialogs
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.5.0-victorialogs
2025-01-13 07:34:33 +01:00
Aliaksandr Valialkin
3e00fae3f4
docs/VictoriaLogs/CHANGELOG.md: cut v1.5.0-victorialogs release 2025-01-13 07:28:08 +01:00
Roman Khavronenko
ee3c0c6a87
make: bump golangci-lint to v1.63.4 (
New version has additional checks and reduced resource consumption, so
it doesn't timeout for our internal repos.

To make linter happy, I addressed "redefinition of the built-in
function" lint error.

----
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2025-01-13 07:18:04 +01:00
Aliaksandr Valialkin
cf7ea78588
lib/logstorage: format pipe: add frequently used formatters
- url encoding / decoding with <urlencode:field> and <urldecode:field>
- base64 encoding / decoding with <base64encode:field> and <base64decode:field>
- hex encoding / decoding with <hexencode:field> and <hexdecode:field>
- hex encoding for integers with <hexnumencode:field> and <hexnumdecode:field>
2025-01-13 07:08:43 +01:00
Aliaksandr Valialkin
186aa3bb0e
lib/logstorage: explicitly pass statsFunc to statsProcessor methods
This allows reducing the state of every statsProcessor by removing pointer to the corresponding statsFunc.
For example, this reduces statsCountProcessor size by 2x.
2025-01-13 04:49:39 +01:00
Aliaksandr Valialkin
e368f687a7
lib/logstorage: stats pipe: stop finalizeStats() as soon as the query is canceled
Previoysly finalizeStats() for some functions such as count_uniq() could run for long periods
of time after the query is canceled, since stopCh wan't propagated to finalizeStats().
2025-01-13 03:38:09 +01:00
Aliaksandr Valialkin
0214aa328e
lib/logstorage: stats pipe: use integer group keys if stats by(...) contains a single field with integer values
This reduces memory usage and improves performance, since access to a map with integer keys
is faster than access to a map with string keys.
2025-01-13 03:22:00 +01:00
Aliaksandr Valialkin
dd919eeee6
lib/logstorage: count_uniq and count_uniq_hash stats functions: avoid converting integer values to strings
Prevsiously integer values were converted to strings before being passed to `updateState()` function at `count_uniq`
and `count_uniq_hash`. Later such values are converted back to integers in order to track them via integer map of unique values.

This commit avoids the int -> string -> int conversion. Instead, it passes integers directly to the integer map of unique values.
This improves performance of `count_uniq` and `count_uniq_hash` functions even further.
2025-01-13 02:45:28 +01:00
Aliaksandr Valialkin
3f22d06b0c
lib/logstorage: add value_type filter to LogsQL
This filter can be used when debugging and exploring logs in order to understand better
which value types are used for storing the particular log fields.

The `value_type` filter complements `block_stats` pipe.
2025-01-12 22:21:39 +01:00
Aliaksandr Valialkin
b812de236b
lib/logstorage: run make fmt after e610edf045 2025-01-12 03:17:57 +01:00