Commit graph

4901 commits

Author SHA1 Message Date
Zakhar Bessarab
eefae85450
vmagent: add support of HTTP2 client for Kubernetes SD (#7114)
### Describe Your Changes

Currently, vmagent always uses a separate `http.Client` for every group
watcher in Kubernetes SD. With a high number of group watchers this
leads to large amount of opened connections.

This PR adds 2 changes to address this:
- re-use of existing `http.Client` - in case `http.Client` is connecting
to the same API server and uses the same parameters it will be re-used
between group watchers
- HTTP2 support - this allows to reuse connections more efficiently due
to ability of using streaming via existing connections.

See this issue for the details and test results -
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5971

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
2024-10-08 10:36:31 +02:00
Zakhar Bessarab
9b6efb5e81
make: add darwin builds for cluster (#7195)
### Describe Your Changes

Add darwin `amd64` and `arm64` builds for cluster binaries build.

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>

(cherry picked from commit b9115d6882)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-10-08 10:19:58 +02:00
Github Actions
5481fa669c
Automatic update operator docs from VictoriaMetrics/operator@80f95d2 (#7191)
Automated changes by
[create-pull-request](https://github.com/peter-evans/create-pull-request)
GitHub action

Signed-off-by: Github Actions <133988544+victoriametrics-bot@users.noreply.github.com>
Co-authored-by: Haleygo <39937150+Haleygo@users.noreply.github.com>
2024-10-07 14:26:39 +02:00
Meysam
5e7b3e035b
docs: fix typo in vmagent docs (#7192) 2024-10-07 14:26:20 +02:00
Aliaksandr Valialkin
492190885d
app/vlogscli: add ability to display query results in logfmt, single-line and multi-line json modes 2024-10-07 12:20:06 +02:00
Artem Navoiev
e144a2b062
docs: change tier menu item name
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2024-10-07 12:12:45 +02:00
Artem Navoiev
2593f32b63
docs: replace cloud url to be consistent with the other ones (#7194)
### Describe Your Changes

Please provide a brief description of the changes you made. Be as
specific as possible to help others understand the purpose and impact of
your modifications.

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2024-10-07 10:44:01 +02:00
Aliaksandr Valialkin
daad96b3a5
app/vlogscli: return back sorting result fields by name
This simplifies locating the needed field when the number of fields per each returned result is big
2024-10-07 10:41:48 +02:00
Artem Navoiev
18dd4105be
docs: add audit logs page for Cloud (#7193)
### Describe Your Changes

Please provide a brief description of the changes you made. Be as
specific as possible to help others understand the purpose and impact of
your modifications.

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2024-10-07 01:38:00 -07:00
Fred Navruzov
5c9bd35eb9
docs/vmanomaly: remove duplicate header in VmWriter docs (#7189)
### Describe Your Changes

docs/vmanomaly: remove duplicate header in VmWriter docs

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-10-06 09:49:04 -07:00
Fred Navruzov
b2e7b05918
docs/vmanomaly: release 1.16.2 (#7188)
### Describe Your Changes

docs for `vmanomaly`, updated after release 1.16.2

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-10-06 18:54:43 +03:00
Aliaksandr Valialkin
596e4de248
app/vlogscli: preserve the original order of fields in the displayed responses 2024-10-05 21:27:32 +02:00
Github Actions
4d9ad9654f
Automatic update helm docs from VictoriaMetrics/helm-charts@65cc293 (#7180)
Automated changes by
[create-pull-request](https://github.com/peter-evans/create-pull-request)
GitHub action

Signed-off-by: Github Actions <133988544+victoriametrics-bot@users.noreply.github.com>
Co-authored-by: AndrewChubatiuk <3162380+AndrewChubatiuk@users.noreply.github.com>
2024-10-04 10:50:23 +02:00
Aliaksandr Valialkin
66645c3dff
docs/VictoriaLogs/LogsQL.md: remove redundant replace word in examples for conditional replace
Thanks to @p5i for the bugreport at https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7168
2024-10-03 20:26:58 +02:00
Aliaksandr Valialkin
364f084b43
lib/logstorage: add len pipe for calculating byte length of log field values 2024-10-03 18:21:10 +02:00
Fred Navruzov
63a9fb34d1
docs/vmanomaly: fix example snippets (#7174)
### Describe Your Changes

fix of typos and improper version references in code snippets of example
usage

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-10-03 18:30:24 +04:00
hagen1778
3f2bfd2ff6
docs: move Retry-After to the 1.104.0 notes
It was mistakenly place to 1.103.0

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-10-03 15:23:03 +02:00
hagen1778
7f47713821
docs: add missing -search.maxDeleteSeries to vmselect flags
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-10-03 14:11:04 +02:00
hagen1778
4086cef01c
docs: rm incorrectly placed bugfix change from v1.103
The change was present in v1.103 by mistake.
In fact, it was released in v1.104
See c193e6d43e

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-10-03 09:55:07 +02:00
hagen1778
80d4acf2cf
docs: add tickets routine to release guide
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-10-03 09:49:07 +02:00
hagen1778
feba481ac2
docs: re-qualify -search.maxDeleteSeries change into feature
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-10-02 21:18:54 +02:00
hagen1778
ce81a86fc2
docs: re-order changes by priority
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-10-02 21:15:59 +02:00
hagen1778
41850995d3
docs: rm vm_rows_ignored_total{reason="nan_value"}
It was reverted in 0d4f4b8f7d

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-10-02 21:13:53 +02:00
Artem Navoiev
ddfb6db8cf
docs: api cloud clarify key value property
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2024-10-02 19:46:11 +02:00
Artem Navoiev
5df015b9db
docs: api cloud fix the naming
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2024-10-02 18:00:43 +02:00
hagen1778
bd84f8a35d
docs: bump VM latest LTS releases
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-10-02 16:34:06 +02:00
hagen1778
4c6b7ce6da
docs: mention to test releases before publishing them in release guide
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-10-02 16:31:11 +02:00
hagen1778
8592fc3162
docs: add link to docs for multitenant reads
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-10-02 16:27:20 +02:00
Fred Navruzov
150ee902fd
docs/vmanomaly: patch release 1.16.1 (#7169)
### Describe Your Changes

`vmanomaly` patch release 1.16.1 updates

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-10-02 18:10:08 +04:00
hagen1778
01bc28eda2
docs: mention version available for multitenant reads
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-10-02 14:58:06 +02:00
hagen1778
cd8a478a8d
docs: actualize multitenancy docs
Follow-up after https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6346

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-10-02 14:35:29 +02:00
hagen1778
36acde1d11
docs: add missing release notes
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-10-02 14:20:55 +02:00
Artem Navoiev
c04c377d09
docs: cloud API remove link to guide as we don't have it
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2024-10-02 11:10:38 +02:00
Artem Navoiev
504da7d02b
docs: add API doc for cloud (#7164)
### Describe Your Changes

Please provide a brief description of the changes you made. Be as
specific as possible to help others understand the purpose and impact of
your modifications.

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2024-10-02 10:58:35 +02:00
Fred Navruzov
dde2a0cb25
docs/vmanomaly - update versions in examples (#7163)
### Describe Your Changes

update `vnanomaly` versions in examples

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-10-02 11:53:05 +03:00
Fred Navruzov
0e54cfe350
docs/vmanomaly - release 1.16.0 docs (#7159)
### Describe Your Changes

doc updates for vmanomaly v1.16.0

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-10-02 00:19:14 +02:00
Aliaksandr Valialkin
3634fefc64
docs/VictoriaLogs/querying/vlogscli.md: typo fixes 2024-10-01 21:10:05 +02:00
f41gh7
776c501cb2
CHANGELOG.md: cut v1.104.0 release 2024-10-01 16:55:04 +02:00
f41gh7
076a1f84e1
vmselect: add support of multi-tenant queries
Added  ability to query data across multiple tenants. See:
VictoriaMetrics/VictoriaMetrics#1434

Currently, the following endpoints work with multi-tenancy:
- /prometheus/api/v1/query
- /prometheus/api/v1/query_range
- /prometheus/api/v1/series
- /prometheus/api/v1/labels
- /prometheus/api/v1/label/<label_name>/values
- /prometheus/api/v1/status/active_queries
- /prometheus/api/v1/status/top_queries
- /prometheus/api/v1/status/tsdb
- /prometheus/api/v1/export
- /prometheus/api/v1/export/csv
- /vmui

A note regarding VMUI: endpoints such as `active_queries` and
`top_queries` have been updated to indicate whether query was a
single-tenant or multi-tenant, but UI needs to be updated to display
this info.
cc: @Loori-R

---------

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
Signed-off-by: f41gh7 <nik@victoriametrics.com>
Co-authored-by: f41gh7 <nik@victoriametrics.com>
2024-10-01 16:49:46 +02:00
Aliaksandr Valialkin
496015aa0e
docs/VictoriaLogs/querying/vlogscli.md: provide instructions on how to download run vlogscli 2024-10-01 16:13:12 +02:00
Aliaksandr Valialkin
b05fbee63d
deployment: update VictoriaLogs docker image from v0.32.1-victorialogs to v0.33.0-victorialogs
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.33.0-victorialogs
2024-10-01 13:57:56 +02:00
Aliaksandr Valialkin
234c81754e
docs/VictoriaLogs/CHANGELOG.md: cut v0.33.0-victorialogs release 2024-10-01 13:42:18 +02:00
Roman Khavronenko
d57d8b5e60
docs: add note that otel is not using streaming parsing (#7148)
### Describe Your Changes

Please provide a brief description of the changes you made. Be as
specific as possible to help others understand the purpose and impact of
your modifications.

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-10-01 13:39:40 +02:00
Aliaksandr Valialkin
a350be48b6
lib/logstorage: do not count dictionary values which have no matching logs in count_uniq stats function
Create blockResultColumn.forEachDictValue* helper functions for visiting matching
dictionary values. These helper functions should prevent from counting dictionary values
without matching logs in the future.

This is a follow-up for 0c0f013a60
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7152
2024-10-01 13:34:45 +02:00
Aliaksandr Valialkin
630211cfed
app/vlogscli: add interactive command-line tool for querying VictoriaLogs 2024-10-01 12:23:07 +02:00
Aliaksandr Valialkin
61ae077e43
docs/VictoriaLogs/FAQ.md: clarify answers a bit 2024-10-01 12:08:27 +02:00
Nikolay
fbaa026ae6
dashboards: updates operator dashboard (#7139)
* Replaces deprecated graphs with Timeseries panels
* Adds new latency dashboards for rest client and golang scheduler
* Adds new overview panels
* Adds VM Datasource version of dashboard

---------

Signed-off-by: f41gh7 <nik@victoriametrics.com>
Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
2024-09-30 15:35:39 +02:00
Artem Navoiev
0b2d3d7752
docs: cloud add support page. remove title as far it's already there
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2024-09-30 14:45:33 +02:00
Zhu Jiekun
7bb8853a5c
feature: [vmagent] Add service discovery support for OVH Cloud VPS and dedicated server (#6160)
### Describe Your Changes
related issue:
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6071

#### Added
- Added service discovery support for OVH Cloud:
    - VPS.
    - Dedicated server.

#### Docs
- `CHANGELOG.md`, `sd_configs.md`, `vmagent.md` are updated.

#### Note
- Useful links: 
    - OVH Cloud VPS API: https://eu.api.ovh.com/console/#/vps~GET
- OVH Cloud Dedicated server API:
https://eu.api.ovh.com/console/#/dedicated/server~GET
    - OVH Cloud SDK: https://github.com/ovh/go-ovh
- Prometheus SD:
https://prometheus.io/docs/prometheus/latest/configuration/configuration/#ovhcloud_sd_config

Tested on OVH Cloud VPS and dedicated server.
<img width="1722" alt="image"
src="https://github.com/VictoriaMetrics/VictoriaMetrics/assets/30280396/d3f0adc8-b0ef-423e-9379-8a9b9b0792ee">

<img width="1724" alt="image"
src="https://github.com/VictoriaMetrics/VictoriaMetrics/assets/30280396/18b5b730-3512-4fc0-8b2c-f2450ac550fd">

---
Signed-off-by: Jiekun <jiekun@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
2024-09-30 14:42:46 +02:00
Aliaksandr Valialkin
aafa9262c5
deployment/docker: update VictoriaLogs from v0.32.0-victorialogs to v0.32.1-victorialogs
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.32.1-victorialogs
2024-09-30 14:35:40 +02:00
Aliaksandr Valialkin
82482fca4b
docs/VictoriaLogs/CHANGELOG.md: cut v0.32.1-victorialogs release 2024-09-30 14:31:17 +02:00
Artem Navoiev
f5f04c903b
docs: cloud add support page (#7143)
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2024-09-30 14:29:26 +02:00
Hui Wang
664f337c70
stream aggregation: fix possible duplicated aggregation results (#7118)
When ingesting samples with the same labels(duplicated samples or
samples with the same labels after `by` or `without` options). They
could register different entries for the same labelset in
LabelsCompressor.
For example, both index 99 and 100 can be assigned to label `foo=1` in
two concurrent pushes. Then due to differing label indexes in encoded
keys, the samples will appear as distinct in aggrState, resulting in
duplicated results after decompressing the label indexes.

fbde238cdc/lib/streamaggr/streamaggr.go (L933)

In this pull request, since we need to store `idxToLabel` first to
ensure the idx can be searched after `lc.labelToIdxStore`,
the `lc.idxToLabel` still could contain a duplicated entries
[100]="foo=1". But given the low likelihood of this issue and the size
of idxToLabel, it should be fine.
2024-09-30 14:24:59 +02:00
Aliaksandr Valialkin
0c0f013a60
lib/logstorage: skip values with zero hits for 'uniq', 'top' and 'field_values' pipes
See https://github.com/VictoriaMetrics/victorialogs-datasource/issues/72#issuecomment-2352078483
2024-09-30 14:15:07 +02:00
f41gh7
758f42fc12
docs: add Update Note for upcoming release changes
Signed-off-by: f41gh7 <nik@victoriametrics.com>
2024-09-30 12:37:30 +02:00
Artem Fetishev
ed5da38ede
Introduce a flag for limiting the number of time series to delete (#7091)
### Describe Your Changes

Introduce the `-search.maxDeleteSeries` flag that limits the number of
time series that can be deleted with a single
`/api/v1/admin/tsdb/delete_series` call.

Currently, any number can be deleted and if the number is big (millions)
then the operation may result in unaccounted CPU and memory usage spikes
which in some cases may result in OOM kill (see #7027). The flag limits
the number to 30k by default and the users may override it if needed at
the vmstorage start time.


---------

Signed-off-by: Artem Fetishev <rtm@victoriametrics.com>
Co-authored-by: Nikolay <nik@victoriametrics.com>
2024-09-30 10:02:21 +02:00
hagen1778
2df2e9b92f
docs: use global dedup flag in vmagent's FlexibleDeduplication docs
Current doc is using per-url deduplication, and users might use this example
when they have more than 1 remoteWrite URL. Which would result into extra resource usage.
Changing the example to use global dedup, as it makes more sense.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-09-30 09:08:56 +02:00
Github Actions
3d01bc3fbe
Automatic update helm docs from VictoriaMetrics/helm-charts@de4356f (#7138)
Automated changes by
[create-pull-request](https://github.com/peter-evans/create-pull-request)
GitHub action

Signed-off-by: Github Actions <133988544+victoriametrics-bot@users.noreply.github.com>
Co-authored-by: f41gh7 <18450869+f41gh7@users.noreply.github.com>
2024-09-30 13:03:15 +08:00
Github Actions
5e67e611b8
Automatic update operator docs from VictoriaMetrics/operator@0e44654 (#7136)
Automated changes by
[create-pull-request](https://github.com/peter-evans/create-pull-request)
GitHub action

Signed-off-by: Github Actions <133988544+victoriametrics-bot@users.noreply.github.com>
Co-authored-by: f41gh7 <18450869+f41gh7@users.noreply.github.com>
2024-09-29 22:30:24 +02:00
Aliaksandr Valialkin
2047ad20ef
deployment: update VictoriaLogs docker image from v0.31.0-victorialogs to v0.32.0-victorialogs
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.32.0-victorialogs
2024-09-29 14:54:21 +02:00
Aliaksandr Valialkin
45cfb6b526
docs/VictoriaLogs/CHANGELOG.md: cut v0.32.0-victorialogs 2024-09-29 14:47:31 +02:00
Nikolay
3bbb2aed72
fscore: rollback trailing space trim (#7106)
Previous commit 201fd6de1e removed
trailing space trim from data read from file. But common practice is to
remove such trailing space. And it leaded to the authorization errors
for the major group of users.

In first place, this change must help to mitigate an issue with
kubernetes. When authorization information was read from Secret content.
Changes to the operator was made to mitigate such problem at commit
1cf64358c8

We could introduce later optional flag for VictoriaMetrics to disable
trim space behavior.

Related issues:
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6986
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7089 
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6947

---------

Signed-off-by: f41gh7 <nik@victoriametrics.com>
Co-authored-by: Zhu Jiekun <jiekun@victoriametrics.com>
2024-09-29 10:59:25 +02:00
Aliaksandr Valialkin
55eb321f77
lib/logstorage: clear hits slice obtained from encoding.GetUint64s() before updating it with hits for valueTypeDict column
encoding.GetUint64s() returns uninitialized slice, which may contain arbitrary values.
So values in this slice must be reset to zero before using it for counting hits in `uniq` and `top` pipes.
2024-09-29 10:29:13 +02:00
Aliaksandr Valialkin
0b91452ca4
lib/logstorage: add non-empty if (...) condition to automatically generated result names in stats pipe
This allows executing queries with `stats` pipe, which calculate multiple results with the same functions,
but with different `if (...)` conditions. For example:

  _time:5m | count(), count() if (error)

Previously such queries couldn't be executed becasue automatically generated name for the second result
didn't include `if (error)`, so names for both results were identical - `count(*)`.
2024-09-29 09:51:28 +02:00
Aliaksandr Valialkin
8772aea24b
lib/logstorage: support order alias for sort pipe
Now the following queries are equivalents:

    _time:5s | sort by (_time)

    _time:5s | order by (_time)

This is needed for convenience, since `order by` is commonly used in other query languages such as SQL.
2024-09-29 09:51:27 +02:00
Artem Navoiev
14a0396f53
docs: changelog fix typo in url
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2024-09-28 23:30:19 +02:00
Artem Navoiev
96efe99eef
docs: mention new create backup api in docs and changelog (#7104)
### Describe Your Changes

Please provide a brief description of the changes you made. Be as
specific as possible to help others understand the purpose and impact of
your modifications.

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2024-09-28 14:28:58 -07:00
Aliaksandr Valialkin
806bc2ac58
app/vlinsert: support unix timestamps in seconds and milliseconds in JSON stream data ingestion API 2024-09-28 21:56:50 +02:00
Aliaksandr Valialkin
7d7d7c03bc
app/vlinsert: accept unix timestamp in seconds additionally to milliseconds at ElasticSearch bulk API
Timestamps in seconds are sometimes used for data ingestion via ElasticSearch bulk API
2024-09-28 21:19:54 +02:00
Github Actions
e62b88b2e0
Automatic update operator docs from VictoriaMetrics/operator@d39fc94 (#7129)
Automated changes by
[create-pull-request](https://github.com/peter-evans/create-pull-request)
GitHub action

Signed-off-by: Github Actions <133988544+victoriametrics-bot@users.noreply.github.com>
Co-authored-by: f41gh7 <18450869+f41gh7@users.noreply.github.com>
2024-09-27 19:03:14 +02:00
Roman Khavronenko
59bc63ebc4
app/vmalert: mention labels conflict resolution strategy (#7085)
The change should help users to understand what happens on labels
conflict.

### Describe Your Changes

Please provide a brief description of the changes you made. Be as
specific as possible to help others understand the purpose and impact of
your modifications.

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-09-27 14:41:33 +02:00
Andrii Chubatiuk
05a64a8c14
victorialogs: marked fluentd support in roadmap, added syslog example (#7098)
### Describe Your Changes

Marked fluentd in victorialogs roadmap
Added fluentd syslog example setup

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-09-27 14:38:39 +02:00
Aliaksandr Valialkin
86c0eb816c
deployment: update VictoriaLogs docker image from v0.30.1-victorialogs to v0.31.0-victorialogs
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.31.0-victorialogs
2024-09-27 13:58:54 +02:00
Aliaksandr Valialkin
58c69386c7
docs/VictoriaLogs/CHANGELOG.md: cut v0.31.0-victorialogs release 2024-09-27 13:54:17 +02:00
Yury Molodov
25a9802ca4
vmui: add link to vmalert (#7088)
### Describe Your Changes

Add link to VMalert when proxy is enabled. The link is displayed when
the `-vmalert.proxyURL` flag is present.

#5924


![image](https://github.com/user-attachments/assets/c45ca884-8912-4bd9-a867-df5919f278a1)

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2024-09-27 13:22:22 +02:00
Yury Molodov
8657d03433
vmui/logs: improve graph usability (#7025)
### Describe Your Changes

- Show the time range in the tooltip when hovering over staircase
graphs.
- Use bolder lines for staircase graphs.
- Increase the number of steps on the staircase graph to 100.
- Reduce the maximum width of the tooltip to 1/3 of the screen.
- Insert only the label name under the cursor into the query input field
when `Ctrl`-clicking the line legend.

See [this
comment](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6545#issuecomment-2336805237).

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2024-09-27 13:19:46 +02:00
Aliaksandr Valialkin
09b309a82e
lib/logstorage: allow using - instead of ! as a shorthand for NOT operator in LogsQL 2024-09-27 13:14:47 +02:00
Aliaksandr Valialkin
76c1b0b8ea
lib/logstorage: support skipping _stream: prefix for stream filters
'_stream:{...}' can be written as '{...}'

This simplifies writing queries with stream filters, and makes them more familier to Loki users.
2024-09-27 13:14:46 +02:00
Hui Wang
fbde238cdc
stream aggregation: support configuring multiple labels per `remoteWrite… (#7073)
….url` using `-remoteWrite.streamAggr.dropInputLabels`

Before, labels were set to all the `remoteWrite.url`.

address https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6780

---------

Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
2024-09-27 12:21:09 +02:00
Yury Molodov
c896bf340d
vmui: add functionality to preserve selected columns (#7037)
### Describe Your Changes

1) Changed table settings from a popup to a modal window to simplify
future functionality additions.
2) Added functionality to save selected columns when data is modified or
the page is reloaded. See #7016.

<details>
  <summary>Example screenshots</summary>
  
<img alt="demo-1" width="600"
src="https://github.com/user-attachments/assets/a5d9a910-363c-4931-8b12-18ea8b3d97d8"/>
  
</details>


### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
2024-09-27 11:52:01 +02:00
Github Actions
2d26d3e3de
Automatic update operator docs from VictoriaMetrics/operator@db0d09f (#7111)
Automated changes by
[create-pull-request](https://github.com/peter-evans/create-pull-request)
GitHub action

Signed-off-by: Github Actions <133988544+victoriametrics-bot@users.noreply.github.com>
Co-authored-by: f41gh7 <18450869+f41gh7@users.noreply.github.com>
2024-09-27 11:42:10 +02:00
Github Actions
ea3b20622a
Automatic update helm docs from VictoriaMetrics/helm-charts@ed0f351 (#7115)
Automated changes by
[create-pull-request](https://github.com/peter-evans/create-pull-request)
GitHub action

Signed-off-by: Github Actions <133988544+victoriametrics-bot@users.noreply.github.com>
Co-authored-by: AndrewChubatiuk <3162380+AndrewChubatiuk@users.noreply.github.com>
2024-09-27 11:41:24 +02:00
Aliaksandr Valialkin
b670b0e9ff
deployment: update VictoriaLogs docker image tag from v0.30.0-victorialogs to v0.30.1-victorialogs
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.30.1-victorialogs
2024-09-27 11:26:15 +02:00
Aliaksandr Valialkin
8077585303
docs/VictoriaLogs/CHANGELOG.md: cut v0.30.1-victorialogs release 2024-09-27 11:20:58 +02:00
Aliaksandr Valialkin
9367a9a6a2
lib/logstorage: consistently sort stream contexts belonging to different streams by the minimum time seen in the matching logs
This should simplify debugging of stream_context output, since it remains stable over repeated requests.
2024-09-27 11:19:26 +02:00
Aliaksandr Valialkin
13cc4a2618
docs/VictoriaLogs/LogsQL.md: use proper heading for blocks_count pipe docs
All the links in docs assume that the heading for pipe docs should end with `pipe` word.
This fixes broken links.

This is a follow-up for e9950f6307
2024-09-27 09:45:46 +02:00
Aliaksandr Valialkin
6f1fde24dc
deployment: update VictoriaLogs docker image from v0.29.0-victorialogs to v0.30.0-victorialogs
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.30.0-victorialogs
2024-09-27 09:34:09 +02:00
Aliaksandr Valialkin
31117c66d5
docs/VictoriaLogs/CHANGELOG.md: cut v0.30.0-victorialogs release 2024-09-27 09:18:31 +02:00
Aliaksandr Valialkin
4b1611267f
lib/logstorage: properly return surrounding logs outside the selected time range by stream_context pipe
Previously only logs inside the selected time range could be returned by stream_context pipe.
For example, the following query could return up to 10 surrounding logs only for the last 5 minutes,
while most users expect this query should return up to 10 surrounding logs without restrictions on the time range.

    _time:5m panic | stream_context before 10

This enables the ability to implement stream context feature at VictoriaLogs web UI: https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7063 .

Reduce memory usage when returning stream context over big log streams with millions of entries.
The new logic scans over all the log messages for the selected log stream, while keeping in memory only
the given number of surrounding logs. Previously all the logs for the given log stream on the selected time range
were loaded in memory before selecting the needed surrounding logs.
This should help https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6730 .

Reduce the scan performance for big log streams by fetching only the requested fields. For example, the following
query should be executed much faster than before if logs contain many fields other than _stream, _msg and _time:

    panic | stream_context after 30 | fields _stream, _msg, _time
2024-09-26 17:03:45 +02:00
Aliaksandr Valialkin
037652d5ae
app/vlinsert: support _time field without timezone information during data ingestion
Use local timezone of the host server in this case. The timezone can be overridden
with TZ environment variable if needed.

While at it, allow using whitespace instead of T as a delimiter between data and time
in the ingested _time field. For example, '2024-09-20 10:20:30' is now accepted
during data ingestion. This is valid ISO8601 format, which is used by some log shippers,
so it should be supported. This format is also known as SQL datetime format.

Also assume local time zone when time without timezone information is passed to querying APIs.
Previously such a time was parsed in UTC timezone. Add `Z` to the end of the time string
if the old behaviour is preferred.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6721
2024-09-26 12:49:35 +02:00
Aliaksandr Valialkin
7c86835f3c
docs/VictoriaLogs/CHANGELOG.md: typo fix: itentifying -> identifying 2024-09-26 09:41:30 +02:00
Zhu Jiekun
7185fe012b
feature: [victorialogs] drop logs without non-empty _msg field (#7056)
### Describe Your Changes

VictoriaLogs allows logs without `_msg` field or `_msg` field is empty.
This lead to incorrect search result. See:
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6785

This pull request search for non-empty `_msg` field before log entry is
added to `LogRows`.

New counter `vl_rows_dropped_total{reason="msg_not_exist"}` is
introduced.

Example log output:
```
2024-09-23T02:33:19.719Z        warn    app/vlinsert/insertutils/common_params.go:189   dropping log line without _msg field; [{@timestamp 2024-09-18T13:42:16.600000000Z} {Attributes.array.attribute ["many","values"]} {Attributes.boolean.attribute true} {Attributes.double.attribute 637.704} {Attributes.int.attribute 10} {Attributes.map.attribute.some.map.key some value} {Attributes.string.attribute some string} {Body Example ddddddddddlog record} {Resource.service.name my.service} {Scope.my.scope.attribute some scope attribute} {Scope.name my.library} {Scope.version 1.0.0} {SeverityNumber 10} {SeverityText Information} {SpanId eee19b7ec3c1b174} {TraceFlags 0} {TraceId 5b8efff798038103d269b633813fc60c}]
```

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
- [ ] Benchmark for potential performance loss.

---------

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2024-09-26 09:35:28 +02:00
Aliaksandr Valialkin
2ec0cfec62
docs/VictoriaLogs/CHANGELOG.md: document the fix for Windows build
This is a follow-up for 264c2ec6bd

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6998
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6973
2024-09-26 09:15:27 +02:00
Aliaksandr Valialkin
c6b2cac892
docs/VictoriaLogs/CHANGELOG.md: typo fix after 255d1d4e13: returns -> return 2024-09-26 09:00:55 +02:00
Aliaksandr Valialkin
255d1d4e13
app/vlselect/logsql: clone the query with the current timestamp when performing live tailing requests in the loop
Previously the original timestamp was used in the copied query, so _time:duration filters
were applied to the original time range: (timestamp-duration ... timestamp]. This resulted
in stopped live tailing, since new logs have timestamps bigger than the original time range.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7028
2024-09-26 08:57:23 +02:00
Github Actions
c89a7a0b62
Automatic update operator docs from VictoriaMetrics/operator@5271a59 (#7099)
Automated changes by
[create-pull-request](https://github.com/peter-evans/create-pull-request)
GitHub action

Signed-off-by: Github Actions <133988544+victoriametrics-bot@users.noreply.github.com>
Co-authored-by: f41gh7 <18450869+f41gh7@users.noreply.github.com>
2024-09-25 23:23:08 +02:00
Aliaksandr Valialkin
c66da8b0ba
docs/LTS-releases.md: consistently use v prefix in front of VictoriaMetrics releases 2024-09-25 19:29:30 +02:00
Aliaksandr Valialkin
e9950f6307
lib/logstorage: add blocks_count pipe
This pipe is useful for debugging purposes when the number of processed blocks must be calculated for the given query:

    <query> | blocks_count

This helps detecting the root cause of query performance slowdown in cases like https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7070
2024-09-25 19:17:48 +02:00
Aliaksandr Valialkin
4599429f51
lib/logstorage: read timestamps column when it is really needed during query execution
Previously timestamps column was read unconditionally on every query.
This could significantly slow down queries, which do not need reading this column
like in https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7070 .
2024-09-25 19:17:47 +02:00
Andrii Chubatiuk
f934f71708
docs/victorialogs/data-ingestion: removed FluentBit Elasticsearch from examples (#7093)
removed FluentBit Elasticsearch example from docs as custom headers are
not supported by elasticsearch output till
https://github.com/fluent/fluent-bit/pull/9416 is merged and released

fixes https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6985

### Describe Your Changes

Please provide a brief description of the changes you made. Be as
specific as possible to help others understand the purpose and impact of
your modifications.

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-09-25 18:43:26 +02:00
Andrii Chubatiuk
e75ae1b274
deployment: restructure victorialogs examples (#6971)
### Describe Your Changes

- Use common compose.yaml file for all victorialogs setups to set
version in a single place and override it on demand for each agent and
protocol
- Replaced multiple victorialogs instances in HA setup with single setup
with `deploy.replica` parameter set
- Added fluentd setup

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-09-25 18:33:26 +02:00
Github Actions
612be0954c
Automatic update operator docs from VictoriaMetrics/operator@1feab7d (#7092)
Automated changes by
[create-pull-request](https://github.com/peter-evans/create-pull-request)
GitHub action

Signed-off-by: Github Actions <133988544+victoriametrics-bot@users.noreply.github.com>
Co-authored-by: f41gh7 <18450869+f41gh7@users.noreply.github.com>
2024-09-25 15:02:24 +02:00
Roman Khavronenko
6b1b47df54
app/vmalert: bump default values for sending data to remoteWrite.url (#7084)
* `remoteWrite.maxQueueSize` from `100_000` to `1_000_000`, this should
improve resiliency of recording rules that produce many series;
* `remoteWrite.maxBatchSize` from `1_000` to `10_000`, this should be
more efficient to send from netwroking perspective;
* `remoteWrite.concurrency` from `1` to `4`, this should imrpove speed
of sending the generated series.

The new settings should improve remote write performance of vmalert with
default settings.

### Describe Your Changes

Please provide a brief description of the changes you made. Be as
specific as possible to help others understand the purpose and impact of
your modifications.

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: Hui Wang <haley@victoriametrics.com>
2024-09-25 15:01:39 +02:00
Roman Khavronenko
9a0f697622
docs: update CONTRIBUTING.md with practical requirements (#7087)
The change supposed to have more practical recommendations and reflect
the real processes for maintaining the project.


Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-09-24 18:22:18 +02:00
Github Actions
e28265fa39
Automatic update operator docs from VictoriaMetrics/operator@27ad7e1 (#7086)
Automated changes by
[create-pull-request](https://github.com/peter-evans/create-pull-request)
GitHub action

Signed-off-by: Github Actions <133988544+victoriametrics-bot@users.noreply.github.com>
Co-authored-by: f41gh7 <18450869+f41gh7@users.noreply.github.com>
2024-09-24 15:47:30 +02:00
Zhu Jiekun
5319acb8ed
vmagent: remote write respect Retry-After in header (#6124)
### Describe Your Changes
related issue:
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6097

#### Changed
- Remote write retry policy in `vmagent` is changed into:
  1. Respect `Retry-After` duration if exists.
2. Otherwise, calculate next retry duration by backoff policy (x2) and
max retry duration limit.
 
#### Docs
- `CHANGELOG.md`.

---
### Checklist
The following checks are mandatory:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Co-authored-by: Zakhar Bessarab <me@zekker-dev.tk>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
2024-09-24 12:44:03 +02:00
Dmytro Kozlov
cbeb7d50e8
lib/promscrape: show only unhealthy targets if show_only_unhealthy filter is enabled (#6960)
### Describe Your Changes

It is better to show only unhealthy targets instead of all of them when
`show_only_unhealthy` filter is enabled.
Related issue:
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3536

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
2024-09-24 12:18:24 +02:00
Phuong Le
df665a13c9
docs: update logos files and usage rules (#6980)
### Describe Your Changes

New logos and usage guideline

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-09-24 11:53:58 +02:00
Zhu Jiekun
fea4433362
docs: [VictoriaLogs] OTel Collector elasticsearchexporter header note (#7074)
### Describe Your Changes

By default, the `elasticsearchexporter` in OTel Collector puts the log
message under a field other than `_msg` (e.g., `Body`). Without
specifying via an HTTP header, those logs may not be queried correctly.
See also:
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6785.

This pull request updates the example configuration and notes for the
`elasticsearchexporter`.

### Checklist

The following checks are **mandatory**:

- [X] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-09-24 11:52:09 +02:00
Github Actions
524579d9bd
Automatic update operator docs from VictoriaMetrics/operator@75bc1b4 (#7080)
Automated changes by
[create-pull-request](https://github.com/peter-evans/create-pull-request)
GitHub action

Signed-off-by: Github Actions <133988544+victoriametrics-bot@users.noreply.github.com>
Co-authored-by: AndrewChubatiuk <3162380+AndrewChubatiuk@users.noreply.github.com>
2024-09-24 11:50:50 +02:00
Roman Khavronenko
4d0b41e63b
deployment: add panel and alerts for displying go scheduler latency (#7078)
The panel and alerting rule should help to understand whether VM
component doesn't have enough CPU resources or gets throttled. The alert
is applicable for all VM components.
The panel was added to vmalert, vmagent, vmsingle, vm clusert and
victorialogs dashes.

-------------------

This alerting rule should have help us identify resource shortage for
sandbox vmagent - see [this
link](https://play.victoriametrics.com/select/accounting/1/6a716b0f-38bc-4856-90ce-448fd713e3fe/prometheus/graph/#/?g0.range_input=23d13h25m25s424ms&g0.end_input=2024-09-23T14%3A11%3A00&g0.relative_time=none&g0.tab=0&g0.expr=histogram_quantile%280.99%2C+sum%28rate%28go_sched_latencies_seconds_bucket%7Bjob%3D%22vmagent-monitoring-vmagent%22%7D%5B5m%5D%29%29+by+%28le%2C+job%2C+instance%29%29+%3E+0.1)
for example. We weren't aware of resource shortage, because VM metrics
assumed this vmagent had 1vCPU while in fact its limit was 0.2vCPU.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-09-23 16:54:42 +02:00
Aliaksandr Valialkin
3964889705
app/vmselect/promql: consistently replace NaN data points with non-NaN values for range_first and range_last functions
It is expected that range_first and range_last functions return non-nan const value across all the points
if the original series contains at least a single non-NaN value. Previously this rule was violated for NaN data points
in the original series. This could confuse users.

While at it, add tests for series with NaN values across all the range_* and running_* functions, in order to maintain
consistent handling of NaN values across these functions.
2024-09-23 14:59:29 +02:00
hagen1778
3ed172eeeb
docs: add note about testing new releases on testing env
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-09-23 11:58:43 +02:00
Zakhar Bessarab
78ecfede06
license: add ability to reload keys (#775)
* lib/license: add support of license key hot-reload

* docs: add info about license key hot reload

---------

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2024-09-23 09:04:17 +02:00
Aliaksandr Valialkin
57183c9b61
docs/changelog/CHANGELOG.md: moved the description of the fix for proper usage of -streamAggr.dedupInterval and -remoteWrite.streamAggr.dedupInterval from FEATURE to BUGFIX section
The previous behaviour was incorrect, since it is unexpected that the -streamAggr.dedupInterval
and -remoteWrite.streamAggr.dedupInterval is applied to processed samples only if -streamAggr.config isn't set.

This is a follow-up for d523015f27
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6711
2024-09-23 08:56:33 +02:00
Github Actions
4dc85613a2
Automatic update helm docs from VictoriaMetrics/helm-charts@428cb36 (#7072)
Automated changes by
[create-pull-request](https://github.com/peter-evans/create-pull-request)
GitHub action

Signed-off-by: Github Actions <133988544+victoriametrics-bot@users.noreply.github.com>
Co-authored-by: AndrewChubatiuk <3162380+AndrewChubatiuk@users.noreply.github.com>
2024-09-23 13:17:34 +08:00
Aliaksandr Valialkin
0ada781cf2
docs/changelog/CHANGELOG.md: document bugfix for https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7009
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/7064
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7009

This is a follow-up for 55febc0920
2024-09-22 21:57:48 +02:00
Github Actions
8ed9978591
Automatic update operator docs from VictoriaMetrics/operator@d661554 (#7069)
Automated changes by
[create-pull-request](https://github.com/peter-evans/create-pull-request)
GitHub action

Signed-off-by: Github Actions <133988544+victoriametrics-bot@users.noreply.github.com>
Co-authored-by: AndrewChubatiuk <3162380+AndrewChubatiuk@users.noreply.github.com>
2024-09-22 18:39:52 +02:00
Artem Navoiev
a2ecba4154
docs: cloud siplify menu items names for notification, user managemenr and notifs
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2024-09-20 16:16:25 +02:00
Github Actions
8645438a79
Automatic update helm docs from VictoriaMetrics/helm-charts@f0e007f (#7059)
Automated changes by
[create-pull-request](https://github.com/peter-evans/create-pull-request)
GitHub action

Signed-off-by: Github Actions <133988544+victoriametrics-bot@users.noreply.github.com>
Co-authored-by: AndrewChubatiuk <3162380+AndrewChubatiuk@users.noreply.github.com>
2024-09-20 16:11:23 +02:00
Artem Navoiev
9e9583357d
docs add root menu to _index.md as homepage
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2024-09-20 16:06:15 +02:00
Github Actions
75a29655f5
Automatic update operator docs from VictoriaMetrics/operator@fdd3f9f (#7065)
Automated changes by
[create-pull-request](https://github.com/peter-evans/create-pull-request)
GitHub action

Signed-off-by: Github Actions <133988544+victoriametrics-bot@users.noreply.github.com>
Co-authored-by: tenmozes <1381404+tenmozes@users.noreply.github.com>
2024-09-20 16:03:52 +02:00
Andrii Chubatiuk
5708a85499
docs: updated root menu items (#7061)
### Describe Your Changes

Please provide a brief description of the changes you made. Be as
specific as possible to help others understand the purpose and impact of
your modifications.

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
Co-authored-by: Artem Navoiev <tenmozes@gmail.com>
2024-09-20 06:14:29 -07:00
Artem Navoiev
7596e239eb
docs: set operator menu item weight to 30
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2024-09-20 11:22:03 +02:00
Artem Navoiev
36d58588a7
docs: clarify operator name.2
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2024-09-20 11:20:16 +02:00
Artem Navoiev
cd384c7547
docs: clarify operator name
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2024-09-20 11:11:38 +02:00
Hui Wang
d6d02d7aeb
vmalert: fix variable $activeAt value when templating rule annotation in replay mode
Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
2024-09-20 11:07:40 +02:00
hagen1778
6167bccc5a
docs: fix more typos in the changelog
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-09-20 10:54:43 +02:00
hagen1778
59281d5358
docs: rm update node about loggerMaxArgLen as it doesn't have incompatibility effect
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-09-20 10:42:07 +02:00
hagen1778
6726a5aaed
docs: fix typo in link in change line about NaN
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-09-20 10:39:13 +02:00
Thomas Danielsson
258201af04
docs: fix typo in the changelog
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-09-20 08:46:29 +02:00
f41gh7
61721303fd
docs/changelog: mention vmagent kafka consumer bugfix
Changes were made to the enteprise repository
2024-09-19 15:35:48 +02:00
Yury Molodov
b0bdb92729
vmui: change the query_range request method from GET to POST (#7039)
### Describe Your Changes

change the `/query_range` and `/query` requests method from `GET` to
`POST`. See #6288.

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
2024-09-19 14:30:54 +02:00
Roman Khavronenko
e115b85770
lib/logger: increase default value of -loggerMaxArgLen cmd-line fla… (#7008)
…g from 1e3 to 5e3

This should improve visibility on errors produced by very long queries.

The change is classified as BUG in order to port it to LTS releases.

### Describe Your Changes

Please provide a brief description of the changes you made. Be as
specific as possible to help others understand the purpose and impact of
your modifications.

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: Mathias Palmersheim <mathias@victoriametrics.com>
2024-09-19 14:29:18 +02:00
Yury Molodov
bc9cb69170
vmui/logs: add auto refresh (#7038)
### Describe Your Changes

Add auto refresh 
#7017


![image](https://github.com/user-attachments/assets/20ed1102-d5e4-4d3f-9c24-7d298d93400a)

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
2024-09-19 11:11:16 +02:00
Aliaksandr Valialkin
e86891b010
app/vlselect/logsql: call Query.Optimize() on the cloned query in order to replace * filter with filterNoop inside getLastNQueryResults()
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6785
2024-09-18 18:24:17 +02:00
Aliaksandr Valialkin
b82e2cabc5
app/vmselect/promql: properly calculate c1 and c2 and c1 or c2 by upgrading github.com/VictoriaMetrics/metricsql to v0.79.0
The fix is in the https://github.com/VictoriaMetrics/metricsql/pull/34
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6637
2024-09-18 17:38:19 +02:00
Github Actions
8bc30b68cd
Automatic update operator docs from VictoriaMetrics/operator@fe90f57 (#7026)
Automated changes by
[create-pull-request](https://github.com/peter-evans/create-pull-request)
GitHub action

Signed-off-by: Github Actions <133988544+victoriametrics-bot@users.noreply.github.com>
Co-authored-by: f41gh7 <18450869+f41gh7@users.noreply.github.com>
2024-09-17 19:30:09 +02:00
Dmytro Kozlov
4c228f1e18
docs/victoriametrics-cloud: updated pictures and descriptions with new functionality (#7014)
Improved VictoriaMetrics documentation for cloud

Related issue: https://github.com/VictoriaMetrics/cloud/issues/2143

### Describe Your Changes

Please provide a brief description of the changes you made. Be as
specific as possible to help others understand the purpose and impact of
your modifications.

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-09-16 23:33:40 -07:00
Nikolay
d8f8822fa5
lib/storage: consistently check for missing metricID index records (#6967)
* Previously, only metricID->metricName missing index records were
tracked with deadline But it was possible a case for missing
metricID->TSID index records. IndexDB metrics fix exposed misleading
metric for such missing records.

* This commit adds check for metricID->TSID missing index records. And
delete missing metricID entry if it hit 60 second deadline.

Related issue
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6931

Signed-off-by: f41gh7 <nik@victoriametrics.com>
2024-09-16 10:05:08 +02:00
Nikolay
264c2ec6bd
lib/fs: properly call windows APIs (#6998)
Previously we manually imported system windows DDLs
and made direct syscall.

 But golang exposes syscall wrappers with sys/windows package.
It seems, that direct syscall was broken at 1.23 golang release. It was
`GetDiskFreeSpace` syscall in our case.

This commit replaces all manual syscalls with wrappers

Related issue:
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6973

Related golang issue:
https://github.com/golang/go/issues/69029

Signed-off-by: f41gh7 <nik@victoriametrics.com>
2024-09-13 12:22:25 +02:00
Dima Lazerka
8207879fa3
docs: fixes misspelled typos
Also tried to make it catch "Authorisation" in the future, fixed a lot
of other misspells along the way, but didn't make it catch
"Authorisation" anyway.

- Fix misspelled "Authorization" header name
- Fix misspelled "organization"
- Fix more misspells
2024-09-13 12:14:24 +02:00
Aliaksandr Valialkin
e92f347336
docs/VictoriaLogs/querying/README.md: typo fixes: use field_values instead field_names where needed 2024-09-11 09:39:22 +02:00
Alexander Marshalov
c0272463d9
[vmcloud]: Cloud tiers assumptions docs (#6948)
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
Co-authored-by: Artem Navoiev <tenmozes@gmail.com>
2024-09-10 12:41:25 +02:00
Artem Navoiev
277fed9990
victorialogs: add HA example for logstash and fluentbit (#6968)
### Describe Your Changes

Please provide a brief description of the changes you made. Be as
specific as possible to help others understand the purpose and impact of
your modifications.

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2024-09-09 14:33:05 -07:00
Artem Navoiev
153cceb124
docs: adjust image size
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2024-09-09 21:30:54 +02:00
Artem Navoiev
ae5f98f46b
docs: fix links to docker
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2024-09-09 19:56:58 +02:00
Artem Navoiev
71df11dfbb
docs: victorialogs just mention telegraf as we support not only http but more
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2024-09-09 17:41:15 +02:00
Artem Navoiev
47cd7bb4c1
docs: update roadmap, mark telegraf and otel as done and link the docs
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2024-09-09 17:35:14 +02:00
Hui Wang
ae4d376e41
vmalert: do not send message to alertmanager when alert has no label … (#6823)
…pair

`alert_relabel_configs` in [notifier
config](https://docs.victoriametrics.com/vmalert/#notifier-configuration-file)
can drop alert labels when used to filter different tenant alert message
to different notifier.
alertmanager would report error like `msg="Failed to validate alerts"
err="at least one label pair required"` in this case, but the rest of
the alerts inside one request would still be valid in alertmanager, so
it's not severe.
2024-09-09 13:34:48 +02:00
Artem Navoiev
f9a8c09fe8
docs: Add VictoriaLogs HA example for single nodes (#6945)
### Describe Your Changes

Please provide a brief description of the changes you made. Be as
specific as possible to help others understand the purpose and impact of
your modifications.

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2024-09-09 03:39:58 -07:00
hagen1778
7983ea4edb
docs: mention -dryRun flag next to config example in vmalert
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-09-09 11:34:52 +02:00
Aliaksandr Valialkin
0ed835026f
deployment: update VictoriaLogs from v0.28.0-victorialogs to v0.29.0-victorialogs
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.29.0-victorialogs
2024-09-08 21:23:16 +02:00
Aliaksandr Valialkin
fb339d7574
docs/VictoriaLogs/CHANGELOG.md: cut v0.29.0 2024-09-08 21:10:17 +02:00
Aliaksandr Valialkin
4fbdde5852
deployment/docker: update base Alpine docker image from 3.20.2 to 3.20.3
See https://alpinelinux.org/posts/Alpine-3.17.10-3.18.9-3.19.4-3.20.3-released.html
2024-09-08 19:26:48 +02:00
Aliaksandr Valialkin
eaee2d7db4
lib/logstorage: improve error logging for incorrect queries passed to /select/logsql/stats_query and /select/logsql/stats_query_range functions 2024-09-08 11:24:44 +02:00
Aliaksandr Valialkin
0a40064a6f
app/vlselect: add /select/logsql/stats_query_range endpoint for building time series panels in VictoriaLogs plugin for Grafana
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6943
Updates https://github.com/VictoriaMetrics/victorialogs-datasource/issues/61
2024-09-07 00:41:47 +02:00
Aliaksandr Valialkin
c9bb4ddeed
app/vlselect: add /select/logsql/stats_query endpoint, which is going to be used by vmalert
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6942
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6706
2024-09-06 23:06:43 +02:00
Aliaksandr Valialkin
5261a84119
deployment: update Go builder from Go1.23.0 to Go1.23.1
See https://github.com/golang/go/issues?q=milestone%3AGo1.23.1+label%3ACherryPickApproved
2024-09-06 22:51:15 +02:00
Aliaksandr Valialkin
00e7d5add3
lib/logstorage: substitute | operator with or operator at math pipe
This is needed for avoiding confusion between the `|` operator at `math` pipe and `|` pipe delimiter.
For example, the following query was parsed unexpectedly:

   * | math foo / bar | fields x

as

   * | math foo / (bar | fields) as x

Substituting `|` with `or` inside `math` pipe fixes this ambiguity.
2024-09-06 22:44:14 +02:00
f41gh7
feafb30266
docs/changelog: mention storage changes
After a5424e95b3

Signed-off-by: f41gh7 <nik@victoriametrics.com>
2024-09-06 18:05:11 +02:00
Zakhar Bessarab
9f7ee4c0bb
Vmgateway no prefix string (#784)
* app/vmgateway: allow skipping Bearer prefix, parsing access as string

- allow disabling of "Bearer" prefix check - This is needed in order to support OIDC systems where identity token is provided separately from access token and it does not contain "Bearer" prefix(such as Azure Entra ID, ex AD).a

- support parsing "vm_access" claim as a string - This is helpful for systems where claims can only be mapped to string.

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* docs/changelog: mention vmgateway updates

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

---------

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2024-09-06 16:20:20 +02:00
Aliaksandr Valialkin
258ccfb953
lib/logstorage: pre-calculate hashes from tokens used in bloom filter search
Previously per-token hashes for per-block bloom filters were re-calculated on every scanned block.
This could be slow when the number of tokens is big or when the number of blocks to scan is big.
Pre-calculate hashes for bloom filters and then use them for searching in bloom filters.
This improves performance by 2.5x for in(...) filters with many values to search inside `in()`.
2024-09-05 19:44:17 +02:00
f41gh7
d7be0e7c9a
docs/changelog: mention storage NaN changes
follow-up after 39294b4919

Signed-off-by: f41gh7 <nik@victoriametrics.com>
2024-09-05 16:56:32 +02:00
Zhu Jiekun
c193e6d43e
lib/discovery/azure: fix host check in next link in Azure SD (#6915)
Previous bugfix at 49f63b2 only partially fixed pagination host validation error.

 Before this fix it was:
```
unexpected nextLink host \"management.azure.com\", expecting \"https://management.azure.com\"
```

Now we only check the `Host` without schema. 

However, when Azure respond `nextLink` in `Host:Port` format, the
`nextLink` check will fail:
```
unexpected nextLink host \"management.azure.com:443\", expecting \"management.azure.com\"
```

This pull request further relaxes the checks by only checking the
`Hostname`.

---

 related issue: https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6912
2024-09-05 16:48:09 +02:00
Hui Wang
b48f5f3e59
lib/storage: fix metric vm_object_references{type="indexdb"} (#6937)
follow up
4ecc370acb

### Describe Your Changes

Please provide a brief description of the changes you made. Be as
specific as possible to help others understand the purpose and impact of
your modifications.

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-09-05 16:42:49 +02:00
Aliaksandr Valialkin
2dd845fa53
lib/logstorage: properly fix incorrect extraction of common tokens for OR filters at distinct log fields
Previously (f1:foo OR f2:bar) was incorrectly returning `foo` token for `f1` and `bar` token for `f2`.
These tokens were used for checking against bloom filter for every data block, so the data block,
which didn't contain simultaneously `foo` token for `f1` field and `bar` token for `f2` field, was skipped.
This was incorrect, since such a block may contain logs matching the original OR filter.

The fix is to return common tokens from `OR`-delimted filters only if these tokens exist at EVERY such filter
for the given field name. If some `OR`-delimited filter misses the given field name, then `OR`-delimited filters
do not contain common tokens, which could be used for checking against bloom filter.

While at it, add more tests covering various edge cases for filters delimited by AND and OR.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6554
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6556
2024-09-05 14:29:50 +02:00
Artem Navoiev
25d433d75f
docs: remove old png file as we have webp now
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2024-09-05 14:20:22 +02:00
Github Actions
da9c41e6b0
Automatic update Grafana datasource docs from VictoriaMetrics/victorialogs-datasource@edd3259 (#6946) 2024-09-05 05:19:03 -07:00
hagen1778
666316465c
docs: fix typo in vmagent.md
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-09-05 13:12:16 +02:00
f41gh7
be66aa5f4e
docs/changelog: mention enterprise changes
Signed-off-by: f41gh7 <nik@victoriametrics.com>
2024-09-04 15:39:34 +02:00
f41gh7
b8bbea8896
docs/changelog: moves victorialogs changes to proper file
Signed-off-by: f41gh7 <nik@victoriametrics.com>
2024-09-04 15:36:33 +02:00
Roman Khavronenko
45f53748ff
docs: clarify why 1.102.1 was publicly available (#6933)
### Describe Your Changes

Please provide a brief description of the changes you made. Be as
specific as possible to help others understand the purpose and impact of
your modifications.

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-09-04 14:42:17 +02:00
hagen1778
06779f0f67
docs: bump last LTS versions
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-09-04 14:10:50 +02:00
Andrii Chubatiuk
01430a155c
vlinsert: added opentelemetry logs support
Commit adds the following changes:

* Adds support of OpenTelemetry logs for Victoria Logs with protobuf encoded messages

*  json encoding is not supported for the following reasons:
   - It brings a lot of fragile code, which works inefficiently.
   - json encoding is impossible to use with language SDK.

* splits metrics and logs structures at lib/protoparser/opentelemetry/pb package.

* adds docs with examples for opentelemetry logs.

---
Related issue: https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4839

Co-authored-by: AndrewChubatiuk <andrew.chubatiuk@gmail.com>
Co-authored-by: f41gh7 <nik@victoriametrics.com>
2024-09-03 20:12:05 +02:00
f41gh7
8b36529b32
follow-up after 1731c0eabf
* updates change log
* adds VL-Debug http header
* updates doc
* extracts only the first value of http headers for VL-Stream-Fields and VL-Ignore-Fields.
  It makes behaviour the same as Query string args. And allows to easily configure client applications.
  Since most of the client collectors don't support multi value headers.

Signed-off-by: f41gh7 <nik@victoriametrics.com>
2024-09-03 19:16:10 +02:00
Andrii Chubatiuk
1731c0eabf
app/vlinsert: support getting _msg_field, _time_field, _stream_fields and _ignore_fields from headers
*  Many collectors don't support forwarding url query params to the remote system. It makes impossible to define stream fields for it. Workaround with proxy between VictoriaLogs and log shipper is too complicated solution.

* This commit adds the following changes:
 * Adds fallback to to headers params, if query param is empty for:
     _msg_field -> VL-Msg-Field
    _stream_fields -> VL-Stream-Fields
    _ignore_fields -> VL-Ignore-Fields
    _time_field -> VL-Time-Field
 * removes deprecations from victorialogs compose files, added more
output format examples for logstash, telegraf, fluent-bit

 related issue: https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5310
2024-09-03 17:43:26 +02:00
hagen1778
4dcb6a3719
dashboards/vmagent: fix legend captions for stream aggregation related panels.
Before they were displaying wrong label names.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-09-03 14:23:35 +02:00
Hui Wang
d523015f27
stream aggregation: perform deduplication for all received data when … (#6711)
…specifying `-streamAggr.dedupInterval` or
`-remoteWrite.streamAggr.dedupInterval` command-line flag

[The
documentation](https://docs.victoriametrics.com/stream-aggregation/)
contains conflicting descriptions regarding deduplication for
non-matched series when `-remoteWrite.streamAggr.config` and / or
`-streamAggr.config` are set:
1. Statement below says **all the received data** is deduplicated:
>[vmagent](https://docs.victoriametrics.com/vmagent/) supports
relabeling, deduplication and stream aggregation for all the received
data, scraped or pushed. Then, the collected data will be forwarded to
specified -remoteWrite.url destinations. The data processing order is
the following:
>1. all the received data is relabeled according to the specified
[-remoteWrite.relabelConfig](https://docs.victoriametrics.com/vmagent/#relabeling)
(if it is set)
>2. all the received data is deduplicated according to specified
[-streamAggr.dedupInterval](https://docs.victoriametrics.com/stream-aggregation/#deduplication)
(if it is set to duration bigger than 0)

2. Another statement says the deduplication is performed individually
for the **matching samples**
>The de-deduplication is performed after applying
[relabeling](https://docs.victoriametrics.com/vmagent/#relabeling) and
before performing the aggregation. If the -remoteWrite.streamAggr.config
and / or -streamAggr.config is set, then the de-duplication is performed
individually per each [stream aggregation
config](https://docs.victoriametrics.com/stream-aggregation/#stream-aggregation-config)
for the matching samples after applying
[input_relabel_configs](https://docs.victoriametrics.com/stream-aggregation/#relabeling).


Considering the following deduplication use cases:
1. To apply deduplication(globally or for specific remoteWrite
destination) for all the received data, scraped or pushed
--- using `-streamAggr.dedupInterval` or
`-remoteWrite.streamAggr.dedupInterval`.
2. To deduplicate and aggregate metrics that match the rule `match`
filters
--- using `-remoteWrite.streamAggr.config` and specifiying
`dedup_interval` option in [stream aggregation
config](https://docs.victoriametrics.com/stream-aggregation/#stream-aggregation-config).
3. To deduplicate all the received data while having `streamAggr.config`
for some metrics
--- no way for a single vmagent now, need to set up two level vmagents

This PR implements case3.

---------

Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
2024-09-03 10:47:05 +02:00
Github Actions
78067ff2c2
Automatic update Grafana datasource docs from VictoriaMetrics/victorialogs-datasource@1f6f9ea (#6925) 2024-09-03 10:23:18 +02:00
Github Actions
fd5469e039
Automatic update Grafana datasource docs from VictoriaMetrics/victoriametrics-datasource@297faf8 (#6926) 2024-09-03 10:23:02 +02:00
jackyin
975ed27a76
lib/logstorage: and filter results in unexpected response (#6556)
fix #6554
andfilter shouldn't return orfilter field which result in bloomfilter
return false.

---------

Co-authored-by: hagen1778 <roman@victoriametrics.com>
2024-09-03 10:17:44 +02:00
Github Actions
f102b14ac9
Automatic update operator docs from VictoriaMetrics/operator@7eaac41 (#6923)
Automated changes by
[create-pull-request](https://github.com/peter-evans/create-pull-request)
GitHub action

Signed-off-by: Github Actions <133988544+victoriametrics-bot@users.noreply.github.com>
Co-authored-by: tenmozes <1381404+tenmozes@users.noreply.github.com>
2024-09-02 14:15:29 -07:00
hagen1778
9d94573572
docs: fix broken links to prev changelogs
The prev links like `/changelog_2021/`
stopped working after 9dc8d1debd
because these files now require specifying the parent `changelog` in the path, like `/changelog/changelog_2021/`.

This fix adds an alias for an old link.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-08-31 16:34:21 +02:00
Ivan Yatskevich
8a8a1d5df2
docs/vmalert: mention VM Cloud where it can be helpful to a user (#6888)
### Describe Your Changes

Add mentions of VictoriaMetrics Cloud to the documentation of vmalert
where this info is helpful to a user.

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
2024-08-30 10:39:05 +02:00
Ivan Yatskevich
f6278d99dd
docs/quickstart: describe steps to run VM on VM Cloud (#6877)
### Describe Your Changes

Describe steps to run VictoriaMetrics Single node or Cluster on
VictoriaMetrics Cloud
### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
2024-08-30 10:36:40 +02:00
Zhu Jiekun
f572365a93
docs: fix incorrect URLs for resetCache and OTel guide (#6906)
### Describe Your Changes

This pull request fixes incorrect URLs in two places:

1. In the OTel guide, which has been corrected in
https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6880, but one
incorrect URL is still missing.
2. In the URL example, the cache reset endpoint for vmselect / Cluster
version is `/internal/resetRollupResultCache`, but it is mistakenly
noted as `/select/internal/resetRollupResultCache`, which misguides the
user. (introduced in #4468)

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-08-30 10:31:14 +02:00
hagen1778
b27c4b198f
docs/guides: fix images in vmgateway guide
Follow-up after ce4cc4cbb2

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-08-30 10:28:38 +02:00
hagen1778
11ae873d0a
docs/release-guide: rm RPM packages from the list
RPM packages are now managed by community at https://github.com/VictoriaMetrics-Community/victoriametrics-rpm

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-08-30 09:49:06 +02:00
Github Actions
f42ef2152f
Automatic update operator docs from VictoriaMetrics/operator@f82db30 (#6910)
Automated changes by
[create-pull-request](https://github.com/peter-evans/create-pull-request)
GitHub action

Signed-off-by: Github Actions <133988544+victoriametrics-bot@users.noreply.github.com>
Co-authored-by: f41gh7 <18450869+f41gh7@users.noreply.github.com>
2024-08-29 19:45:09 +04:00
hagen1778
00d09d67dc
docs/victorialogs/CHANGELOG.md: mention issue #4750 in the change
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-08-29 15:09:52 +02:00
hagen1778
d5755e55ef
docs/CHANGELOG.md: update changelog with LTS release notes
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-08-29 13:27:27 +02:00
Github Actions
982c907ee1
Automatic update operator docs from VictoriaMetrics/operator@bf5e160 (#6901)
Automated changes by
[create-pull-request](https://github.com/peter-evans/create-pull-request)
GitHub action

Signed-off-by: Github Actions <133988544+victoriametrics-bot@users.noreply.github.com>
Co-authored-by: f41gh7 <18450869+f41gh7@users.noreply.github.com>
2024-08-28 17:56:11 +02:00
hagen1778
5aeb759df9
docs/CHANGELOG.md: cut v1.103.0
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-08-28 13:48:31 +02:00
hagen1778
e71cfdcfa5
docs: pre-release doc update
* typo fix
* mention version starting from features are available

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-08-28 13:47:29 +02:00
hagen1778
d9982520a8
docs: update dedup docs for cluster
* remove repeating sentences
* clarify why identical dedup config is important

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-08-28 13:00:30 +02:00
f41gh7
40d55199fd
docs/changelog: mention bugfix
Signed-off-by: f41gh7 <nik@victoriametrics.com>
2024-08-28 11:51:02 +02:00
Nikolay
4ecc370acb
lib/storage: properly add previous indexDB metrics (#6890)
Previously, some extIndexDB metrics were not registered. It resulted
into missing metrics, if metric value was added to the extIndexDB. It's
a usual case for search requests at both indexes.

 Current commit updates all metrics from extIndexDB according to the
current IndexDB. It must fix such cases

Related issue:
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6868

### Describe Your Changes

Please provide a brief description of the changes you made. Be as
specific as possible to help others understand the purpose and impact of
your modifications.

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-08-28 11:14:28 +02:00
Cuong Le
6154bc9466
VictoriaLogs/keyConcepts.md: fix broken anchor to #other-fields (#6896)
### Describe Your Changes

The anchor to "Other fields" section should be #other-fields (instead of
#other-field)

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

Signed-off-by: Cuong Le <cuongleqq@gmail.com>
2024-08-28 01:02:01 -07:00
Zhu Jiekun
52043796c9
docs: Add backup and restore doc for VictoriaLogs
This commit adds doc for VictoriaLogs backup and restore based on `rsync`.
2024-08-27 21:41:42 +02:00
rtm0
9fcfba3927
lib/storage: properly handle maxMetrics limit at metricID search
`TL;DR` This PR improves the metric IDs search in IndexDB:

- Avoid seaching for metric IDs twice when `maxMetrics` limit is
exceeded
- Use correct error type for indicating that the `maxMetrics` limit is
exceded
- Simplify the logic of deciding between per-day and global index search

A unit test has been added to ensure that this refactoring does not
break anything.

---

Function calls before the fix:

```
idb.searchMetricIDs
    |__ is.searchMetricIDs
        |__ is.searchMetricIDsInternal
            |__ is.updateMetricIDsForTagFilters
                |__ is.tryUpdatingMetricIDsForDateRange
                |                       |
                |__ is.getMetricIDsForDateAndFilters
```

- `searchMetricIDsInternal` searches metric IDs for each filter set. It
maintains a metric ID set variable which is updated every time the
`updateMetricIDsForTagFilters` function is called. After each successful
call, the function checks the length of the updated metric ID set and if
it is greater than `maxMetrics`, the function returns `too many
timeseries` error.
- `updateMetricIDsForTagFilters` uses either per-day or global index to
search metric IDs for the given filter set. The decision of which index
to use is made is made within the `tryUpdatingMetricIDsForDateRange`
function and if it returns `fallback to global search` error then the
function uses global index by calling `getMetricIDsForDateAndFilters`
with zero date.
- `tryUpdatingMetricIDsForDateRange` first checks if the given time
range is larger than 40 days and if so returns `fallback to global
search` error. Otherwise it proceeds to searching for metric IDs within
that time range by calling `getMetricIDsForDateAndFilters` for each
date.
- `getMetricIDsForDateAndFilters` searches for metric IDs for the given
date and returns `fallback to global search` error if the number of
found metric IDs is greater than `maxMetrics`.

Problems with this solution:

1. The `fallback to global search` error returned by
`getMetricIDsForDateAndFilters` in case when maxMetrics is exceeded is
misleading.
2. If `tryUpdatingMetricIDsForDateRange` proceeds to date range search
and returns `fallback to global search` error (because
`getMetricIDsForDateAndFilters` returns it) then this will trigger
global search in `updateMetricIDsForTagFilters`. However the global
search uses the same maxMetrics value which means this search is
destined to fail too. I.e. the same search is performed twice and fails
twice.
3. `too many timeseries` error is already handled in
`searchMetricIDsInternal` and therefore handing this error in
`updateMetricIDsForTagFilters` is redundant
4. updateMetricIDsForTagFilters is a better place to make a decision on
whether to use per-day or global index.

Solution:

1.  Use a dedicated error for `too many timeseries` case
2. Handle `too many timeseries` error in  `searchMetricIDsInternal` only
3. Move the per-day or global search decision from
`tryUpdatingMetricIDsForDateRange` to `updateMetricIDsForTagFilters` and
remove `fallback to global search` error.


---------

Signed-off-by: Artem Fetishev <wwctrsrx@gmail.com>
Co-authored-by: Nikolay <nik@victoriametrics.com>
2024-08-27 21:39:03 +02:00
rtm0
eef6943084
lib/storage: properly register index records with RegisterMetricNames
Once the timeseries is in tsidCache, new entries won't be created in
per-day index because the RegisterMetricNames() code does consider
different dates for the same timeseries. So this case has been added.

The same bug exists for AddRows() but it is not manifested because the
index entries are finally created in updatePerDateData().

RegisterMetricNames also updated to increase the newTimeseriesCreated
counter because it actually creates new time series in index.

A unit tests has been added that check all possible data patterns
(different metric names and dates) and code branches in both
RegisterMetricNames and AddRows. The total number of new unit tests is
around 100 which increaded the running time of storage tests by 50%. 

---------

Signed-off-by: Artem Fetishev <wwctrsrx@gmail.com>
Co-authored-by: Roman Khavronenko <hagen1778@gmail.com>
2024-08-27 21:33:53 +02:00
hagen1778
6f17ee0d0f
docs: typo fix
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-08-27 15:49:29 +02:00
rtm0
e7f1297517
docs: document IndexDB (#6840)
### Describe Your Changes

This is an attempt to document IndexDB. I guess I was trying to touch
the important points that might be of interest for the end users while
refraining from making it too detailed (such as I did not enumerate and
describe all the specific record types).

Please take a look and any suggestions are very welcome.

### Checklist

The following checks are **mandatory**:

- [x ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Signed-off-by: Artem Fetishev <wwctrsrx@gmail.com>
2024-08-27 13:52:46 +02:00
Zhu Jiekun
e97e966f82
lib/promrelabel: follow-up for 8958cecad6
In the previous commit 8958cecad6
the default ports (80/443) were removed for both the `scrapeURL` and
`instance` label values for those targets without a port in
`__address__`. Different values in the `instance` label generate new
time series.

This commit reverts the changes made to the `instance` label. Now,
for those targets:
- `scrapeURL` will remain unchanged.
- The `instance` label value will include the default port.

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6792
2024-08-27 13:04:26 +02:00
Zhu Jiekun
9c3b5116dd
docs: fix example links in getting started with OpenTelemetry (#6880)
### Describe Your Changes

Fix broken link on:
https://docs.victoriametrics.com/guides/getting-started-with-opentelemetry/index.html#building-the-go-application-instrumented-with-metrics
- https://docs.victoriametrics.com/guides/app.go-collector.example
- https://docs.victoriametrics.com/guides/app.go.example

The valid links **could be**:
-
https://docs.victoriametrics.com/guides/getting-started-with-opentelemetry/app.go-collector.example
-
https://docs.victoriametrics.com/guides/getting-started-with-opentelemetry/app.go.example

Also went throught `/guides` folder and did not found doc with same
issue. Please advice if you know more.

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-08-27 10:36:52 +02:00
Fred Navruzov
ac485a8486
docs: vmanomaly - release notes v1.15.[6-9] (#6879)
### Describe Your Changes

release notes for patches 1.15.6 - 9

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-08-27 10:30:48 +02:00
YuDong Tang
295f2aa8ca
app/vmselect:add command-line flag -search.inmemoryBufSizeBytes (#6869)
add command-line flag `-search.inmemoryBufSizeBytes` for configuring size of in-memory buffers used by vmselect during processing of vmstorage responses. A new summary metric `vm_tmp_blocks_inmemory_file_size_bytes` is exposed to show the size of the buffer during requests processing.

The new setting can be used by experienced users to adjust memory usage by vmselect when processing
many small read requests. Instead of allocating 4MB buffers each time, vmselect can be instructed to lower
the buffer size via `-search.inmemoryBufSizeBytes`. To make the decision whether this flag needs to be adjusted
users can consult with `vm_tmp_blocks_inmemory_file_size_bytes` which shows the actual size of buffers used
during query processing.

----------

The detailed information of this PR can be found in
https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6851

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Co-authored-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit cab3ef8294)
2024-08-26 14:48:53 +02:00
Github Actions
e7b3a996da
Automatic update operator docs from VictoriaMetrics/operator@3ba727f (#6875)
Automated changes by
[create-pull-request](https://github.com/peter-evans/create-pull-request)
GitHub action

Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-08-26 14:39:12 +02:00
Github Actions
84ca4d18f0
Automatic update operator docs from VictoriaMetrics/operator@369ab8d (#6866)
Automated changes by
[create-pull-request](https://github.com/peter-evans/create-pull-request)
GitHub action

Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-08-23 14:09:13 +02:00
Yury Akudovich
d0f5a9d77a
app/vmagent: add remoteWrite.retryMinInterval and remoteWrite.retryMaxTime flags (#6289)
## Describe Your Changes

Add RemoteWrite Retry Controls

This PR introduces two new flags to the remote write functionality:
- remoteWrite.retryMinInterval
- remoteWrite.retryMaxTime
 
These flags provide finer control over the retry behavior for
remoteWrite operations, allowing users to customize the minimum interval
between retries and the maximum duration for retry attempts.

Fixes #5486.

## Checklist

- [x] The following checks are mandatory:

My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Signed-off-by: Yury Akudovich <ya@matterlabs.dev>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
2024-08-23 14:05:51 +02:00
Denys Holius
336406e2e1
docs/guides/guide-delete-or-replace-metrics/README.md: adds a link to the API examples (#6863)
### Describe Your Changes

Adds a link to the API example section that describes how to delete
metrics on VM Single.

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
2024-08-23 10:01:03 +02:00
Roman Khavronenko
70a94ea492
app/vmalert: update parsing for instant responses (#6859)
This change is made in attempt to reduce memory usage by vmalert when
parsing big instant responses from VM/Prometheus.

In
a5c427bac4
vmalert switched from std json lib to fastjson lib in order to reduce
amount of allocations, as according to highloaded profiles of vmalert
the CPU is mostly spent on GC.

But switching to fastjson resulted into excessive memory usage for cases
when vmalert has to parse long json lines, which usually happens when
instant response contains many `metric` objects.

In this change we do a mixed parsing:
1. Slice of `metric` objects is parsed with std lib to keep mem low
2. Each `metric` object is parsed with fastjson to reduce allocs

The benchmark results are the following:
```
pkg: github.com/VictoriaMetrics/VictoriaMetrics/app/vmalert/datasource
BenchmarkParsePrometheusResponse/Instant_std+fastjson-10                    1760            668959 ns/op          280147 B/op       5781 allocs/op
MBs allocated at heap: 493.078392
mallocs: 18655472
BenchmarkParsePrometheusResponse/Instant_fastjson-10                        6109            198258 ns/op          172839 B/op       5548 allocs/op
MBs allocated at heap: 1056.384464
mallocs: 34457184
BenchmarkParsePrometheusResponse/Instant_std-10                             1287            950987 ns/op          451677 B/op       9619 allocs/op
MBs allocated at heap: 580.802976
mallocs: 13351636
```
The benchmark function code with mem measurement is available here
https://gist.github.com/hagen1778/b9c3ca7f8ca7d6b21aec9777112c5810

The benchmark contains 3 results:
1. Instant_std+fastjson is the implementation in this change
2. Instant_fastjson-10 is the implementation from
a5c427bac4
3. BenchmarkParsePrometheusResponse/Instant_std-10 is implementation
before
a5c427bac4

According to these results, this new implementation is slower than
previous, but faster than before switching to fastjson. It also has
lower number of allocations and roughly the same memory allocation on
heap with GC turned off.

---------

Other changes:
1. rm BenchmarkMetrics as it doesn't measure anything
2. simplify BenchmarkParsePrometheusResponse into
BenchmarkPromInstantUnmarshal

### Describe Your Changes

Please provide a brief description of the changes you made. Be as
specific as possible to help others understand the purpose and impact of
your modifications.

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-08-22 17:36:11 +02:00
Yury Molodov
e35237920a
vmui: add column search in table settings (#6804)
### Describe Your Changes

Add search functionality to the column display settings in the table
 #6668
 

![image](https://github.com/user-attachments/assets/e9bd52c3-6428-4d4f-8b7f-d83dd80b6912)

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
2024-08-22 16:57:26 +02:00
hagen1778
fa9de9b45c
docs: fix ordering in vmagent docs
Looks like auto-numeration doesn't work if item in the list
takes more than one line.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-08-22 11:58:27 +02:00
hagen1778
dbb4899d71
docs: fix root page in keyConcepts
Add _index.md to fix `http://docs/keyconcepts` link.
Without this change only `http://docs/keyconcepts/keyconcepts` was working.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-08-22 09:41:30 +02:00
Roman Khavronenko
22975d28ae
docs: move keyconcepts to separate dir (#6855)
Moving key-concepts-related docs to a separate dir should make it easier
to navigate in `docs/` folder and helps to avoid adding prefixes to
image assets.

-----------

The change shouldn't have any visual changes or changes to the links.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-08-21 17:27:24 +02:00
Roman Khavronenko
9dc8d1debd
docs: move changelog to dir (#6853)
Moving changelog-related docs to a separate dir should make it easier to
navigate in `docs/` folder.

-----------

The change shouldn't have any visual changes or changes to the links.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-08-21 17:26:54 +02:00
hagen1778
74e8d6ad55
docs: rm mentions of google group as it was deprecated
See https://docs.victoriametrics.com/#community-and-contributions

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-08-21 14:08:27 +02:00
Github Actions
a82dda1d18
Automatic update operator docs from VictoriaMetrics/operator@5fd7d6e (#6845)
Automated changes by
[create-pull-request](https://github.com/peter-evans/create-pull-request)
GitHub action

Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-08-21 11:27:42 +02:00
Github Actions
37654e0604
Automatic update Grafana datasource docs from VictoriaMetrics/victoriametrics-datasource@c23c4d1 (#6844) 2024-08-21 11:26:44 +02:00
Zakhar Bessarab
654c1cb9d1
make: reset permissions when creating release archive (#6846)
### Describe Your Changes

Forcefully set permissions for release binaries to 1000:1000. This helps
to avoid issues such as
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6788 on
systems with limitations around UID:GID configuration.

"1000" UID and GID is widely used by linux distributions for the first
user in the system.

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
2024-08-21 11:25:13 +02:00
Ayush Chauhan
af54ddea23
docs: add zomato case study (#6848) 2024-08-21 09:54:10 +02:00
Dima Lazerka
535a9ed059
vmui: Fix initial serverUrl for vmanomaly (#6834)
- fix TS lint
- anomaly: remove /vmui
- anomaly: minor inspections fix
- docs: fix broken links to headings

### Describe Your Changes

Initially vmanomaly opened with `/vmui` in serverUrl, remove it.
2024-08-20 22:30:38 +03:00
hagen1778
2dba4165e8
docs: move change from VM changelog to VL changelog
The change for https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6764
was added to VM changelog by mistake

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-08-20 17:16:04 +02:00
jackyin
3ebdd3bcb8
vmui: fix not found index.js in VictoriaLogs (#6770)
fix #6764
the index.js file is for [this
feature](https://github.com/VictoriaMetrics/VictoriaMetrics/tree/master/app/vmui#predefined-dashboards),
the feature is just for victoriametrics. so the index.js is deleted in
victorialogs.
i just add an empty index.js to fix it.

---------

Co-authored-by: hagen1778 <roman@victoriametrics.com>
2024-08-20 14:50:37 +02:00
Nikolay
9feee15493
lib/promscrape: fixes proxy autorization (#6783)
* Adds custom dial func for HTTP-Connect and socks5 proxy tunnels.
  Standard golang http.transport exposes GetProxyConnectHeader function,
  but it doesn't allow to use separate tls config for proxy.
  It also not possible to enforce HTTP-Connect with standard http lib.
* For http scrape targets, by default http.Transport.Proxy function must
  be used. Since it has special case with full uri forward.
* Adds proxy.URL json methods that allow to properly copy internal
fields, like User/Password.
It should fix bug with proxy_url. When credentials specified at URL was
ignored.
* Adds tests for scrape client proxy requests

related issue https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6771
2024-08-19 22:31:18 +02:00
Zhu Jiekun
723d834c1a
lib/promrelabel: stop adding default port 80/433 to address label
*  It was necessary to add default ports for fasthttp client. After migration to the std.httpclient it's no longer needed.
* An additional configuration is required at proxy servers with implicitly set 80/443 ports to the host header (such as HA proxy.

It's expected that after upgrade __address_ label may change. But it should be rare case. 80/443 ports are not widely used at monitoring ecosystem. And it shouldn't have much impact. 

Related issue https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6792

Co-authored-by: Nikolay <nik@victoriametrics.com>
2024-08-19 22:28:49 +02:00
Hui Wang
0f1ec33892
vmalert: add command line flag -notifier.headers (#6751)
to allow configuring additional headers in each request to the
corresponding notifier.
Other flags like `-datasource.headers`, `-remoteWrite.headers` already
use `^^` as delimiter, it's consistent to use it in `-notifier.headers`
as well.


related https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3260
vmalert can integrate with alertmanager that supports multi-tenant by
adding tenantID header`X-Scope-OrgID` in requests.
In multitenancy, vmalert can also filter alerts which send to different
notifier addresses(or with different header settings) using
`alert_relabel_configs`.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3260

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
2024-08-19 21:40:57 +02:00
Github Actions
0ef59bf7b3
Automatic update Grafana datasource docs from VictoriaMetrics/victoriametrics-datasource@de3f649 (#6837) 2024-08-19 21:31:59 +02:00
Hui Wang
0fc1130f47
vmalert-tool: add -external.label and -external.url command-line … (#6766)
…flags to perform the same as vmalert

address https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6735

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
2024-08-19 21:29:28 +02:00
Andrii Chubatiuk
36bc458e9e
docs: reuse hugo image for webp conversion (#6825)
Use same hugo docker image for webp conversion.
While there, remove unused *.png images.
2024-08-19 14:38:56 +02:00
Fred Navruzov
0e464a3a4f
docs: vmanomaly - v1.15.5 patch notes (#6835)
### Describe Your Changes

docs: vmanomaly - v1.15.5 patch notes

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-08-19 11:14:57 +04:00
Daria Karavaieva
7279899a8a
docs/vmanomaly: updated model list in Overview (#6832)
### Describe Your Changes

Updated model list in Anomaly Detection Overview

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-08-17 13:43:28 +03:00
Github Actions
015f0b0424
Automatic update operator docs from VictoriaMetrics/operator@64879fb (#6831)
Automated changes by
[create-pull-request](https://github.com/peter-evans/create-pull-request)
GitHub action

Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Signed-off-by: f41gh7 <nik@victoriametrics.com>
2024-08-16 16:32:25 +02:00
Roman Khavronenko
e58dde6925
lib/httputils: parse URL before creating HTTP transport (#6820)
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6740

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-08-16 11:32:04 +02:00
Zhu Jiekun
a0d82a2a83
docs: add more details to -cacheDataPath vmselect flag (#6708)
vmselect will create `./tmp` dir under `cacheDataPath`. If
`cacheDataPath` is set to `/`, vmselect will use `/tmp`.

content under `/tmp` dir might be auto removed based on the OS
behaviour. See:
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5770

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
2024-08-15 21:37:03 +02:00
Fred Navruzov
f523a8adb2
docs: vmanomaly - release notes for 1.15.4 (#6813)
### Describe Your Changes

release notes for 1.15.4 patch

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-08-15 12:47:35 +04:00
Zakhar Bessarab
5390ee2413
app/vmseleсt/promql: fix calculation of histogram buckets
This issue was introduced in 6a4bd5049b

See: https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6714

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2024-08-15 10:11:41 +02:00
Daria Karavaieva
e702321cdc
docs/vmanomaly:change links from relative to absolute (#6809)
### Describe Your Changes

- change links from relative to absolute under Anomaly Detection section

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-08-14 23:00:57 +02:00
Fred Navruzov
6e7fe4841b
docs/vmanomaly - changelog update to v1.15.3 (#6808)
### Describe Your Changes

changelog updates to v1.15.3 patch of `vmanomaly`

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-08-14 20:34:40 +04:00
Fred Navruzov
9a64184fc7
docs: vmanomaly - reader page update (#6806)
### Describe Your Changes

small update to `data_range` parameter in uppermost config conversion
example

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-08-14 18:34:54 +03:00
Fred Navruzov
538e779b8b
docs: vmanomaly - release v1.15.2 (#6802)
### Describe Your Changes

update vmanomaly docs to forthcoming release v1.15.2

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-08-14 12:09:57 +03:00
Nikolay
9f42fccfc2
app/vminsert: returns back memory optimisation (#6794)
Production workload shows that it's useful optimisation.

Channel based objects pool allows to handle irregural data ingestion
requests and make memory allocations more smooth.
It's improves sync.Pool efficiency, since objects from sync.Pool removed
after 2 GC cycles. With GOGC=30 value, GC runs significantly more often.

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6733

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Signed-off-by: f41gh7 <nik@victoriametrics.com>
Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit f255800da3)
Signed-off-by: hagen1778 <roman@victoriametrics.com>

# Conflicts:
#	app/vminsert/common/insert_ctx_pool.go
2024-08-13 10:56:33 -04:00
ccliu
d134a310f3
vmagent: resolve the issue where usePromCompatibleNaming is not working (#6776)
Describe Your Changes
When I use usePromCompatibleNaming with vmagent to process data that
needs to be formatted from different sources such as InfluxDB, I find
that it doesn’t work

However, it works in vminsert. I found that vminsert uses the
HasRelabeling method to determine whether to relabel.
```go
func HasRelabeling() bool {
	pcs := pcsGlobal.Load()
	return pcs.Len() > 0 || *usePromCompatibleNaming
}
```
in vmagent, the decision to relabel is determined only by
pcsGlobal.Len() > 0. However, in the applyRelabeling method, the
usePromCompatibleNaming logic is also used to determine whether to
relabel in the error handling.
```go
func (rctx *relabelCtx) applyRelabeling(tss []prompbmarshal.TimeSeries, pcs *promrelabel.ParsedConfigs) []prompbmarshal.TimeSeries {
	if pcs.Len() == 0 && !*usePromCompatibleNaming {
		// Nothing to change.
		return tss
	}
```
So I think that the logic for determining whether to relabel in vmagent
is not as expected.

Checklist
The following checks are mandatory:

[]My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Co-authored-by: Roman Khavronenko <hagen1778@gmail.com>
2024-08-13 10:32:05 -04:00
jackyin
5f5bc46b3e
vlogs: add select/deselect all button to table settings in UI (#6680)
fix #6668, just add **select all** and "unselect all" func.


https://github.com/user-attachments/assets/0c31385b-def0-4618-aa9c-5ba4bb6f56c3

---------

Co-authored-by: Yury Molodov <yurymolodov@gmail.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
2024-08-13 10:20:07 -04:00
Hui Wang
62d19369a3
stream aggregation: do not allow to enable -stream.keepInput and `k… (#6723)
…eep_metric_names` options in stream aggregation config together

With aggregated data and raw data under the same metric, results would
be confusing.

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
2024-08-13 08:54:35 -04:00
hagen1778
331573b0bb
docs: mention https://blog.zomato.com/migrating-to-victoriametrics-a-complete-overhaul-for-enhanced-observability
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-08-13 08:26:20 -04:00
Fred Navruzov
a99fcfbf1a
docs/vmanomaly - typos fix & clarity (#6793)
### Describe Your Changes

typos fix & clarity improvement of vmanomaly docs after v1.15.1 release

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-08-10 17:43:05 +03:00
Fred Navruzov
985e4f0b99
docs: vmanomaly - release v1.15.1 (#6782)
### Describe Your Changes

 vmanomaly - release v1.15.1 updates to docs:
- changelog page
- reader page (new arguments docs)
- typos & fixes

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-08-10 16:54:27 +03:00