Commit graph

9547 commits

Author SHA1 Message Date
f41gh7
f97489dcf9
app/vmagent: follow-up 430163d and 680b8c2
Removes global defaultAuthToken, since it's no longer needed.
It was added as fallback for 'remoteWrite.multitenantURL' feature.
This feature was deprecated at v1.102 version and removed.

 Updates newRemoteWriteCtxs function, it shouldn't accept auth.Token no longer.
This was also a part of remove feature.

Signed-off-by: f41gh7 <nik@victoriametrics.com>
2024-11-29 15:23:21 +01:00
Nikolay
d0a508d1ca
app/vmagent: fixes multitenant token parse
Previously, vmagent produced parsing error for 'multitenant' auth token
value for the cases:
* data ingestion with enableMultitentEndpoints
* data scrapping at promscrape

 It's inconsistent to the other VictoriaMetrics components.
Since 'multitenant' is well-known token value for multitenancy via
labels. And vmagent is intended to be compatible with vminsert ingestion
endpoints.

 This commit replaces NewToken with NewTokenPossibleMultitenant function
for token parsing. It allows to use multitenant value for it. And it
makes token values consistent for the all components.

Related issue:
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7694
2024-11-29 15:23:20 +01:00
Aliaksandr Valialkin
87ffaf4bb5
vendor: run make vendor-update 2024-11-29 13:48:58 +01:00
f41gh7
92d884eeb7
app/vmauth: fixes ip_filters typo at example_config
url_map section cannot have ip_filters. Only top level config section
have it.
2024-11-29 11:01:07 +01:00
f41gh7
e1d9c32f04
app/vmauth: fixes ip_filters init for uauthorized_user
Previously ip_filters wasn't properly inited for this part of config.
It resulted to bypass requests for this section.

 This commit properly inits `ip_filter`.
2024-11-29 11:01:07 +01:00
f41gh7
2f916f25bc
app/vmauth: add ip_filter deny metrics
Previously, all requests rejected by `ip_filter` were silently aborted.

This commit adds new metrics:
* vmauth_user_ip_denies_total
* vmauth_global_ip_denies_total
* vmauth_unauthorized_user_ip_denies_total

 It adds observability to this feature and allow to measure rejected requests.

Related issue:
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6883

Signed-off-by: f41gh7 <nik@victoriametrics.com>
2024-11-29 11:01:07 +01:00
Github Actions
41b90cadb0
Automatic update helm docs from VictoriaMetrics/helm-charts@f5f1b9c (#7691)
Automated changes by
[create-pull-request](https://github.com/peter-evans/create-pull-request)
GitHub action

Signed-off-by: Github Actions <133988544+victoriametrics-bot@users.noreply.github.com>
Co-authored-by: AndrewChubatiuk <3162380+AndrewChubatiuk@users.noreply.github.com>
2024-11-29 10:34:25 +01:00
f41gh7
e25e67b300
app/vmalert-tool: print an error message if no rule group is found under 'rule_files'
Both vmalert and vmalert-tool support multiple `rule_files` and use
directory as a file, so it's ok if some files don't contain any rule
group. But vmalert-tool should warn the user if no rule group is found
in any of the `rule_files`.

Related issue:
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7663
2024-11-29 10:34:14 +01:00
Nikolay
f4c597da88
app/vmauth: add removeXFFHTTPHeaderValue flag
Previously, there was no option to replace value of `X-Forwarded-For`
HTTP Header. It was only possible to completely remove it. It's not good
solution, since backend may require this information. But using direct
value of this header is insecure. And requires complex knowledge of
infrastruce at backend side (see spoofing X-Forwarded-For articles).

This commit adds new flag, that replaces content of `X-Forwarded-For`
HTTP Header value with current `RemoteAddress` of client that send
request.
It should be used if `vmauth` is directly attached to the internet.

 Related issue:
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6883

---------

Signed-off-by: f41gh7 <nik@victoriametrics.com>
2024-11-29 10:33:19 +01:00
Nikolay
1d339e8146
app/vmauth: add real_ip_header to ip_filters
This commit allows vmauth to obtain client IP address from HTTP Headers.
Main scenario for it is vmauth located behind reverse-proxy.

 It adds both global and per user configuration settings: -httpRealIPHeader and `real_ip_header` config option.

vmauth try to obtain IP from header if this setting is set. If header is not exists, vmauth fallbacks to `remoteAddress`.

Commit also updates incorrect benchmarks and align test package naming for ip_filters

Related issue:
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6883

Signed-off-by: f41gh7 <nik@victoriametrics.com>
2024-11-28 14:55:49 +01:00
Yury Molodov
0d951a35f2
vmui: fix predefined panels
### Describe Your Changes

- Fixes the handling of the `showLegend` flag.  
- Fixes the handling of `alias`.  
- Adds support for alias templates, allowing dynamic substitutions like
`{{label_name}}`.

Related issue:
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7565
2024-11-28 13:50:02 +01:00
Andrii Chubatiuk
8271094c98
deployment/docker: use vector for local setup (#7681) 2024-11-28 13:50:02 +01:00
Github Actions
19589fdeb1
Automatic update helm docs from VictoriaMetrics/helm-charts@a5bc12b (#7685)
Automated changes by
[create-pull-request](https://github.com/peter-evans/create-pull-request)
GitHub action

Signed-off-by: Github Actions <133988544+victoriametrics-bot@users.noreply.github.com>
Co-authored-by: AndrewChubatiuk <3162380+AndrewChubatiuk@users.noreply.github.com>
2024-11-28 13:50:02 +01:00
Fred Navruzov
ae673e8b34
docs/vmanomaly - release 1.18.5 (#7684)
### Describe Your Changes

docs/vmanomaly - release 1.18.5 doc updates

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-11-28 13:50:02 +01:00
Nikolay
879bba11ba
app/vmauth: adds dryRun flag for config validation
This flag only validates configuration file pointed by `auth.config`
flag.

 Related issue:
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7505

Signed-off-by: f41gh7 <nik@victoriametrics.com>
2024-11-28 13:50:02 +01:00
Yury Molodov
633a7e7245
vmui/logs: optimize memory consumption (#7524)
### Describe Your Changes

- **Memory Optimization**: Reduced memory consumption on the "Group" and
"JSON" tabs by approximately 30%.

- **Table Pagination**: Added pagination to the "Table" view with an
option to select the number of rows displayed (from 10 to 1000 items per
page, with a default of 1000). This change significantly reduced memory
usage by approximately 75%.

Related to #7185

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
2024-11-28 13:50:02 +01:00
Andrii Chubatiuk
7ded82d663
docs/vmgateway: added to documentation a case, when vm_access claim value is of string type 2024-11-28 13:50:01 +01:00
Viet Hung Nguyen
b5a40f4be8
docs: fix rule unittest rule_files point to alerts.yaml (#7664)
### Describe Your Changes

Fix wrong path to rules_file in vmalert-tool doc

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

Signed-off-by: Viet Hung Nguyen <hvn@familug.org>
Co-authored-by: Hui Wang <haley@victoriametrics.com>
2024-11-28 13:50:01 +01:00
Hui Wang
275b776e02
docs: add multitenat endpoints usage for vmalert multitenancy (#7665)
### Describe Your Changes

Please provide a brief description of the changes you made. Be as
specific as possible to help others understand the purpose and impact of
your modifications.

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-11-28 13:50:01 +01:00
Zhu Jiekun
44d856922a
lib/promscrape/discovery: properly apply the resource_group filter for Azure service discovery
Previously, this filter did not apply to virtual
machine scale sets, causing all virtual machines to be discovered.

 This commit conditionally adds `resource_group` filter for Azure service discovery on virtual
machine scale sets. 

 Related issue:
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7630.
2024-11-26 19:08:31 +01:00
Nikolay
04ec09f6ce
apptest: adds cluster test for multitenant API requests
This commit adds integration test for multitenant via labels feature -
https://docs.victoriametrics.com/cluster-victoriametrics/#multitenancy-via-labels

It also extends current test models in order to:
- accept float timestamps returned from /api/v1/query_range and query
api
https://github.com/VictoriaMetrics/VictoriaMetrics/blob/master/app/vmselect/prometheus/util.qtpl#L43
- accept arbitrary query url params for requests. It simplifies testing
for the different VM API extensions


---------

Signed-off-by: f41gh7 <nik@victoriametrics.com>
2024-11-26 19:03:56 +01:00
f41gh7
ed9ab2ea73
refactoring: changed prompb to prompbmarshal everythere where internal series transformations are happening (#7409)
doing similar changes for both vmagent and vminsert (like one in
https://github.com/VictoriaMetrics/VictoriaMetrics/pull/7399) ends up
with almost same implementations for each of packages instead of having
this shared code in one place. one of the reasons is the same Timeseries
and Labels structure from different prompb and prompbmarshal packages.
My proposal is to use structures from prompb package only to
marshal/unmarshal sent/received data, but for internal transformations
use only structures from prompbmarshal package

Another example, where it already can help to simplify code is streaming
aggregation pipeline for vmsingle (now it first marshals
prompb.Timeseries to storage.MetricRow and then if streaming aggregation
or deduplication is enabled it unmarshals all the series back but to
prompbmarshal.Timeseries)

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-11-26 19:02:06 +01:00
Roman Khavronenko
1753c3850b
app/vmauth: dump requests that failed the route rules to stderr (#7649)
Additional info from the dump can be used to debug rotuing rules.

https://pkg.go.dev/net/http/httputil#DumpRequest

### Describe Your Changes

Please provide a brief description of the changes you made. Be as
specific as possible to help others understand the purpose and impact of
your modifications.

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-11-26 18:50:31 +01:00
Artem Navoiev
d8b4faf50c
docs: update formatting on cloud billing page
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2024-11-26 18:50:31 +01:00
matty
ecbdbb8e49
docs: Bump LTS versions (#7658)
### Describe Your Changes

LTS versions are out of date, linking to old releases. This updates them
to the latest.

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-11-26 18:50:31 +01:00
Hui Wang
5f9db9a61f
alerts-vmalert: reserve rule name for description (#7659) 2024-11-26 18:50:30 +01:00
Artem Navoiev
103bd5bd88
docs: add cloud billing page (#7655)
### Describe Your Changes

Please provide a brief description of the changes you made. Be as
specific as possible to help others understand the purpose and impact of
your modifications.

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2024-11-26 18:50:30 +01:00
Github Actions
f122f35a37
Automatic update helm docs from VictoriaMetrics/helm-charts@ebc6d67 (#7653)
Automated changes by
[create-pull-request](https://github.com/peter-evans/create-pull-request)
GitHub action

Signed-off-by: Github Actions <133988544+victoriametrics-bot@users.noreply.github.com>
Co-authored-by: AndrewChubatiuk <3162380+AndrewChubatiuk@users.noreply.github.com>
2024-11-25 18:08:32 +01:00
Andrei Baidarov
728ceefca1
vmagent: set up a timeout for tcp connection establishment during k8s discovery
Previously, default dial timeout was used for kubernetes API server connection.

 This commit changes it for custom dialer used by the all VictoriaMetrics components. It has lower connection timeout (30s by default). 


 Related issue:
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7127

---------
Co-authored-by: f41gh7 <nik@victoriametrics.com>
2024-11-25 18:08:32 +01:00
hagen1778
7a06f05221
docs: re-structure multitenancy docs
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-11-25 18:08:32 +01:00
hagen1778
a099f0a70e
docs: update otel docs
* cross-link related doc chapters and guides about otel
* mention different URL format for cluster version of VM

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-11-25 18:08:32 +01:00
hagen1778
4545c39b21
dashboards: fix Ingestion row for vmagent dashboard
Previously, clicking on Ingestion row could result in a visual blip.
Re-ordering panels within the row seems to fix it.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-11-25 18:08:32 +01:00
hagen1778
9a0b263e53
docs: update link for tsdb stats word
The link was pointing to itself, which doesn't make sense.
Changed it to point to https://docs.victoriametrics.com/url-examples/#apiv1statustsdb

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-11-25 18:08:31 +01:00
Hui Wang
82e42b1d4a
vmalert-tool: exit immediately with error message if no test file found in -files (#7627) 2024-11-25 11:52:33 +01:00
Nikolay
93061dfa7b
app/vmselect: fixes multitenant cache init
Previously multitenant cache was inited before flag.Parse call. It
didn't allow to change cache expiration value and default value was
always used.

 This commit adds cache init at the first time cache was called.

 Also this commit adds small cache improvements:
* chore for cleanup cache, it now uses common pattern for in-place items
filtering
* fail cache request fast if item is already expired


---------
Signed-off-by: f41gh7 <nik@victoriametrics.com>
Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
2024-11-25 11:49:34 +01:00
Andrei Baidarov
439d1b932e
app/vmselect: fix panic/incorrect tenant in key
This is a follow-up after 3120dc2
  
- Consistently use key for rollupCache in multitenant mode cache keys use different authTokens. Previously it could lead to panic in rare cases when cache state was inconsistent. 
- Do not share `err` variable across goroutines for `processBlock` function. It could lead to data races. 

Related issue https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7549
---------
Signed-off-by: Andrei Baidarov <abaidarov@yandex.ru>
Co-authored-by: f41gh7 <nik@victoriametrics.com>
2024-11-25 11:47:24 +01:00
Github Actions
ba5e0c8745
Automatic update helm docs from VictoriaMetrics/helm-charts@6a5b14b (#7642)
Automated changes by
[create-pull-request](https://github.com/peter-evans/create-pull-request)
GitHub action

Signed-off-by: Github Actions <133988544+victoriametrics-bot@users.noreply.github.com>
Co-authored-by: AndrewChubatiuk <3162380+AndrewChubatiuk@users.noreply.github.com>
Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
2024-11-25 11:42:16 +01:00
dmitry-shur
857144826f
docs: fixed typo in FAQ (#7643)
### Describe Your Changes

Please provide a brief description of the changes you made. Be as
specific as possible to help others understand the purpose and impact of
your modifications.

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-11-25 11:42:16 +01:00
Github Actions
65ed9759ed
Automatic update helm docs from VictoriaMetrics/helm-charts@a582432 (#7641)
Automated changes by
[create-pull-request](https://github.com/peter-evans/create-pull-request)
GitHub action

Signed-off-by: Github Actions <133988544+victoriametrics-bot@users.noreply.github.com>
Co-authored-by: AndrewChubatiuk <3162380+AndrewChubatiuk@users.noreply.github.com>
2024-11-25 11:42:16 +01:00
hagen1778
65a2752cd8
docs: add vmui section to cluster docs
It looks like users get confused with vmui availability in vmselect.
Adding a note about it to cluster docs should improve situation.

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7631

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-11-25 11:42:16 +01:00
Artem Navoiev
a1b2cf43c4
docs: cloud mention silver level in support
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2024-11-25 11:42:16 +01:00
Hui Wang
af4c6f3a29
vmalert: fix alert states restoration (#7624)
Previously, when the alert got resolved shortly before the vmalert
process shuts down, this could result in false alerts.

This change switches vmalert to use MetricsQL function during alerts state restore, which makes it
incompatible for state restoration with PromQL.

---------

Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
2024-11-22 17:38:14 +01:00
Artem Fetishev
d238011327
apptest: add tests for stale nans in instant query (#7621)
### Describe Your Changes

These are the integration tests that confirm that instant queries may
return stale NaNs when the query contains a rollup function.

The bug was reported at #5806. There is also a fix: #7275. The tests in
this PR will be used co confirm that the fix works.

Some test refactoring has been done along the way. Sorry, couldn't
resist.

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Signed-off-by: Artem Fetishev <rtm@victoriametrics.com>
2024-11-22 17:38:13 +01:00
Github Actions
53c2214940
Automatic update Grafana datasource docs from VictoriaMetrics/victoriametrics-datasource@a5e2cb4 (#7622) 2024-11-22 17:38:13 +01:00
Dmytro Kozlov
78175a45e0
deployment/docker: update victoriametrics datasource versions to the latest releases (#7623)
### Describe Your Changes

Updated victoriametrics-datasource to the newest
[release](https://github.com/VictoriaMetrics/victoriametrics-datasource/releases/tag/v0.10.3)
version

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-11-22 17:38:13 +01:00
hagen1778
1dd56f022d
docs: update differences between vmagent and Prometheus in FAQ
* mention stream aggregation
* rm statement that Prometheus can only pull data, which is not true anymore
* mention absence of backfilling limitations

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-11-22 17:38:13 +01:00
Ivan Yurochko
60ac4e0c00
lib/streamaggr: add ignore_first_sample_interval param for streamaggr cfg (#7313)
### Describe Your Changes

As of right now by default aggregated output in streaming aggregation
takes a staleness interval and only starts sending first samples after
the staleness interval passes. We have a use case where we prefer to
start sending data as soon as we have any. This adds the option to
configure when we start sending first samples

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7116

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Co-authored-by: hagen1778 <roman@victoriametrics.com>
2024-11-22 17:38:13 +01:00
Github Actions
66cd3f0a9f
Automatic update helm docs from VictoriaMetrics/helm-charts@b0095d3 (#7618)
Automated changes by
[create-pull-request](https://github.com/peter-evans/create-pull-request)
GitHub action

Signed-off-by: Github Actions <133988544+victoriametrics-bot@users.noreply.github.com>
Co-authored-by: AndrewChubatiuk <3162380+AndrewChubatiuk@users.noreply.github.com>
2024-11-22 17:38:13 +01:00
hagen1778
257b0dc10c
docs: mention Raw Query tab in vmui
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7024
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-11-22 17:38:12 +01:00
Yury Molodov
db917d522d
vmui: add Raw Query tab (#7575)
### Describe Your Changes

1. **Add new `Raw Query` tab**  
A new `Raw Query` tab has been added to the
[vmui](https://docs.victoriametrics.com/#vmui) interface for displaying
raw data. The tab uses the `/api/v1/export` API endpoint. Related issue:
[#7024](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7024)


2. **Fix rendering of isolated points on the graph**  
Previously, isolated points (not connected to other points on the left
or right) were not visible on the graph. Now, they are rendered
correctly.
 

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
2024-11-22 17:38:12 +01:00