github-mirrors/VictoriaMetrics
1
0
Fork 0
mirror of https://github.com/VictoriaMetrics/VictoriaMetrics.git synced 2024-11-21 14:44:00 +00:00
Code Issues Projects Releases Packages Wiki Activity
7708 commits 150 branches 748 tags 347 MiB
Commit graph

3631 commits

Author SHA1 Message Date
Dmytro Kozlov
38b2a5bc44
deployment/docker: add grafana datasource to the docker-compose files (#5363) 
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3920
https://github.com/VictoriaMetrics/grafana-datasource/issues/113
 2024-01-22 15:45:31 +01:00
hagen1778
da556cc329
docs: fix Grafana link example for vmalert 
Signed-off-by: hagen1778 <roman@victoriametrics.com>
 2024-01-22 09:35:18 +01:00
Aliaksandr Valialkin
d3ee3e0ef5
Revert "lib/promscrape: do not store last scrape response when stale markers … (#5577)" 
This reverts commit cfec258803.

Reason for revert: the original code already doesn't store the last scrape response when stale markers are disabled.
The scrapeWork.areIdenticalSeries() function always returns true is stale markers are disabled.
This prevents from storing the last response at scrapeWork.processScrapedData().

It looks like the reverted commit could also return back the issue https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3660

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5577
 2024-01-22 00:43:48 +02:00
Aliaksandr Valialkin
9c0863babc
docs: use persistent links to Grafana dashboards 
These links do not depend on the dashboard name, so they do not break after the renaming of the dashboard.

This is a follow-up for ff33e60a3d
 2024-01-22 00:17:17 +02:00
Aliaksandr Valialkin
1c7f990fad
app/vmselect: handle negative time range start in a generic manner inside NewSearchQuery() 
This is a follow-up for cf03e11d89

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5553
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5630
 2024-01-21 23:45:31 +02:00
Artem Navoiev
3f7ed7e6b2
docs vmanomaly fix anchor 
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
 2024-01-21 22:21:37 +01:00
Hui Wang
4e3242b02d
lib/promscrape/discovery/kubernetes: fix watcher start order for roles endpoints and endpointslice (#5557) 
* lib/promscrape/discovery/kubernetes: fix watcher start order for roles endpoints and endpointslice

Previously the groupWatcher could be mistakenly stopped when requests for pod or services resources take too long.

* remove mislead comment

* docs/sd_configs.md: mention -promscrape.kubernetes.attachNodeMetadataAll flag in the description for attach_metadata section

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4640

* wip

* lib/promscrape/kubernetes: prevent from stopping groupWatcher when there are in-flight apiWatcher.mustStart() calls

groupWatcher is stopped if it has zero registered apiWatchers during 14 seconds.
But such a groupWatcher can be still in use if apiWatcher for `role: endpoints` or `role: endpointslice`
is being registered and the discovery of the associated `pod` and/or `service` objects takes longer
than 14 seconds - see the beginning of groupWatcher.startWatchersForRole() function for details.

Track the number of in-flight calls to apiWatcher.mustStart() and prevent from stopping the associated groupWatcher
if the number of in-flight calls is non-zero.

P.S. postponing the discovery of `pod` and/or `service` objects associated with `endpoints` or `endpointslice` roles
isn't the best solution, since it slows down initial discovery of `endpoints` and `endpointslice` targets.

* typo fix

---------

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2024-01-21 23:13:15 +02:00
Aliaksandr Valialkin
1f105dde98
all: allow dynamically reading *AuthKey flag values from files and urls 
Examples:

1) -metricsAuthKey=file:///abs/path/to/file - reads flag value from the given absolute filepath
2) -metricsAuthKey=file://./relative/path/to/file - reads flag value from the given relative filepath
3) -metricsAuthKey=http://some-host/some/path?query_arg=abc - reads flag value from the given url

The flag value is automatically updated when the file contents changes.
 2024-01-21 22:03:38 +02:00
Fred Navruzov
7e68722686
- fix 404 links after renaming (#5653) 
- improve wording on diagram
- swap enterprise/about chapters for page clarity
 2024-01-21 21:24:29 +02:00
Fred Navruzov
0038102b98
docs: vmanomaly - add component interaction diagram (#5652) 
* add interaction diagram for vmanomaly components
* small docs fixes
* resolve suggestions
 2024-01-21 19:33:59 +02:00
Aliaksandr Valialkin
3d83f3347d
docs/sd_configs.md: mention -promscrape.kubernetes.attachNodeMetadataAll flag in the description for attach_metadata section 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4640
 2024-01-21 13:26:03 +02:00
Nikolay
b3598ba2c1
app/vmauth: adds metric_labels and backend_errors counter (#5585) 
* app/vmauth: adds metric_labels and backend_errors counter
it must improve observability for user requests with new metric - per user backend errors counter.
it's needed to calculate requests fail rate to the configured backends.
metric_labels configuration allows to perform additional aggregations on top of multiple users from configuration section.
It could be multiple clients or clients with separate read/write tokens
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5565

* wip

---------

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2024-01-21 04:40:52 +02:00
Aliaksandr Valialkin
7fba73ce11
lib/promscrape/discovery/kubernetes: add -promscrape.kubernetes.attachNodeMetadataAll command-line flag 
This flag allows setting attach_metadata.node=true for all the kubernetes_sd_configs defined at -promscrape.config

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4640

Thanks to wasim-nihal for the initial implementation at https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5593
 2024-01-21 03:13:56 +02:00
Hui Wang
fad212c39c
app/vmselect/promql: properly handle possible negative results caused… (#5608) 
* app/vmselect/promql: properly handle possible negative results caused by float operations precision error in rollup functions like rate() or increase()

* fix test
 2024-01-21 02:53:29 +02:00
Nikolay
c9f39fd51f
app/vmselect/netstorage (#5649) 
* app/vmselect/netstorage

correctly handle errGlobal set

* wip

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5649

---------

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2024-01-21 02:47:29 +02:00
Nikolay
8ab0ce3ded
app/vmselect: abort streaming connections for vmselect (#5650) 
* app/vmselect: abort streaming connections for vmselect
due to streaming nature of export APIs, curl and simmilr tools cannot
detect errors that happened after http.Header with status 200 was
written to it.

This PR tracks if body write was already started and closes connection.

It allows client to detect not expected chunk sequence and return error
to the caller.

Mostly it affects vmselect at cluster version

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5645

* wip

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5645
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5650

---------

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2024-01-21 02:12:51 +02:00
Aliaksandr Valialkin
74448a7e57
lib/promscrape/discovery/hetzner: follow-up after 03a97dc678 
- docs/sd_configs.md: moved hetzner_sd_configs docs to the correct place according to alphabetical order of SD names,
  document missing __meta_hetzner_role label.
- lib/promscrape/config.go: added missing MustStop() call for Hetzner SD,
  and moved the code to the correct place according to alphabetical order of SD names.
- lib/promscrape/discovery/hetzner: properly handle pagination for hloud API responses,
  populate missing __meta_hetzner_role label like Prometheus does.
- Properly populate __meta_hetzner_public_ipv6_network label like Prometheus does.
- Remove unused SDConfig.Token.
- Remove "omitempty" annotation from SDConfig.Role field, since this field is mandatory.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5550
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3154
 2024-01-20 17:01:53 +02:00
Artem Navoiev
873483a782
vmanomly use proper title for overiview doc 
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
 2024-01-19 19:59:17 +01:00
Hui Wang
cfec258803
lib/promscrape: do not store last scrape response when stale markers … (#5577) 
* lib/promscrape: do not store last scrape response when stale markers are disabled

* update changelog
 2024-01-20 00:53:41 +08:00
Artem Navoiev
6a2a8cd426
docs add docs titles 
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
 2024-01-19 17:02:20 +01:00
Artem Navoiev
dfa43da1a2
vmanonamly docs fix sorting for jekill as far it doesnt support of sorting the folders 
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
 2024-01-19 16:55:44 +01:00
Artem Navoiev
1af5faa4af
vmanomaly remove unused pages from menu 
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
 2024-01-19 16:50:12 +01:00
Artem Navoiev
5e17636994
vmanomly specify the right menu parent for overview page 
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
 2024-01-19 16:43:18 +01:00
Artem Navoiev
c425ec3088
docs vmanmoly fix sorting 
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
 2024-01-19 16:15:40 +01:00
Artem Navoiev
ec85d32e21
Move vmanomaly page to its own section (#5646) 
* docs: move vmanomaly overview page to its section

Signed-off-by: Artem Navoiev <tenmozes@gmail.com>

* add alias for backward compatibility

Signed-off-by: Artem Navoiev <tenmozes@gmail.com>

* fix links

Signed-off-by: Artem Navoiev <tenmozes@gmail.com>

* change title

Signed-off-by: Artem Navoiev <tenmozes@gmail.com>

---------

Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
 2024-01-19 07:00:41 -08:00
Roman Khavronenko
7e374c227f
app/vmui: send step param for instant queries (#5639) 
The change reverts https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3896
 due to reasons explained in https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3896#issuecomment-1896704401

Signed-off-by: hagen1778 <roman@victoriametrics.com>
 2024-01-19 08:48:16 +01:00
Fred Navruzov
69ae1d30bf
docs: vmanomaly slight improvements (#5637) 
* - better messaging
- update links to dockerhub in guides
- added anomaly_score to FAQ
- improve model section (sort + use cases)
- slight refactor of a guide

* rename guide & change refs

* change wording in installation options

* - update remaining text reference
- add cross-link to component sections in guide

* add docs/.jekyll-metadata to .gitignore
 2024-01-18 02:37:36 -08:00
hagen1778
0a5ffb3bc1
docs: remove slug from Grafana dashboard URLs 
Each Grafana dashboard has unique ID which can be used to fetch the dashboard
from grafana.com: https://grafana.com/grafana/dashboards/11176
The same dashboard can be accessed via URL with slug: https://grafana.com/grafana/dashboards/11176-victoriametrics-cluster/
But using slug implies that any change to dashboard name will break the link.
So it is better to just use ID, so the dashboard URL will never break.

This is follow-up for ff33e60a3d

Signed-off-by: hagen1778 <roman@victoriametrics.com>
 2024-01-18 11:19:53 +01:00
Artem Navoiev
f89d16fc4c
docs: vmanomaly update vmanomaly + vmalert guide (#5636) 
* docs: vmanomaly update vmanomaly + vmalert guide

Signed-off-by: Artem Navoiev <tenmozes@gmail.com>

* docs: vmanomaly update vmanomaly + vmalert guide. Update docker compose and monitoring section

Signed-off-by: Artem Navoiev <tenmozes@gmail.com>

* typos and fixes

Signed-off-by: Artem Navoiev <tenmozes@gmail.com>

---------

Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
 2024-01-17 11:49:51 -08:00
Artem Navoiev
ff33e60a3d
fix link for grafana dashbaord for single node after its renaming 
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
 2024-01-17 16:00:33 +01:00
Artem Navoiev
dab160cd74
docs: changelog fix the link to cluster 
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
 2024-01-17 15:44:56 +01:00
Artem Navoiev
a3b3ea4d73
vmanomaly docs fix broken relative links 
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
 2024-01-17 15:41:03 +01:00
Artem Navoiev
9a353ee695
docs/anomaly-detection/components/models.md add sort:1 
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
 2024-01-17 15:26:38 +01:00
Artem Navoiev
0c06934a59
vmanonaly docs add .html for the section document models 
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
 2024-01-17 15:25:30 +01:00
Artem Navoiev
5b419cfb2b
vmanomaly docs simplify the strcuture (#5634) 
* vmanomaly docs simplify the strcuture

Signed-off-by: Artem Navoiev <tenmozes@gmail.com>

* fix links

Signed-off-by: Artem Navoiev <tenmozes@gmail.com>

---------

Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
 2024-01-17 06:20:27 -08:00
hagen1778
71681fd1ca
docs/setup-size: rm tolerable churn rate % 
It is likely this value was borrowed from panel `Slow inserts` panel
from Grafana dasbhoard for VM single/cluster installations. This is a mistake.
There is no such thing as "tolerable churn rate", as tolerancy depends on the amount
of allocated resources.

Although, it is unclear what is meant by 5%. If it refers to 5% of new time series per second,
then such churn rate is extremely high. It would mean that the avg life of a time series is 20s.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
 2024-01-17 14:44:28 +01:00
Roman Khavronenko
cf03e11d89
app/vmselect: properly calculate start param for queries with too big look-behind window (#5630) 
Properly determine time range search for instant queries with too big look-behind window like `foo[100y]`.
 Previously, such queries could return empty responses even if `foo` is present in database.

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5553

Signed-off-by: hagen1778 <roman@victoriametrics.com>
 2024-01-17 13:48:06 +01:00
Aliaksandr Valialkin
3c3450fc53
docs/keyConcepts.md: typo fixes after b7ffee2644 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5555
 2024-01-17 13:27:47 +02:00
Aliaksandr Valialkin
befcd93305
docs/keyConcepts.md: document /internal/force_flush handler 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5555
 2024-01-17 13:23:32 +02:00
Aliaksandr Valialkin
cc6819869a
docs/CHANGELOG*: move changes for 2023 year to docs/CHANGELOG_2023.md 2024-01-17 13:10:32 +02:00
hagen1778
8040bdc1d6
docs/troubleshooting: mention query latency in unexpected query results 
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5555

Signed-off-by: hagen1778 <roman@victoriametrics.com>
 2024-01-17 11:59:47 +01:00
Aliaksandr Valialkin
1683df11f0
docs/CHANGELOG.md: document v1.93.10 LTS release 
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.93.10
 2024-01-17 01:45:06 +02:00
Aliaksandr Valialkin
f2f0468ae7
docs/keyConcepts.md: clarify which values can be stored in VictoriaMetrics without precision loss 
This is a follow-up for 43d7de4afe

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5485
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5503
 2024-01-17 01:09:16 +02:00
Aliaksandr Valialkin
ecce2d6db1
docs/CHANGELOG.md: document v1.87.13 LTS release 
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.87.13
 2024-01-17 01:04:05 +02:00
Aliaksandr Valialkin
c830064c2f
docs/{vmbackup,vmbackupmanager}.md: clarify why storing backups to S3 Glacier can be time-consuming and expensive 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5614
This is a follow-up for e14e3d9c8c
 2024-01-16 21:50:43 +02:00
hagen1778
b0287867fe
deployment/dashboards: change title VictoriaMetrics to VictoriaMetrics - single-node 
The new title should provide better understanding of this dashboard purpose.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
 2024-01-16 20:39:52 +01:00
Aliaksandr Valialkin
30d77393a5
docs/CHANGELOG.md: fix a link in the description of 70cd09e736 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5581
 2024-01-16 20:54:44 +02:00
Aliaksandr Valialkin
19c04549a5
docs/Single-server-VictoriaMetrics.md: explain why staleness markers are treated as an ordinary values during de-duplication 
This is a follow-up for d374595e31
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5587
 2024-01-16 20:10:12 +02:00
Aliaksandr Valialkin
db6495560c
docs/vmagent.md: mention the minumum value for -remoteWrite.vmProtoCompressLevel 
Recommend using the default value for -remoteWrite.vmProtoCompressLevel

This is a follow-up for 095d982976
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5575
 2024-01-16 19:46:13 +02:00
Artem Navoiev
846d5a3ab8
docs: vmanomaly fix font matter 
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
 2024-01-16 16:58:48 +01:00
Artem Navoiev
481471b872
docs: vmanomaly remove commented text 
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
 2024-01-16 16:43:59 +01:00
Aliaksandr Valialkin
a74f6d63e0
deployment/docker: update Go builder from Go1.21.5 to Go1.21.6 2024-01-16 17:00:16 +02:00
Aliaksandr Valialkin
9d886a2eb0
lib/storage: follow-up for 4b8088e377 
- Clarify the bugfix description at docs/CHANGELOG.md
- Simplify the code by accessing prefetchedMetricIDs struct under the lock
  instead of using lockless access to immutable struct.
  This shouldn't worsen code scalability too much on busy systems with many CPU cores,
  since the code executed under the lock is quite small and fast.
  This allows removing cloning of prefetchedMetricIDs struct every time
  new metric names are pre-fetched. This should reduce load on Go GC,
  since the cloning of uin64set.Set struct allocates many new objects.
 2024-01-16 15:29:57 +02:00
Hui Wang
3ac44baebe
exit vmagent if there is config syntax error in scrape_config_files when -promscrape.config.strictParse=true (#5560) 2024-01-16 17:30:02 +08:00
hagen1778
d0e4190969
deployment/alerts: add job label to DiskRunsOutOfSpace alerting rule 
So it is easier to understand to which installation the triggered instance belongs.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
 2024-01-16 09:49:39 +01:00
Aliaksandr Valialkin
388d020b7c
app/vmselect/promql: follow-up for ce4f26db02 
- Document the bugfix at docs/CHANGELOG.md
- Filter out NaN values before sorting as suggested at https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5509#discussion_r1447369218
- Revert unrelated changes in lib/filestream and lib/fs
- Use simpler test at app/vmselect/promql/exec_test.go

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5509
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5506
 2024-01-16 02:55:08 +02:00
Aliaksandr Valialkin
190a6565ae
app/vmselect/promql: consistently sort results of a or b query 
Previously the order of results returned from `a or b` query could change with each request
because the sorting for such query has been disabled in order to satisfy
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4763 .

This commit executes `a or b` query as `sortByMetricName(a) or sortByMetricName(b)`.
This makes the order of returned time series consistent across requests,
while maintaining the requirement from https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4763 ,
e.g. `b` results are consistently put after `a` results.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5393
 2024-01-16 01:30:10 +02:00
Artem Navoiev
51cdf3676b
docs: vmanomaly fix image size 
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
 2024-01-15 23:10:52 +01:00
Artem Navoiev
74219a1727
vmanomly: guide add diagramm 
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
 2024-01-15 23:07:49 +01:00
Artem Navoiev
041a1966c5
docs: vmanomaly fix formatting and remove unnecessary elements 
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
 2024-01-15 22:45:02 +01:00
Artem Navoiev
191e322879
vmanomaly - models a bit pretify docs (#5618) 
* vmanomaly - models a bit pretify docs

Signed-off-by: Artem Navoiev <tenmozes@gmail.com>

* typi

Signed-off-by: Artem Navoiev <tenmozes@gmail.com>

* fix formatting

Signed-off-by: Artem Navoiev <tenmozes@gmail.com>

---------

Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
 2024-01-15 13:26:38 -08:00
Artem Navoiev
544da241e8
vmanomaly guides - fix formating, add missing piece, clarify statement, use common languagefor VM ecosystem (#5620) 
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
 2024-01-15 13:26:26 -08:00
Artem Navoiev
0c78b891b0
clarify cluster multitenacy in vmanomaly docs (#5619) 
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
 2024-01-15 12:36:28 -08:00
Artem Navoiev
f51b7fda8e
docs: fix markdown in how-to-monitor-k8s 
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
 2024-01-15 18:16:26 +01:00
Roman Khavronenko
e14e3d9c8c
docs: mention requirements for latest backups (#5614) 
Add docs to explain that `latest` backup folder can be accessed
more frequently than others. Hence, it is recommended to keep it
on storages with low access price.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
 2024-01-15 15:30:55 +01:00
Aliaksandr Valialkin
be509b3995
lib/pushmetrics: wait until the background goroutines, which push metrics, are stopped at pushmetrics.Stop() 
Previously the was a race condition when the background goroutine still could try collecting metrics
from already stopped resources after returning from pushmetrics.Stop().
Now the pushmetrics.Stop() waits until the background goroutine is stopped before returning.

This is a follow-up for https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5549
and the commit fe2d9f6646 .

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5548
 2024-01-15 13:50:36 +02:00
Aleksandr Stepanov
03a97dc678
vmagent: added hetzner sd config (#5550) 
* added hetzner robot and hetzner cloud sd configs

* remove gettoken fun and update docs

* Updated CHANGELOG and vmagent docs

* Updated CHANGELOG and vmagent docs

---------

Co-authored-by: Nikolay <nik@victoriametrics.com>
 2024-01-15 10:13:22 +01:00
Roman Khavronenko
4b8088e377
lib/storage: properly check for storage/prefetchedMetricIDs cache expiration deadline (#5607) 
Before, this cache was limited only by size.
Cache invalidation by time happens with jitter to prevent thundering herd problem.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
 2024-01-15 10:03:06 +01:00
rbizos
70cd09e736
Handling negative index in Graphite groupByNode/aliasByNode (#5581) 
Handeling the error case with -1

Signed-off-by: Raphael Bizos <r.bizos@criteo.com>
Co-authored-by: Nikolay <nik@victoriametrics.com>
 2024-01-15 09:57:15 +01:00
Aliaksandr Valialkin
a47127c1a6
app/vmalert/remotewrite: properly calculate vmalert_remotewrite_dropped_rows_total 
It was calculating the number of dropped time series instead of the number of dropped samples.

While at it, drop vmalert_remotewrite_dropped_bytes_total metric, since it was inconsistently calculated -
at one place it was calculating raw protobuf-encoded sample sizes, while at another place it was calculating
the size of snappy-compressed prompbmarshal.WriteRequest protobuf message.
Additionally, this metric has zero practical sense, so just drop it in order to reduce the level of confusion.
 2024-01-14 22:55:11 +02:00
Aliaksandr Valialkin
0597718435
lib/protoparser/datadogv2: add support for reading protobuf-encoded requests at /api/v2/series endpoint 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4451
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5094
 2024-01-14 21:09:05 +02:00
Dmytro Kozlov
828aca82e9
app/vmctl: add insecure skip verify flags for source and destination addresses for native protocol (#5606) 
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5595
 2024-01-11 14:04:32 +01:00
Zakhar Bessarab
e5a767cff8
docs: explicitly mention "delete_series" endpoint accepts any HTTP method (#5605) 
See: #5552

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
 2024-01-11 15:39:11 +04:00
hagen1778
eae585e8de
docs: make docs-sync 
Signed-off-by: hagen1778 <roman@victoriametrics.com>
 2024-01-11 11:54:40 +01:00
hagen1778
91ccea236f
app/all: follow-up after 84d710beab 
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5548
Signed-off-by: hagen1778 <roman@victoriametrics.com>
 2024-01-09 13:34:54 +01:00
Dan Dascalescu
b79d4cc988
docs: fix English in keyConcepts.md, add instant query use case (#5547) 2024-01-09 11:29:21 +01:00
Fred Navruzov
e34f77aed4
docs: vmanomaly chapter hotfixes (#5583) 
* update link to book a demo for AD & RCA

* fix invalid refs in components/model

* - fix staging -> prod links
- replace capitalized FAQ headers
- change the section order on main page
- replace :latest tag with current stable
 2024-01-09 11:04:24 +01:00
Fred Navruzov
bbea02f82b
- fix 404 in /guides page (#5582) 
- change back AD section title
 2024-01-08 11:27:16 -08:00
Artem Navoiev
fdefc8a816
docs: properly close diff 
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
 2024-01-08 20:20:09 +01:00
Artem Navoiev
095d982976
docs: vmagent specify default value for the compression level and its maximum. The question appers too often in our channel (#5575) 
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
 2024-01-08 20:14:23 +01:00
Dmytro Kozlov
105c6b2eb7
app/vmui: fix broken link for the statistic inaccuracy explanation (#5568) 2024-01-08 20:13:45 +01:00
Dan Dascalescu
33df9bee22
Link to start/end timestamp formats in url-examples (#5531) 2024-01-08 18:23:38 +04:00
hagen1778
463455665b
dashboards: update cluster dashboard 
* add panels for detailed visualization of traffic usage between vmstorage, vminsert, vmselect
components and their clients. New panels are available in the rows dedicated to specific components.

* update "Slow Queries" panel to show percentage of the slow queries to the total number of read queries
served by vmselect. The percentage value should make it more clear for users whether there is a service degradation.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
 2024-01-08 11:58:31 +01:00
Denys Holius
eb08f5c7e5
docs/anomaly-detection/guides/README.md: fixed markdown link to the vmanomaly guide (#5578) 2024-01-08 02:41:52 -08:00
Artem Navoiev
1aa39efec1
docs: vmanomaly add list of guides to guides page 
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
 2024-01-08 10:44:48 +01:00
Artem Navoiev
07e5d6f0fb
docs: change sorting of anomaly deteciton 
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
 2024-01-08 10:39:38 +01:00
Fred Navruzov
f75874f5df
docs: vmanomaly part 1 (#5558) 
* add `AD` section, fix links, release docs and changelog

* - connect sections, refactor structure

* - resolve suggestions
- add FAQ section
- fix dead links

* - fix incorrect render of tables for Writer
- comment out internal readers/writers
- fix page ordering to some extent

* - link licensing requirements from v1.5.0 to main page

---------

Co-authored-by: Artem Navoiev <tenmozes@gmail.com>
 2024-01-08 01:31:36 -08:00
Denys Holius
aecfabe318
CHANGELOG.md: fixed wrong links to vmalert-tool documentation page (#5570) 2024-01-05 07:16:46 -08:00
Artem Navoiev
47307c7a37
docs: specify right link to grafana operator docs 
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
 2023-12-24 22:51:43 +01:00
hagen1778
d0ca448093
docs: fix typo in VictoriaLogs upgrading procedure 
Signed-off-by: hagen1778 <roman@victoriametrics.com>
 2023-12-22 21:41:27 +01:00
hagen1778
35dd6e5e8e
docs: docs-sync after 52692d001a 
Signed-off-by: hagen1778 <roman@victoriametrics.com>
 2023-12-22 21:34:26 +01:00
hagen1778
95edeffbc6
docs: add link to sandbox to the Grafana section 
Signed-off-by: hagen1778 <roman@victoriametrics.com>
 2023-12-22 16:42:33 +01:00
Hui Wang
1f477aba41
vmalert: automatically add exported_ prefix for original evaluation… (#5398) 
automatically add `exported_` prefix for original evaluation result label if it's conflicted with external or reserved one,
previously it was overridden.

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5161

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
 2023-12-22 16:07:47 +01:00
Daria Karavaieva
be20501376
docs: vmanomaly guide v1.7.0 changes (#5505) 2023-12-22 16:06:41 +01:00
Dmytro Kozlov
43d7de4afe
docs: clarify information about values (#5503) 2023-12-22 16:05:16 +01:00
hagen1778
8ba483eca3
docs: fix unauthorizedAccessConfig filed names for operator 
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5520

Signed-off-by: hagen1778 <roman@victoriametrics.com>
 2023-12-22 10:50:21 +01:00
Aliaksandr Valialkin
b4ba8d0d76
lib/protoparser: add missing /datadog/ prefix to the /api/v2/series path in the description for -datadog.maxInsertRequestSize command-line flag 2023-12-21 21:04:53 +02:00
Aliaksandr Valialkin
9678235eea
docs/CHANGELOG.md: typo fix after fb90a56de2: supperted -> supported 2023-12-21 21:01:42 +02:00
Aliaksandr Valialkin
fb90a56de2
app/{vminsert,vmagent}: preliminary support for /api/v2/series ingestion from new versions of DataDog Agent 
This commit adds only JSON support - https://docs.datadoghq.com/api/latest/metrics/#submit-metrics ,
while recent versions of DataDog Agent send data to /api/v2/series in undocumented Protobuf format.
The support for this format will be added later.

Thanks to @AndrewChubatiuk for the initial implementation at https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5094

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4451
 2023-12-21 20:50:55 +02:00
Aliaksandr Valialkin
160cc9debd
app/{vmagent,vmalert}: add the ability to set OAuth2 endpoint params via the corresponding *.oauth2.endpointParams command-line flags 
This is a follow-up for 5ebd5a0d7b

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5427
 2023-12-20 21:35:28 +02:00
Aliaksandr Valialkin
67160d08a2
docs/Articles.md: add two articles about VictoriaMetrics from zetablogs.medium.com 2023-12-20 21:24:51 +02:00
Morgan
5ebd5a0d7b
Expose OAuth2 Endpoint Parameters to cli (#5427) 
The user may which to control the endpoint parameters for instance to
set the audience when requesting an access token. Exposing the
parameters as a map allows for additional use cases without requiring
modification.
 2023-12-20 20:16:43 +02:00
Aliaksandr Valialkin
7a31f8a6c9
app/vmselect/netstorage: make sure that at least a single result is collected from every storage group before deciding whether it is OK to skip results from the remaining storage nodes 2023-12-20 19:55:13 +02:00
Nikolay
7cfde237ec
lib/awsapi: properly assume role with webIdentity token (#5495) 
* lib/awsapi: properly assume role with webIdentity token
introduce new irsaRoleArn param for config. It's only needed for authorization with webIdentity token.
First credentials obtained with irsa role and the next sts assume call for an actual roleArn made with those credentials.
Common use case for it - cross AWS accounts authorization
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3822

* wip

---------

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2023-12-20 19:05:39 +02:00
Dmytro Kozlov
df012f1553
docs: remove default value from the maxConcurrentInserts flag (#5494) 2023-12-19 15:57:33 +01:00
Aliaksandr Valialkin
326a77c697
all: add -metrics.exposeMetadata command-line flag, which can be used for adding TYPE and HELP metadata for metrics exposed at /metrics page 
This may be needed for systems, which require this metadata such as Google Cloud Managed Prometheus.
See https://cloud.google.com/stackdriver/docs/managed-prometheus/troubleshooting#missing-metric-type
 2023-12-19 03:20:40 +02:00
Artem Navoiev
bc3feebf69
docs: vmalert exapand HA abbreviation 
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
 2023-12-18 15:46:40 +01:00
hagen1778
11e2d41c77
docs: mention Splitting data streams in Multi-retention guide 
Signed-off-by: hagen1778 <roman@victoriametrics.com>
 2023-12-18 13:23:16 +01:00
hagen1778
afaf7f0b74
docs: add example for Splitting data streams among multiple systems 
Signed-off-by: hagen1778 <roman@victoriametrics.com>
 2023-12-18 13:19:24 +01:00
Aliaksandr Valialkin
4b529562ce
lib/pushmetrics: add -pushmetrics.header and -pushmetrics.disableCompression command-line flags 2023-12-17 19:56:46 +02:00
Aliaksandr Valialkin
873f0deaa6
docs/CHANGELOG.md: typo fix: use proper backtick closing quote instead of single quote 2023-12-17 19:28:15 +02:00
Aliaksandr Valialkin
0379a0eb82
lib/protoparser/opentelemetry: allow ingesting metrics without resource labels 
Some clients may ingest samples via OpenTelemetry protocol without Resource labels.
Previously VictoriaMetrics was silently dropping such samples.

The commit 317834f876 added vm_protoparser_rows_dropped_total{type="opentelemetry",reason="resource_not_set"}
counter for tracking of such dropped samples. See https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5459

It is better from usability PoV to accept such samples instead of dropping them and incrementing the corresponding counter.
 2023-12-17 19:12:58 +02:00
Aliaksandr Valialkin
5ddccbc2b9
docs/CHANGELOG.md: move the description of the bugfix from 9253c24dd6 into correct place 
The description of the bugfix was incorrectly placed in already released v1.96.0,
while it should be placed in tip.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5450
 2023-12-17 18:58:42 +02:00
Roman Khavronenko
779bbc2e91
vmctl: rename vm-native-disable-retries to vm-native-disable-per-metric-migration (#5476) 
The change supposed to better reflect the meaning of this flag.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
 2023-12-15 12:36:28 +01:00
Roman Khavronenko
664fa5cb78
vmctl: retry requests that failed in the very end for vm-native (#5475) 
Before, retries happened only on writes into a network connection
between source and destination. But errors returned by server after
all the data was transmitted were logged, but not retried.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
 2023-12-15 11:43:41 +01:00
Hui Wang
9253c24dd6
vmalert: validate schema for -external.url (#5450) 
Requests with wrong or no schema in  `-external.url` could be rejected by alertmanager.
So we validate schema on start up.
 2023-12-15 11:13:56 +01:00
Aliaksandr Valialkin
0a6a2e455d
app/vmstorage: addd missing -inmemoryDataFlushInterval command-line flag 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3337
 2023-12-14 20:52:39 +02:00
Artem Navoiev
5896fb129d
add link to RELEX solutions case study to home and signle pages 
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
 2023-12-14 19:11:38 +01:00
nemobis
0b0f565c31
Add RELEX Solutions to case studies (#5469) 
Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
 2023-12-14 10:07:22 -08:00
Aliaksandr Valialkin
df88baef07
docs/CHANGELOG.md: document the bugfix at 66c76a4d4d 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5414
 2023-12-14 12:48:37 +02:00
Aliaksandr Valialkin
68be182075
app/vmauth: add ability to route requests to different backends depending on the request host 2023-12-14 00:46:36 +02:00
Aliaksandr Valialkin
6f15ca4a16
docs/Single-server-VictoriaMetrics.md: cross-link HA docs with the corresponding chapter at vmauth docs 2023-12-13 23:06:46 +02:00
Aliaksandr Valialkin
463a6e9ac6
deployment: update VictoriaMetrics images from v1.95.1 to v1.96.0 
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.96.0
 2023-12-13 01:20:51 +02:00
Aliaksandr Valialkin
304fe05650
docs/CHANGELOG.md: cut v1.96.0 release 2023-12-13 00:42:56 +02:00
Yury Molodov
1a5cdb4790
vmui: autocomplete usability improvements (#5422) 
* vmui: add show quick tip for autocomplete

* vmui: auto-completion usability improvements #5348

* vmui: add const for min symbols in autocomplete

* Use proper queries to VictoriaMetrics

* vmui: fix comments for autocomplete

* app/vmselect: run `make vmui-update`

---------

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2023-12-13 00:32:41 +02:00
Aliaksandr Valialkin
0f91f83639
app/vmselect: add support for vmstorage groups with independent -replicationFactor per group 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5197

See https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#vmstorage-groups-at-vmselect

Thanks to @zekker6 for the initial pull request at https://github.com/VictoriaMetrics/VictoriaMetrics-enterprise/pull/718
 2023-12-13 00:14:45 +02:00
hagen1778
242472086b
app/vmctl: follow-up after 6af732b6f7 
Make docs more clear about new feature.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
 2023-12-12 13:44:21 +01:00
Dmytro Kozlov
6af732b6f7
app/vmctl: enable range steps in reverse order (#5444) 
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5376
 2023-12-12 13:05:44 +01:00
hagen1778
e0fc5ef140
lib/promscrape: comsetic changes after e373bb84d5 
* fix typos in docs
* add `shard-` prefix to generated links when `-promscrape.cluster.memberURLTemplate` is enabled

Signed-off-by: hagen1778 <roman@victoriametrics.com>
 2023-12-12 11:28:18 +01:00
hagen1778
1e02efd511
docs: fix formatting after a list in CHANGELOG.md 
Signed-off-by: hagen1778 <roman@victoriametrics.com>
 2023-12-12 10:28:11 +01:00
hagen1778
39c405ed4d
app/vmctl: follow-up after 27668c9d01 
* remove duplications in error messages
* mention the change in CHANGELOG.md

27668c9d01
Signed-off-by: hagen1778 <roman@victoriametrics.com>
 2023-12-11 15:30:47 +01:00
hagen1778
e13dc04fbf
docs: mention alerts change in CHANGELOG.md 
Signed-off-by: hagen1778 <roman@victoriametrics.com>
 2023-12-11 15:23:09 +01:00
Aliaksandr Valialkin
635da5fab7
docs/CHANGELOG.md: document v1.93.9 LTS release 2023-12-11 10:39:28 +02:00
Aliaksandr Valialkin
ce8ae450fc
docs/CHANGELOG.md: document v1.87.12 2023-12-10 14:30:22 +02:00
Aliaksandr Valialkin
6d03779870
deployment/docker: update base Docker image from alpine:3.18.5 to alpine:3.19.0 
See https://www.alpinelinux.org/posts/Alpine-3.19.0-released.html
 2023-12-10 02:28:19 +02:00
Aliaksandr Valialkin
a5bc9d93cc
docs: sync -help output for VictoriaMetrics components after recent changes 2023-12-10 01:13:41 +02:00
Aliaksandr Valialkin
3d3b0e31e0
app/vmselect: add -search.maxResponseSeries command-line flag for limiting the number of time series a single response can return 
This limit can be used for preventing from high memory usage at Grafana when the response returns too many series.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5372
 2023-12-10 00:54:42 +02:00
Aliaksandr Valialkin
b1fed78e0b
app: make more clear that -tls enables https at -httpListenAddr 2023-12-10 00:25:01 +02:00
Aliaksandr Valialkin
c7504daa7a
docs: follow-up after 49552eaa15 
Link to the related issue - https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4792
Fix heading for `Modifying HTTP headers` chapter at docs/vmagent.md
 2023-12-08 23:56:56 +02:00
Aliaksandr Valialkin
042267541f
app/vmauth: add support for hot standby mode via first_available load balancing policy 
vmauth in `hot standby` mode sends requests to the first url_prefix while it is available.
If the first url_prefix becomes unavailable, then vmauth falls back to the next url_prefix.
This allows building highly available setup as described at https://docs.victoriametrics.com/vmauth.html#high-availability

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4893
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4792
 2023-12-08 23:31:07 +02:00
Roman Khavronenko
74b09ab4de
app/vmalert: sanitize label names before sending to Alertmanager (#5442) 
Before, vmalert would send notifications with labels containing characters
  not supported by Alertmanager validator, resulting into validation errors
  like `msg="Failed to validate alerts" err="invalid label set: invalid name "foo.bar"`

Signed-off-by: hagen1778 <roman@victoriametrics.com>
 2023-12-08 16:53:35 +03:00
hagen1778
6acf28715b
docs: mentioned that re-routing increases number of active time series 
Signed-off-by: hagen1778 <roman@victoriametrics.com>
 2023-12-08 13:29:23 +01:00
Aliaksandr Valialkin
adc69b872c
docs: run make docs-images-to-webp after 02a0a7f428 
This reduces added image sizes by 3x

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5437
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5370
 2023-12-07 16:40:37 +02:00
Alexander Marshalov
02a0a7f428
added field version to the response for /api/v1/status/buildinfo API for using more efficient API in Grafana for receiving label values, added additional info about setup Grafana datasource (#5370) (#5437) 2023-12-07 16:37:36 +02:00
Aliaksandr Valialkin
e373bb84d5
lib/promscrape: add -promscrape.cluster.memberURLTemplate command-line flag for creating direct links to vmagent instances at /service-discovery page 
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4018#issuecomment-1843811569
 2023-12-07 16:04:21 +02:00
Aliaksandr Valialkin
b39e9257eb
app/vmselect/prometheus: properly encode Prometheus label values at /federate endpoint 
Prometheus spec says that only \, \n and " must be escaped inside label values.
See 995743836e/content/docs/instrumenting/exposition_formats.md (L90)

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5431
 2023-12-07 15:36:01 +02:00
hagen1778
cb90d09c9d
docs: fix wrong link in Troubleshooting.md 
Signed-off-by: hagen1778 <roman@victoriametrics.com>
 2023-12-07 12:54:55 +01:00
Aliaksandr Valialkin
3a8b9cc81e
docs/vmagent.md: mention that vmagent displays up to -promscrape.maxDroppedTargets at /service-discovery page 
Suggest increasing `-promscrape.maxDroppedTargets` command-line flag value if /service-discovery page
misses some dropped targets.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5389
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4018
 2023-12-07 00:34:02 +02:00
Aliaksandr Valialkin
7cb8ed8271
lib/promscrape: show -promscrape.cluster.memberNum values for vmagent instances, which scrape the given dropped target at /service-discovery page 
The /service-discovery page contains the list of all the discovered targets
after the commit 487f6380d0 on all the vmagent instances
in cluster mode ( https://docs.victoriametrics.com/vmagent.html#scraping-big-number-of-targets ).

This commit improves debuggability of targets in cluster mode by providing a list of -promscrape.cluster.memberNum
values per each target at /service-discovery page, which has been dropped becasue of sharding,
e.g. if this target is scraped by other vmagent instances in the cluster.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5389
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4018
 2023-12-07 00:05:32 +02:00
Aliaksandr Valialkin
efbe25a678
deployment/docker: update Go builder from Go1.21.4 to Go1.21.5 
See https://github.com/golang/go/issues?q=milestone%3AGo1.21.5+label%3ACherryPickApproved
 2023-12-06 22:33:40 +02:00