github-mirrors/VictoriaMetrics
1
0
Fork 0
mirror of https://github.com/VictoriaMetrics/VictoriaMetrics.git synced 2024-12-11 14:53:49 +00:00
Code Issues Projects Releases Packages Wiki Activity
4673 commits 131 branches 758 tags 363 MiB
Commit graph

465 commits

Author SHA1 Message Date
Zakhar Bessarab
e0ba243a22
lib/promscrape/discovery/azure: remove API server from URL returned by azure (#3403) 
* lib/promscrape/discovery/azure: remove API server from URL returned by azure

* lib/promscrape/discovery/azure: validate nextLink contains same URL as apiServer
 2022-12-10 21:34:43 -08:00
Aliaksandr Valialkin
4df78e8814
lib/promscrape/discovery/gce: do not pass filter arg when discovering zones 
The filter arg isn't supported by zones API in GCE.

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3202
 2022-12-10 21:14:42 -08:00
Aliaksandr Valialkin
4d8aa13ec1
lib/promscrape/discovery/kubernetes: add missing __meta_kubernetes_ingress_class_name label for role: ingress 
See 7e65ad3e43
and 7e1111ff14
 2022-08-08 15:51:31 +03:00
Aliaksandr Valialkin
ef3fab515d
lib/promscrape/discovery/consul: allow stale responses from Consul service discovery by default 
This aligns with Prometheus behaviour.

See `allow_stale` option description at https://prometheus.io/docs/prometheus/latest/configuration/configuration/#consul_sd_config
 2022-08-08 15:50:05 +03:00
Aliaksandr Valialkin
3764ba98a5
lib/promscrape/discovery/dockerswarm: properly set __meta_dockerswarm_container_label_* labels instead of __meta_dockerswarm_task_label_* labels 
See https://github.com/prometheus/prometheus/issues/9187
 2022-08-08 15:46:59 +03:00
Aliaksandr Valialkin
ce00b3862f
lib/promscrape: reload all the scrape configs when the global section is changed inside -promscrape.config 
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2884
 2022-08-08 15:33:47 +03:00
Aliaksandr Valialkin
d5bb9ca25a
lib/promscrape: set up=0 for partially failed scrape in stream parsing mode 
This behaviour aligns with Prometheus behavior
 2022-08-08 15:30:52 +03:00
Aliaksandr Valialkin
1005d5a186
lib/promscrape/discovery/ec2: properly handle custom endpoint option in ec2_sd_configs 
This option was ignored since d289ecded1

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1287
 2022-08-08 15:26:50 +03:00
Nikolay
c007b129cb
lib/promscrape: adds azure service discovery (#2743) 
* lib/promscrape: adds azure service discovery
Adds azure service discovery mechanism
implements authorization with oauth and msi
lists virtual machines and virtual machines managed by scaleSet

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1364

* makes linter happy

* Apply suggestions from code review

Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>

* wip

Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2022-07-13 23:45:43 +03:00
Aliaksandr Valialkin
5794886662
lib/promscrape: properly set Host header when sending requests via http proxy 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2794
 2022-07-07 02:28:47 +03:00
Aliaksandr Valialkin
4d03ac90fc
lib/promscrape/discovery/kubernetes: properly populate service-level labels for role: endpointslice targets 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2823
 2022-07-07 00:36:25 +03:00
Aliaksandr Valialkin
c4cc45d7f8
lib/promscrape/discovery/kubernetes: allow attaching node-level labels to role: endpoints and role: endpointlice targets in the same way as Prometheus does 
See https://github.com/prometheus/prometheus/pull/10759
 2022-07-07 00:36:24 +03:00
Aliaksandr Valialkin
f9303e494c
lib/promscrape: fix a test after c66f676f3b 2022-07-06 13:25:17 +03:00
Aliaksandr Valialkin
498c6d6e72
lib/promscrape: push scrape_samples_limit metric to remote storage if sample_limit option is set in scrape_config for this target 
See https://github.com/VictoriaMetrics/operator/issues/497
 2022-07-06 12:46:23 +03:00
Aliaksandr Valialkin
ecc11dc32d
lib/promauth: refactor NewConfig in order to improve maintainability 
1. Split NewConfig into smaller functions
2. Introduce Options struct for simplifying construction of the Config with various options

This commit is based on https://github.com/VictoriaMetrics/VictoriaMetrics/pull/2684
 2022-07-04 14:31:43 +03:00
ttyv
00956e585d
lib/promscrape: fix vmagent tickerCh reload behaviour (#2786) 
Co-authored-by: Dmitriy <dab@ttyv.ru>
 2022-06-30 13:52:44 +03:00
Aliaksandr Valialkin
71b0dfdefa
lib/promscrape: always send stale markers with the real scrape timestamp 
This guarantees that query won't return data just after the series is disappeared.
 2022-06-23 11:49:13 +03:00
Aliaksandr Valialkin
3ae6300497
lib/promauth: add ability to send additional http headers in requests to scrape targets 
This solves https://stackoverflow.com/questions/66032498/prometheus-scrape-metric-with-custom-header
 2022-06-22 20:40:50 +03:00
Aliaksandr Valialkin
fe2269b999
all: remove explicit "xxhash" name when importing github.com/cespare/xxhash/v2 package 
This package already has the same name, so there is no need in explicit name
 2022-06-21 20:24:28 +03:00
Roman Khavronenko
2b5e1dee91
vmagent: update SD duration histogram metric if SD is active (#2677) 
The change updates histogram for registering SD update duration
only SD is considered as `active`. SD is active if at least
one scraper for this SD has started.

This change supposed to reduce metrics cardinality produced
by duration histogram which gets updated even if SD isn't configured.

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2671

Signed-off-by: hagen1778 <roman@victoriametrics.com>
 2022-06-07 15:53:06 +03:00
Aliaksandr Valialkin
b6e3c12811
lib/promscrape/discovery/kubernetes: use unsupportedFieldError() function instead of errContext string 
This improves code readability and maintainability a bit, since the format string
is passed as string literal into fmt.Errorf.
 2022-06-07 01:24:14 +03:00
Aliaksandr Valialkin
68b6ddfb14
all: follow-up after 8edb390e21 
- Remove unused js bloatware from /targets page. This strips down binary size by more than 100Kb
- Add /service-discovery page for API compatibility with Prometheus
- Properly load bootstrap.min.css from /prometheus/targets
- Serve static contents for /targets page from app/vminsert instead of app/vmselect, because /targets page is served from there
 2022-06-07 01:05:53 +03:00
Aliaksandr Valialkin
3dbb19d624
lib/promscrape/discovery/kubernetes: follow-up after 006b8c7534 
- make more clear error logs
- simplify testing for newKubeConfig by passing only the path to kube_config file instead of SDConfig struct
 2022-06-06 14:41:28 +03:00
Aliaksandr Valialkin
6c2fb9d8c4
lib/promscrape: add -promscrape.cluster.name command-line flag 
This flag is used for proper data de-duplication when the same target is scraped
from multiple vmagent clusters.

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2679
 2022-06-04 01:11:23 +03:00
Dmytro Kozlov
ce8aade80e
lib/promscrape: adds service discovery visualization for /targets page(#2675) 
* lib/promscrape: updated template

* lib/promscrape: fixed click on unhealthy and all btns

* app/vmselect: jquery scripts into static folder

Co-authored-by: f41gh7 <nik@victoriametrics.com>
 2022-06-04 01:11:23 +03:00
Nikolay
72e43ef2fe
lib/promscrape/discovery/kubernetes: follow-up after 0b5c874911 (#2672) 2022-06-04 01:11:23 +03:00
hadesy
28d4624f60
promscrape/discovery: support kubeconfig (#2533) 2022-06-04 01:11:23 +03:00
Aliaksandr Valialkin
afced37c0b
all: add initial support for query tracing 
See https://docs.victoriametrics.com/Single-server-VictoriaMetrics.html#query-tracing

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1403
 2022-06-01 02:31:44 +03:00
Aliaksandr Valialkin
386f6110ec
lib/promscrape: use strconv.Atoi instead of strconv.ParseInt for parsing -promscrape.cluster.memberNum 
In this case there is no need in converting int64 to int
 2022-06-01 01:43:25 +03:00
Dmytro Kozlov
cd1fa2e4cd
issue-2594: use embedded for static files (#2650) 
embed static js and css files from CDN into vmalert, vmagent and vmsingle binaries.

Co-authored-by: f41gh7 <nik@victoriametrics.com>

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2594
 2022-05-31 12:42:48 +03:00
Dmytro Kozlov
6add79143b
removed redundant return (fixed linter) (#2647) 
* removed redundant return

* updated lint package version
 2022-05-30 12:25:58 +03:00
Aliaksandr Valialkin
f149d56ac2
lib/promscrape: add -promscrape.suppressScrapeErrorsDelay command-line flag 
This flag can be used for reducing the amounts of logs when scraping unreliable scrape targets.

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2575

The patch is based on https://github.com/VictoriaMetrics/VictoriaMetrics/pull/2576 .
Thanks to @jelmd .
 2022-05-25 23:00:30 +03:00
Roman Khavronenko
7406665fc3
lib/promscrape/discovery/kubernetes: fixes kubernetes service discovery (#2615) 
* lib/promscrape/discovery/kubernetes: properly updates discovered scrape works
previously, added or updated scrapeworks may override previuosly
discovered.
it happens because swosByKey may contain small subset of kubernetes
objects with it's labels.
It happens for objectsUpdated and objectsAdded maps, which include only changed elements

* Properly calculate vm_promscrape_discovery_kubernetes_scrape_works

Co-authored-by: f41gh7 <nik@victoriametrics.com>
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2022-05-21 01:17:21 +03:00
Boris Petersen
3a8b4fab97
Add ability to sign requests for all AWS services (#2604) 
This adds the ability to utilize sigv4 signing for all AWS services not
just "aps". When the newly introduced property "service" is not set it
will default to "aps".

Signed-off-by: Boris Petersen <boris.petersen@idealo.de>
 2022-05-20 14:20:00 +03:00
Aliaksandr Valialkin
116c0b8f2e
docs/vmagent.md: typo fix in the description for -promscrape.cluster.replicationFactor command-line flag 2022-05-12 18:51:20 +03:00
Aliaksandr Valialkin
0d0561ca8c
lib/awsapi: remove whitelist arg from GetFiltersQueryString(), since it may break new filters in the future 
Let users decide which filters to use. If users start using disallowed filters, then AWS will return an error.
 2022-05-09 15:34:56 +03:00
Aliaksandr Valialkin
810dd74fb9
lib/promscrape: properly implement ScrapeConfig.clone() 
Previously ScrapeConfig.clone() was improperly copying promauth.Secret fields -
their contents was replaced with `<secret>` value.

This led to inability to use passwords and secrets in `-promscrape.config` file.
The bug has been introduced in v1.77.0 in the commit 67b10896d2

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2551
 2022-05-07 00:06:19 +03:00
Aliaksandr Valialkin
af0da45d3e
lib/promscrape: rename promscrape_stale_samples_created_total metric to vm_promscrape_stale_samples_created_total, so its name is consistent with the rest of vm_promscrape_ metrics 2022-05-06 15:33:43 +03:00
Aliaksandr Valialkin
9d40bb7137
lib/promscrape/discovery/ec2: add ability to filter Availability Zones in ec2_sd_config via az_filters section 2022-05-06 12:44:01 +03:00
Aliaksandr Valialkin
2ce1d09135
lib/promscrape/discovery/ec2: properly pass filters to DescribeAvailabilityZones API call 
Previously filters wheren't passed to this call after the commit 0e09fdb8b0

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1626
 2022-05-05 11:01:17 +03:00
Aliaksandr Valialkin
873f55bac5
lib/awsapi: pass filtersQueryString arg to GetEC2APIResponse() function, so the caller could decide whether to use the filters during the AWS API query 
The filters shouldn't be passed to DescribeAvailabilityZones API call.
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1626
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1287

Related commits:
0e09fdb8b0
d289ecded1
 2022-05-05 10:29:47 +03:00
Nikolay
7e58cba6cf
{lib/promscrape,app/vmagent}: adds sigv4 support for vmagent remoteWrite (#2458) 
* {lib/promscrape,app/vmagent}: adds sigv4 support for vmagent remoteWrite
moves aws related code into separate lib from lib/promscrape
it allows to write data from vmagent to the AWS managed prometheus (cortex)

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1287

* Apply suggestions from code review

* wip

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2022-05-04 20:28:37 +03:00
Nikolay
51a77759c1
lib/promscrape: adds correct http status codes for redirect (#2530) 
standard http client accepts multiple http status codes as redirect
it should fix issue with incorrect redirects
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2482
 2022-05-03 14:01:57 +03:00
Dima Lazerka
837e440865
Fix targetstatus qtpl paths (#2517) 
Ran `make quicktemplate-gen` from the root directory
 2022-04-29 11:18:14 +03:00
Aliaksandr Valialkin
aa82987d70
lib/promscrape/discovery/kubernetes: do not drop pod meta-labels even if the corresponding node objects are missing 
This reflects the logic used in Prometheus.

See https://github.com/prometheus/prometheus/pull/10080
 2022-04-26 15:27:42 +03:00
Aliaksandr Valialkin
c2b13e6a04
lib/promscrape/discovery/kubernetes: limit the minimum sleep time between updating dependent ScrapeWork objects 
Previously the sleep time could be dropped to nanoseconds, which could result in CPU time waste
 2022-04-22 23:15:34 +03:00
Aliaksandr Valialkin
a89e31b304
lib/promscrape/discovery/kubernetes: allow attaching node-level labels and annotations to discovered pod targets in the same way as Prometheus 2.35 does 
See https://github.com/prometheus/prometheus/issues/9510
and https://github.com/prometheus/prometheus/pull/10080
 2022-04-22 20:15:34 +03:00
Aliaksandr Valialkin
cc6eae6992
lib/promscrape/discovery/kubernetes: improve the performance of urlWatcher.reloadObjects() on multi-CPU systems 
Parallelize the generation of ScrapeWork objects there. Previously they were generated in a single goroutine.
 2022-04-22 13:23:39 +03:00
Aliaksandr Valialkin
60f74dab56
lib/promscrape: prevent from memory leaks on -promscrape.config reload when only a small part of scrape jobs is updated 
This is a follow-up after 26b78ad707
 2022-04-22 13:23:37 +03:00
Aliaksandr Valialkin
fea9d1e6ee
lib/promscrape/discovery/kubernetes: properly update endpoints and endpointslice objects when the related pod or service objects are updated 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1240

This is a follow-up for 2341bd48d7
 2022-04-21 13:06:49 +03:00