github-mirrors/VictoriaMetrics

mirror of https://github.com/VictoriaMetrics/VictoriaMetrics.git synced 2024-12-01 14:47:38 +00:00

Author	SHA1	Message	Date
Aliaksandr Valialkin	1005d5a186	lib/promscrape/discovery/ec2: properly handle custom `endpoint` option in ec2_sd_configs This option was ignored since `d289ecded1` Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1287	2022-08-08 15:26:50 +03:00
Nikolay	c007b129cb	lib/promscrape: adds azure service discovery (#2743 ) * lib/promscrape: adds azure service discovery Adds azure service discovery mechanism implements authorization with oauth and msi lists virtual machines and virtual machines managed by scaleSet https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1364 * makes linter happy * Apply suggestions from code review Co-authored-by: Roman Khavronenko <roman@victoriametrics.com> * wip Co-authored-by: Roman Khavronenko <roman@victoriametrics.com> Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2022-07-13 23:45:43 +03:00
Aliaksandr Valialkin	4d03ac90fc	lib/promscrape/discovery/kubernetes: properly populate service-level labels for `role: endpointslice` targets Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2823	2022-07-07 00:36:25 +03:00
Aliaksandr Valialkin	c4cc45d7f8	lib/promscrape/discovery/kubernetes: allow attaching node-level labels to `role: endpoints` and `role: endpointlice` targets in the same way as Prometheus does See https://github.com/prometheus/prometheus/pull/10759	2022-07-07 00:36:24 +03:00
Aliaksandr Valialkin	ecc11dc32d	lib/promauth: refactor NewConfig in order to improve maintainability 1. Split NewConfig into smaller functions 2. Introduce Options struct for simplifying construction of the Config with various options This commit is based on https://github.com/VictoriaMetrics/VictoriaMetrics/pull/2684	2022-07-04 14:31:43 +03:00
Aliaksandr Valialkin	3ae6300497	lib/promauth: add ability to send additional http headers in requests to scrape targets This solves https://stackoverflow.com/questions/66032498/prometheus-scrape-metric-with-custom-header	2022-06-22 20:40:50 +03:00
Aliaksandr Valialkin	b6e3c12811	lib/promscrape/discovery/kubernetes: use unsupportedFieldError() function instead of errContext string This improves code readability and maintainability a bit, since the format string is passed as string literal into fmt.Errorf.	2022-06-07 01:24:14 +03:00
Aliaksandr Valialkin	3dbb19d624	lib/promscrape/discovery/kubernetes: follow-up after `006b8c7534` - make more clear error logs - simplify testing for newKubeConfig by passing only the path to kube_config file instead of SDConfig struct	2022-06-06 14:41:28 +03:00
Nikolay	72e43ef2fe	lib/promscrape/discovery/kubernetes: follow-up after `0b5c874911` (#2672 )	2022-06-04 01:11:23 +03:00
hadesy	28d4624f60	promscrape/discovery: support kubeconfig (#2533 )	2022-06-04 01:11:23 +03:00
Roman Khavronenko	7406665fc3	lib/promscrape/discovery/kubernetes: fixes kubernetes service discovery (#2615 ) * lib/promscrape/discovery/kubernetes: properly updates discovered scrape works previously, added or updated scrapeworks may override previuosly discovered. it happens because swosByKey may contain small subset of kubernetes objects with it's labels. It happens for objectsUpdated and objectsAdded maps, which include only changed elements * Properly calculate vm_promscrape_discovery_kubernetes_scrape_works Co-authored-by: f41gh7 <nik@victoriametrics.com> Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2022-05-21 01:17:21 +03:00
Boris Petersen	3a8b4fab97	Add ability to sign requests for all AWS services (#2604 ) This adds the ability to utilize sigv4 signing for all AWS services not just "aps". When the newly introduced property "service" is not set it will default to "aps". Signed-off-by: Boris Petersen <boris.petersen@idealo.de>	2022-05-20 14:20:00 +03:00
Aliaksandr Valialkin	0d0561ca8c	lib/awsapi: remove whitelist arg from GetFiltersQueryString(), since it may break new filters in the future Let users decide which filters to use. If users start using disallowed filters, then AWS will return an error.	2022-05-09 15:34:56 +03:00
Aliaksandr Valialkin	810dd74fb9	lib/promscrape: properly implement ScrapeConfig.clone() Previously ScrapeConfig.clone() was improperly copying promauth.Secret fields - their contents was replaced with `<secret>` value. This led to inability to use passwords and secrets in `-promscrape.config` file. The bug has been introduced in v1.77.0 in the commit `67b10896d2` Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2551	2022-05-07 00:06:19 +03:00
Aliaksandr Valialkin	9d40bb7137	lib/promscrape/discovery/ec2: add ability to filter Availability Zones in `ec2_sd_config` via `az_filters` section	2022-05-06 12:44:01 +03:00
Aliaksandr Valialkin	2ce1d09135	lib/promscrape/discovery/ec2: properly pass filters to DescribeAvailabilityZones API call Previously filters wheren't passed to this call after the commit `0e09fdb8b0` Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1626	2022-05-05 11:01:17 +03:00
Aliaksandr Valialkin	873f55bac5	lib/awsapi: pass `filtersQueryString` arg to GetEC2APIResponse() function, so the caller could decide whether to use the filters during the AWS API query The filters shouldn't be passed to DescribeAvailabilityZones API call. See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1626 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1287 Related commits: `0e09fdb8b0` `d289ecded1`	2022-05-05 10:29:47 +03:00
Nikolay	7e58cba6cf	{lib/promscrape,app/vmagent}: adds sigv4 support for vmagent remoteWrite (#2458 ) * {lib/promscrape,app/vmagent}: adds sigv4 support for vmagent remoteWrite moves aws related code into separate lib from lib/promscrape it allows to write data from vmagent to the AWS managed prometheus (cortex) https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1287 * Apply suggestions from code review * wip Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2022-05-04 20:28:37 +03:00
Aliaksandr Valialkin	aa82987d70	lib/promscrape/discovery/kubernetes: do not drop pod meta-labels even if the corresponding node objects are missing This reflects the logic used in Prometheus. See https://github.com/prometheus/prometheus/pull/10080	2022-04-26 15:27:42 +03:00
Aliaksandr Valialkin	c2b13e6a04	lib/promscrape/discovery/kubernetes: limit the minimum sleep time between updating dependent ScrapeWork objects Previously the sleep time could be dropped to nanoseconds, which could result in CPU time waste	2022-04-22 23:15:34 +03:00
Aliaksandr Valialkin	a89e31b304	lib/promscrape/discovery/kubernetes: allow attaching node-level labels and annotations to discovered pod targets in the same way as Prometheus 2.35 does See https://github.com/prometheus/prometheus/issues/9510 and https://github.com/prometheus/prometheus/pull/10080	2022-04-22 20:15:34 +03:00
Aliaksandr Valialkin	cc6eae6992	lib/promscrape/discovery/kubernetes: improve the performance of urlWatcher.reloadObjects() on multi-CPU systems Parallelize the generation of ScrapeWork objects there. Previously they were generated in a single goroutine.	2022-04-22 13:23:39 +03:00
Aliaksandr Valialkin	fea9d1e6ee	lib/promscrape/discovery/kubernetes: properly update endpoints and endpointslice objects when the related pod or service objects are updated Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1240 This is a follow-up for `2341bd48d7`	2022-04-21 13:06:49 +03:00
Aliaksandr Valialkin	e9f08b1e6a	lib/promscrape/discovery/kubernetes: do not pre-allocate memory for ScrapeWork objects There is high chance that ScrapeWork objects won't be generated because of relabeling	2022-04-20 16:42:41 +03:00
Aliaksandr Valialkin	909a3ee0e4	lib/promscrape: follow-up after `91e290a8ff`	2022-04-20 16:12:26 +03:00
Nikolay	429848a67d	lib/promscrape: reduce latency for k8s GetLabels (#2454 ) replaces internStringMap with sync.Map - it greatly reduces lock contention concurently reload scrape work for api watcher - each object labels added by dedicated CPU changes can be tested with following script https://gist.github.com/f41gh7/6f8f8d8719786aff1f18a85c23aebf70	2022-04-20 16:12:25 +03:00
Aliaksandr Valialkin	d0bac8e224	all: typo fix: Kuberntes -> Kubernetes	2022-04-20 10:51:41 +03:00
Roman Khavronenko	5a4b16794d	Consul SD - update services on the watcher's start (#2202 ) * lib/discovery/consul: update services on the watcher's start Previously, watcher's start was only initing goroutines for discovery but not waiting for the first iteration to end. It means first Consul discovery wasn't returning discovered targets until the next iteration. The change makes the watcher's start blocking until we get first discovery iteration done and all registries updated. Signed-off-by: hagen1778 <roman@victoriametrics.com> * vmalert: remove workarounds for consul SD Now when consul SD lib properly updates services on the first start, we don't need workarounds in vmalert. Signed-off-by: hagen1778 <roman@victoriametrics.com> * lib/discovery/consul: update after review Signed-off-by: hagen1778 <roman@victoriametrics.com> * wip Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2022-02-21 15:33:33 +02:00
Nikolay	c11d0949c8	fixes all_tenants query option usage for openstack service discovery (#2184 ) explicit use configuration parametr instead of conditional add https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2182	2022-02-14 13:13:03 +02:00
artifactori	943fc056ca	Show gce sdconfig zone on vmagent:8429/config (#2178 ) * vmagent: add test for marshalling gce sdconfig with ZoneYAML * vmagent: implement MarshalYAML for ZoneYAML on gce sdconfig	2022-02-12 00:48:38 +02:00
Aliaksandr Valialkin	727b29d4a3	lib/promscrape/discovery/kubernetes: add `__meta_kubernetes_endpointslice_{label,annotation}*` labels to be consistent with other `role` values for Kubernetes service discovery	2022-02-11 14:56:10 +02:00
Nikolay	265938a385	fixes service discovery for kubernetes (#2173 ) * fixes service discovery for kubernetes now it must take in account all pods that belong to the discovered endpoint and endpointslice adds simple test for endpoints https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2134 * wip * docs/CHANGELOG.md: document the change Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2022-02-11 13:35:34 +02:00
Aliaksandr Valialkin	dd91759f1f	lib/promscrape/discovery/kubernetes: add `__meta_kubernetes_node_provider_id` label for discovered Kubernetes nodes in the same way as Prometheus does See https://github.com/prometheus/prometheus/pull/9603	2022-01-13 23:17:24 +02:00
Aliaksandr Valialkin	bc18368c15	lib/promscrape/discovery/kubernetes: add the ability to limit service discovery to the current namespace See https://github.com/prometheus/prometheus/issues/9782 and https://github.com/prometheus/prometheus/pull/9881	2022-01-13 22:44:59 +02:00
Aliaksandr Valialkin	de8299f465	lib/promscrape/discovery/dockerswarm: follow up after `68a117a25a` - Document the bugfix at docs/CHANGELOG.md - Set __address__ field after copying commonLabels to the resulting map of discovered labels. This makes sure that the correct __address__ label is used.	2022-01-11 09:22:03 +02:00
Alexander Shtuchkin	45a92e6ce1	Fix for #2038 : Make correct __address__ value for dockerswarm promscrape (#2041 )	2022-01-11 09:22:02 +02:00
Aliaksandr Valialkin	847004fa77	app/{vminsert,vmagent}: hide passwords and auth tokens by default at `/config` page Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1764	2021-11-05 14:42:13 +02:00
Aliaksandr Valialkin	ad445a06cd	lib/promscrape: properly show `proxy_url` option value at `/config` page Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1755	2021-10-26 21:24:22 +03:00
Aliaksandr Valialkin	ea69eef375	lib/promscrape/discovery/kubernetes: log a warning if `role: endpoints` discovers more than 1000 targets per a single endpoint In this case `role: endpointslice` must be used instead. See the following references: * https://kubernetes.io/docs/reference/labels-annotations-taints/#endpoints-kubernetes-io-over-capacity * https://github.com/kubernetes/kubernetes/pull/99975 * https://github.com/prometheus/prometheus/issues/7572#issuecomment-934779398	2021-10-19 13:22:28 +03:00
Aliaksandr Valialkin	3e9beb0f8d	lib/promscrape/discovery/kubernetes: rename endpointslices.go -> endpointslice.go in order to be consistent with EndpointSlice struct name This is a follow-up for `31b42b30b6`	2021-10-15 12:27:31 +03:00
Nikolay	9be5689b3f	changes auth validation for openstack (#1663 ) * changes auth validation for openstack must fix https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1655 * Apply suggestions from code review Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2021-09-29 00:33:38 +03:00
Nikolay	1ab2f844a2	makes filters optional for ec2 api requests (#1627 ) filters can be applied only for DescribeInstances requests, like prometheus does. related issue https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1626	2021-09-17 18:12:25 +03:00
Aliaksandr Valialkin	b684624f67	lib/promscrape/discovery/docker: support host networking mode See https://github.com/prometheus/prometheus/issues/9116	2021-09-13 13:30:55 +03:00
Aliaksandr Valialkin	6ed9f10da5	lib/promscrape/discovery/kubernetes: properly use https scheme for wildcard TLS certificates in ingress target discovery See https://github.com/prometheus/prometheus/issues/8902	2021-09-13 13:04:43 +03:00
Aliaksandr Valialkin	146c14d879	lib/promscrape/discovery/kubernetes: return back support `role: endpointslices`, since it is used by VictoriaMetrics operator This is a follow up commit after `31b42b30b6`	2021-08-29 12:37:36 +03:00
Aliaksandr Valialkin	ca61d7c82b	lib/promscrape/discovery/kubernetes: rename `role: endpointslices` to `role: endpointslice` to be consistent with Prometheus See `2ec6c7dbb8/discovery/kubernetes/kubernetes.go (L99)`	2021-08-29 11:23:59 +03:00
Aliaksandr Valialkin	327034b54f	lib/promscrape/discovery/kubernetes: use v1 API instead of v1beta1 API for `role: ingress` and `role: endpointslices` This should fix service discovery for these roles in Kubernetes v1.22 and newer versions. See https://kubernetes.io/docs/reference/using-api/deprecation-guide/#ingress-v122 The corresponding change in Prometheus - https://github.com/prometheus/prometheus/pull/9205	2021-08-29 11:23:58 +03:00
Aliaksandr Valialkin	4a2d7aec7f	lib/promscrape: expose promscrape_discovery_http_errors_total metric for tracking errors per each http_sd config	2021-08-25 13:05:29 +03:00
Aliaksandr Valialkin	77bb9e1656	lib/promscrape/discovery/gce: add __meta_gce_interface_ipv4_<name> labels as in Prometheus 2.29 See https://github.com/prometheus/prometheus/pull/8978	2021-08-03 15:51:45 +03:00
Aliaksandr Valialkin	336a2aa2e0	lib/promscrape/discovery/ec2: add `__meta_ec2_availability_zone_id` label as Prometheus 2.29 does	2021-08-03 13:28:13 +03:00

1 2 3 4

186 commits