github-mirrors/VictoriaMetrics

mirror of https://github.com/VictoriaMetrics/VictoriaMetrics.git synced 2024-11-21 14:44:00 +00:00

Author	SHA1	Message	Date
Aliaksandr Valialkin	f9303e494c	lib/promscrape: fix a test after `c66f676f3b`	2022-07-06 13:25:17 +03:00
Aliaksandr Valialkin	498c6d6e72	lib/promscrape: push `scrape_samples_limit` metric to remote storage if `sample_limit` option is set in `scrape_config` for this target See https://github.com/VictoriaMetrics/operator/issues/497	2022-07-06 12:46:23 +03:00
Aliaksandr Valialkin	ecc11dc32d	lib/promauth: refactor NewConfig in order to improve maintainability 1. Split NewConfig into smaller functions 2. Introduce Options struct for simplifying construction of the Config with various options This commit is based on https://github.com/VictoriaMetrics/VictoriaMetrics/pull/2684	2022-07-04 14:31:43 +03:00
ttyv	00956e585d	lib/promscrape: fix vmagent tickerCh reload behaviour (#2786 ) Co-authored-by: Dmitriy <dab@ttyv.ru>	2022-06-30 13:52:44 +03:00
Aliaksandr Valialkin	71b0dfdefa	lib/promscrape: always send stale markers with the real scrape timestamp This guarantees that query won't return data just after the series is disappeared.	2022-06-23 11:49:13 +03:00
Aliaksandr Valialkin	3ae6300497	lib/promauth: add ability to send additional http headers in requests to scrape targets This solves https://stackoverflow.com/questions/66032498/prometheus-scrape-metric-with-custom-header	2022-06-22 20:40:50 +03:00
Aliaksandr Valialkin	fe2269b999	all: remove explicit "xxhash" name when importing github.com/cespare/xxhash/v2 package This package already has the same name, so there is no need in explicit name	2022-06-21 20:24:28 +03:00
Roman Khavronenko	2b5e1dee91	vmagent: update SD duration histogram metric if SD is active (#2677 ) The change updates histogram for registering SD update duration only SD is considered as `active`. SD is active if at least one scraper for this SD has started. This change supposed to reduce metrics cardinality produced by duration histogram which gets updated even if SD isn't configured. See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2671 Signed-off-by: hagen1778 <roman@victoriametrics.com>	2022-06-07 15:53:06 +03:00
Aliaksandr Valialkin	b6e3c12811	lib/promscrape/discovery/kubernetes: use unsupportedFieldError() function instead of errContext string This improves code readability and maintainability a bit, since the format string is passed as string literal into fmt.Errorf.	2022-06-07 01:24:14 +03:00
Aliaksandr Valialkin	68b6ddfb14	all: follow-up after `8edb390e21` - Remove unused js bloatware from /targets page. This strips down binary size by more than 100Kb - Add /service-discovery page for API compatibility with Prometheus - Properly load bootstrap.min.css from /prometheus/targets - Serve static contents for /targets page from app/vminsert instead of app/vmselect, because /targets page is served from there	2022-06-07 01:05:53 +03:00
Aliaksandr Valialkin	3dbb19d624	lib/promscrape/discovery/kubernetes: follow-up after `006b8c7534` - make more clear error logs - simplify testing for newKubeConfig by passing only the path to kube_config file instead of SDConfig struct	2022-06-06 14:41:28 +03:00
Aliaksandr Valialkin	6c2fb9d8c4	lib/promscrape: add `-promscrape.cluster.name` command-line flag This flag is used for proper data de-duplication when the same target is scraped from multiple vmagent clusters. See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2679	2022-06-04 01:11:23 +03:00
Dmytro Kozlov	ce8aade80e	lib/promscrape: adds service discovery visualization for /targets page(#2675 ) * lib/promscrape: updated template * lib/promscrape: fixed click on unhealthy and all btns * app/vmselect: jquery scripts into static folder Co-authored-by: f41gh7 <nik@victoriametrics.com>	2022-06-04 01:11:23 +03:00
Nikolay	72e43ef2fe	lib/promscrape/discovery/kubernetes: follow-up after `0b5c874911` (#2672 )	2022-06-04 01:11:23 +03:00
hadesy	28d4624f60	promscrape/discovery: support kubeconfig (#2533 )	2022-06-04 01:11:23 +03:00
Aliaksandr Valialkin	afced37c0b	all: add initial support for query tracing See https://docs.victoriametrics.com/Single-server-VictoriaMetrics.html#query-tracing Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1403	2022-06-01 02:31:44 +03:00
Aliaksandr Valialkin	386f6110ec	lib/promscrape: use strconv.Atoi instead of strconv.ParseInt for parsing -promscrape.cluster.memberNum In this case there is no need in converting int64 to int	2022-06-01 01:43:25 +03:00
Dmytro Kozlov	cd1fa2e4cd	issue-2594: use embedded for static files (#2650 ) embed static js and css files from CDN into vmalert, vmagent and vmsingle binaries. Co-authored-by: f41gh7 <nik@victoriametrics.com> https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2594	2022-05-31 12:42:48 +03:00
Dmytro Kozlov	6add79143b	removed redundant return (fixed linter) (#2647 ) * removed redundant return * updated lint package version	2022-05-30 12:25:58 +03:00
Aliaksandr Valialkin	f149d56ac2	lib/promscrape: add -promscrape.suppressScrapeErrorsDelay command-line flag This flag can be used for reducing the amounts of logs when scraping unreliable scrape targets. See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2575 The patch is based on https://github.com/VictoriaMetrics/VictoriaMetrics/pull/2576 . Thanks to @jelmd .	2022-05-25 23:00:30 +03:00
Roman Khavronenko	7406665fc3	lib/promscrape/discovery/kubernetes: fixes kubernetes service discovery (#2615 ) * lib/promscrape/discovery/kubernetes: properly updates discovered scrape works previously, added or updated scrapeworks may override previuosly discovered. it happens because swosByKey may contain small subset of kubernetes objects with it's labels. It happens for objectsUpdated and objectsAdded maps, which include only changed elements * Properly calculate vm_promscrape_discovery_kubernetes_scrape_works Co-authored-by: f41gh7 <nik@victoriametrics.com> Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2022-05-21 01:17:21 +03:00
Boris Petersen	3a8b4fab97	Add ability to sign requests for all AWS services (#2604 ) This adds the ability to utilize sigv4 signing for all AWS services not just "aps". When the newly introduced property "service" is not set it will default to "aps". Signed-off-by: Boris Petersen <boris.petersen@idealo.de>	2022-05-20 14:20:00 +03:00
Aliaksandr Valialkin	116c0b8f2e	docs/vmagent.md: typo fix in the description for `-promscrape.cluster.replicationFactor` command-line flag	2022-05-12 18:51:20 +03:00
Aliaksandr Valialkin	0d0561ca8c	lib/awsapi: remove whitelist arg from GetFiltersQueryString(), since it may break new filters in the future Let users decide which filters to use. If users start using disallowed filters, then AWS will return an error.	2022-05-09 15:34:56 +03:00
Aliaksandr Valialkin	810dd74fb9	lib/promscrape: properly implement ScrapeConfig.clone() Previously ScrapeConfig.clone() was improperly copying promauth.Secret fields - their contents was replaced with `<secret>` value. This led to inability to use passwords and secrets in `-promscrape.config` file. The bug has been introduced in v1.77.0 in the commit `67b10896d2` Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2551	2022-05-07 00:06:19 +03:00
Aliaksandr Valialkin	af0da45d3e	lib/promscrape: rename `promscrape_stale_samples_created_total` metric to `vm_promscrape_stale_samples_created_total`, so its name is consistent with the rest of `vm_promscrape_` metrics	2022-05-06 15:33:43 +03:00
Aliaksandr Valialkin	9d40bb7137	lib/promscrape/discovery/ec2: add ability to filter Availability Zones in `ec2_sd_config` via `az_filters` section	2022-05-06 12:44:01 +03:00
Aliaksandr Valialkin	2ce1d09135	lib/promscrape/discovery/ec2: properly pass filters to DescribeAvailabilityZones API call Previously filters wheren't passed to this call after the commit `0e09fdb8b0` Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1626	2022-05-05 11:01:17 +03:00
Aliaksandr Valialkin	873f55bac5	lib/awsapi: pass `filtersQueryString` arg to GetEC2APIResponse() function, so the caller could decide whether to use the filters during the AWS API query The filters shouldn't be passed to DescribeAvailabilityZones API call. See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1626 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1287 Related commits: `0e09fdb8b0` `d289ecded1`	2022-05-05 10:29:47 +03:00
Nikolay	7e58cba6cf	{lib/promscrape,app/vmagent}: adds sigv4 support for vmagent remoteWrite (#2458 ) * {lib/promscrape,app/vmagent}: adds sigv4 support for vmagent remoteWrite moves aws related code into separate lib from lib/promscrape it allows to write data from vmagent to the AWS managed prometheus (cortex) https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1287 * Apply suggestions from code review * wip Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2022-05-04 20:28:37 +03:00
Nikolay	51a77759c1	lib/promscrape: adds correct http status codes for redirect (#2530 ) standard http client accepts multiple http status codes as redirect it should fix issue with incorrect redirects https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2482	2022-05-03 14:01:57 +03:00
Dima Lazerka	837e440865	Fix targetstatus qtpl paths (#2517 ) Ran `make quicktemplate-gen` from the root directory	2022-04-29 11:18:14 +03:00
Aliaksandr Valialkin	aa82987d70	lib/promscrape/discovery/kubernetes: do not drop pod meta-labels even if the corresponding node objects are missing This reflects the logic used in Prometheus. See https://github.com/prometheus/prometheus/pull/10080	2022-04-26 15:27:42 +03:00
Aliaksandr Valialkin	c2b13e6a04	lib/promscrape/discovery/kubernetes: limit the minimum sleep time between updating dependent ScrapeWork objects Previously the sleep time could be dropped to nanoseconds, which could result in CPU time waste	2022-04-22 23:15:34 +03:00
Aliaksandr Valialkin	a89e31b304	lib/promscrape/discovery/kubernetes: allow attaching node-level labels and annotations to discovered pod targets in the same way as Prometheus 2.35 does See https://github.com/prometheus/prometheus/issues/9510 and https://github.com/prometheus/prometheus/pull/10080	2022-04-22 20:15:34 +03:00
Aliaksandr Valialkin	cc6eae6992	lib/promscrape/discovery/kubernetes: improve the performance of urlWatcher.reloadObjects() on multi-CPU systems Parallelize the generation of ScrapeWork objects there. Previously they were generated in a single goroutine.	2022-04-22 13:23:39 +03:00
Aliaksandr Valialkin	60f74dab56	lib/promscrape: prevent from memory leaks on -promscrape.config reload when only a small part of scrape jobs is updated This is a follow-up after `26b78ad707`	2022-04-22 13:23:37 +03:00
Aliaksandr Valialkin	fea9d1e6ee	lib/promscrape/discovery/kubernetes: properly update endpoints and endpointslice objects when the related pod or service objects are updated Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1240 This is a follow-up for `2341bd48d7`	2022-04-21 13:06:49 +03:00
Aliaksandr Valialkin	1e0517b9cd	lib/promscrape: remove possible data race when cleaning up internStringsMap	2022-04-20 18:41:23 +03:00
Aliaksandr Valialkin	1ae16bf671	lib/promscrape: zero out labels after duplicate removal inside mergeLabels()	2022-04-20 18:35:27 +03:00
Aliaksandr Valialkin	e9f08b1e6a	lib/promscrape/discovery/kubernetes: do not pre-allocate memory for ScrapeWork objects There is high chance that ScrapeWork objects won't be generated because of relabeling	2022-04-20 16:42:41 +03:00
Aliaksandr Valialkin	909a3ee0e4	lib/promscrape: follow-up after `91e290a8ff`	2022-04-20 16:12:26 +03:00
Nikolay	429848a67d	lib/promscrape: reduce latency for k8s GetLabels (#2454 ) replaces internStringMap with sync.Map - it greatly reduces lock contention concurently reload scrape work for api watcher - each object labels added by dedicated CPU changes can be tested with following script https://gist.github.com/f41gh7/6f8f8d8719786aff1f18a85c23aebf70	2022-04-20 16:12:25 +03:00
Dmytro Kozlov	9dbfd99777	lib/promscrape: simply update UI (#2479 ) * lib/promscrape: simply update UI * lib/promscrape: added vm icon	2022-04-20 15:38:04 +03:00
Aliaksandr Valialkin	45385a5dc6	lib/promscrape: optimize getScrapeWork() function Reduce the number of memory allocations in this function. This improves its performance by up to 50%. This should improve service discovery speed when big number of potential targets with big number of meta-labels are generated by service discovery. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2270	2022-04-20 15:34:18 +03:00
Aliaksandr Valialkin	bfa0b8f710	lib/promscrape: use a hash over target labels as a key for dropped targets' map This reduces the number of allocations and improves the performance for updating dropped targets' map. This map is exposed at /api/v1/targets as in droppedTargets list.	2022-04-20 15:23:54 +03:00
Aliaksandr Valialkin	d0bac8e224	all: typo fix: Kuberntes -> Kubernetes	2022-04-20 10:51:41 +03:00
Dmytro Kozlov	17552dba8b	lib/promscrape: Enable filters for endpoint and labels (#2466 ) * lib/promscrape: Enable filters for endpoint and labels * lib/promscrape: cleanup * lib/promscrape: update template * lib/promscrape: move logic filter logic to backend * lib/promscrape: updated placeholder * lib/promscrape: updated placeholder * lib/promscrape: use two different fields for filters, updated form, added error on parsing queries * lib/promscrape: rename functions * lib/promscrape: removed unused values * wip * wip * wip Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2022-04-19 18:27:44 +03:00
Nikolay	628905f080	lib/promscrape: adds job restart method (#2455 ) * lib/promscrape: adds job restart method it must restart only ScrapeConfig with changed content this change greatly reduce time, that needed for job restart and it should decrease possible data loss when config frequently changed at kubernetes based deployments Apply suggestions from code review Co-authored-by: Roman Khavronenko <roman@victoriametrics.com> * wip Co-authored-by: Roman Khavronenko <roman@victoriametrics.com> Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2022-04-16 20:29:33 +03:00
Aliaksandr Valialkin	c50e48a74c	lib/promscrape: follow-up after `baa1c24b36`	2022-04-16 14:26:38 +03:00

1 2 3 4 5 ...

453 commits