VictoriaMetrics/lib
Hui Wang 4e3242b02d
lib/promscrape/discovery/kubernetes: fix watcher start order for roles endpoints and endpointslice (#5557)
* lib/promscrape/discovery/kubernetes: fix watcher start order for roles endpoints and endpointslice

Previously the groupWatcher could be mistakenly stopped when requests for pod or services resources take too long.

* remove mislead comment

* docs/sd_configs.md: mention -promscrape.kubernetes.attachNodeMetadataAll flag in the description for attach_metadata section

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4640

* wip

* lib/promscrape/kubernetes: prevent from stopping groupWatcher when there are in-flight apiWatcher.mustStart() calls

groupWatcher is stopped if it has zero registered apiWatchers during 14 seconds.
But such a groupWatcher can be still in use if apiWatcher for `role: endpoints` or `role: endpointslice`
is being registered and the discovery of the associated `pod` and/or `service` objects takes longer
than 14 seconds - see the beginning of groupWatcher.startWatchersForRole() function for details.

Track the number of in-flight calls to apiWatcher.mustStart() and prevent from stopping the associated groupWatcher
if the number of in-flight calls is non-zero.

P.S. postponing the discovery of `pod` and/or `service` objects associated with `endpoints` or `endpointslice` roles
isn't the best solution, since it slows down initial discovery of `endpoints` and `endpointslice` targets.

* typo fix

---------

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2024-01-21 23:13:15 +02:00
..
appmetrics all: add -metrics.exposeMetadata command-line flag, which can be used for adding TYPE and HELP metadata for metrics exposed at /metrics page 2023-12-19 03:20:40 +02:00
auth lib/auth: add NewTokenPossibleMultitenant() for parsing auth token, which can be multitenant 2023-08-30 14:17:55 +02:00
awsapi lib/awsapi: properly assume role with webIdentity token (#5495) 2023-12-20 19:05:39 +02:00
backup vendor: run make vendor-update 2023-12-11 10:48:36 +02:00
blockcache lib/blockcache: do not cache entries, which were attempted to be accessed 1 or 2 times 2023-11-10 22:28:03 +01:00
bloomfilter all: call atomic.Load* in front of atomic.CompareAndSwap* at places where the atomic.CompareAndSwap* returns false most of the time 2024-01-21 14:04:54 +02:00
bufferedwriter app/vmselect: move common http functionality from app/vmselect/searchutils to lib/httputils 2023-06-19 22:34:20 -07:00
buildinfo all: open-sourcing single-node version 2019-05-23 00:18:06 +03:00
bytesutil lib/logger: add -loggerMaxArgLen command-line flag for fine-tuning the maximum length of logged args 2023-11-11 12:30:08 +01:00
cgroup lib/cgroup: add SetGOGC() function 2023-06-19 22:39:00 -07:00
decimal lib/decimal: use consistent randomizer in tests 2023-01-23 19:23:39 -08:00
encoding all: cleanup: remove // +build ... lines, since they are no longer needed after Go1.17, and the minimum supported Go version for VictoriaMetrics source code is Go1.20 2023-11-13 19:12:51 +01:00
envflag lib/envflag: do not allow unsupported form for boolean command-line flags in the form -boolFlag value 2023-08-17 13:26:53 +02:00
envtemplate allowed using dashes and dots in environment variables names (#4009) 2023-03-24 15:43:05 -07:00
fastnum Makefile: add build and test rules with enabled race detector. These rules have -race suffix 2020-03-05 12:03:38 +02:00
fasttime lib: extract common code for returning fast unix timestamp into lib/fasttime 2020-05-14 23:02:07 +03:00
filestream app/vmselect/promql: follow-up for ce4f26db02 2024-01-16 02:55:08 +02:00
flagutil all: allow dynamically reading *AuthKey flag values from files and urls 2024-01-21 22:03:38 +02:00
formatutil app/vmbackupmanager: add metrics for better observability (#488) 2022-12-20 14:18:06 -08:00
fs all: allow dynamically reading *AuthKey flag values from files and urls 2024-01-21 22:03:38 +02:00
htmlcomponents lib/htmlcomponents: use relative links for the top page and for favicon.ico 2023-11-13 20:29:05 +01:00
httpserver all: allow dynamically reading *AuthKey flag values from files and urls 2024-01-21 22:03:38 +02:00
httputils lib/httputils: handle step=undefined query arg as an empty value 2024-01-16 18:59:26 +02:00
influxutils lib/flagutil: rename Array to ArrayString 2022-10-01 18:26:36 +03:00
ingestserver lib/ingestserver: properly log the number of closed connections 2023-11-14 21:53:24 +01:00
leveledbytebufferpool all: make fmt via the upcoming Go1.19 2022-07-11 19:22:15 +03:00
logger lib/logger: increase default -loggerMaxArgLen command-line flag value from 500 to 1000 2023-11-14 19:52:27 +01:00
logjson app/vlinsert/jsonline: code prettifying 2023-06-21 19:39:22 -07:00
logstorage lib/logstorage: always check the previous indexBlockHeader for blocks with matching tenantID and/or streamID 2023-11-13 23:13:53 +01:00
lrucache Makefile: update golangci-lint from v1.51.2 to v1.54.2 2023-09-01 10:16:42 +02:00
memory all: cleanup: remove // +build ... lines, since they are no longer needed after Go1.17, and the minimum supported Go version for VictoriaMetrics source code is Go1.20 2023-11-13 19:12:51 +01:00
mergeset all: call atomic.Load* in front of atomic.CompareAndSwap* at places where the atomic.CompareAndSwap* returns false most of the time 2024-01-21 14:04:54 +02:00
metricsql all: make fmt via the upcoming Go1.19 2022-07-11 19:22:15 +03:00
netutil all: consistently use %w instead of %s in when error is passed to fmt.Errorf() 2023-10-25 21:24:03 +02:00
persistentqueue app/vmagent: follow-up for 090cb2c9de 2023-11-25 12:09:44 +02:00
procutil all: cleanup: remove // +build ... lines, since they are no longer needed after Go1.17, and the minimum supported Go version for VictoriaMetrics source code is Go1.20 2023-11-13 19:12:51 +01:00
promauth all: allow dynamically reading *AuthKey flag values from files and urls 2024-01-21 22:03:38 +02:00
prompb lib/prompbmarshal: switch to github.com/VictoriaMetrics/easyproto 2024-01-14 23:04:45 +02:00
prompbmarshal lib/prompbmarshal: move WriteRequest proto definition to the correct place 2024-01-16 00:20:37 +02:00
promrelabel all: allow dynamically reading *AuthKey flag values from files and urls 2024-01-21 22:03:38 +02:00
promscrape lib/promscrape/discovery/kubernetes: fix watcher start order for roles endpoints and endpointslice (#5557) 2024-01-21 23:13:15 +02:00
promutils all: consistently clear prompbmarshal.Label by assigning an empty struct instead of zeroing Name and Value individually 2024-01-21 05:11:05 +02:00
protoparser all: consistently clear prompbmarshal.Label by assigning an empty struct instead of zeroing Name and Value individually 2024-01-21 05:11:05 +02:00
proxy lib/promauth: follow-up for e16d3f5639 2023-10-25 23:19:37 +02:00
pushmetrics lib/pushmetrics: wait until the background goroutines, which push metrics, are stopped at pushmetrics.Stop() 2024-01-15 13:50:36 +02:00
querytracer lib/querytracer: add missing blank comment line after 3121d76bee 2023-11-15 16:10:43 +01:00
regexutil lib/regexutil: properly handle alternate regexps surrounded by .+ or .* 2023-11-13 18:23:38 +01:00
snapshot Makefile: update golangci-lint version from v1.54.2 to v1.55.1 2023-11-10 20:23:48 +01:00
storage all: call atomic.Load* in front of atomic.CompareAndSwap* at places where the atomic.CompareAndSwap* returns false most of the time 2024-01-21 14:04:54 +02:00
streamaggr all: allow dynamically reading *AuthKey flag values from files and urls 2024-01-21 22:03:38 +02:00
stringsutil lib/stringsutil: add tests for LimitStringLen() function 2023-11-13 10:32:33 +01:00
syncwg all: open-sourcing single-node version 2019-05-23 00:18:06 +03:00
tenantmetrics lib/encoding/zstd: switch back from atomic.Pointer to atomic.Value for map[...]... 2023-07-20 20:56:11 -07:00
timerpool lib/timerpool: use timer pool in concurrency limiters 2019-05-28 17:20:10 +03:00
uint64set lib/uint64set: use repeatable randomizer in tests 2023-01-23 19:22:58 -08:00
workingsetcache lib/workingsetcache: fix outdated comments for Load() and New() functions 2023-10-25 21:04:20 +02:00
writeconcurrencylimiter lib/writeconcurrencylimiter: initialize concurrencyLimitCh before exporting vm_concurrent_insert_capacity and vm_concurrent_insert_current metrics 2023-02-07 11:08:17 -08:00