Roman Khavronenko 8e2a8a6ae2 lib/promscrape: limit number of sent stale series at once (#3686) ... Stale series are sent when there is a difference between current and previous scrapes. Those series which disappeared in the current scrape are marked as stale and sent to the remote storage. Sending stale series requires memory allocation and in case when too many series disappear in the same it could result in noticeable memory spike. For example, re-deploy of a big fleet of service can result into excessive memory usage for vmagent, because all the series with old pod name will be marked as stale and sent to the remote write storage. This change limits the number of stale series which can be sent at once, so memory usage remains steady. https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3668 https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3675 Signed-off-by: hagen1778 <roman@victoriametrics.com> Signed-off-by: hagen1778 <roman@victoriametrics.com>		2023-01-23 21:56:17 -08:00
..
appmetrics	all: add ability to push internal metrics to remote storage system specified via -pushmetrics.url	2022-07-21 20:15:29 +03:00
auth	app/vminsert: allows parsing tenant id from labels (#3009)	2022-09-30 17:28:35 +03:00
awsapi	lib/promscrape/discovery/ec2: expose __meta_ec2_region label in the same way as Prometheus 2.39 does	2022-09-30 20:49:08 +03:00
backup	app/vmbackupmanager: add metrics for better observability (#488)	2022-12-20 14:18:43 -08:00
blockcache	all: remove explicit "xxhash" name when importing github.com/cespare/xxhash/v2 package	2022-06-21 20:24:28 +03:00
bloomfilter	lib/promscrape: add the ability to limit the number of unique series per each scrape target	2021-09-01 16:08:12 +03:00
buildinfo	all: open-sourcing single-node version	2019-05-23 00:18:06 +03:00
bytesutil	lib/bytesutil: add InternBytes() function as a shortcut to InternString(ToUnsafeString(..))	2023-01-03 22:15:49 -08:00
cgroup	all: use os.{Read|Write}File instead of ioutil.{Read|Write}File	2022-08-21 23:55:20 +03:00
consts	app/vminsert: reduce the max packet size, which vminsert can send to vmstorage	2022-04-05 15:39:58 +03:00
decimal	lib/decimal: use consistent randomizer in tests	2023-01-23 19:24:05 -08:00
encoding	lib/encoding: make deterministic tests which rely on math/rand	2023-01-23 18:43:49 -08:00
envflag	lib/envflag: small refactoring after 518c340ae3 and 02096e06d0	2022-10-29 02:29:19 +03:00
envtemplate	lib/envtemplate: allow non-env var names inside "%{ ... }"	2022-11-07 13:16:00 +02:00
fastnum	Makefile: add build and test rules with enabled race detector. These rules have -race suffix	2020-03-05 12:05:16 +02:00
fasttime	lib: extract common code for returning fast unix timestamp into lib/fasttime	2020-05-14 23:06:50 +03:00
filestream	lib/filestream: remove logging redundant path values in a single error message	2022-12-03 22:02:04 -08:00
flagutil	lib/flagutil/bytes.go: properly handle values bigger than 2GiB on 32-bit architectures	2022-12-14 19:29:57 -08:00
formatutil	app/vmbackupmanager: add metrics for better observability (#488)	2022-12-20 14:18:43 -08:00
fs	lib/fs: use unix.Statfs() / unix.Statvfs() when using a path (#3663)	2023-01-17 21:22:02 -08:00
handshake	all: substitute zeroTime with time.Time{}, since this generates more optimal binary code	2022-02-07 14:36:41 +02:00
htmlcomponents	app/vmselect: remove dependency on lib/promscrape from app/vmselect	2023-01-03 23:27:36 -08:00
httpserver	lib/httpserver: directly pass flag value to CheckAuthFlag()	2023-01-10 15:59:55 -08:00
influxutils	lib/flagutil: rename Array to ArrayString	2022-10-01 18:28:19 +03:00
ingestserver	Use httpAuth.* flags as a fallback for endpoints protected by *AuthKey flags (#3582)	2023-01-10 15:57:55 -08:00
leveledbytebufferpool	all: make fmt via the upcoming Go1.19	2022-07-11 19:23:25 +03:00
logger	lib/logger: follow-up for 72f8fce107	2022-12-16 17:44:05 -08:00
lrucache	all: remove explicit "xxhash" name when importing github.com/cespare/xxhash/v2 package	2022-06-21 20:24:28 +03:00
memory	lib/flagutil/bytes.go: properly handle values bigger than 2GiB on 32-bit architectures	2022-12-14 19:29:57 -08:00
mergeset	lib/mergeset: use deterministic random generator in tests	2023-01-23 19:44:10 -08:00
metricsql	all: make fmt via the upcoming Go1.19	2022-07-11 19:23:25 +03:00
netutil	app/{vminsert,vmselect}/netstorage: allow calling Init()+MustStop() in a loop	2022-10-25 14:43:05 +03:00
persistentqueue	lib/flagutil/bytes.go: properly handle values bigger than 2GiB on 32-bit architectures	2022-12-14 19:29:57 -08:00
procutil	lib/procutil: stop immediately after receiving the second SIGINT or SIGTERM signal	2022-10-20 21:58:49 +03:00
promauth	lib/{httpserver,netutil}: allow to define min and max TLS version of the http server (#3109)	2022-09-26 17:38:43 +03:00
prompb	app/vminsert: moved -maxInsertRequestSize command-line flag out of lib/prompb in order to prevent its inclusion in vmselect and vmstorage apps	2020-01-28 22:53:50 +02:00
prompbmarshal	all: use %w instead of %s for wrapping errors in fmt.Errorf	2020-06-30 23:33:46 +03:00
promrelabel	lib/promrelabel: pass query args via query string at /metric-relabel-debug and /target-relabel-debug pages if their length doesnt exceed 1000	2023-01-05 16:45:42 -08:00
promscrape	lib/promscrape: limit number of sent stale series at once (#3686)	2023-01-23 21:56:17 -08:00
promutils	lib/promscrape: implement target-level and metric-level relabel debugging	2022-12-10 02:25:56 -08:00
protoparser	lib/writeconcurrencylimiter: improve the logic behind -maxConcurrentInserts limit	2023-01-06 22:07:16 -08:00
proxy	lib/promauth: add ability to send additional http headers in requests to scrape targets	2022-06-22 20:40:50 +03:00
pushmetrics	lib/flagutil: rename Array to ArrayString	2022-10-01 18:28:19 +03:00
querytracer	lib/querytracer: fix remaining tests after 49ebc48809	2022-12-08 18:18:50 -08:00
regexutil	lib/promrelabel: there is no need in calling regex.HasPrefix() after the optimization at 17289ff481	2022-09-30 12:28:25 +03:00
snapshot	app/vmbackup: prevent password leaks (#3672)	2023-01-18 11:40:52 -08:00
storage	lib/storage: use deterministic random generator in tests	2023-01-23 20:12:32 -08:00
streamaggr	lib/streamaggr: limit the the number of concurrent flushes of the aggregate data to the exact number of available CPUs	2023-01-07 00:19:34 -08:00
syncwg	all: open-sourcing single-node version	2019-05-23 00:18:06 +03:00
tenantmetrics	app/vminsert: allows parsing tenant id from labels (#3009)	2022-09-30 17:28:35 +03:00
timerpool	lib/timerpool: use timer pool in concurrency limiters	2019-05-28 17:30:10 +03:00
uint64set	lib/uint64set: use repeatable randomizer in tests	2023-01-23 19:24:05 -08:00
vmselectapi	lib/vmselectapi: propagate timeout errors from vmselect to vmstorage instead of closing the connection established from vmselect to vmstorage	2023-01-20 19:30:22 -08:00
workingsetcache	lib/workingsetcache: expose -cacheExpireDuration command-line flag for fine-tuning of the cache expiration	2022-11-17 21:55:11 +02:00
writeconcurrencylimiter	all: small improvements in error messages and command-line flag descriptions related to concurrency limiters	2023-01-07 00:12:24 -08:00