Roman Khavronenko 7cb894a777 lib/streamaggr: reduce number of inuse objects (#6402) ... The main change is getting rid of interning of sample key. It was discovered that for cases with many unique time series aggregated by vmagent interned keys could grow up to hundreds of millions of objects. This has negative impact on the following aspects: 1. It slows down garbage collection cycles, as GC has to scan all inuse objects periodically. The higher is the number of inuse objects, the longer it takes/the more CPU it takes. 2. It slows down the hot path of samples aggregation where each key needs to be looked up in the map first. The change makes code more fragile, but suppose to provide performance optimization for heavy-loaded vmagents with stream aggregation enabled. --------- Signed-off-by: hagen1778 <roman@victoriametrics.com> Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>		2024-06-07 15:45:52 +02:00
..
appmetrics	all: add -metrics.exposeMetadata command-line flag, which can be used for adding TYPE and HELP metadata for metrics exposed at /metrics page	2023-12-19 03:20:40 +02:00
auth	lib/auth: add NewTokenPossibleMultitenant() for parsing auth token, which can be multitenant	2023-08-30 14:17:55 +02:00
awsapi	lib/awsapi: properly assume role with webIdentity token (#5495)	2023-12-20 19:05:39 +02:00
backup	lib/backup: add -s3TLSInsecureSkipVerify command-line flag (#6318)	2024-05-22 13:58:39 +02:00
blockcache	lib: consistently use atomic.* types instead of atomic.* functions	2024-02-24 02:07:53 +02:00
bloomfilter	lib: consistently use atomic.* types instead of atomic.* functions	2024-02-24 02:07:53 +02:00
bufferedwriter	app/vmselect: move common http functionality from app/vmselect/searchutils to lib/httputils	2023-06-19 22:34:20 -07:00
buildinfo	all: open-sourcing single-node version	2019-05-23 00:18:06 +03:00
bytesutil	lib/slicesutil: add helper functions for setting slice length and extending its capacity	2024-05-12 11:32:17 +02:00
cgroup	lib/cgroup: remove SetGOGC() function	2024-02-05 12:11:08 +02:00
decimal	lib/slicesutil: add helper functions for setting slice length and extending its capacity	2024-05-12 11:32:17 +02:00
encoding	lib/encoding: optimize UnmarshalVarUint64, UnmarshalVarInt64 and UnmarshalBytes a bit	2024-05-14 01:23:54 +02:00
envflag	lib/envflag: do not allow unsupported form for boolean command-line flags in the form -boolFlag value	2023-08-17 13:26:53 +02:00
envtemplate	allowed using dashes and dots in environment variables names (#4009)	2023-03-24 15:43:05 -07:00
fastnum	lib/fastnum: use unsafe.Slice() instead of deprecated reflect.SliceHeader	2024-02-29 17:17:13 +02:00
fasttime	lib: consistently use atomic.* types instead of atomic.* functions	2024-02-24 02:07:53 +02:00
filestream	lib/filestream: do not measure read / write duration from / to in-memory buffers	2024-01-23 14:52:22 +02:00
flagutil	Revert "app/vmbackup: introduce new flag type URL (#6152)"	2024-04-24 13:47:57 +02:00
formatutil	app/vmbackupmanager: add metrics for better observability (#488)	2022-12-20 14:18:06 -08:00
fs	lib/fs: support NFS implementations that return EEXIST instead of ENOTEMPTY (#6398)	2024-06-04 15:17:38 +02:00
htmlcomponents	lib/htmlcomponents: use relative links for the top page and for favicon.ico	2023-11-13 20:29:05 +01:00
httpserver	lib/httpserver: add support for automatic issuing of TLS certificates via Lets Encrypt service	2024-04-17 23:50:57 +02:00
httputils	app/vmalert: support DNS SRV record in -remoteWrite.url (#6299)	2024-05-22 10:52:51 +02:00
influxutils	lib/flagutil: rename Array to ArrayString	2022-10-01 18:26:36 +03:00
ingestserver	Statsd protocol compatibility (#5053)	2024-05-07 21:46:08 +02:00
leveledbytebufferpool	lib/slicesutil: add helper functions for setting slice length and extending its capacity	2024-05-12 11:32:17 +02:00
logger	lib/logger: increase default -loggerMaxArgLen command-line flag value from 500 to 1000	2023-11-14 19:52:27 +01:00
logstorage	lib/logstorage: work-in-progress	2024-06-06 12:27:05 +02:00
lrucache	lib: consistently use atomic.* types instead of atomic.* functions	2024-02-24 02:07:53 +02:00
memory	all: cleanup: remove // +build ... lines, since they are no longer needed after Go1.17, and the minimum supported Go version for VictoriaMetrics source code is Go1.20	2023-11-13 19:12:51 +01:00
mergeset	lib/mergeset: adds tracking for indexdb records drop (#6297)	2024-05-24 14:55:20 +02:00
metricsql	all: make fmt via the upcoming Go1.19	2022-07-11 19:22:15 +03:00
netutil	lib/netutil: move creation of GetCertificate callback into a separate function	2024-04-17 22:10:43 +02:00
persistentqueue	Feature allow configuring disableOnDiskQueue and dropSamplesOnOverload per url (#6248)	2024-05-10 12:09:21 +02:00
procutil	all: cleanup: remove // +build ... lines, since they are no longer needed after Go1.17, and the minimum supported Go version for VictoriaMetrics source code is Go1.20	2023-11-13 19:12:51 +01:00
promauth	all: replace old https://docs.victoriametrics.com/sd_configs.html url with the new one - https://docs.victoriametrics.com/sd_configs/	2024-04-18 02:27:47 +02:00
prompb	Exemplar support (#5982)	2024-05-07 12:09:44 +02:00
prompbmarshal	lib/slicesutil: add helper functions for setting slice length and extending its capacity	2024-05-12 11:32:17 +02:00
promrelabel	lib/promrelabel: speedup label match by __name__ (#6432)	2024-06-07 15:44:48 +02:00
promscrape	app/vmalert: support DNS SRV record in -remoteWrite.url (#6299)	2024-05-22 10:52:51 +02:00
promutils	lib/logstorage: work-in-progress	2024-06-04 01:49:02 +02:00
protoparser	follow-up for c6c5a5a186 (#6265)	2024-05-16 09:25:42 +02:00
proxy	lib/promscrape: use the standard net/http.Client instead of fasthttp.Client for scraping targets in non-streaming mode	2024-01-30 18:39:10 +02:00
pushmetrics	lib/pushmetrics: wait until the background goroutines, which push metrics, are stopped at pushmetrics.Stop()	2024-01-15 13:50:36 +02:00
querytracer	lib/querytracer: add missing blank comment line after 3121d76bee	2023-11-15 16:10:43 +01:00
ratelimiter	app/vmagent: properly shutdown when -maxIngestionRate limit is reached	2024-03-30 06:43:48 +02:00
regexutil	lib/logstorage: work-in-progress	2024-05-25 22:59:13 +02:00
slicesutil	lib/slicesutil: add helper functions for setting slice length and extending its capacity	2024-05-12 11:32:17 +02:00
snapshot	Revert "app/vmbackup: introduce new flag type URL (#6152)"	2024-04-24 13:47:57 +02:00
storage	lib/storage: filter deleted label names and values from `/api/v1/labe… (#6342)	2024-05-29 14:07:44 +02:00
streamaggr	lib/streamaggr: reduce number of inuse objects (#6402)	2024-06-07 15:45:52 +02:00
stringsutil	lib/logstorage: work-in-progress	2024-05-22 21:01:20 +02:00
syncwg	all: open-sourcing single-node version	2019-05-23 00:18:06 +03:00
tenantmetrics	lib/encoding/zstd: switch back from atomic.Pointer to atomic.Value for map[...]...	2023-07-20 20:56:11 -07:00
timerpool	lib/timerpool: use timer pool in concurrency limiters	2019-05-28 17:20:10 +03:00
timeutil	all: add up to 10% random jitter to the interval between periodic tasks performed by various components	2024-01-22 18:40:32 +02:00
uint64set	lib/slicesutil: add helper functions for setting slice length and extending its capacity	2024-05-12 11:32:17 +02:00
workingsetcache	lib: consistently use atomic.* types instead of atomic.* functions	2024-02-24 02:07:53 +02:00
writeconcurrencylimiter	app/vmagent/remotewrite: clarify the reason behind the default value for -remoteWrite.queues in the same way as the reason for -maxConcurrentInserts is defined at 73f5fb0f0c	2024-03-06 13:43:08 +02:00