Aliaksandr Valialkin 8669584e9f lib/{storage,mergeset}: convert beffered items into searchable in-memory parts exactly once per the given flush interval ... Previously the interval between item addition and its conversion to searchable in-memory part could vary significantly because of too coarse per-second precision. Switch from fasttime.UnixTimestamp() to time.Now().UnixMilli() for millisecond precision. It is OK to use time.Now() for tracking the time when buffered items must be converted to searchable in-memory parts, since time.Now() calls aren't located in hot paths. Increase the flush interval for converting buffered samples to searchable in-memory parts from one second to two seconds. This should reduce the number of blocks, which are needed to be processed during high-frequency alerting queries. This, in turn, should reduce CPU usage. While at it, hardcode the maximum size of rawRows shard to 8Mb, since this size gives the optimal data ingestion pefromance according to load tests. This reduces memory usage and CPU usage on systems with big amounts of RAM under high data ingestion rate.		2024-02-23 01:11:57 +02:00
..
appmetrics	all: add -metrics.exposeMetadata command-line flag, which can be used for adding TYPE and HELP metadata for metrics exposed at /metrics page	2023-12-19 03:26:02 +02:00
auth	lib/auth: add NewTokenPossibleMultitenant() for parsing auth token, which can be multitenant	2023-08-30 14:13:51 +02:00
awsapi	lib/awsapi: properly assume role with webIdentity token (#5495)	2023-12-20 19:07:04 +02:00
backup	lib/snapshot: move Time, Validate and NewName into lib/snapshot/snapshotutil package	2024-02-09 04:19:30 +02:00
blockcache	all: add up to 10% random jitter to the interval between periodic tasks performed by various components	2024-01-22 18:39:16 +02:00
bloomfilter	all: call atomic.Load* in front of atomic.CompareAndSwap* at places where the atomic.CompareAndSwap* returns false most of the time	2024-01-22 01:13:41 +02:00
bufferedwriter	app/vmselect: move common http functionality from app/vmselect/searchutils to lib/httputils	2023-07-06 17:22:23 -07:00
buildinfo	all: open-sourcing single-node version	2019-05-23 00:18:06 +03:00
bytesutil	lib/logger: add -loggerMaxArgLen command-line flag for fine-tuning the maximum length of logged args	2023-11-13 09:43:49 +01:00
cgroup	lib/cgroup: remove SetGOGC() function	2024-02-05 12:13:08 +02:00
consts	app/vminsert: reduce the max packet size, which vminsert can send to vmstorage	2022-04-05 15:39:58 +03:00
decimal	lib/decimal: follow-up for e6bad5174f	2024-01-27 00:08:32 +01:00
encoding	lib/encoding: follow-up for 49e3665d6d	2024-02-01 05:38:58 +02:00
envflag	lib/envflag: do not allow unsupported form for boolean command-line flags in the form -boolFlag value	2023-08-17 13:37:05 +02:00
envtemplate	allowed using dashes and dots in environment variables names (#4009)	2023-03-24 17:57:19 -07:00
fastnum	Makefile: add build and test rules with enabled race detector. These rules have -race suffix	2020-03-05 12:05:16 +02:00
fasttime	lib: extract common code for returning fast unix timestamp into lib/fasttime	2020-05-14 23:06:50 +03:00
filestream	lib/filestream: do not measure read / write duration from / to in-memory buffers	2024-01-23 14:53:35 +02:00
flagutil	all: add support for specifying multiple -httpListenAddr options	2024-02-09 03:22:49 +02:00
formatutil	app/vmbackupmanager: add metrics for better observability (#488)	2022-12-20 14:18:43 -08:00
fs	lib/fs: lazily open the file at ReaderAt on the first access	2024-02-06 21:10:00 +02:00
handshake	lib/handshake: substitute time.Now() with fastttime.UnixTimestamp(), since profiling shows time.Now() is slow	2024-01-23 18:39:28 +02:00
htmlcomponents	lib/htmlcomponents: use relative links for the top page and for favicon.ico	2023-11-13 20:28:17 +01:00
httpserver	all: add support for specifying multiple -httpListenAddr options	2024-02-09 03:22:49 +02:00
httputils	[lib/httputils] fixed floating-point error when parsing time in RFC3339 format (#5814)	2024-02-23 00:58:26 +02:00
influxutils	lib/flagutil: rename Array to ArrayString	2022-10-01 18:28:19 +03:00
ingestserver	lib/ingestserver: properly log the number of closed connections	2023-11-14 21:53:10 +01:00
leveledbytebufferpool	all: make fmt via the upcoming Go1.19	2022-07-11 19:23:25 +03:00
logger	lib/logger: increase default -loggerMaxArgLen command-line flag value from 500 to 1000	2023-11-14 19:55:55 +01:00
logjson	app/vlinsert/jsonline: code prettifying	2023-07-06 21:35:55 -07:00
logstorage	app/vlselect: follow-up for 451d2abf50	2024-02-18 23:06:08 +02:00
lrucache	all: add up to 10% random jitter to the interval between periodic tasks performed by various components	2024-01-22 18:39:16 +02:00
memory	all: cleanup: remove // +build ... lines, since they are no longer needed after Go1.17, and the minimum supported Go version for VictoriaMetrics source code is Go1.20	2023-11-13 19:15:42 +01:00
mergeset	lib/{storage,mergeset}: convert beffered items into searchable in-memory parts exactly once per the given flush interval	2024-02-23 01:11:57 +02:00
metricsql	all: make fmt via the upcoming Go1.19	2022-07-11 19:23:25 +03:00
netutil	all: upgrade Go builder from Go1.21.7 to Go1.22.0	2024-02-12 22:14:00 +02:00
persistentqueue	app/vmagent: follow-up for 090cb2c9de	2023-11-25 12:13:39 +02:00
procutil	all: cleanup: remove // +build ... lines, since they are no longer needed after Go1.17, and the minimum supported Go version for VictoriaMetrics source code is Go1.20	2023-11-13 19:15:42 +01:00
promauth	lib/promauth: follow-up for fca3b14b7b	2024-01-31 19:47:53 +02:00
prompb	lib/prompbmarshal: switch to github.com/VictoriaMetrics/easyproto	2024-01-16 20:48:30 +02:00
prompbmarshal	lib/prompbmarshal: code cleanup after 8aaa828ba3	2024-02-01 21:41:10 +02:00
promrelabel	lib/promrelabel: store the original labels before returning them them to promutils.PutLabels()	2024-02-14 16:09:38 +02:00
promscrape	lib/promscrape: add support for enable_compression option in the same way as Prometheus does	2024-02-18 19:42:09 +02:00
promutils	all: consistently clear prompbmarshal.Label by assigning an empty struct instead of zeroing Name and Value individually	2024-01-22 01:11:59 +02:00
protoparser	lib/protoparser/datadogsketches: use math.RoundToEven() for calculating the rank	2024-02-07 21:45:05 +02:00
proxy	lib/promscrape: use the standard net/http.Client instead of fasthttp.Client for scraping targets in non-streaming mode	2024-01-30 18:39:55 +02:00
pushmetrics	lib/pushmetrics: wait until the background goroutines, which push metrics, are stopped at pushmetrics.Stop()	2024-01-16 21:18:22 +02:00
querytracer	lib/querytracer: add missing blank comment line after 3121d76bee	2023-11-15 16:11:50 +01:00
regexutil	all: upgrade Go builder from Go1.21.7 to Go1.22.0	2024-02-12 22:14:00 +02:00
snapshot	lib/snapshot: move Time, Validate and NewName into lib/snapshot/snapshotutil package	2024-02-09 04:19:30 +02:00
storage	lib/{storage,mergeset}: convert beffered items into searchable in-memory parts exactly once per the given flush interval	2024-02-23 01:11:57 +02:00
streamaggr	lib/streamaggr: fix incorrect err message for min interval value	2024-01-29 17:27:23 +01:00
stringsutil	lib/stringsutil: add tests for LimitStringLen() function	2023-11-13 10:33:07 +01:00
syncwg	all: open-sourcing single-node version	2019-05-23 00:18:06 +03:00
tenantmetrics	lib/encoding/zstd: switch back from atomic.Pointer to atomic.Value for map[...]...	2023-07-20 21:54:51 -07:00
timerpool	lib/timerpool: use timer pool in concurrency limiters	2019-05-28 17:30:10 +03:00
timeutil	all: add up to 10% random jitter to the interval between periodic tasks performed by various components	2024-01-22 18:39:16 +02:00
uint64set	lib/uint64set: go fmt after c0a9b87f46	2024-02-15 14:52:53 +02:00
vmselectapi	vmcluster: re-routing enhancement (#5293)	2023-11-14 01:00:42 +01:00
workingsetcache	all: add up to 10% random jitter to the interval between periodic tasks performed by various components	2024-01-22 18:39:16 +02:00
writeconcurrencylimiter	lib/writeconcurrencylimiter: initialize concurrencyLimitCh before exporting vm_concurrent_insert_capacity and vm_concurrent_insert_current metrics	2023-02-07 11:08:39 -08:00