450 commits

Author	SHA1	Message	Date
Aliaksandr Valialkin	e72518e8c6	lib/backup: donload only the remaining parts for partially downloaded files after vmrestore restart ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/487	2020-05-15 17:03:25 +03:00
Aliaksandr Valialkin	1e5c1d7eaa	app/vmstorage: add vm_slow_metric_name_loads_total metric, which could be used as an indicator when more RAM is needed for improving query performance	2020-05-15 14:12:24 +03:00
Aliaksandr Valialkin	d6b9a49481	app/vmstorage: add vm_slow_row_inserts_total and vm_slow_per_day_index_inserts_total metrics for determining whether VictoriaMetrics required more RAM for the current number of active time series	2020-05-15 13:46:57 +03:00
Aliaksandr Valialkin	a72f18e821	lib/{storage,mergeset}: further tuning of compression levels depending on block size ... This should improve performance for querying newly added data, since it can be unpacked faster.	2020-05-15 13:12:28 +03:00
Aliaksandr Valialkin	2cf2e9955b	lib/storage: wait for all the goroutines to finish in TestSearch in order to prevent racy behavior on test finish	2020-05-15 12:12:20 +03:00
Aliaksandr Valialkin	67e331ac62	lib/storage: optimize ingestion pefrormance for new time series	2020-05-15 12:12:19 +03:00
Aliaksandr Valialkin	6838fa876c	lib/mergeset: tune compression levels in order to improve ingestion performance a bit	2020-05-15 12:12:15 +03:00
Aliaksandr Valialkin	1b5d272e07	lib/storage: reduce indentation in Storage.add	2020-05-14 23:23:56 +03:00
Aliaksandr Valialkin	71d29a8fa1	lib/storage: return the first error instead of the last error, since the first error usually points to the root cause	2020-05-14 23:18:59 +03:00
Aliaksandr Valialkin	3845420a8f	lib: extract common code for returning fast unix timestamp into lib/fasttime	2020-05-14 23:06:50 +03:00
Aliaksandr Valialkin	7e831741f9	lib/{storage,mergeset}: return dst on error from unmarshalBlockHeaders, so it could be reused	2020-05-14 15:32:23 +03:00
Aliaksandr Valialkin	2f42b85e0e	lib/storage: document that getnerateUniqueMetricID should return dense ids	2020-05-14 14:08:59 +03:00
Aliaksandr Valialkin	f442d81648	lib/{storage,mergeset}: cleanup: remove unused partSearch.indexBlockReuse	2020-05-14 14:03:15 +03:00
Aliaksandr Valialkin	8bb44a5d09	lib/storage: optimize label matching for regexp ending with literal suffix ... For example, `{label=~"foo.*bar.+baz"}` contains literal suffix `baz`, so it should work faster now.	2020-05-13 11:39:05 +03:00
Aliaksandr Valialkin	3b0f66a227	app/vmagent: fix a bug with improper relabeling when multiple -remoteWrite.urlRelableConfig args are set ... This bug could result in incorrect relabeling and metrics' drop. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/467	2020-05-12 22:03:45 +03:00
Aliaksandr Valialkin	c9ab6dc532	lib/fs: do not use mmap for 32-bit arches by default, since they cannot map files bigger than 4GB in RAM	2020-05-12 20:21:39 +03:00
Aliaksandr Valialkin	d54a93fc81	app/vmagent: fix scraping mTLS targets, which has been broken in v1.35.1 ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/470	2020-05-12 17:23:43 +03:00
Aliaksandr Valialkin	405cf44aed	app/vmagent,lib/promscrape: do not set HostClient.DialDualStack, since it isnt used if HostClient.Dial is set	2020-05-12 15:24:53 +03:00
Aliaksandr Valialkin	bd5f4e0344	lib/storage: properly initialize part struct before trying to close it on error ... This should prevent from nil pointer dereference bug at https://github.com/VictoriaMetrics/VictoriaMetrics/issues/468 .	2020-05-12 14:54:16 +03:00
Aliaksandr Valialkin	f7753b1469	lib/storage: gradually pre-populate per-day inverted index for the next day ... This should prevent from CPU usage spikes at 00:00 UTC every day when inverted index for new day must be quickly created for all the active time series. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/430	2020-05-12 12:13:32 +03:00
Aliaksandr Valialkin	8c77cb436a	lib/storage: typo fixes in error messages: or -> of	2020-05-12 12:12:33 +03:00
Aliaksandr Valialkin	bbf06a4248	lib/storage: speed up matching for common regexps in label filters ... The following regexps have been optimized: * 'foo.+bar' * 'foo.+bar.+baz' This should improve performance for matching Graphite-like metrics.	2020-05-11 22:49:01 +03:00
Aliaksandr Valialkin	37254a139a	lib/storage: add a benchmark for Graphite-like regexps for metric names	2020-05-11 22:49:00 +03:00
Aliaksandr Valialkin	2f28e945b8	lib/httpserver: add -http.shutdownDelay flag for a grace period before http server shutdown ... The http server returns 503 non-OK error at `/health` page during grace period, so load balancers in front of the http server could re-route incoming requests to other servers. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/463	2020-05-07 15:25:51 +03:00
Aliaksandr Valialkin	3052b479b7	lib/httpserver: reduce typical duration for http server graceful shutdown ... Previously the duration for graceful shutdown for http server could take more than a minute because of imporperly set timeouts in setNetworkTimeout. Now typical duration for graceful shutdown should be reduced to less than 5 seconds.	2020-05-07 14:16:38 +03:00
Aliaksandr Valialkin	c43a265716	lib/flagutil: make errcheck happy by explicitly ignoring Array.Set result in tests	2020-05-06 22:37:28 +03:00
Aliaksandr Valialkin	15e3682b40	lib/flagutil: properly parse quoted flag values for flagutil.Array	2020-05-06 22:28:15 +03:00
Aliaksandr Valialkin	20538a2a5d	app/vmagent: allow setting independent auth configs per each configured -remoteWrite.url	2020-05-06 16:52:32 +03:00
Aliaksandr Valialkin	9f39e618ed	lib/promscrape/discovery/gce: discover per-zone instances for gce_sd_config in parallel. This should reduce discovery latency	2020-05-06 15:00:23 +03:00
Aliaksandr Valialkin	8ab5e47b5c	lib/promscrape: add Prometheus-compatible DNS-based service discovery aka dns_sd_configs	2020-05-06 00:02:41 +03:00
Aliaksandr Valialkin	42d563934b	lib/promscrape: properly connect to TCP6 addresses if -enableTCP6 is set	2020-05-06 00:02:40 +03:00
Aliaksandr Valialkin	1c8e97c8a0	lib/procutil: add NewSighupChan function, which returns a channel, which is triggered on every SIGHUP	2020-05-05 10:56:15 +03:00
Aliaksandr Valialkin	054457d1f4	lib/promscrape: allow explicitly setting empty token via token: "" in consul_sd_config	2020-05-05 07:49:54 +03:00
Aliaksandr Valialkin	89aa6dbf56	lib/promscrape: add Prometheus-compatible service discovery for Consul aka consul_sd_configs ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/330	2020-05-04 20:53:06 +03:00
Aliaksandr Valialkin	28e0e8fd88	lib/promauth: properly set up client certificate in tls.Config ... Previously the client certificate has been mistakenly set up as a server certificate	2020-05-04 20:53:04 +03:00
Aliaksandr Valialkin	ed91fe1d9b	lib/promscrape: move common code for discovery api config map handling into discoveryutils	2020-05-04 20:52:58 +03:00
Aliaksandr Valialkin	c50fd219dc	lib/promscrape/discovery/kubernetes/: unify apiConfig creation	2020-05-04 20:52:53 +03:00
Aliaksandr Valialkin	a5880f17af	lib/promscrape: remove debug line left after the commit e4aac6ea40	2020-05-03 17:16:19 +03:00
Aliaksandr Valialkin	1f0e8fdc0d	lib/promscrape: fix tests after the commit 658a8742ac ... The original commit copies `__address__` label to `instance` label when generating per-target labels as Prometheus does. See https://www.robustperception.io/life-of-a-label for details. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/453	2020-05-03 16:59:29 +03:00
DexterZhang	317688f144	fix(vmagent): different behavior as how prometheus deal with labels. [Issue#453] (#454)	2020-05-03 16:59:28 +03:00
Aliaksandr Valialkin	ab1e6a76bb	lib/promscrape: make consistent scrape time offsets across reloads for the same ScrapeURL and Labels ... This should make consistent intervals between data points for scrape targets across reloads. Previously these intervals were random.	2020-05-03 14:31:22 +03:00
Aliaksandr Valialkin	f25416984b	lib/promscrape: fix TestGetFileSDScrapeWorkSuccess after 3b234d82e5	2020-05-03 14:31:20 +03:00
Aliaksandr Valialkin	f422203e10	lib/promscrape: reload only modified scrapers on config changes ... This should improve scrape stability when big number of targets are scraped and these targets are frequently changed. Thanks to @xbsura for the idea and initial implementation attempts at the following pull requests: - https://github.com/VictoriaMetrics/VictoriaMetrics/pull/449 - https://github.com/VictoriaMetrics/VictoriaMetrics/pull/458 - https://github.com/VictoriaMetrics/VictoriaMetrics/pull/459 - https://github.com/VictoriaMetrics/VictoriaMetrics/pull/460	2020-05-03 12:47:16 +03:00
Aliaksandr Valialkin	bbaca16ce8	lib/httpserver: rename http.externalURL to http.pathPrefix and improve help message for this flag ... The `http.externalURL` flag name was slightly misleading, so it has been renamed to `http.pathPrefix`.	2020-05-02 13:12:24 +03:00
DexterZhang	a0589f2ca5	feat(httpserver): add http.externalUrl config to http server, it adds prefix to http path automatically (#452)	2020-05-02 13:12:23 +03:00
Aliaksandr Valialkin	b21b73115a	app/vminsert: add /-/reload handler in the same way as for vmagent	2020-04-30 02:18:08 +03:00
Aliaksandr Valialkin	a970705d8e	lib/procutil: prevent from app termination on SIGHUP signal, since this signal is frequently used for config reload	2020-04-30 02:18:06 +03:00
Aliaksandr Valialkin	d99f48aa48	lib/httpserver: mention that -http.maxGracefulShutdownDuration command-line flag value can be increased on shutdown timeout	2020-04-30 01:37:02 +03:00
Aliaksandr Valialkin	de5f923476	lib/promscrape: set 30 seconds timeout for discovery api requests ... Previously such requests could hang for long time. This could make debugging harder.	2020-04-29 17:29:03 +03:00
Aliaksandr Valialkin	b6d88bac04	vendor: use github.com/VictoriaMetrics/fasthttp instead of github.com/fasthttp/fasthttp ... The upstream fasthttp may contain issues like 996610f021 , plus a code that isn't used by VictoriaMetrics. So let's use a private copy under our control instead.	2020-04-29 16:43:09 +03:00