Aliaksandr Valialkin
c14dafce43
lib/promscrape: an attempt to reduce memory usage when vmagent scrapes targets with varying number of metrics
...
Do not cache too big byte buffers and too big writeRequestCtx objects,
since it is cheaper to re-create them instead of wasting RAM for their caching.
This reverts 7f6f350ee1
2021-03-15 11:45:39 +02:00
Aliaksandr Valialkin
7f6f350ee1
lib/promscrape: return back the logic for flushing big buffers to storage from the commit 3fd8653b40
...
This should reduce memory usage when vmagent scrapes targets with big number of metrics and `-promscrape.streamParse` isn't enabled
2021-03-14 22:26:00 +02:00
Aliaksandr Valialkin
b88806ecbf
lib/promscrape/discovery/kubernetes: do not start object watcher until initial objects are loaded
2021-03-14 21:55:00 +02:00
Aliaksandr Valialkin
83edbb7cab
lib/promscrape: retry service discovery in a few seconds if it starts returning 0 targets
...
This should reduce recovery time from temporary issues during service discovery
2021-03-14 21:53:23 +02:00
Aliaksandr Valialkin
bf15d6a6a2
lib/promscrape: remove duplicate target word in error message
2021-03-14 21:52:02 +02:00
Aliaksandr Valialkin
d409898515
lib/promscrape/discovery/kubernetes: further optimize kubernetes service discovery for the case with many scrape jobs
...
Do not re-calculate labels per each scrape job - reuse them instead for scrape jobs with identical Kubernetes role
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1113
2021-03-14 21:14:53 +02:00
Aliaksandr Valialkin
7a16e8e3a2
lib/promscrape/discovery: fixes after 133b288681
...
- Removed a deadlock in addAPIWatcher
- Do not create unused ScrapeWork objects
- Do not spend CPU resources on creating objectByKey map in addAPIWatcher
This work is based on https://github.com/VictoriaMetrics/VictoriaMetrics/pull/1125
2021-03-13 15:18:51 +02:00
Aliaksandr Valialkin
2096c6e464
app/vmselect/prometheus: typo fix after 7c002023d7
2021-03-12 12:19:36 +02:00
Aliaksandr Valialkin
7c002023d7
app/vmselect/prometheus: do not include datapoints with timestamps matching t-d when returning results from /api/v1/query?query=m[d]&time=t as Prometheus does
2021-03-12 12:16:50 +02:00
Aliaksandr Valialkin
43552fa8d3
docs/CaseStudies.md: fix incorrect number of active time series for Zhihu
2021-03-12 11:45:38 +02:00
Aliaksandr Valialkin
def014eb75
lib/promscrape/discovery/kubernetes: remove debug lines left after the commit 133b288681
2021-03-12 11:22:33 +02:00
Aliaksandr Valialkin
ca4d5ce037
lib/proxy: there is no need in cloning tlsCfg, which has been created two lines above
2021-03-12 10:47:02 +02:00
Aliaksandr Valialkin
895d5d1355
lib/proxy: set proxy address in tls.Config.ServerName instead of the target address
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1116
2021-03-12 10:41:33 +02:00
Aliaksandr Valialkin
a6a71ef861
lib/promscrape: add ability to configure proxy options via proxy_tls_config, proxy_basic_auth, proxy_bearer_token and proxy_bearer_token_file
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1116
2021-03-12 03:36:19 +02:00
Aliaksandr Valialkin
fa448806a5
deployment/docker: update Go builder from 1.16.1 to 1.16.2
...
See https://github.com/golang/go/issues?q=milestone%3AGo1.16.2+label%3ACherryPickApproved
2021-03-12 01:53:48 +02:00
Aliaksandr Valialkin
f3d712724c
docs/Articles.md: add https://www.sensedia.com/post/monitoring-with-prometheus-alertmanager
2021-03-12 01:22:51 +02:00
Aliaksandr Valialkin
f669531506
lib/storage: further tune filters sorting logic
2021-03-12 00:53:04 +02:00
Aliaksandr Valialkin
133b288681
lib/promscrape/discovery/kubernetes: use a single watcher per apiURL
...
Previously multiple scrape jobs could create multiple watchers for the same apiURL. Now only a single watcher is used.
This should reduce load on Kubernetes API server when many scrape job configs use Kubernetes service discovery.
2021-03-11 16:43:04 +02:00
Aliaksandr Valialkin
dc0cb54d41
deployment/docker: update Go builder from 1.16.0 to 1.16.1
...
See https://github.com/golang/go/issues?q=milestone%3AGo1.16.1+label%3ACherryPickApproved
2021-03-11 16:43:03 +02:00
Aliaksandr Valialkin
c0ac740f93
lib/proxy: do not show inline basic auth passwords when logging errors related to proxy_url
2021-03-11 13:43:36 +02:00
Aliaksandr Valialkin
bebcb8130c
lib/promscrape/discovery/kubernetes: localize Bookmark parsing code
...
This is a follow-up for e772d1c920
2021-03-11 13:08:08 +02:00
Aliaksandr Valialkin
971e3d83f7
docs/ExttendedPromQL.md: remove outdated doc
2021-03-11 12:41:20 +02:00
Brensted
238fe7d4e8
Update BestPractices.md ( #1123 )
...
update lists, hyperlinks fixed.
2021-03-11 11:39:14 +02:00
Aliaksandr Valialkin
e772d1c920
lib/promscrape/discovery/kubernetes: reduce load on Kubernetes API server by using watch bookmarks
...
This allows continuing object watch from the last bookbark instead of reloading all the objects
on watch errors or timeouts.
See https://kubernetes.io/docs/reference/using-api/api-concepts/#watch-bookmarks
2021-03-10 15:06:35 +02:00
Aliaksandr Valialkin
bc5c4add89
lib/httpserver: export vm_available_memory_bytes and vm_available_cpu_cores metrics
...
These metrics are useful for tracking the available memory and CPU cores for VictoriaMetrics apps.
2021-03-10 12:02:42 +02:00
Brensted
e02265cfa7
Add files via upload
2021-03-10 10:16:59 +02:00
Brensted
314f28fb38
Add files via upload
2021-03-10 10:14:34 +02:00
Brensted
45ede6ba98
Add files via upload
2021-03-10 10:08:38 +02:00
Brensted
09dc49e942
Add files via upload
2021-03-10 10:06:49 +02:00
Brensted
96dbe9bcbd
Add files via upload
2021-03-10 10:05:18 +02:00
Ihor Borodin
d212f35ae8
Fixing examples of external.alert.source in documentation ( #1120 )
...
* Fixing examples of external.alert.source in documentation
2021-03-09 20:49:50 +00:00
Aliaksandr Valialkin
48fb0d1c4b
vendor: update github.com/VictoriaMetrics/fasthttp from v1.0.13 to v1.0.14
2021-03-09 21:34:07 +02:00
Aliaksandr Valialkin
2728b25783
docs/CHANGELOG.md: mention about the bugfix from 787242d7b0
2021-03-09 20:56:06 +02:00
Aliaksandr Valialkin
787242d7b0
lib/proxy: pass proxy hostname in Host header of the CONNECT request
...
This should resolve the following issue when connecting to tls proxy:
cannot validate certificate for ... because it doesn't contain any IP SANs
2021-03-09 20:39:40 +02:00
Aliaksandr Valialkin
36fd007247
lib/proxy: set missing ServerName in TLS config for proxy_url.
...
While at it, allow setting Proxy-Authorization for `proxy_url` via `basic_auth` and `bearer_token` configs.
2021-03-09 18:58:18 +02:00
Nikolay
ad34f42467
Changes tlsConfig init for proxy connections ( #1121 )
...
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1116
2021-03-09 18:51:00 +02:00
Aliaksandr Valialkin
3fd8653b40
lib/promscrape: apply sample_limit after metric relabeling is applied as Prometheus does
...
See the description for `sample_limit` option from Prometheus docs:
Per-scrape limit on number of scraped samples that will be accepted.
If more than this number of samples are present after metric relabeling
the entire scrape will be treated as failed. 0 means no limit.
https://prometheus.io/docs/prometheus/latest/configuration/configuration/#scrape_config
2021-03-09 15:47:18 +02:00
Aliaksandr Valialkin
4b18a4f026
lib/promscrape/discovery/kubernetes: remove too verbose logs about starting and stopping the watchers
...
Log the number of objects loaded per each watch url
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1113
2021-03-09 15:05:14 +02:00
Aliaksandr Valialkin
e44532b760
docs/CHANGELOG.md: mention about improved query performance at 18fe0ff14b
2021-03-09 13:12:25 +02:00
Aliaksandr Valialkin
fe8b12fbad
app/vmselect/promql: follow up for 433fff0006
2021-03-09 12:48:44 +02:00
Nikolay
433fff0006
duplicate timeseries fix for prometheus_buckets function ( #1119 )
...
* try fix for prometheus_buckets
* merge possible end of the bucket collision
2021-03-09 12:26:23 +02:00
Aliaksandr Valialkin
534944b671
vendor: make vendor-update
2021-03-09 12:01:55 +02:00
Aliaksandr Valialkin
f6878eac36
vendor: update github.com/VictoriaMetrics/fasthttp from 1.0.12 to 1.0.13
...
This should fix a bug in vmagent with high CPU usage during failed scrapes with small `scrape_timeout`.
2021-03-09 11:45:02 +02:00
John Belmonte
364fdf4a56
spelling fix: adjacent ( #1115 )
2021-03-09 09:18:19 +02:00
Aliaksandr Valialkin
14a399dd06
lib/promscrape: add scrape_offset option to scrape_config
...
This option can be used for specifying the particular offset per each scrape interval for target scraping
2021-03-08 12:03:33 +02:00
Aliaksandr Valialkin
345980f78f
lib/storage: go fmt
2021-03-08 12:03:31 +02:00
Aliaksandr Valialkin
18fe0ff14b
lib/storage: tune loopsCount estimations in getMetricIDsForTagFilterSlow
...
The adjusted estmations give up to 2x lower median response times on 200qps /api/v1/query_range workload
2021-03-07 21:12:35 +02:00
Aliaksandr Valialkin
ab4f090c63
lib/promscrape/discovery/kubernetes: reduce memory usage further when big number of scrape jobs are configured for the same kubernetes_sd_config role
...
Serialize reloading per-role objects, so they don't occupy too much memory when objects for many scrape jobs are simultaneously refreshed.
Do not reload per-role objects if they were already refreshed by concurrent goroutines. This should reduce load on Kubernetes API server
when big number of scrape jobs are configured for the same Kubernetes role.
This is a follow-up for 17b87725edhttps://github.com/VictoriaMetrics/VictoriaMetrics/issues/1113
2021-03-07 19:51:03 +02:00
Aliaksandr Valialkin
1187ee5e16
lib/decimal: prevent exponent overflow when processing values close to zero
...
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1114
2021-03-05 18:52:47 +02:00
Aliaksandr Valialkin
47ac2051bb
app/vmauth: allow using regexps in url_map paths
...
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1112
2021-03-05 18:21:36 +02:00
