Commit graph

3050 commits

Author SHA1 Message Date
Aliaksandr Valialkin
72de54f93e lib/promauth: cache the client TLS certificate for up to a second
This should reduce CPU usage when TLS connections are established at a high rate.
2021-07-02 13:20:18 +03:00
Aliaksandr Valialkin
a12c6a52c9 vendor: update github.com/VictoriaMetrics/fasthttp from v1.0.15 to v1.0.16
This should help with proper copying of tls.Config struct after it has been set up in lib/promauth.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1420
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/470
2021-07-02 12:05:23 +03:00
Aliaksandr Valialkin
4b4e69791f docs/Cluster-VictoriaMetrics.md: typo fix: siplify -> simplify 2021-07-02 10:50:26 +03:00
Aliaksandr Valialkin
9ec08213be docs/Cluster-VictoriaMetrics.md: add a chapter describing a toy cluster setup on a single host
While at it, refer to available tools, which can simplify cluster setup
2021-07-02 10:48:13 +03:00
Aliaksandr Valialkin
1c12c0f79c lib/promauth: reload TLS certificates from disk on every mTLS connection as Prometheus does
This allows updating client certificates without the need to restart vmagent and/or single-node VictoriaMetrics.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1420
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/470
2021-07-01 15:43:43 +03:00
Nikolay
6bd2309449 fixes /targets button style (#1423)
* fixes /targets button style
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1422

* updates boostrap version
2021-07-01 11:52:47 +03:00
Aliaksandr Valialkin
c1b15c7764 docs/CHANGELOG.md: document ae485c2bfd 2021-07-01 11:52:47 +03:00
Aliaksandr Valialkin
71c856beb8 lib/workingsetcache: reset the cache mode when the cache is reset
This should reduce memory usage if the working set is reduced after the cache reset.
2021-07-01 11:52:47 +03:00
Aliaksandr Valialkin
225ec527df deployment/docker: upgrade alpine image from v3.13.5 to v3.14.0 2021-07-01 10:57:32 +03:00
Roman Khavronenko
c6cf821600
dashboard: several minor fixes (#1418)
* move panel `Disk writes/reads` to `Resource usage` row
* rename row `storage` to `vmstorage`
* remove cumulative display for `Storage ETA` panel
2021-07-01 05:45:35 +03:00
Aliaksandr Valialkin
bced9ee666 lib/{mergeset,storage}: reduce the maximum lifetime for cached indexdb and data blocks from 2 minutes to a minute
This should reduce memory usage on a system with high number of active time series and a high churn rate.
One minute is enough for caching the blocks needed for repeated queries (e.g. alerting rules, recording rules and dashboard refreshes).
2021-06-29 19:57:53 +03:00
Aliaksandr Valialkin
b7c0b3dde3 lib/mergeset: switch from sync.Pool to a channel for a pool for inmemoryBlock structs
This should reduce memory usage for the pool on systems with big number of CPU cores.

The sync.Pool maintains per-CPU pools, so the total number of objects in the pool
is proportional to the number of available CPU cores. The channel limits the number
of pooled objects by its own capacity. This means smaller number of pooled objects on average.
2021-06-29 19:57:52 +03:00
Aliaksandr Valialkin
2edfea8c36 lib/promscrape/discovery/docker: fix golint warning: struct field Id should be ID 2021-06-29 13:11:33 +03:00
Aliaksandr Valialkin
609ad6d9bf lib/storage: put indexDBName into the key for dateTagFilter cache and for uselessTagFilters cache
This should prevent from stats overwriting when the previous indexdb is queried.
2021-06-29 13:11:32 +03:00
Aliaksandr Valialkin
0c4c630839 lib/promscrape: typo fix in /targets output
The typo has been introduced in fb72a2133f

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1408
2021-06-28 21:27:22 +03:00
Aliaksandr Valialkin
4d8ab5d9fa docs/vmagent.md: mention about docker_sd_config support 2021-06-25 20:53:09 +03:00
Aliaksandr Valialkin
db66ea9a5f docs/CHANGELOG.md: cut v1.62.0 2021-06-25 13:29:48 +03:00
Aliaksandr Valialkin
97d1ccfc8e lib/promscrape: split docker and dockerswarm service discovery code bases, since they have very little in common
This is a follow up after c85a5b7fcb
2021-06-25 13:22:16 +03:00
Aliaksandr Valialkin
4461e20e7d lib/promscrape: consistently sort service discovery routines
This should simplify further maintenance of the code
2021-06-25 13:22:16 +03:00
Lu Jiajing
12b4cbb68f Support Docker ServiceDiscovery (#1402)
* add docker discovery

* add test

* add labels test and add scrape work

* remove TODO

* refactor to merge apiConfig and sdConfig

* apply suggestion
2021-06-25 13:22:16 +03:00
Nikolay
501429c3ff adds missing MustStop call to do and http sd (#1404) 2021-06-25 11:43:32 +03:00
Aliaksandr Valialkin
f1a2aa0992 vendor: make vendor-update 2021-06-24 17:28:37 +03:00
Aliaksandr Valialkin
01af676436 docs: consistently put the link to articles and slides about VictoriaMetrics after the links to case studies 2021-06-24 15:38:49 +03:00
Aliaksandr Valialkin
634f2128d8 docs/CaseStudies.md: add a case study for DFKI 2021-06-24 15:25:19 +03:00
Aliaksandr Valialkin
b054563703 Add case study for Groove X 2021-06-24 15:06:19 +03:00
Aliaksandr Valialkin
65576ebb5b docs/CaseStudies.md: add Sensedia case study 2021-06-24 14:35:47 +03:00
Aliaksandr Valialkin
1b6850bab3 docs/CHANGELOG.md: document the bugfix in increase_pure() function from the commit fb4f758715 2021-06-24 12:06:15 +03:00
Aliaksandr Valialkin
b84aea1e6e lib/protoparser/clusternative: do not pool unmarshalWork structs, since they can occupy big amounts of memory (more than 100MB per each struct)
This should reduce memory usage for vmstorage under high ingestion rate when the vmstorage runs on a system with big number of CPU cores
2021-06-23 15:45:08 +03:00
Aliaksandr Valialkin
856aecae05 app/vmselect/promql: return the last timestamp for the max / min value from tmax_over_time() and tmin_over_time() function as most users expect 2021-06-23 14:18:37 +03:00
Aliaksandr Valialkin
dffa2afefe docs/CHANGELOG.md: document the bugfix for incorrect stats collection for concurrently executed tag filter
Follow up for c22114c6f0
2021-06-23 14:06:33 +03:00
Aliaksandr Valialkin
c18017a9c3 app/vminsert/netstorage: sort the -storageNode list passed to vminsert nodes
This should reduce resource usage (CPU, RAM, disk IO) at vmstorage nodes
if the addresses of vmstorage nodes are passed in random order to vminsert nodes.
2021-06-23 14:00:08 +03:00
Aliaksandr Valialkin
a22f37599b lib/storage: tune tag filters search logic
Tune the logic according to the logs provided at https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1338#issuecomment-864293624

The previous logic had a race when multiple concurrent queries execute the same tag filter without prior stats.
This could result in incorrectly stored stats for such tag filter, which then could result in non-optimal sorting of tag filters
for further queries.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1338
2021-06-23 13:30:36 +03:00
Aliaksandr Valialkin
f10fa0d1d7 lib/promscrape/discovery/consul: properly pass namespace to Consul watcher
Follow-up for 58a2989fe7
2021-06-22 17:43:20 +03:00
Aliaksandr Valialkin
4adf6c9766 lib/promscrape/discovery/http: follow up after e307bbb29a 2021-06-22 13:42:10 +03:00
Nikolay
e03a3d3a36 adds http_sd (#1399)
* adds http_sd

* adds X-Prometheus-Refresh-Interval-Seconds header

* Update lib/promscrape/discovery/http/api.go

Co-authored-by: Aliaksandr Valialkin <valyala@gmail.com>
2021-06-22 13:42:09 +03:00
Aliaksandr Valialkin
3ab3902f17 lib/promscrape/discovery: support generic auth configs in Consul service discovery in the same way as Prometheus 2.28 does 2021-06-22 13:18:51 +03:00
Aliaksandr Valialkin
61fc9b98e5 docs/CHANGELOG.md: document the support for Consul namepsace
See 58a2989fe7
2021-06-22 12:56:12 +03:00
Nikolay
827a2396d2 adds consul enterprise namespace support (#1400)
* adds consul enterprise namespace support

* Update lib/promscrape/discovery/consul/consul.go

Co-authored-by: Aliaksandr Valialkin <valyala@gmail.com>
2021-06-22 12:56:11 +03:00
Aliaksandr Valialkin
a575882ca2 docs/PerTenantStatistic.md: document that the per-tenant statistic is a part of cluster version of VictoriaMetrics 2021-06-22 12:43:30 +03:00
Roman Khavronenko
79474baf99 vmctl: add more context to flags description in vm-native mode (#1395) 2021-06-18 19:20:52 +03:00
Aliaksandr Valialkin
23fcafd437 docs/CHANGELOG.md: typo fixes 2021-06-18 19:15:29 +03:00
Aliaksandr Valialkin
b92d110cad app/vmselect: log slow requests to all the /api/v1/* handlers if their execution time exceeds -search.logSlowQueryDuration 2021-06-18 19:07:03 +03:00
Aliaksandr Valialkin
d29e130181 docs/Single-server-VictoriaMetrics.md: mention that it is recommended to use a single scrape_interval across all the scrape targets 2021-06-18 15:41:35 +03:00
Aliaksandr Valialkin
4acc4602b3 app/vmctl: limit JSON line size by 10K samples (#1394)
This should reduce the maximum memory usage at VictoriaMetrics when importing time series with big number of samples.
2021-06-18 15:41:34 +03:00
Aliaksandr Valialkin
3ec3705943 docs/Cluster-VictoriaMetrics.md: clarify docs about VictoriaMetrics cluster architecture 2021-06-18 14:35:55 +03:00
Aliaksandr Valialkin
cd697b88c5 docs/CHANGELOG.md: document the reduced disk write IO usage
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1338
2021-06-18 14:03:42 +03:00
Aliaksandr Valialkin
f9069ba32a lib/promscrape: show jobs with empty scrape targets on /targets page 2021-06-18 10:54:12 +03:00
Roman Khavronenko
5cb378f5b5
dasbhoard: display tweaks (#1387)
* rm cumulative visualisation for panel `Disk space used`.
It uses % threshold and cumulative display breaks it.
* remove area filling for resource usage row;
* add job name for panels in resource usage row.
2021-06-18 10:48:40 +03:00
Nikolay
9ea1dca3dd fixes DO service discovery labels (#1389)
adds test for digitalocean sd
2021-06-17 17:21:10 +03:00
Aliaksandr Valialkin
60bc35f550 docs/{vmgateway,vmbackupmanager}: explicitly mention that these components are a part of an enterprise package 2021-06-17 17:19:13 +03:00