Commit graph

702 commits

Author SHA1 Message Date
Aliaksandr Valialkin
8d8e2ccf5f Merge branch 'public-single-node' into pmm-6401-read-prometheus-data-files 2020-11-04 19:10:41 +02:00
Aliaksandr Valialkin
b2042a1c30 lib/promscrape: docs update after e4182dd896
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/878
2020-11-04 17:12:30 +02:00
Aliaksandr Valialkin
caeb74f068 app/vmselect: reduce memory usage when query touches big number of time series 2020-11-04 17:04:04 +02:00
Aliaksandr Valialkin
ae91a6883c lib/{storage,mergeset}: clean cached index blocks and inmemory blocks more aggressively
Previously such blocks were cleaned after they weren't accessed during 10 minutes.
Now they are cleaned after one minute of missing access. This should reduce memory usage in general case.
2020-11-04 17:04:04 +02:00
Nikolay
e4182dd896
reduces memory usage for vmagent, (#880)
* reduces memory usage for vmagent,
limits count of droppedTarget, that can be stored for /api/v1/targets page up to 999 items,
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/878

* Update app/vmagent/README.md

* Update app/vmagent/README.md

Co-authored-by: Aliaksandr Valialkin <valyala@gmail.com>
2020-11-04 17:03:43 +02:00
Aliaksandr Valialkin
b9e5172aa2 Revert "lib/promscrape: add -promscrape.dropOriginalLabels command-line flag for reducing memory usage when discovering big number of scrape targets"
This reverts commit 5a9743211f.
2020-11-04 11:45:02 +02:00
Aliaksandr Valialkin
5a9743211f lib/promscrape: add -promscrape.dropOriginalLabels command-line flag for reducing memory usage when discovering big number of scrape targets
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/878
2020-11-04 11:08:57 +02:00
Aliaksandr Valialkin
ca8b5745b5 lib/promscrape: reduce memory allocations in promLabelsString() function
This should help with reducing memory usage in https://github.com/VictoriaMetrics/VictoriaMetrics/issues/878
2020-11-04 10:38:44 +02:00
Aliaksandr Valialkin
f3f62ab04e lib/storage: do not report about the need of free disk space if parts cannot be merged due to too big write amplification 2020-11-03 15:32:02 +02:00
Aliaksandr Valialkin
c87fb9191e lib/storage: remove unneeded fmt.Sprintf 2020-11-03 14:20:31 +02:00
Aliaksandr Valialkin
8772cb617c Merge branch 'public-single-node' into pmm-6401-read-prometheus-data-files 2020-11-03 14:15:45 +02:00
John Belmonte
51e661ecfe
add short_version label to vm_app_version metric (#877)
* add short_version label to vm_app_version metric

use case:  Version panel of Grafana dashboard should use a live query, but currently it uses a template query which becomes stale.  Grafana is not able to preform regex substitution on labels.

* Update metrics.go

* fix compile
2020-11-03 14:10:42 +02:00
Aliaksandr Valialkin
1b389674c0 Merge branch 'public-single-node' into pmm-6401-read-prometheus-data-files 2020-11-02 21:58:57 +02:00
Aliaksandr Valialkin
fe289331dd lib/storage: remove obsolete code 2020-11-02 19:11:59 +02:00
Aliaksandr Valialkin
98529e16ee Merge branch 'public-single-node' into pmm-6401-read-prometheus-data-files 2020-11-02 02:12:19 +02:00
Aliaksandr Valialkin
c0f3be824d lib/promscrape: properly handle response body after 301 redirect
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/869
2020-11-02 01:09:52 +02:00
Aliaksandr Valialkin
1b112405a8 Merge branch 'public-single-node' into pmm-6401-read-prometheus-data-files 2020-11-02 00:47:34 +02:00
Aliaksandr Valialkin
e277c3d07b lib/promscrape: add stream parse mode for efficient scraping of targets that expose millions of metrics 2020-11-01 23:35:06 +02:00
Aliaksandr Valialkin
29e4e7f422 lib/storage: drop more samples outside the given retention during background merge
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/17
2020-11-01 23:35:06 +02:00
Aliaksandr Valialkin
b7638f04a7 app/vmagent: expose /api/v1/targets page according to https://prometheus.io/docs/prometheus/latest/querying/api/#targets
This page is exposed by vmagent and by a single-node VictoriaMetrics

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/643
2020-11-01 23:35:06 +02:00
Aliaksandr Valialkin
d12c4914f0 lib/storage: properly handle the case when key="__name__" is passed to MetricName.AddTag* 2020-11-01 23:35:06 +02:00
Aliaksandr Valialkin
64e2d66014 lib/storage: code cleanup after 5bfd4e6218 2020-11-01 23:35:06 +02:00
Nikolay
67059caa12
fixes panic at scrape error body formating, (#868)
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/864
regression after body reuse improvements
2020-10-29 17:17:52 +03:00
Nikolay
de3fe22815
adds leading forward slash check for scrapeURL path (#855)
* fixes in-consistency with prometheus behaviour for scrape targets url path.
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/835
2020-10-29 08:39:42 +03:00
Aliaksandr Valialkin
0729cc36b2 lib/memory: do not print trailing zeroes in logs for -memory.allowedPercent command-line flag 2020-10-20 14:32:07 +03:00
Aliaksandr Valialkin
5bfd4e6218 app/vmstorage: support for -retentionPeriod smaller than one month
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/173
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/17
2020-10-20 14:31:44 +03:00
faceair
84e3881c0b
disable response compression on websocket (#841) 2020-10-17 13:32:34 +03:00
Aliaksandr Valialkin
8bbc83e85e Merge branch 'public-single-node' into pmm-6401-read-prometheus-data-files 2020-10-17 12:13:56 +03:00
Aliaksandr Valialkin
01987f8c77 lib/storage: small code adjustements after d2960a20e0
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/781
2020-10-17 01:16:54 +03:00
faceair
d2960a20e0
evaluate the execution cost of all tag filters (#824)
* evaluate the execution cost of all tag filters

* fix suffixes typo
2020-10-17 00:46:55 +03:00
Nikolay Khramchikhin
e6ab69dd88
fixes openstack api endpoint with suffix trim adds openstack (#840)
api v2.0 check
2020-10-16 21:20:57 +03:00
Aliaksandr Valialkin
4dc13754d8 Merge branch 'public-single-node' into pmm-6401-read-prometheus-data-files 2020-10-13 18:37:48 +03:00
Aliaksandr Valialkin
63c4999e06 lib/promscrape: code prettifying after 9bd9f67718 2020-10-12 16:12:36 +03:00
Nikolay Khramchikhin
9bd9f67718
Adds dockerswarm sd (#818)
* adds dockerswarm service discovery

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/656

 Following roles supported: services, tasks and nodes.
 Basic, token and tls auth supported.
 Added tests for labels generation.

* added unix socket support to discovery utils

Co-authored-by: Aliaksandr Valialkin <valyala@gmail.com>
2020-10-12 13:38:21 +03:00
Aliaksandr Valialkin
b449607181 lib/backup: add MustStop() method for all remote filesystems 2020-10-09 15:32:19 +03:00
Aliaksandr Valialkin
cf5f2874cd lib/backup/fslocal: add FS.MustStop() method for stopping bandwidth limiter 2020-10-09 15:12:03 +03:00
Aliaksandr Valialkin
68f0e00761 app/vmstorage: add vm_rows_added_to_storage_total metric, which shows the total number of rows added to storage since app start 2020-10-09 13:35:48 +03:00
Aliaksandr Valialkin
83b7eb8ca6 Merge branch 'public-single-node' into pmm-6401-read-prometheus-data-files 2020-10-09 12:36:16 +03:00
Aliaksandr Valialkin
561a7619a5 lib/promscrape: fix tests after 71ea4935de 2020-10-08 19:32:36 +03:00
Aliaksandr Valialkin
e5ef3288dd Merge branch 'public-single-node' into pmm-6401-read-prometheus-data-files 2020-10-08 19:27:30 +03:00
Aliaksandr Valialkin
71ea4935de lib/promscrape: add -promscrape.suppressDuplicateScrapeTargetErrors command-line flag in order to suppress duplicate scrape target errors
Show also original labels for duplicate targets in error message in order to simplify debugging the issue.

Now `/targets` endpoint accepts optional `show_original_labels=1` query arg, which shows original labels for each target.
This may simplify debugging for target relabeling.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/651
2020-10-08 18:58:30 +03:00
Aliaksandr Valialkin
9b0a5c1028 lib/backup/actions: improve logging to be more clear to humans 2020-10-08 14:23:07 +03:00
Aliaksandr Valialkin
e7f2907138 Merge branch 'public-single-node' into pmm-6401-read-prometheus-data-files 2020-10-07 21:40:45 +03:00
Aliaksandr Valialkin
af90b3121c app/vmstorage: add -finalMergeDelay command-line flag for configuring the delay before final merge for per-month partitions after no new data is ingested to it 2020-10-07 17:35:44 +03:00
Aliaksandr Valialkin
317ddb84b9 Merge branch 'public-single-node' into pmm-6401-read-prometheus-data-files 2020-10-06 16:18:01 +03:00
Aliaksandr Valialkin
e19d400230 lib/protoparser/graphite: support parsing floating-point timestamp like Graphite does
Such timestamps are rounded to seconds like Carbon does.
See b0ba62a62d/lib/carbon/protocols.py (L197)
2020-10-06 11:38:29 +03:00
Aliaksandr Valialkin
90aa2a8ffd lib/promscrape/discovery/openstack: show expiration time for refreshed OpenStack token in seconds - this is easier to interpret by human 2020-10-06 11:34:09 +03:00
Aliaksandr Valialkin
40d2f6fee4 Merge branch 'public-single-node' into pmm-6401-read-prometheus-data-files 2020-10-05 18:14:51 +03:00
Aliaksandr Valialkin
aba899c298 lib/promscrape/discovery/openstack: code prettifying after cbe3cf683b
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/728
2020-10-05 18:11:55 +03:00
Nikolay Khramchikhin
cbe3cf683b
Adds openstack sd (#811)
* adds openstack service discovery

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/728

 implemented hypervisors and instance discovery with openstack v3 api.
 Added tests for labeling and data parsing.
 Added token refresh.

* Apply suggestions from code review

* Apply suggestions from code review

Co-authored-by: Aliaksandr Valialkin <valyala@gmail.com>
2020-10-05 16:45:33 +03:00