github-mirrors/VictoriaMetrics
1
0
Fork 0
mirror of https://github.com/VictoriaMetrics/VictoriaMetrics.git synced 2024-11-21 14:44:00 +00:00
Code Issues Projects Releases Packages Wiki Activity
1852 commits 150 branches 748 tags 347 MiB
Commit graph

685 commits

Author SHA1 Message Date
Nikolay
4c808d58bf
Adds ready probe (#874) 
* adds leading forward slash check for scrapeURL path
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/835

* adds ready probe for scrape config initialization,
it should prevent metrics loss during vmagent rolling update,
/ready api will return 425 http code, if some scrape config still waits for initialization.

* updates docs

* Update app/vmagent/README.md

* renames var

* Update app/vmagent/README.md

Co-authored-by: Aliaksandr Valialkin <valyala@gmail.com>
 2020-11-04 20:29:18 +02:00
Aliaksandr Valialkin
b2042a1c30 lib/promscrape: docs update after e4182dd896 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/878
 2020-11-04 17:12:30 +02:00
Aliaksandr Valialkin
caeb74f068 app/vmselect: reduce memory usage when query touches big number of time series 2020-11-04 17:04:04 +02:00
Aliaksandr Valialkin
ae91a6883c lib/{storage,mergeset}: clean cached index blocks and inmemory blocks more aggressively 
Previously such blocks were cleaned after they weren't accessed during 10 minutes.
Now they are cleaned after one minute of missing access. This should reduce memory usage in general case.
 2020-11-04 17:04:04 +02:00
Nikolay
e4182dd896
reduces memory usage for vmagent, (#880) 
* reduces memory usage for vmagent,
limits count of droppedTarget, that can be stored for /api/v1/targets page up to 999 items,
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/878

* Update app/vmagent/README.md

* Update app/vmagent/README.md

Co-authored-by: Aliaksandr Valialkin <valyala@gmail.com>
 2020-11-04 17:03:43 +02:00
Aliaksandr Valialkin
b9e5172aa2 Revert "lib/promscrape: add -promscrape.dropOriginalLabels command-line flag for reducing memory usage when discovering big number of scrape targets" 
This reverts commit 5a9743211f.
 2020-11-04 11:45:02 +02:00
Aliaksandr Valialkin
5a9743211f lib/promscrape: add -promscrape.dropOriginalLabels command-line flag for reducing memory usage when discovering big number of scrape targets 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/878
 2020-11-04 11:08:57 +02:00
Aliaksandr Valialkin
ca8b5745b5 lib/promscrape: reduce memory allocations in promLabelsString() function 
This should help with reducing memory usage in https://github.com/VictoriaMetrics/VictoriaMetrics/issues/878
 2020-11-04 10:38:44 +02:00
Aliaksandr Valialkin
f3f62ab04e lib/storage: do not report about the need of free disk space if parts cannot be merged due to too big write amplification 2020-11-03 15:32:02 +02:00
Aliaksandr Valialkin
c87fb9191e lib/storage: remove unneeded fmt.Sprintf 2020-11-03 14:20:31 +02:00
John Belmonte
51e661ecfe
add short_version label to vm_app_version metric (#877) 
* add short_version label to vm_app_version metric

use case:  Version panel of Grafana dashboard should use a live query, but currently it uses a template query which becomes stale.  Grafana is not able to preform regex substitution on labels.

* Update metrics.go

* fix compile
 2020-11-03 14:10:42 +02:00
Aliaksandr Valialkin
fe289331dd lib/storage: remove obsolete code 2020-11-02 19:11:59 +02:00
Aliaksandr Valialkin
c0f3be824d lib/promscrape: properly handle response body after 301 redirect 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/869
 2020-11-02 01:09:52 +02:00
Aliaksandr Valialkin
e277c3d07b lib/promscrape: add stream parse mode for efficient scraping of targets that expose millions of metrics 2020-11-01 23:35:06 +02:00
Aliaksandr Valialkin
29e4e7f422 lib/storage: drop more samples outside the given retention during background merge 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/17
 2020-11-01 23:35:06 +02:00
Aliaksandr Valialkin
b7638f04a7 app/vmagent: expose /api/v1/targets page according to https://prometheus.io/docs/prometheus/latest/querying/api/#targets 
This page is exposed by vmagent and by a single-node VictoriaMetrics

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/643
 2020-11-01 23:35:06 +02:00
Aliaksandr Valialkin
d12c4914f0 lib/storage: properly handle the case when key="__name__" is passed to MetricName.AddTag* 2020-11-01 23:35:06 +02:00
Aliaksandr Valialkin
64e2d66014 lib/storage: code cleanup after 5bfd4e6218 2020-11-01 23:35:06 +02:00
Nikolay
67059caa12
fixes panic at scrape error body formating, (#868) 
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/864
regression after body reuse improvements
 2020-10-29 17:17:52 +03:00
Nikolay
de3fe22815
adds leading forward slash check for scrapeURL path (#855) 
* fixes in-consistency with prometheus behaviour for scrape targets url path.
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/835
 2020-10-29 08:39:42 +03:00
Aliaksandr Valialkin
0729cc36b2 lib/memory: do not print trailing zeroes in logs for -memory.allowedPercent command-line flag 2020-10-20 14:32:07 +03:00
Aliaksandr Valialkin
5bfd4e6218 app/vmstorage: support for -retentionPeriod smaller than one month 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/173
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/17
 2020-10-20 14:31:44 +03:00
faceair
84e3881c0b
disable response compression on websocket (#841) 2020-10-17 13:32:34 +03:00
Aliaksandr Valialkin
01987f8c77 lib/storage: small code adjustements after d2960a20e0 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/781
 2020-10-17 01:16:54 +03:00
faceair
d2960a20e0
evaluate the execution cost of all tag filters (#824) 
* evaluate the execution cost of all tag filters

* fix suffixes typo
 2020-10-17 00:46:55 +03:00
Nikolay Khramchikhin
e6ab69dd88
fixes openstack api endpoint with suffix trim adds openstack (#840) 
api v2.0 check
 2020-10-16 21:20:57 +03:00
Aliaksandr Valialkin
63c4999e06 lib/promscrape: code prettifying after 9bd9f67718 2020-10-12 16:12:36 +03:00
Nikolay Khramchikhin
9bd9f67718
Adds dockerswarm sd (#818) 
* adds dockerswarm service discovery

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/656

 Following roles supported: services, tasks and nodes.
 Basic, token and tls auth supported.
 Added tests for labels generation.

* added unix socket support to discovery utils

Co-authored-by: Aliaksandr Valialkin <valyala@gmail.com>
 2020-10-12 13:38:21 +03:00
Aliaksandr Valialkin
b449607181 lib/backup: add MustStop() method for all remote filesystems 2020-10-09 15:32:19 +03:00
Aliaksandr Valialkin
cf5f2874cd lib/backup/fslocal: add FS.MustStop() method for stopping bandwidth limiter 2020-10-09 15:12:03 +03:00
Aliaksandr Valialkin
68f0e00761 app/vmstorage: add vm_rows_added_to_storage_total metric, which shows the total number of rows added to storage since app start 2020-10-09 13:35:48 +03:00
Aliaksandr Valialkin
561a7619a5 lib/promscrape: fix tests after 71ea4935de 2020-10-08 19:32:36 +03:00
Aliaksandr Valialkin
71ea4935de lib/promscrape: add -promscrape.suppressDuplicateScrapeTargetErrors command-line flag in order to suppress duplicate scrape target errors 
Show also original labels for duplicate targets in error message in order to simplify debugging the issue.

Now `/targets` endpoint accepts optional `show_original_labels=1` query arg, which shows original labels for each target.
This may simplify debugging for target relabeling.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/651
 2020-10-08 18:58:30 +03:00
Aliaksandr Valialkin
9b0a5c1028 lib/backup/actions: improve logging to be more clear to humans 2020-10-08 14:23:07 +03:00
Aliaksandr Valialkin
af90b3121c app/vmstorage: add -finalMergeDelay command-line flag for configuring the delay before final merge for per-month partitions after no new data is ingested to it 2020-10-07 17:35:44 +03:00
Aliaksandr Valialkin
e19d400230 lib/protoparser/graphite: support parsing floating-point timestamp like Graphite does 
Such timestamps are rounded to seconds like Carbon does.
See b0ba62a62d/lib/carbon/protocols.py (L197)
 2020-10-06 11:38:29 +03:00
Aliaksandr Valialkin
90aa2a8ffd lib/promscrape/discovery/openstack: show expiration time for refreshed OpenStack token in seconds - this is easier to interpret by human 2020-10-06 11:34:09 +03:00
Aliaksandr Valialkin
aba899c298 lib/promscrape/discovery/openstack: code prettifying after cbe3cf683b 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/728
 2020-10-05 18:11:55 +03:00
Nikolay Khramchikhin
cbe3cf683b
Adds openstack sd (#811) 
* adds openstack service discovery

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/728

 implemented hypervisors and instance discovery with openstack v3 api.
 Added tests for labeling and data parsing.
 Added token refresh.

* Apply suggestions from code review

* Apply suggestions from code review

Co-authored-by: Aliaksandr Valialkin <valyala@gmail.com>
 2020-10-05 16:45:33 +03:00
Aliaksandr Valialkin
f42194d817 lib/promrelabel: make a copy of label with new name for action: labelmap in the same way as Prometheus does 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/812
 2020-10-05 16:19:19 +03:00
Aliaksandr Valialkin
bbeac0ba46 lib/protoparser/influx: add -influx.maxLineSize command-line flag for configuring the maximum size for a single Influx line during parsing 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/807
 2020-10-05 15:19:05 +03:00
Aliaksandr Valialkin
47db9bb24a lib/decimal: add tests for negative values passed to maxUpExponent 2020-10-05 14:56:45 +03:00
Aliaksandr Valialkin
bc7d67cee2 lib/decimal: properly calibrate scale for blocks with Inf values 
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/805
 2020-10-05 14:52:44 +03:00
Aliaksandr Valialkin
764dc2499f lib/storage: code cleanup after 10f2eedee0 
Remove the code that uses metricIDs caches for the current and the previous hour during metricIDs search,
since this code became unused after implementing per-day inverted index almost a year ago.

While at it, fix a bug, which could prevent from finding time series with names containing dots (aka Graphite-like names
such as `foo.bar.baz`).
 2020-10-01 19:06:23 +03:00
Aliaksandr Valialkin
10f2eedee0 lib/storage: imrpove cache effectiveness for time series ids matching the given filters 
Previously the maximum cache lifetime has been limited by 10 seconds. Now it is extended up to a day.
This should reduce CPU usage in the following cases:

* when querying recently added data with small churn rate for time series
* when querying historical data
 2020-10-01 14:38:25 +03:00
Aliaksandr Valialkin
bec9b31b81 lib/storage: allow set values higher than 1 for vm_merge_need_free_disk_space if there are multiple partitions with deferred merges due to disk space shortage 2020-09-29 22:51:43 +03:00
Aliaksandr Valialkin
a9db81c4ab app/vmstorage: add metrics for determining whether background merges need additional disk space to complete 
These metrics are:

* vm_small_merge_need_free_disk_space
* vm_big_merge_need_free_disk_space

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/686
 2020-09-29 21:48:33 +03:00
Aliaksandr Valialkin
127537d631 app/vmagent/remotewrite: do not show -remoteWrite.url in logs if -remoteWrite.showURL isn't set 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/773
 2020-09-29 19:49:12 +03:00
Aliaksandr Valialkin
76b244cfcf lib/cgroup: do not adjust the number of detected CPU cores via /sys/devices/system/cpu/online 
The adjustement increases the resulting GOMAXPROC by 1, which looks confusing to users
as outlined at https://github.com/VictoriaMetrics/VictoriaMetrics/issues/685#issuecomment-698595309
 2020-09-29 13:55:26 +03:00
Aliaksandr Valialkin
1659135752 lib/storage: fix tests for 32-bit arches such as GOARCH=386 and GOARCH=arm 2020-09-29 13:10:22 +03:00