github-mirrors/VictoriaMetrics
1
0
Fork 0
mirror of https://github.com/VictoriaMetrics/VictoriaMetrics.git synced 2024-11-21 14:44:00 +00:00
Code Issues Projects Releases Packages Wiki Activity
1764 commits 150 branches 748 tags 347 MiB
Commit graph

663 commits

Author SHA1 Message Date
Nikolay Khramchikhin
a5d842caf8 fixes openstack api endpoint with suffix trim adds openstack (#840) 
api v2.0 check

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/728
 2020-10-16 23:01:45 +03:00
Aliaksandr Valialkin
938b3b7ed1 lib/promscrape: code prettifying after 9bd9f67718 2020-10-12 16:13:59 +03:00
Nikolay Khramchikhin
7f96712b38 Adds dockerswarm sd (#818) 
* adds dockerswarm service discovery

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/656

 Following roles supported: services, tasks and nodes.
 Basic, token and tls auth supported.
 Added tests for labels generation.

* added unix socket support to discovery utils

Co-authored-by: Aliaksandr Valialkin <valyala@gmail.com>
 2020-10-12 16:13:58 +03:00
Aliaksandr Valialkin
de1c07b937 lib/backup: add MustStop() method for all remote filesystems 2020-10-09 15:32:13 +03:00
Aliaksandr Valialkin
bf6d523bef lib/backup/fslocal: add FS.MustStop() method for stopping bandwidth limiter 2020-10-09 15:11:55 +03:00
Aliaksandr Valialkin
d2e917d1cb app/vmstorage: add vm_rows_added_to_storage_total metric, which shows the total number of rows added to storage since app start 2020-10-09 13:36:17 +03:00
Aliaksandr Valialkin
a93c62cd60 lib/promscrape: fix tests after 71ea4935de 2020-10-08 19:32:48 +03:00
Aliaksandr Valialkin
0d44e371f3 lib/promscrape: add -promscrape.suppressDuplicateScrapeTargetErrors command-line flag in order to suppress duplicate scrape target errors 
Show also original labels for duplicate targets in error message in order to simplify debugging the issue.

Now `/targets` endpoint accepts optional `show_original_labels=1` query arg, which shows original labels for each target.
This may simplify debugging for target relabeling.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/651
 2020-10-08 18:59:25 +03:00
Aliaksandr Valialkin
ad0950f630 lib/backup/actions: improve logging to be more clear to humans 2020-10-08 14:23:00 +03:00
Aliaksandr Valialkin
b51fa16177 app/vmstorage: add -finalMergeDelay command-line flag for configuring the delay before final merge for per-month partitions after no new data is ingested to it 2020-10-07 17:42:31 +03:00
Aliaksandr Valialkin
54ff78c6c9 lib/protoparser/graphite: support parsing floating-point timestamp like Graphite does 
Such timestamps are rounded to seconds like Carbon does.
See b0ba62a62d/lib/carbon/protocols.py (L197)
 2020-10-06 11:38:35 +03:00
Aliaksandr Valialkin
1da41177a8 lib/promscrape/discovery/openstack: show expiration time for refreshed OpenStack token in seconds - this is easier to interpret by human 2020-10-06 11:20:36 +03:00
Aliaksandr Valialkin
97b836a6f4 lib/fs: fix GOOS=openbsd build by adding fadviseSequentialRead implementation. 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/785
 2020-10-05 23:32:28 +03:00
Aliaksandr Valialkin
50316070d6 lib/promscrape/discovery/openstack: code prettifying after cbe3cf683b 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/728
 2020-10-05 18:12:31 +03:00
Nikolay Khramchikhin
b4c77fc6d2 Adds openstack sd (#811) 
* adds openstack service discovery

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/728

 implemented hypervisors and instance discovery with openstack v3 api.
 Added tests for labeling and data parsing.
 Added token refresh.

* Apply suggestions from code review

* Apply suggestions from code review

Co-authored-by: Aliaksandr Valialkin <valyala@gmail.com>
 2020-10-05 16:48:54 +03:00
Aliaksandr Valialkin
7505e4f390 lib/promrelabel: make a copy of label with new name for action: labelmap in the same way as Prometheus does 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/812
 2020-10-05 16:20:22 +03:00
Aliaksandr Valialkin
9a7c863bd8 lib/protoparser/influx: add -influx.maxLineSize command-line flag for configuring the maximum size for a single Influx line during parsing 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/807
 2020-10-05 15:19:11 +03:00
Aliaksandr Valialkin
11ddb79aeb lib/decimal: add tests for negative values passed to maxUpExponent 2020-10-05 14:57:03 +03:00
Aliaksandr Valialkin
d448e07546 lib/decimal: properly calibrate scale for blocks with Inf values 
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/805
 2020-10-05 14:52:59 +03:00
Aliaksandr Valialkin
fd7dd5064a lib/storage: code cleanup after 10f2eedee0 
Remove the code that uses metricIDs caches for the current and the previous hour during metricIDs search,
since this code became unused after implementing per-day inverted index almost a year ago.

While at it, fix a bug, which could prevent from finding time series with names containing dots (aka Graphite-like names
such as `foo.bar.baz`).
 2020-10-01 19:12:04 +03:00
Aliaksandr Valialkin
3ad7566a87 lib/storage: imrpove cache effectiveness for time series ids matching the given filters 
Previously the maximum cache lifetime has been limited by 10 seconds. Now it is extended up to a day.
This should reduce CPU usage in the following cases:

* when querying recently added data with small churn rate for time series
* when querying historical data
 2020-10-01 14:39:46 +03:00
Aliaksandr Valialkin
7c2e4e267a lib/storage: allow set values higher than 1 for vm_merge_need_free_disk_space if there are multiple partitions with deferred merges due to disk space shortage 2020-09-29 22:53:34 +03:00
Aliaksandr Valialkin
097a4c10dd app/vmstorage: add metrics for determining whether background merges need additional disk space to complete 
These metrics are:

* vm_small_merge_need_free_disk_space
* vm_big_merge_need_free_disk_space

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/686
 2020-09-29 21:47:47 +03:00
Aliaksandr Valialkin
5dca7bbe85 app/vmagent/remotewrite: do not show -remoteWrite.url in logs if -remoteWrite.showURL isn't set 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/773
 2020-09-29 19:49:19 +03:00
Aliaksandr Valialkin
5e998e597a lib/cgroup: do not adjust the number of detected CPU cores via /sys/devices/system/cpu/online 
The adjustement increases the resulting GOMAXPROC by 1, which looks confusing to users
as outlined at https://github.com/VictoriaMetrics/VictoriaMetrics/issues/685#issuecomment-698595309
 2020-09-29 13:55:53 +03:00
Aliaksandr Valialkin
338a53ccf9 lib/storage: fix tests for 32-bit arches such as GOARCH=386 and GOARCH=arm 2020-09-29 13:10:37 +03:00
Aliaksandr Valialkin
ef416c72c2 lib/storage: fix 32-bit builds for GOARH=386 or GOARCH=arm 2020-09-29 12:42:25 +03:00
Aliaksandr Valialkin
0b0259c42c lib/protoparser/prometheus: sort rows before comparing them in TestParseStream, since the order for callback calls is non-deterministic 2020-09-29 12:29:50 +03:00
Aliaksandr Valialkin
8f25206d8c lib/protoparser/prometheus: fix TestParseStream after 124f78857b 2020-09-29 12:12:33 +03:00
Aliaksandr Valialkin
81cdf2fa14 lib/{fs,filestream}: small consistency-related updates after cc90a548b1 2020-09-29 00:43:20 +03:00
Nikolay Khramchikhin
658a05ef0f added openbsd implementations (#790) 
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/785

removed fadvise for openbsd, added freespace implemenation for openbsd
 2020-09-29 00:43:19 +03:00
Aliaksandr Valialkin
1481d6d8ff lib/protoparser: avoid copying of buffer read from the network to unmarshal buffer 2020-09-28 17:19:04 +03:00
Aliaksandr Valialkin
8df33bd5c1 app/{vminsert,vmagent}: improve data ingestion speed over a single connection 
Process data obtianed from a single connection on all the available CPU cores.
 2020-09-28 04:14:51 +03:00
Aliaksandr Valialkin
7072db75cb lib/protoparser: use 64KB read buffer instead of default 4KB buffer provided by net/http.Server 
This should reduce syscall overhead when reading big amounts of data
 2020-09-28 02:07:19 +03:00
Aliaksandr Valialkin
6d8c23fdbd app/{vminsert,vmselect}: skip accountID and projectID when marshaling/unmarshaling MetricName in /api/v1/export/native and /api/v1/import/native 
This is needed in order to be able to migrate native data from/to single-node VictoriaMetrics
 2020-09-28 00:58:58 +03:00
Aliaksandr Valialkin
aadbd014ff all: add native format for data export and import 
The data can be exported via [/api/v1/export/native](https://victoriametrics.github.io/#how-to-export-data-in-native-format) handler
and imported via [/api/v1/import/native](https://victoriametrics.github.io/#how-to-import-data-in-native-format) handler.
 2020-09-27 17:36:38 +03:00
Aliaksandr Valialkin
00ec2b7189 lib/protoparser: use all the available CPU cores for processing ingested data from a single /api/v1/import stream 
Previously a single data ingestion stream to /api/v1/import could load only a single CPU core.
 2020-09-26 04:22:06 +03:00
Aliaksandr Valialkin
533bf76a12 lib/storage: correctly use maxBlockSize in various checks 
Previously `maxBlockSize` has been multiplied by 8 in certain checks. This is unnecessary.
 2020-09-24 18:13:15 +03:00
Aliaksandr Valialkin
543f3aea97 all: consistently use "%w" formatting in fmt.Errorf for wrapped errors 2020-09-23 22:48:21 +03:00
Aliaksandr Valialkin
90d2549428 lib/persistentqueue: protect from multiple concurrent opening for the same persistent queue 2020-09-23 02:17:53 +03:00
Aliaksandr Valialkin
89d652b583 lib/cgroup: attempt to obtain available CPU cores via /sys/devices/system/cpu/online 
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/685#issuecomment-674423728
 2020-09-22 23:27:26 +03:00
Aliaksandr Valialkin
31e341371b lib/storage: code prettifying after be5e1222f3 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/781
 2020-09-22 00:42:20 +03:00
faceair
ad41e39350 add filter to getMetricIDs (#783) 
* add getMetricIDs filter

* check nil filter before use
 2020-09-22 00:42:19 +03:00
Aliaksandr Valialkin
d32c3f747c lib/logger: add -loggerDisableTimestamps command-line flag for disabling timestamps in logs 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/778
 2020-09-21 19:25:50 +03:00
Aliaksandr Valialkin
f961838290 lib/promscrape/discovery/ec2: code prettifying after 312fead9a2 2020-09-21 18:44:04 +03:00
Nikolay Khramchikhin
0069353d5e Add improvements to ec2_sd_discovery (#775) 
* Add improvements to ec2 discovery

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/771

 role_arn support with aws sts
 instance iam_role support
 refreshing temporary tokens

* Apply suggestions from code review

Co-authored-by: Roman Khavronenko <hagen1778@gmail.com>

* changed implementation, removed tests, clean up code

* moved endpoint builder into getEC2APIResponse

Co-authored-by: Roman Khavronenko <hagen1778@gmail.com>
 2020-09-21 16:05:01 +03:00
Aliaksandr Valialkin
a9321f6a60 lib/storage: reduce CPU load for idle VictoriaMetrics by reducing the frequency for the need for background merges 2020-09-21 15:51:26 +03:00
Aliaksandr Valialkin
604e8f6114 lib/decimal: optimize maxUpExponent() by eliminating division from hot path 2020-09-19 13:49:43 +03:00
Aliaksandr Valialkin
df547bf345 lib/persistentqueue: sync data to file inside filestream.Writer.MustFlush 2020-09-19 12:51:46 +03:00
Aliaksandr Valialkin
778ea183ca lib/decimal: properly store Inf values 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/752
 2020-09-18 19:08:53 +03:00