github-mirrors/VictoriaMetrics
1
0
Fork 0
mirror of https://github.com/VictoriaMetrics/VictoriaMetrics.git synced 2024-11-21 14:44:00 +00:00
Code Issues Projects Releases Packages Wiki Activity
1718 commits 150 branches 748 tags 347 MiB
Commit graph

700 commits

Author SHA1 Message Date
Aliaksandr Valialkin
b51fa16177 app/vmstorage: add -finalMergeDelay command-line flag for configuring the delay before final merge for per-month partitions after no new data is ingested to it 2020-10-07 17:42:31 +03:00
Aliaksandr Valialkin
c4d309aa41 app/vmagent: add a link to https://victoriametrics.github.io/vmagent.html from main page 2020-10-06 15:30:07 +03:00
Aliaksandr Valialkin
abfd3a8fab app/{vminsert,vmselect,vmstorage}: add a link to https://victoriametrics.github.io/Cluster-VictoriaMetrics.html from main page of every cluster component 2020-10-06 15:30:07 +03:00
Nikolay Khramchikhin
b4c77fc6d2 Adds openstack sd (#811) 
* adds openstack service discovery

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/728

 implemented hypervisors and instance discovery with openstack v3 api.
 Added tests for labeling and data parsing.
 Added token refresh.

* Apply suggestions from code review

* Apply suggestions from code review

Co-authored-by: Aliaksandr Valialkin <valyala@gmail.com>
 2020-10-05 16:48:54 +03:00
Aliaksandr Valialkin
3e0c473cc9 app/vmselect/promql: fill gaps on graphs for range_* and running_* functions 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/806
 2020-10-02 14:02:47 +03:00
Aliaksandr Valialkin
fd7dd5064a lib/storage: code cleanup after 10f2eedee0 
Remove the code that uses metricIDs caches for the current and the previous hour during metricIDs search,
since this code became unused after implementing per-day inverted index almost a year ago.

While at it, fix a bug, which could prevent from finding time series with names containing dots (aka Graphite-like names
such as `foo.bar.baz`).
 2020-10-01 19:12:04 +03:00
Roman Khavronenko
368b890e11 vmalert: make maxIdleConnections configurable for datasource HTTP client (#797) 
Address https://github.com/VictoriaMetrics/VictoriaMetrics/issues/795
 2020-09-30 09:51:14 +03:00
Aliaksandr Valialkin
536aa8779a app/vmstorage: rename vm_{big|small}_merge_need_free_disk_space to vm_merge_need_free_disk_space 
This simplifies alerting.
 2020-09-29 22:53:33 +03:00
Aliaksandr Valialkin
097a4c10dd app/vmstorage: add metrics for determining whether background merges need additional disk space to complete 
These metrics are:

* vm_small_merge_need_free_disk_space
* vm_big_merge_need_free_disk_space

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/686
 2020-09-29 21:47:47 +03:00
Aliaksandr Valialkin
5dca7bbe85 app/vmagent/remotewrite: do not show -remoteWrite.url in logs if -remoteWrite.showURL isn't set 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/773
 2020-09-29 19:49:19 +03:00
Aliaksandr Valialkin
61f4801b93 app/vmselect/graphite: do not substitute path and path. with path.. in /metrics/find/?format=completer output 2020-09-29 18:03:45 +03:00
Aliaksandr Valialkin
f4d74ccfec docs/{vmbackup,vmrestore}: formatting fixes 2020-09-29 13:19:13 +03:00
Aliaksandr Valialkin
5179df7082 docs/vmbackup.md: make docs about minio config more prominent 2020-09-29 13:16:34 +03:00
Aliaksandr Valialkin
79b00f7b6b docs: improve readability a bit 2020-09-29 13:05:05 +03:00
Nikolay Khramchikhin
d77f56fd97 update vmbackup/vmrestore README usage (#794) 
* update vmbackup/vmrestore README usage

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/381

with minio and configuration file examples.

* Apply suggestions from code review

Co-authored-by: Roman Khavronenko <hagen1778@gmail.com>

* added backup/restore docs changes

* added example for relabelConfig flag

* Apply suggestions from code review

Co-authored-by: Aliaksandr Valialkin <valyala@gmail.com>
Co-authored-by: Roman Khavronenko <hagen1778@gmail.com>
 2020-09-29 13:05:04 +03:00
Aliaksandr Valialkin
0d082cdf53 app/vmselect/netstorage: remove unused code 2020-09-29 11:55:08 +03:00
Aliaksandr Valialkin
c47972d843 app/vmselect/prometheus: check for errors returned from bufferedwriter.Write 
This makes `make errcheck` happy
 2020-09-29 11:36:54 +03:00
Aliaksandr Valialkin
e66f7edfc9 app/vmselect/graphite: properly handle case when /metrics/find finds both leaf and node for the given query=prefix.* 
In this case only node must be returned with stripped dot in the end of id as carbonapi does
 2020-09-29 11:29:38 +03:00
Aliaksandr Valialkin
ffa6581c46 app/vminsert: refresh the list of healthy storage nodes only if the the row cannot be sent to destination storage node 
Previously the list had been generated for each rerouted row. This could consume additional CPU time during rerouting,
which could lead to rerouting slowdown.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/791
 2020-09-29 01:29:24 +03:00
Aliaksandr Valialkin
bc37f1cbec app/vminsert: do not pollute logs with repated cannot dial storageNode errors 
Log only the first error per -storageNode
 2020-09-29 00:20:32 +03:00
Aliaksandr Valialkin
2ee0dc27a6 app/vmstorage: parallelize data processing obtained from a single connection from vminsert 
Previously vmstorage could use only a single CPU core for data processing from a single connection from vminsert.
Now all the CPU cores can be used for data processing from a single connection from vminsert.
This should improve the maximum data ingestion performance for a single vminsert->vmstorage connection.
 2020-09-28 21:41:16 +03:00
Aliaksandr Valialkin
9d123eb22a app/vminsert: remove useless delays when sending data to vmstorage 
This improves the maximum data ingestion performance for cluster VictoriaMetrics

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/791
 2020-09-28 21:41:15 +03:00
Aliaksandr Valialkin
8df33bd5c1 app/{vminsert,vmagent}: improve data ingestion speed over a single connection 
Process data obtianed from a single connection on all the available CPU cores.
 2020-09-28 04:14:51 +03:00
Aliaksandr Valialkin
6d8c23fdbd app/{vminsert,vmselect}: skip accountID and projectID when marshaling/unmarshaling MetricName in /api/v1/export/native and /api/v1/import/native 
This is needed in order to be able to migrate native data from/to single-node VictoriaMetrics
 2020-09-28 00:58:58 +03:00
Aliaksandr Valialkin
db14f22fc0 app/vmselect: stop /api/v1/export/* execution if client disconnects 2020-09-27 23:52:42 +03:00
Aliaksandr Valialkin
aadbd014ff all: add native format for data export and import 
The data can be exported via [/api/v1/export/native](https://victoriametrics.github.io/#how-to-export-data-in-native-format) handler
and imported via [/api/v1/import/native](https://victoriametrics.github.io/#how-to-import-data-in-native-format) handler.
 2020-09-27 17:36:38 +03:00
Aliaksandr Valialkin
e83947a882 app/vminsert: code prettifying 2020-09-26 04:16:14 +03:00
Aliaksandr Valialkin
8fc9b77496 app/vmagent: reduce memory usage when importing data via /api/v1/import 
Previously vmagent could use big amounts of RAM when each ingested JSON line
contained many samples.
 2020-09-26 04:10:13 +03:00
Aliaksandr Valialkin
973df09686 app/vmselect/netstorage: do not spend CPU time on unpacking empty blocks during /api/v1/series calls 2020-09-24 20:44:15 +03:00
Aliaksandr Valialkin
b8bce348c5 app/vmselect/promql: properly limit implicitly set rollup window to -search.maxStalenessInterval 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/784
 2020-09-23 23:24:09 +03:00
Aliaksandr Valialkin
543f3aea97 all: consistently use "%w" formatting in fmt.Errorf for wrapped errors 2020-09-23 22:48:21 +03:00
Aliaksandr Valialkin
8627365b48 app/vmselect/prometheus: code cleanup after 3ba507000c 2020-09-23 13:04:51 +03:00
Aliaksandr Valialkin
1fce79518a app/vmselect/prometheus: return timestamps from /api/v1/query, which match the time query arg 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/720
 2020-09-23 12:59:23 +03:00
Aliaksandr Valialkin
0468cdf33e app/vmselect/netstorage: properly pre-allocate space for sbs 2020-09-22 23:51:01 +03:00
Aliaksandr Valialkin
69eb9783e6 app/vmselect: make sure the request doesnt wait in pending queue more than the configured timeout 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/711
 2020-09-22 01:21:42 +03:00
Aliaksandr Valialkin
ed473c94ff docs/vmagent.md: typo fix 2020-09-21 21:49:08 +03:00
Aliaksandr Valialkin
e564725641 app/vmselect/searchutils: fixed tests after 2eb72e09ab 2020-09-21 21:31:28 +03:00
Aliaksandr Valialkin
07c6226334 app/vmselect: use time value rounded to seconds if it isnt passed to /api/v1/query 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/720
 2020-09-21 21:24:46 +03:00
Nikolay Khramchikhin
0069353d5e Add improvements to ec2_sd_discovery (#775) 
* Add improvements to ec2 discovery

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/771

 role_arn support with aws sts
 instance iam_role support
 refreshing temporary tokens

* Apply suggestions from code review

Co-authored-by: Roman Khavronenko <hagen1778@gmail.com>

* changed implementation, removed tests, clean up code

* moved endpoint builder into getEC2APIResponse

Co-authored-by: Roman Khavronenko <hagen1778@gmail.com>
 2020-09-21 16:05:01 +03:00
Aliaksandr Valialkin
8cd89cb847 app/vmalert: remove unneeded UTC() call 
UTC() doesn't change the underlying timestamp, so the call isn't needed here
 2020-09-21 15:56:48 +03:00
Roman Khavronenko
d111969d39 vmalert: add support for datasource.lookback flag (#779) 
New datasource flag `datasource.lookback` defines how far to look into
past when evaluating queries.

Address https://github.com/VictoriaMetrics/VictoriaMetrics/issues/668
 2020-09-21 15:56:47 +03:00
Roman Khavronenko
0042b0f307 vmalert: fix the typo in error message (#782) 
The error will be always nil so no sense in printing it.
 2020-09-21 11:36:09 +03:00
Aliaksandr Valialkin
1c07d7bee7 app/vmagent: increase default value for -remoteWrite.queues from 1 to 4, since it has been appeared that many users hit this limit 2020-09-18 14:22:02 +03:00
Aliaksandr Valialkin
a1bebb660c app/vmselect/graphite: return proper results /metrics/find?query=foo.*.bar according to Graphite Metrics API 2020-09-18 11:53:28 +03:00
Aliaksandr Valialkin
9b15b11f74 app/vmstorage: added -forceMergeAuthKey command-line flag for protecting /internal/force_merge endpoint 2020-09-17 14:24:20 +03:00
Aliaksandr Valialkin
d96858b921 lib/storage: add /internal/force_merge handler for running forced compactions on historical per-month partitions 
This may be useful for freeing up storage space after time series deletion.

See https://victoriametrics.github.io/#force-merge for more details.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/686
 2020-09-17 12:20:56 +03:00
Aliaksandr Valialkin
406f4fe445 app/vmagent: substitute -remoteWrite.url with secret-url value in logs, since it may contain sensitive info such as passwords or auth tokens 
Pass `-remoteWrite.showURL` command-line flag in order to see real `-remoteWrite.url` values in logs and at `/metrics` page.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/773
 2020-09-16 22:36:18 +03:00
Aliaksandr Valialkin
9793008734 app/vmselect: add -search.storageTimeout command-line flag for limiting the maximum duration of query execution per each -storageNode 
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/711
 2020-09-16 21:33:47 +03:00
Aliaksandr Valialkin
a9205fe308 app/vmselect: prevent from closing connection to vmstorage on query timeout by setting +2 secs deadline on connection comparing to query deadline 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/711
 2020-09-16 21:14:00 +03:00
Aliaksandr Valialkin
1587f83fa0 app/vmselect/netstorage: typo fix after 03dfccfbed 2020-09-16 00:10:33 +03:00