github-mirrors/VictoriaMetrics
1
0
Fork 0
mirror of https://github.com/VictoriaMetrics/VictoriaMetrics.git synced 2024-12-01 14:47:38 +00:00
Code Issues Projects Releases Packages Wiki Activity
2353 commits 119 branches 748 tags 355 MiB
Commit graph

2353 commits

This branch
This branch
All branches
Author SHA1 Message Date
Aliaksandr Valialkin
8fc9b77496 app/vmagent: reduce memory usage when importing data via /api/v1/import 
Previously vmagent could use big amounts of RAM when each ingested JSON line
contained many samples.
 2020-09-26 04:10:13 +03:00
Aliaksandr Valialkin
973df09686 app/vmselect/netstorage: do not spend CPU time on unpacking empty blocks during /api/v1/series calls 2020-09-24 20:44:15 +03:00
Aliaksandr Valialkin
533bf76a12 lib/storage: correctly use maxBlockSize in various checks 
Previously `maxBlockSize` has been multiplied by 8 in certain checks. This is unnecessary.
 2020-09-24 18:13:15 +03:00
Aliaksandr Valialkin
b8bce348c5 app/vmselect/promql: properly limit implicitly set rollup window to -search.maxStalenessInterval 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/784
 2020-09-23 23:24:09 +03:00
Aliaksandr Valialkin
543f3aea97 all: consistently use "%w" formatting in fmt.Errorf for wrapped errors 2020-09-23 22:48:21 +03:00
Aliaksandr Valialkin
265d892a8c vendor: make vendor-update 2020-09-23 14:23:51 +03:00
Aliaksandr Valialkin
8627365b48 app/vmselect/prometheus: code cleanup after 3ba507000c 2020-09-23 13:04:51 +03:00
Aliaksandr Valialkin
1fce79518a app/vmselect/prometheus: return timestamps from /api/v1/query, which match the time query arg 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/720
 2020-09-23 12:59:23 +03:00
Aliaksandr Valialkin
90d2549428 lib/persistentqueue: protect from multiple concurrent opening for the same persistent queue 2020-09-23 02:17:53 +03:00
Aliaksandr Valialkin
0468cdf33e app/vmselect/netstorage: properly pre-allocate space for sbs 2020-09-22 23:51:01 +03:00
Aliaksandr Valialkin
89d652b583 lib/cgroup: attempt to obtain available CPU cores via /sys/devices/system/cpu/online 
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/685#issuecomment-674423728
 2020-09-22 23:27:26 +03:00
Aliaksandr Valialkin
4ebd2fa560 vendor: make vendor-update 2020-09-22 17:08:54 +03:00
Aliaksandr Valialkin
00ff01f766 docs/Single-server-VictoriaMetrics.md: VictoriaMetrics properly stores Inf values after 26115891db 2020-09-22 17:03:16 +03:00
Aliaksandr Valialkin
69eb9783e6 app/vmselect: make sure the request doesnt wait in pending queue more than the configured timeout 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/711
 2020-09-22 01:21:42 +03:00
Aliaksandr Valialkin
31e341371b lib/storage: code prettifying after be5e1222f3 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/781
 2020-09-22 00:42:20 +03:00
faceair
ad41e39350 add filter to getMetricIDs (#783) 
* add getMetricIDs filter

* check nil filter before use
 2020-09-22 00:42:19 +03:00
Aliaksandr Valialkin
ed473c94ff docs/vmagent.md: typo fix 2020-09-21 21:49:08 +03:00
Aliaksandr Valialkin
bedc971398 docs: make docs-sync 2020-09-21 21:48:05 +03:00
Aliaksandr Valialkin
e564725641 app/vmselect/searchutils: fixed tests after 2eb72e09ab 2020-09-21 21:31:28 +03:00
Aliaksandr Valialkin
07c6226334 app/vmselect: use time value rounded to seconds if it isnt passed to /api/v1/query 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/720
 2020-09-21 21:24:46 +03:00
Aliaksandr Valialkin
d32c3f747c lib/logger: add -loggerDisableTimestamps command-line flag for disabling timestamps in logs 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/778
 2020-09-21 19:25:50 +03:00
Aliaksandr Valialkin
f961838290 lib/promscrape/discovery/ec2: code prettifying after 312fead9a2 2020-09-21 18:44:04 +03:00
Nikolay Khramchikhin
0069353d5e Add improvements to ec2_sd_discovery (#775) 
* Add improvements to ec2 discovery

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/771

 role_arn support with aws sts
 instance iam_role support
 refreshing temporary tokens

* Apply suggestions from code review

Co-authored-by: Roman Khavronenko <hagen1778@gmail.com>

* changed implementation, removed tests, clean up code

* moved endpoint builder into getEC2APIResponse

Co-authored-by: Roman Khavronenko <hagen1778@gmail.com>
 2020-09-21 16:05:01 +03:00
Aliaksandr Valialkin
8cd89cb847 app/vmalert: remove unneeded UTC() call 
UTC() doesn't change the underlying timestamp, so the call isn't needed here
 2020-09-21 15:56:48 +03:00
Roman Khavronenko
d111969d39 vmalert: add support for datasource.lookback flag (#779) 
New datasource flag `datasource.lookback` defines how far to look into
past when evaluating queries.

Address https://github.com/VictoriaMetrics/VictoriaMetrics/issues/668
 2020-09-21 15:56:47 +03:00
Aliaksandr Valialkin
a9321f6a60 lib/storage: reduce CPU load for idle VictoriaMetrics by reducing the frequency for the need for background merges 2020-09-21 15:51:26 +03:00
Roman Khavronenko
0042b0f307 vmalert: fix the typo in error message (#782) 
The error will be always nil so no sense in printing it.
 2020-09-21 11:36:09 +03:00
Aleksey Shirokih
a7e17f14f5
dashboard: add account id to datapoints ingestion rate (#772) 2020-09-19 22:00:16 +01:00
Aliaksandr Valialkin
604e8f6114 lib/decimal: optimize maxUpExponent() by eliminating division from hot path 2020-09-19 13:49:43 +03:00
Aliaksandr Valialkin
df547bf345 lib/persistentqueue: sync data to file inside filestream.Writer.MustFlush 2020-09-19 12:51:46 +03:00
Aliaksandr Valialkin
778ea183ca lib/decimal: properly store Inf values 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/752
 2020-09-18 19:08:53 +03:00
Aliaksandr Valialkin
1c07d7bee7 app/vmagent: increase default value for -remoteWrite.queues from 1 to 4, since it has been appeared that many users hit this limit 2020-09-18 14:22:02 +03:00
Aliaksandr Valialkin
5bd30171b3 vendor: update github.com/valyala/quicktemplate from v1.6.2 to v1.6.3 2020-09-18 13:10:55 +03:00
Aliaksandr Valialkin
adb4d3b75c lib/persistentqueue: flush data to disk every second 
Previously small amounts of data may be left unflushed for extended periods of time if vmagent collects small amounts of data.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/687
 2020-09-18 13:06:03 +03:00
Aliaksandr Valialkin
8fd791d399 lib/promscrape: avoid copying response body when scraping targets. 
This should reduce memory usage when scraping targets with millions of metrics.
 2020-09-18 13:06:00 +03:00
Aliaksandr Valialkin
e25b90849f vendor: udpate github.com/VictoriaMetrics/fasthttp from v1.0.5 to v1.0.7 2020-09-18 12:20:48 +03:00
Aliaksandr Valialkin
a1bebb660c app/vmselect/graphite: return proper results /metrics/find?query=foo.*.bar according to Graphite Metrics API 2020-09-18 11:53:28 +03:00
Aliaksandr Valialkin
9b15b11f74 app/vmstorage: added -forceMergeAuthKey command-line flag for protecting /internal/force_merge endpoint 2020-09-17 14:24:20 +03:00
Aliaksandr Valialkin
d96858b921 lib/storage: add /internal/force_merge handler for running forced compactions on historical per-month partitions 
This may be useful for freeing up storage space after time series deletion.

See https://victoriametrics.github.io/#force-merge for more details.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/686
 2020-09-17 12:20:56 +03:00
Aliaksandr Valialkin
3abbb38254 lib/{mergeset,storage}: compare errors with errors.Is() 2020-09-17 03:03:10 +03:00
Aliaksandr Valialkin
ddb3519e17 lib/{mergeset,storage}: code prettifying 2020-09-17 02:06:37 +03:00
Aliaksandr Valialkin
bf826dd828 lib/storage: removed duplicate checks for empty parts during merge - another check is in the beginning of mergeParts functions 2020-09-17 01:49:08 +03:00
Aliaksandr Valialkin
7429fb2301 vendor: make vendor-update 2020-09-17 01:44:16 +03:00
Aliaksandr Valialkin
2084496462 docs/Single-server-VictoriaMetrics.md: document that /api/v1/series/count may count delete time series 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/770
 2020-09-17 01:39:09 +03:00
Aliaksandr Valialkin
406f4fe445 app/vmagent: substitute -remoteWrite.url with secret-url value in logs, since it may contain sensitive info such as passwords or auth tokens 
Pass `-remoteWrite.showURL` command-line flag in order to see real `-remoteWrite.url` values in logs and at `/metrics` page.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/773
 2020-09-16 22:36:18 +03:00
Aliaksandr Valialkin
9793008734 app/vmselect: add -search.storageTimeout command-line flag for limiting the maximum duration of query execution per each -storageNode 
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/711
 2020-09-16 21:33:47 +03:00
Aliaksandr Valialkin
9705ac5d7a lib/persistentqueue: code simplification after d455764a6f 2020-09-16 21:14:01 +03:00
Aliaksandr Valialkin
a9205fe308 app/vmselect: prevent from closing connection to vmstorage on query timeout by setting +2 secs deadline on connection comparing to query deadline 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/711
 2020-09-16 21:14:00 +03:00
Aliaksandr Valialkin
eee6f1e56d lib/persistentqueue: make the persistent queue more durable against unclean shutdown (kill -9, OOM, hard reset) 
The strategy is:

- Periodical flushing of inmemory blocks to files, so they aren't lost on unclean shutdown.
- Periodical syncing of metadata for persisted queues, so the metadata remains in sync with the persisted data.
- Automatic adjusting of too big chunk size when opening the queue. The chunk size may be bigger than the writer offset after unclean shutdown.
- Skipping of broken chunk file if it cannot be read.
- Fsyncing finalized chunk files.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/687
 2020-09-16 18:13:24 +03:00
Aliaksandr Valialkin
6ce52e3702 lib/protoparser/vmimport: add more testcases for invalid timestamps and values 
Updates https://github.com/VictoriaMetrics/vmctl/issues/25
 2020-09-16 02:21:53 +03:00