Aliaksandr Valialkin
95688cbfc5
all: add native format for data export and import
...
The data can be exported via [/api/v1/export/native](https://victoriametrics.github.io/#how-to-export-data-in-native-format ) handler
and imported via [/api/v1/import/native](https://victoriametrics.github.io/#how-to-import-data-in-native-format ) handler.
2020-09-27 19:54:07 +03:00
Aliaksandr Valialkin
b4bf722d8f
lib/protoparser: use all the available CPU cores for processing ingested data from a single /api/v1/import stream
...
Previously a single data ingestion stream to /api/v1/import could load only a single CPU core.
2020-09-26 04:21:32 +03:00
Aliaksandr Valialkin
c00627c103
app/vminsert: code prettifying
2020-09-26 04:13:18 +03:00
Aliaksandr Valialkin
b6a976b98d
app/vmagent: reduce memory usage when importing data via /api/v1/import
...
Previously vmagent could use big amounts of RAM when each ingested JSON line
contained many samples.
2020-09-26 04:10:24 +03:00
Aliaksandr Valialkin
82973f8ae7
Revert "lib/storage: remove unused fetchData
arg from BlockRef.MustReadBlock"
...
This reverts commit bab6a15ae0
.
Reason for revert: the `fetchData` arg is used in cluster branch.
Leaving this arg in master branch makes smaller the diff with cluster branch.
2020-09-24 22:44:23 +03:00
Aliaksandr Valialkin
9d3eb3f4b8
Merge branch 'public-single-node' into pmm-6401-read-prometheus-data-files
2020-09-24 20:49:16 +03:00
Aliaksandr Valialkin
bab6a15ae0
lib/storage: remove unused fetchData
arg from BlockRef.MustReadBlock
...
This arg became unused after 23bdc1f107
2020-09-24 20:48:40 +03:00
Aliaksandr Valialkin
23bdc1f107
app/vmselect/netstorage: do not spend CPU time on unpacking empty blocks during /api/v1/series
calls
2020-09-24 20:18:10 +03:00
Aliaksandr Valialkin
2cd48959d4
lib/storage: correctly use maxBlockSize in various checks
...
Previously `maxBlockSize` has been multiplied by 8 in certain checks. This is unnecessary.
2020-09-24 20:17:51 +03:00
Aliaksandr Valialkin
24ca30bf66
lib/storage: correctly use maxBlockSize in various checks
...
Previously `maxBlockSize` has been multiplied by 8 in certain checks. This is unnecessary.
2020-09-24 18:12:56 +03:00
Aliaksandr Valialkin
8fc8874db4
Merge branch 'public-single-node' into pmm-6401-read-prometheus-data-files
2020-09-23 23:25:34 +03:00
Aliaksandr Valialkin
c584aece38
app/vmselect/promql: properly limit implicitly set rollup window to -search.maxStalenessInterval
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/784
2020-09-23 23:23:59 +03:00
Aliaksandr Valialkin
2985077c35
all: consistently use "%w" formatting in fmt.Errorf for wrapped errors
2020-09-23 22:46:34 +03:00
Aliaksandr Valialkin
ff1cbb524e
app/vmselect: obtain labels and label values from Prometheus storage inside netstorage package
2020-09-23 22:43:01 +03:00
Aliaksandr Valialkin
a70df4bd83
PMM-6401 Initial implementaton for reading data from Prometheus files
2020-09-23 14:26:39 +03:00
Aliaksandr Valialkin
30c7269814
vendor: make vendor-update
2020-09-23 14:23:39 +03:00
Aliaksandr Valialkin
27500d7d4c
app/vmselect/prometheus: code cleanup after 3ba507000c
2020-09-23 13:04:17 +03:00
Aliaksandr Valialkin
3ba507000c
app/vmselect/prometheus: return timestamps from /api/v1/query
, which match the time
query arg
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/720
2020-09-23 12:58:48 +03:00
Aliaksandr Valialkin
c5ef0e6327
lib/persistentqueue: protect from multiple concurrent opening for the same persistent queue
2020-09-23 02:17:47 +03:00
Aliaksandr Valialkin
bed25e3c24
app/vmselect/netstorage: properly pre-allocate space for sbs
2020-09-22 23:49:55 +03:00
Aliaksandr Valialkin
5c42965853
lib/cgroup: attempt to obtain available CPU cores via /sys/devices/system/cpu/online
...
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/685#issuecomment-674423728
2020-09-22 23:27:19 +03:00
Aliaksandr Valialkin
09b0f7c202
app/vmselect/netstorage: release search resources on timeout errors
...
Previously these resources weren't released, which could lead to resource leaks.
2020-09-22 22:57:38 +03:00
Aliaksandr Valialkin
36eb5427eb
vendor: make vendor-update
2020-09-22 17:07:37 +03:00
Aliaksandr Valialkin
31ce0e29cd
docs/Single-server-VictoriaMetrics.md: VictoriaMetrics properly stores Inf values after 26115891db
2020-09-22 17:02:39 +03:00
Aliaksandr Valialkin
3b1e3a03e0
app/vmselect: make sure the request doesnt wait in pending queue more than the configured timeout
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/711
2020-09-22 01:23:19 +03:00
Aliaksandr Valialkin
a69234ed18
lib/storage: code prettifying after be5e1222f3
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/781
2020-09-22 00:36:45 +03:00
faceair
be5e1222f3
add filter to getMetricIDs ( #783 )
...
* add getMetricIDs filter
* check nil filter before use
2020-09-22 00:33:43 +03:00
Aliaksandr Valialkin
94f7d00537
docs/vmagent.md: typo fix
2020-09-21 21:49:22 +03:00
Aliaksandr Valialkin
f6f5c4118c
docs: make docs-sync
2020-09-21 21:47:47 +03:00
Aliaksandr Valialkin
00b5145c69
app/vmselect/searchutils: fixed tests after 2eb72e09ab
2020-09-21 21:31:38 +03:00
Aliaksandr Valialkin
2eb72e09ab
app/vmselect: use time
value rounded to seconds if it isnt passed to /api/v1/query
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/720
2020-09-21 21:24:40 +03:00
Aliaksandr Valialkin
29108cc53e
lib/logger: add -loggerDisableTimestamps
command-line flag for disabling timestamps in logs
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/778
2020-09-21 19:28:04 +03:00
Aliaksandr Valialkin
964bc7595c
lib/promscrape/discovery/ec2: code prettifying after 312fead9a2
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/771
2020-09-21 18:43:34 +03:00
Nikolay Khramchikhin
312fead9a2
Add improvements to ec2_sd_discovery ( #775 )
...
* Add improvements to ec2 discovery
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/771
role_arn support with aws sts
instance iam_role support
refreshing temporary tokens
* Apply suggestions from code review
Co-authored-by: Roman Khavronenko <hagen1778@gmail.com>
* changed implementation, removed tests, clean up code
* moved endpoint builder into getEC2APIResponse
Co-authored-by: Roman Khavronenko <hagen1778@gmail.com>
2020-09-21 16:04:15 +03:00
Aliaksandr Valialkin
1e1a27d803
app/vmalert: remove unneeded UTC() call
...
UTC() doesn't change the underlying timestamp, so the call isn't needed here
2020-09-21 15:55:59 +03:00
Aliaksandr Valialkin
9739283dad
lib/storage: reduce CPU load for idle VictoriaMetrics by reducing the frequency for the need for background merges
2020-09-21 15:54:11 +03:00
Roman Khavronenko
5dffc7a553
vmalert: add support for datasource.lookback
flag ( #779 )
...
New datasource flag `datasource.lookback` defines how far to look into
past when evaluating queries.
Address https://github.com/VictoriaMetrics/VictoriaMetrics/issues/668
2020-09-21 15:53:49 +03:00
Roman Khavronenko
82c3bbce34
vmalert: fix the typo in error message ( #782 )
...
The error will be always nil so no sense in printing it.
2020-09-21 11:34:23 +03:00
Aliaksandr Valialkin
3e8569f456
lib/decimal: optimize maxUpExponent() by eliminating division from hot path
2020-09-19 13:50:09 +03:00
Aliaksandr Valialkin
f00e0e0103
lib/persistentqueue: sync data to file inside filestream.Writer.MustFlush
2020-09-19 12:51:41 +03:00
Aliaksandr Valialkin
26115891db
lib/decimal: properly store Inf values
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/752
2020-09-18 19:07:07 +03:00
Aliaksandr Valialkin
d50165ad59
app/vmagent: increase default value for -remoteWrite.queues
from 1 to 4, since it has been appeared that many users hit this limit
2020-09-18 14:21:54 +03:00
Aliaksandr Valialkin
63d3c88c3b
vendor: update github.com/valyala/quicktemplate from v1.6.2 to v1.6.3
2020-09-18 13:10:48 +03:00
Aliaksandr Valialkin
1a9ee39b0e
lib/promscrape: avoid copying response body when scraping targets.
...
This should reduce memory usage when scraping targets with millions of metrics.
2020-09-18 13:05:43 +03:00
Aliaksandr Valialkin
70c721c01b
lib/persistentqueue: flush data to disk every second
...
Previously small amounts of data may be left unflushed for extended periods of time if vmagent collects small amounts of data.
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/687
2020-09-18 13:05:40 +03:00
Aliaksandr Valialkin
74e3198281
vendor: udpate github.com/VictoriaMetrics/fasthttp from v1.0.5 to v1.0.7
2020-09-18 12:20:29 +03:00
Aliaksandr Valialkin
98d1cd0971
app/vmselect/graphite: return proper results /metrics/find?query=foo.*.bar
according to Graphite Metrics API
2020-09-18 11:00:00 +03:00
Aliaksandr Valialkin
7a134b0fd7
app/vmstorage: added -forceMergeAuthKey
command-line flag for protecting /internal/force_merge
endpoint
2020-09-17 14:21:53 +03:00
Aliaksandr Valialkin
1f33dd717f
lib/storage: add /internal/force_merge
handler for running forced compactions on historical per-month partitions
...
This may be useful for freeing up storage space after time series deletion.
See https://victoriametrics.github.io/#force-merge for more details.
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/686
2020-09-17 12:20:40 +03:00
Aliaksandr Valialkin
8beb0da6ad
lib/{mergeset,storage}: compare errors with errors.Is()
2020-09-17 03:03:02 +03:00