Commit graph

356 commits

Author SHA1 Message Date
Aliaksandr Valialkin
8bbc83e85e Merge branch 'public-single-node' into pmm-6401-read-prometheus-data-files 2020-10-17 12:13:56 +03:00
Aliaksandr Valialkin
28353e48ca app/vmselect/promql: an attempt to improve heuristics for dropping trailing data points in time series
Now trailing data points are additionally dropped for time series with a single raw sample

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/748
2020-10-17 10:44:34 +03:00
Aliaksandr Valialkin
4dc13754d8 Merge branch 'public-single-node' into pmm-6401-read-prometheus-data-files 2020-10-13 18:37:48 +03:00
Aliaksandr Valialkin
a2b9476897 app/vmselect/promql: return a single time series at max from absent() function like Prometheus does 2020-10-13 15:56:04 +03:00
Aliaksandr Valialkin
9aa3b65766 app/vmselect/promql: improve time series staleness detection
This should prevent from double counting for time series at the time when it changes label.
The most common case is in K8S, which changes pod uid label with each new deployment.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/748
2020-10-13 12:19:57 +03:00
Aliaksandr Valialkin
d8af290947 app/vmselect/promql: fix mode_over_time calculations
Previously `mode_over_time` could return garbage due to improper shuffling of input data points.
2020-10-13 11:58:25 +03:00
Aliaksandr Valialkin
1e27420243 app/vmselect/prometheus: fix golangci-lint warning 2020-10-13 09:36:11 +03:00
Aliaksandr Valialkin
4f16a964e3 app/vmselect: add ability to export data in CSV format via /api/v1/export/csv 2020-10-12 20:08:17 +03:00
Aliaksandr Valialkin
762c967855 app/vmselect/promql: keep metric name after applying more functions, which dont change time series meaning
Functions are:

* keep_last_value
* keep_next_value
* interpolate
* running_min
* running_max
* running_avg
* range_min
* range_max
* range_avg
* range_first
* range_last
* range_quantile
* smooth_exponential

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/674
2020-10-12 11:47:06 +03:00
Aliaksandr Valialkin
45f7cdc532 Revert "app/vmselect/promql: remove metric name after applying ceil, floor and round functions in order to be more consistent with Prometheus"
This reverts commit ac45082216.

Reason for revert: the previous behavior for VictoriaMetrics is easier to understand and use by users -
functions, which don't change the meaning of the time series shouldn't drop metric name.

Now the following functions do not drop metric names:

* ceil
* floor
* round

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/674
2020-10-12 11:40:34 +03:00
Aliaksandr Valialkin
a94825b169 Revert "app/vmselect/promql: remove metric name after applying clamp_min and clamp_max functions in order to be consistent with Prometheus"
This reverts commit bb61a4769b.

Reason for revert: the previous behavior for VictoriaMetrics is easier to understand and use by users -
functions, which don't change the meaning of the time series shouldn't drop metric name.

Now the following functions do not drop metric name:

* clamp_min
* clamp_max

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/674
2020-10-12 11:38:27 +03:00
Aliaksandr Valialkin
f7d28bddbf Revert "app/vmselect/promql: remove metric name from results of certain rollup functions in order to be consistent with Prometheus"
This reverts commit e5202a4eae.

Reason for revert: the previous behavior for VictoriaMetrics is easier to understand and use by users -
functions, which don't change the meaning of the time series shouldn't drop metric name.

Now the following functions do not drop metric name:

* max_over_time
* min_over_time
* avg_over_time
* quantile_over_time
* geomean_over_time
* mode_over_time
* holt_winters
* predict_linear

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/674
2020-10-12 11:35:18 +03:00
Aliaksandr Valialkin
e7f2907138 Merge branch 'public-single-node' into pmm-6401-read-prometheus-data-files 2020-10-07 21:40:45 +03:00
Aliaksandr Valialkin
e9f2e2cbc9 app/vmselect/promql: add missing label filters to binary operands before query execution
This implements the optimization described at https://utcc.utoronto.ca/~cks/space/blog/sysadmin/PrometheusLabelNonOptimization

See also https://github.com/cortexproject/cortex/issues/3253
2020-10-07 21:15:09 +03:00
Aliaksandr Valialkin
59c26feefa app/vmselect/promql: fill gaps on graphs for range_* and running_* functions
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/806
2020-10-02 13:59:45 +03:00
Aliaksandr Valialkin
9fbb84d5c2 app/vmselect/promql: fill gaps on graphs for range_* and running_* functions
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/806
2020-10-02 13:58:19 +03:00
Aliaksandr Valialkin
8df5a3c5f6 Merge branch 'public-single-node' into pmm-6401-read-prometheus-data-files 2020-09-30 08:55:57 +03:00
Aliaksandr Valialkin
f7636b0342 app/vmselect/graphite: do not substitute path and path. with path.. in /metrics/find/?format=completer output 2020-09-29 18:03:26 +03:00
Aliaksandr Valialkin
96ee276e6e app/vmselect/prometheus: check for errors returned from bufferedwriter.Write
This makes `make errcheck` happy
2020-09-29 11:37:01 +03:00
Aliaksandr Valialkin
6fdfc67620 app/vmselect/graphite: properly handle case when /metrics/find finds both leaf and node for the given query=prefix.*
In this case only node must be returned with stripped dot in the end of id as carbonapi does
2020-09-29 11:01:59 +03:00
Aliaksandr Valialkin
1b3efccb24 app/vmselect: stop /api/v1/export/* execution if client disconnects 2020-09-27 23:53:13 +03:00
Aliaksandr Valialkin
95688cbfc5 all: add native format for data export and import
The data can be exported via [/api/v1/export/native](https://victoriametrics.github.io/#how-to-export-data-in-native-format) handler
and imported via [/api/v1/import/native](https://victoriametrics.github.io/#how-to-import-data-in-native-format) handler.
2020-09-27 19:54:07 +03:00
Aliaksandr Valialkin
82973f8ae7 Revert "lib/storage: remove unused fetchData arg from BlockRef.MustReadBlock"
This reverts commit bab6a15ae0.

Reason for revert: the `fetchData` arg is used in cluster branch.
Leaving this arg in master branch makes smaller the diff with cluster branch.
2020-09-24 22:44:23 +03:00
Aliaksandr Valialkin
9d3eb3f4b8 Merge branch 'public-single-node' into pmm-6401-read-prometheus-data-files 2020-09-24 20:49:16 +03:00
Aliaksandr Valialkin
bab6a15ae0 lib/storage: remove unused fetchData arg from BlockRef.MustReadBlock
This arg became unused after 23bdc1f107
2020-09-24 20:48:40 +03:00
Aliaksandr Valialkin
23bdc1f107 app/vmselect/netstorage: do not spend CPU time on unpacking empty blocks during /api/v1/series calls 2020-09-24 20:18:10 +03:00
Aliaksandr Valialkin
8fc8874db4 Merge branch 'public-single-node' into pmm-6401-read-prometheus-data-files 2020-09-23 23:25:34 +03:00
Aliaksandr Valialkin
c584aece38 app/vmselect/promql: properly limit implicitly set rollup window to -search.maxStalenessInterval
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/784
2020-09-23 23:23:59 +03:00
Aliaksandr Valialkin
ff1cbb524e app/vmselect: obtain labels and label values from Prometheus storage inside netstorage package 2020-09-23 22:43:01 +03:00
Aliaksandr Valialkin
a70df4bd83 PMM-6401 Initial implementaton for reading data from Prometheus files 2020-09-23 14:26:39 +03:00
Aliaksandr Valialkin
27500d7d4c app/vmselect/prometheus: code cleanup after 3ba507000c 2020-09-23 13:04:17 +03:00
Aliaksandr Valialkin
3ba507000c app/vmselect/prometheus: return timestamps from /api/v1/query, which match the time query arg
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/720
2020-09-23 12:58:48 +03:00
Aliaksandr Valialkin
bed25e3c24 app/vmselect/netstorage: properly pre-allocate space for sbs 2020-09-22 23:49:55 +03:00
Aliaksandr Valialkin
09b0f7c202 app/vmselect/netstorage: release search resources on timeout errors
Previously these resources weren't released, which could lead to resource leaks.
2020-09-22 22:57:38 +03:00
Aliaksandr Valialkin
3b1e3a03e0 app/vmselect: make sure the request doesnt wait in pending queue more than the configured timeout
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/711
2020-09-22 01:23:19 +03:00
Aliaksandr Valialkin
00b5145c69 app/vmselect/searchutils: fixed tests after 2eb72e09ab 2020-09-21 21:31:38 +03:00
Aliaksandr Valialkin
2eb72e09ab app/vmselect: use time value rounded to seconds if it isnt passed to /api/v1/query
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/720
2020-09-21 21:24:40 +03:00
Aliaksandr Valialkin
98d1cd0971 app/vmselect/graphite: return proper results /metrics/find?query=foo.*.bar according to Graphite Metrics API 2020-09-18 11:00:00 +03:00
Aliaksandr Valialkin
9f79bcf64a app/vmselect: improve description for -search.maxQueryDuration 2020-09-16 21:15:41 +03:00
Aliaksandr Valialkin
1fec47a289 app/vmselect/netstorage: reduce memory usage when the time range from query touches big number of samples per each time series 2020-09-15 21:08:28 +03:00
Aliaksandr Valialkin
8c3d7c1a59 app/vmselect: typo fix in -search.maxStalenessInterval description 2020-09-15 14:24:27 +03:00
Aliaksandr Valialkin
0e533d1a9c app/vmselect/promql: support composite durations like Prometheus 2.21 does
The following durations are supported now: `1h5m35s` or `1s543ms`

See https://github.com/prometheus/prometheus/releases/tag/v2.21.0
and https://github.com/prometheus/prometheus/pull/7713
2020-09-11 23:39:13 +03:00
Aliaksandr Valialkin
a2f647d142 app/vmselect/prometheus: typo fix in the description for -search.latencyOffset command-line flag 2020-09-11 14:16:46 +03:00
Aliaksandr Valialkin
f0005c3007 app/vmselect: move Deadline from netstorage to searchutils
This removes dependency on netstorage from searchutils.
2020-09-11 13:27:13 +03:00
Aliaksandr Valialkin
2114179e19 app/vmselect: substitute inf values at smooth_exponential with the previous values
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/757
2020-09-11 12:24:14 +03:00
Aliaksandr Valialkin
204ec415b4 app/vmselect: skip infinite values when calculating smooth_exponential
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/757
2020-09-11 11:29:58 +03:00
Aliaksandr Valialkin
8a8b5a73d3 app/vmselect/graphite: typo fix in label name for vm_request_duration_seconds metric 2020-09-11 01:58:28 +03:00
Aliaksandr Valialkin
f6bc608e86 app/vmselect: initial implementation of Graphite Metrics API
See https://graphite-api.readthedocs.io/en/latest/api.html#the-metrics-api
2020-09-11 00:30:01 +03:00
Aliaksandr Valialkin
62919eaf7e app/vmselect/promql: go fmt 2020-09-08 15:19:59 +03:00
Aliaksandr Valialkin
e6da63dffe app/vmselect/promql: adjust integrate() calculations to be more similar to calculations from InfluxDB: attempt #2
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/701
2020-09-08 14:35:50 +03:00