github-mirrors/VictoriaMetrics
1
0
Fork 0
mirror of https://github.com/VictoriaMetrics/VictoriaMetrics.git synced 2024-12-11 14:53:49 +00:00
Code Issues Projects Releases Packages Wiki Activity
3293 commits 131 branches 758 tags 363 MiB
Commit graph

563 commits

Author SHA1 Message Date
Aliaksandr Valialkin
d86e9b49c4 app/vmselect/promql: increase accuracy for buckets_limit() function for small limits by skipping the first and the last buckets during merge 
The first and the last buckets are usually `[0 ... leMin]` and `(leMax ... +Inf)`. If they are merged with adjancent buckets,
then the resulting accuracy can suffer.
 2021-02-26 22:56:36 +02:00
Aliaksandr Valialkin
98854e5f2b app/vmselect: add sign(q) and clamp(q, min, max) functions, which will be added in the upcoming Prometheus release 
See https://twitter.com/roidelapluie/status/1363428376162295811

The `last_over_time(m[d])` function already exists in MetricsQL.
 2021-02-24 17:24:56 +02:00
Aliaksandr Valialkin
f4135b0d14 app/vmselect/promql: properly calculate histogram_quantile() over zero buckets and only a single non-zero le="+Inf"` bucket like Prometheus does 2021-02-24 00:42:22 +02:00
Aliaksandr Valialkin
fa03e0d210 app/vmselect/promql: add increase_pure() function to MetricsQL 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/962
 2021-02-22 19:14:15 +02:00
Aliaksandr Valialkin
9c70c1f21f app/vmselect/promql: reduce the probability of duplicate time series errors when querying Kubernetes metrics 2021-02-18 22:07:29 +02:00
Aliaksandr Valialkin
687eb4ab00 Merge branch 'public-single-node' into pmm-6401-read-prometheus-data-files 2021-02-16 22:29:45 +02:00
Aliaksandr Valialkin
c4756f94da app/vmselect/netstorage: reuse timeseriesWork objects in order to reduce memory allocations 2021-02-16 16:08:53 +02:00
Aliaksandr Valialkin
5a401225c7 app/vmselect/netstorage: use unsafe string as a key for a map when the map already contains the given key 
This should prevent from a memory allocation and a string copy.
 2021-02-16 15:43:10 +02:00
Aliaksandr Valialkin
4e39bf148c vendor: update github.com/VictoriaMetrics/metrics from v1.13.1 to v1.14.0 
The new version switches from log-linear histograms to log-based histograms,
which provide up to 3.6 times better accuracy.
 2021-02-15 15:12:29 +02:00
Aliaksandr Valialkin
d0c364d93d Merge branch 'public-single-node' into pmm-6401-read-prometheus-data-files 2021-02-15 01:45:39 +02:00
Aliaksandr Valialkin
acdb401585 app/vmselect/prometheus: treat match query arg in the same way as match[] query arg 2021-02-11 15:02:21 +02:00
Aliaksandr Valialkin
2d33230793 app/vmselect/promql: properly make copies of EvalConfig 2021-02-11 12:41:15 +02:00
Aliaksandr Valialkin
7a3a9421f3 app/vmselect/promql: make a copy of EvalConfig when executing q1 and q2 in parallel for q1 binary_op q2 
This should prevent from data races if the underlying functions modify EvalConfig contents.
 2021-02-10 23:05:55 +02:00
Aliaksandr Valialkin
04faea8b45 app/vmselect: parallelize q1 <binary_op> q2 queries by running q1 and q2 in parallel 
This should reduce query execution times.
 2021-02-10 22:59:39 +02:00
Aliaksandr Valialkin
553016ea99 lib/storage: disable composite index usage when querying old data 2021-02-10 14:57:50 +02:00
Aliaksandr Valialkin
c7dccebaef lib/storage: optimize search by label filters matching big number of time series 2021-02-10 00:44:54 +02:00
Aliaksandr Valialkin
803a00102a app/{vminsert,vmselect}: accept requests to paths with /graphite and /prometheus prefixes 
This should improve compatibility with path prefixes from VictoriaMetrics cluster.
See https://victoriametrics.github.io/Cluster-VictoriaMetrics.html#url-format
 2021-02-04 20:01:18 +02:00
Aliaksandr Valialkin
4d43ab0875 app/vmselect: typo fix when stripping url path prefixes 2021-02-04 19:29:25 +02:00
Aliaksandr Valialkin
63c88d8ea2 Merge branch 'public-single-node' into pmm-6401-read-prometheus-data-files 2021-02-03 23:52:43 +02:00
Aliaksandr Valialkin
8629fd8a72 app/vmselect: deprecate -search.treatDotsAsIsInRegexps in favor to {__graphite__="foo.*.bar"} syntax 2021-02-03 20:36:01 +02:00
Aliaksandr Valialkin
d16f22f3a1 app/vmselect,lib/storage: properly parse Graphite selectors with inner wildcards 
Example: foo{bar{x,yz},a[b-c],*de}
 2021-02-03 20:14:22 +02:00
Aliaksandr Valialkin
dc6636e2b2 Merge branch 'public-single-node' into pmm-6401-read-prometheus-data-files 2021-02-03 12:30:44 +02:00
Aliaksandr Valialkin
157c02622b app/vmselect: add ability to set Graphite-compatible filter via {__graphite__="foo.*.bar"} syntax 2021-02-03 01:21:54 +02:00
Aliaksandr Valialkin
c13f1d99e0 Merge branch 'public-single-node' into pmm-6401-read-prometheus-data-files 2021-02-01 20:19:50 +02:00
Aliaksandr Valialkin
b43b498fd8 app/vmselect: add ability to pass extra_label=<label>=<value> query arg to Prometheus Querying API 
This enforced `{label="value"}` label filter to the query.

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1021
 2021-02-01 18:04:17 +02:00
Nikolay
195341a7cf Graphite vmalert wip (#112) 
* init implementation for graphite alerts

* adds graphite support for vmalert

* small fix

* changes vmalert graphite api with type

* updates tests

* small fix

* fixes graphite parse

* Fixes graphite from time
 2021-02-01 15:05:32 +02:00
Aliaksandr Valialkin
079888f719 Merge branch 'public-single-node' into pmm-6401-read-prometheus-data-files 2021-01-27 01:12:45 +02:00
Aliaksandr Valialkin
c164a8d231 app/vmselect/promql: improve documentation for -search.maxPointsPertimeseries command-line flag 
This should reduce incorrect usage and assumptions for this flag.

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1020
 2021-01-22 13:00:10 +02:00
Aliaksandr Valialkin
b68264b4f5 Merge branch 'public-single-node' into pmm-6401-read-prometheus-data-files 2021-01-22 12:10:57 +02:00
Aliaksandr Valialkin
8749c2dd92 app/vmselect: add -search.maxStepForPointsAdjustment command-line flag, which can be used for disabling adjustment for points returned from /api/v1/query_range handler if they have timestamps closer than -search.latencyOffset to the current time 2021-01-19 22:56:32 +02:00
Aliaksandr Valialkin
011c5da785 app/vmselect/graphite: extract getCanonicalPath() function from loop body inside getCanonicalPaths() 2021-01-18 17:30:26 +02:00
Aliaksandr Valialkin
7fcc0a1ef0 Merge branch 'public-single-node' into pmm-6401-read-prometheus-data-files 2021-01-13 12:55:45 +02:00
Aliaksandr Valialkin
1051d8aa2d app/vmselect/promql: add ability to pass multiple labels to sort_by_label and sort_by_label_desc functions 
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/992
 2021-01-13 12:44:51 +02:00
Nikolay
4cf7a55808
fixes tmpBlockFile remove on prometheus search error (#109) 2021-01-13 11:53:11 +02:00
Aliaksandr Valialkin
d72fc60108 Merge branch 'public-single-node' into pmm-6401-read-prometheus-data-files 2021-01-13 11:41:15 +02:00
Aliaksandr Valialkin
acf1a2c72b app/vmselect/promql: properly parse escaped multibyte utf8 code sequences in metric names and labels names 
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/990
 2021-01-13 10:59:42 +02:00
Aliaksandr Valialkin
1837f2f7d3 app/vmselect/promql: add tfirst_over_time(m[d]) and tlast_over_time(m[d]) MetricsQL functions for returning timestamps for the first and the last samples in m over d 2021-01-12 16:12:12 +02:00
Aliaksandr Valialkin
ba4c49dde6 Merge branch 'public-single-node' into pmm-6401-read-prometheus-data-files 2020-12-27 14:10:59 +02:00
Aliaksandr Valialkin
c86286ec1d app/vmselect/promql: do not ajdust offset value provided in the query 
Previously it could be modified in order to improve response cache hit ratio.
This is unneeded, since cache hit ratio should remain good because the query time range
should be already aligned to multiple of `step` values.

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/976
 2020-12-27 14:09:25 +02:00
Aliaksandr Valialkin
35e5e8ff1e Merge branch 'public-single-node' into pmm-6401-read-prometheus-data-files 2020-12-27 13:32:54 +02:00
Aliaksandr Valialkin
4b7105a65b app/vmselect: sync query stats handling with cluster version 2020-12-27 13:00:29 +02:00
Aliaksandr Valialkin
df0309eae0 app/vmselect/promql: simplify defer call for querystats.RegisterQuery 2020-12-27 12:06:04 +02:00
Aliaksandr Valialkin
4cdbc4642d Merge branch 'public-single-node' into pmm-6401-read-prometheus-data-files 2020-12-25 17:41:24 +02:00
Aliaksandr Valialkin
ad4e6a9283 app/vmselect/querystats: reduce the default number of last queries to track from 100K to 20K 
This should reduce memory usage in constrained environments
 2020-12-25 17:40:47 +02:00
Aliaksandr Valialkin
59183f66d0 app/vmselect: refactor /api/v1/stats/top_queries 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/907
 2020-12-25 16:44:29 +02:00
Nikolay
86630350bf
Adds query stats handler (#945) 
* Adds query stat handler,
for query and query_range api, victoriametrics tracks query execution time,
stats are expored at /api/v1/status/queries endpoint with topN param
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/907

* fixed query stats bugs

* improves queryStats tracker

* improves query stat

* small fix

* fix tests

* added more tests

* fixes 386 tests

* naming fixes

* adds drop for outdated records
 2020-12-25 16:42:05 +02:00
Aliaksandr Valialkin
8593358965 Merge branch 'public-single-node' into pmm-6401-read-prometheus-data-files 2020-12-19 16:43:03 +02:00
Aliaksandr Valialkin
88ac4dfc07 app/vmselect: properly parse negative combined offsets such as -1h2m3s 
Previously such offsets were parsed as `-1h + 2m + 3s`. Now they are parsed as `-(1h + 2m + 3s)`.
 2020-12-19 01:23:46 +02:00
Aliaksandr Valialkin
56054f4eb7 Merge branch 'public-single-node' into pmm-6401-read-prometheus-data-files 2020-12-15 14:33:39 +02:00
Aliaksandr Valialkin
11fa458e39 app/vmselect/promql: return expected increase() result for the first point on the graph with value not exceeding 100 2020-12-15 13:40:46 +02:00
Aliaksandr Valialkin
4f311e5827 Merge branch 'public-single-node' into pmm-6401-read-prometheus-data-files 2020-12-14 14:20:52 +02:00
Aliaksandr Valialkin
5ebfc275e6 app/victoria-metrics: automatically reset response cache when samples with too timestamps older than now - search.cacheTimestampOffset are ingested 2020-12-14 13:08:28 +02:00
Aliaksandr Valialkin
8beb1f9519 Merge branch 'public-single-node' into pmm-6401-read-prometheus-data-files 2020-12-11 21:29:22 +02:00
Aliaksandr Valialkin
ba8ac08739 app/vmselect/graphite: properly handle wildcards and charsets inside curly braces 
For example, `foo{bar*,[a-f]a*b}` should match `foobar`, `foobar123`, `foofab`, etc.

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/952
 2020-12-11 17:24:12 +02:00
Aliaksandr Valialkin
501fd8efd9 Merge branch 'public-single-node' into pmm-6401-read-prometheus-data-files 2020-12-11 12:10:21 +02:00
Aliaksandr Valialkin
4146fc4668 all: properly handle CPU limits set on the host system/container 
This can reduce memory usage on systems with enabled CPU limits.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/946
 2020-12-08 21:07:29 +02:00
Aliaksandr Valialkin
1906f841c9 app/vmselect/promql: do not reduce lookbehind window for any_rollup_func(m) to -search.maxStalenessInterval. It should equal to step value passed to /api/v1/query_range as most users expect 2020-12-08 15:16:17 +02:00
Aliaksandr Valialkin
007dbf273d app/vmselect/graphite: remove duplicate name tag from /tags/autoComplete/tags handler 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/942
 2020-12-07 01:08:52 +02:00
Aliaksandr Valialkin
cb2342029e Merge branch 'public-single-node' into pmm-6401-read-prometheus-data-files 2020-12-05 13:24:28 +02:00
Aliaksandr Valialkin
e2e8ef86d9 app/vmselect/promql: add count_eq_over_time(m[d], N) and count_ne_over_time(m[d], N) for calculating the number of samples in m over d that are equal / not equal to N 2020-12-05 12:30:46 +02:00
Aliaksandr Valialkin
66379cc69f app/vmselect/promql: add label_uppercase(q, label1, ... labelN) and label_lowercase(q, label1, ... labelN) functions 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/936
 2020-12-03 21:47:36 +02:00
Aliaksandr Valialkin
44a34a0f5f app/vmselect/promql: make fmt 2020-12-02 21:33:35 +02:00
Aliaksandr Valialkin
1982505c2b app/vmselect/promql: return nan from minute(m) when m equals to nan 
This aligns VictoriaMetrics behaviour with Prometheus behaviour.

The issue has been spotted in https://promlabs.com/promql-compliance-test-results/2020-12-01/victoriametrics/
 2020-12-02 20:16:58 +02:00
Aliaksandr Valialkin
9d87496b50 app/vmselect/promql: do not return 0 value from sum_over_time(m[d]) when there are no samples on the given d window. 
This aligns the behaviour of `sum_over_time()` with other `_over_time()` functions and with Prometheus behavior.
 2020-12-02 13:12:50 +02:00
Aliaksandr Valialkin
91a4c279cc app/vmselect: return metric values from time() cmp_op metric query when cmp_op comparison is true 
This aligns MetricsQL behavior to Prometheus' one.

The issue has been identified at https://promlabs.com/promql-compliance-test-results/2020-12-01/victoriametrics/
 2020-12-02 12:09:34 +02:00
Aliaksandr Valialkin
700bda8e2e app/vmselect/promql: return nan from a >bool b if a is nan in the same way as Prometheus does 2020-12-02 00:28:26 +02:00
Aliaksandr Valialkin
efdc3c71af app/vmselect/searchutils: return elapsed time in Deadline.String() output 
This should improve debuggability for error messages containing Deadline.String() output
 2020-12-01 00:15:18 +02:00
Aliaksandr Valialkin
afe6d2e736 Merge branch 'public-single-node' into pmm-6401-read-prometheus-data-files 2020-11-25 23:03:44 +02:00
Aliaksandr Valialkin
a724dde90a app/vmselect: protect /tags/delSeries with -deleteAuthKey in the same way as /api/v1/admin/tsdb/delete_series 2020-11-23 15:35:59 +02:00
Aliaksandr Valialkin
f0c207fae2 app/vmselect: add /tags/delSeries handler from Graphite Tags API 
See https://graphite.readthedocs.io/en/stable/tags.html#removing-series-from-the-tagdb
 2020-11-23 15:27:21 +02:00
Aliaksandr Valialkin
d3794eb994 app/{vminsert,vmselect}: move /tags/tagSeries and /tags/tagMultiSeries api from vminsert to vmselect 
This is needed for consistency, since all the `/tags*` api handlers are located in vmselect.
 2020-11-23 12:33:19 +02:00
Aliaksandr Valialkin
f765985947 lib/fs: replace fs.OpenReaderAt with fs.MustOpenReaderAt 
All the callers for fs.OpenReaderAt expect that the file will be opened.
So it is better to log fatal error inside fs.MustOpenReaderAt instead of leaving this to the caller.
 2020-11-23 09:57:21 +02:00
Aliaksandr Valialkin
e1a6262302 lib/fs: replace fs.OpenReaderAt with fs.MustOpenReaderAt 
All the callers for fs.OpenReaderAt expect that the file will be opened.
So it is better to log fatal error inside fs.MustOpenReaderAt instead of leaving this to the caller.
 2020-11-23 09:55:41 +02:00
Aliaksandr Valialkin
fb2d4e56ce Merge branch 'public-single-node' into pmm-6401-read-prometheus-data-files 2020-11-19 18:56:02 +02:00
Aliaksandr Valialkin
2859a452d4 app/vmselect: add remoteAddr to slow query log in order to improve debuggability 
This will simplify identifying the client that sends slow queries to VictoriaMetrics.
 2020-11-18 20:38:32 +02:00
Aliaksandr Valialkin
36b748dfc7 Merge branch 'public-single-node' into pmm-6401-read-prometheus-data-files 2020-11-16 21:06:21 +02:00
Aliaksandr Valialkin
266788be14 app/vmselect: use storage.NewSearchQuery() instead of constructing storage.SearchQuery in-place 
This should prevent from bugs when AccountID and ProjectID aren't set in storage.SearchQuery.
 2020-11-16 18:24:00 +02:00
Aliaksandr Valialkin
cf18df367d app/vmselect/netstorage: apply Graphite filter after substituting __name__ with name 2020-11-16 15:52:16 +02:00
Aliaksandr Valialkin
2c67232565 app/vmselect/graphite: add /tags/autoComplete/values handler from Graphite Tags API 2020-11-16 15:29:35 +02:00
Aliaksandr Valialkin
86f99c6b55 app/vmselect/graphite: add /tags/autoComplete/tags handler from Graphite Tags API 
See https://graphite.readthedocs.io/en/stable/tags.html#auto-complete-support
 2020-11-16 14:50:05 +02:00
Aliaksandr Valialkin
3c1434118e app/vmselect/prometheus: return __name__ label if match[] query to /api/v1/labels matches at least a single time series 2020-11-16 13:54:34 +02:00
Aliaksandr Valialkin
27a417bcd3 app/vmselect/prometheus: improve performance for /api/v1/labels and /api/v1/label/<labelName>/values on time ranges exceeding one day when match[] query arg is set 2020-11-16 13:51:59 +02:00
Aliaksandr Valialkin
6fa806f1ca app/vmselect/prometheus: fix deadlock in /api/v1/series on a time range exceeding one day 2020-11-16 13:30:47 +02:00
Aliaksandr Valialkin
465923b181 app/vmselect/graphite: add /tags/findSeries handler from Graphite Tags API 
See https://graphite.readthedocs.io/en/stable/tags.html#exploring-tags
 2020-11-16 12:53:13 +02:00
Aliaksandr Valialkin
414cd39659 app/vmselect/graphite: apply filter then limit 2020-11-16 04:09:14 +02:00
Aliaksandr Valialkin
d100341394 app/vmselect/graphite: add /tags/<tag_name> handler for Graphite Tags API 2020-11-16 03:42:25 +02:00
Aliaksandr Valialkin
6251762787 app/vmselect/graphite: add /tags handler from Graphite Tags API 
See https://graphite.readthedocs.io/en/stable/tags.html#exploring-tags
 2020-11-16 03:00:01 +02:00
Aliaksandr Valialkin
d9d01f976b app/vmselect/promql: remove spikes from increase() and delta() results on time series with spare irregular data points 
Do not take into account spare data point value if the next point will is located too far from the current point.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/894
 2020-11-13 15:23:44 +02:00
Aliaksandr Valialkin
1f19c167a4 app/vmselect/promql: assume that time series value doesnt change during gaps when calculating increase() and delta() 
This should remove unexpected spikes at the end of gaps.
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/894
 2020-11-13 14:59:24 +02:00
Aliaksandr Valialkin
47a038401b all: consistently return text-based HTTP responses with charset=utf-8 
This is a follow-up for https://github.com/VictoriaMetrics/VictoriaMetrics/pull/897
 2020-11-13 10:35:41 +02:00
Aliaksandr Valialkin
348edd92fe app/vmselect: add -search.treatDotsAsIsInRegexps command-line flag for automatic escaping of dots in regexp label filters 2020-11-11 12:39:07 +02:00
Aliaksandr Valialkin
47390d8947 app/vmselect/promql: do not return data points in the end of the selected time range for time series ending in the middle of the selected time range 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/887
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/845
 2020-11-10 14:51:44 +02:00
Aliaksandr Valialkin
ba4a2c8bca app/vmselect: typo fix in a description for -search.minStalenessInterval: mimimum->minimum 2020-11-10 01:18:08 +02:00
Aliaksandr Valialkin
7a473798b7 Merge branch 'public-single-node' into pmm-6401-read-prometheus-data-files 2020-11-07 01:05:52 +02:00
Aliaksandr Valialkin
dd6bfa50e9 app/vmselect/promql: code cleanup after 43823addea 2020-11-06 01:30:50 +02:00
n4mine
43823addea
app/vmselect/promql: fix when the parameter of maxValue(), minValue() leading by NaN. it will cause {top,bottom}k_{max,min} return inappropriate result (#883) 2020-11-06 01:29:24 +02:00
Aliaksandr Valialkin
56303aee5b Merge branch 'public-single-node' into pmm-6401-read-prometheus-data-files 2020-11-05 02:51:08 +02:00
Aliaksandr Valialkin
368b69b4c4 app/vmselect: properly handle errors in GetLabelsOnTimeRange and GetLabelValuesOnTimeRange 2020-11-05 01:38:38 +02:00
Aliaksandr Valialkin
b378cd6ed8 app/vmselect: optimize querying for /api/v1/labels and /api/v1/label/<name>/values when start and end args are set 2020-11-05 01:01:33 +02:00
Aliaksandr Valialkin
8d8e2ccf5f Merge branch 'public-single-node' into pmm-6401-read-prometheus-data-files 2020-11-04 19:10:41 +02:00
Aliaksandr Valialkin
caeb74f068 app/vmselect: reduce memory usage when query touches big number of time series 2020-11-04 17:04:04 +02:00
Aliaksandr Valialkin
98529e16ee Merge branch 'public-single-node' into pmm-6401-read-prometheus-data-files 2020-11-02 02:12:19 +02:00
Aliaksandr Valialkin
5e75c389e6 app/vmselect/promql: allow dropping trailing sample only for default_rollup function 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/850
 2020-11-02 02:10:59 +02:00
Aliaksandr Valialkin
1b112405a8 Merge branch 'public-single-node' into pmm-6401-read-prometheus-data-files 2020-11-02 00:47:34 +02:00
Aliaksandr Valialkin
cb71af216a app/vmselect/promql: go fmt 2020-11-02 00:15:29 +02:00
Aliaksandr Valialkin
daacbc7e34 app/vmselect/promql: do not drop trailing datapoints for instant queries 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/845
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/748
 2020-11-02 00:12:37 +02:00
Aliaksandr Valialkin
c539494b36 app/vmselect/promql: allow passing optional third argument to topk_* and bottomk_* functions in order to obtain sum of time series outside top/bottom K 2020-11-01 23:35:06 +02:00
Aliaksandr Valialkin
8bbc83e85e Merge branch 'public-single-node' into pmm-6401-read-prometheus-data-files 2020-10-17 12:13:56 +03:00
Aliaksandr Valialkin
28353e48ca app/vmselect/promql: an attempt to improve heuristics for dropping trailing data points in time series 
Now trailing data points are additionally dropped for time series with a single raw sample

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/748
 2020-10-17 10:44:34 +03:00
Aliaksandr Valialkin
4dc13754d8 Merge branch 'public-single-node' into pmm-6401-read-prometheus-data-files 2020-10-13 18:37:48 +03:00
Aliaksandr Valialkin
a2b9476897 app/vmselect/promql: return a single time series at max from absent() function like Prometheus does 2020-10-13 15:56:04 +03:00
Aliaksandr Valialkin
9aa3b65766 app/vmselect/promql: improve time series staleness detection 
This should prevent from double counting for time series at the time when it changes label.
The most common case is in K8S, which changes pod uid label with each new deployment.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/748
 2020-10-13 12:19:57 +03:00
Aliaksandr Valialkin
d8af290947 app/vmselect/promql: fix mode_over_time calculations 
Previously `mode_over_time` could return garbage due to improper shuffling of input data points.
 2020-10-13 11:58:25 +03:00
Aliaksandr Valialkin
1e27420243 app/vmselect/prometheus: fix golangci-lint warning 2020-10-13 09:36:11 +03:00
Aliaksandr Valialkin
4f16a964e3 app/vmselect: add ability to export data in CSV format via /api/v1/export/csv 2020-10-12 20:08:17 +03:00
Aliaksandr Valialkin
762c967855 app/vmselect/promql: keep metric name after applying more functions, which dont change time series meaning 
Functions are:

* keep_last_value
* keep_next_value
* interpolate
* running_min
* running_max
* running_avg
* range_min
* range_max
* range_avg
* range_first
* range_last
* range_quantile
* smooth_exponential

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/674
 2020-10-12 11:47:06 +03:00
Aliaksandr Valialkin
45f7cdc532 Revert "app/vmselect/promql: remove metric name after applying ceil, floor and round functions in order to be more consistent with Prometheus" 
This reverts commit ac45082216.

Reason for revert: the previous behavior for VictoriaMetrics is easier to understand and use by users -
functions, which don't change the meaning of the time series shouldn't drop metric name.

Now the following functions do not drop metric names:

* ceil
* floor
* round

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/674
 2020-10-12 11:40:34 +03:00
Aliaksandr Valialkin
a94825b169 Revert "app/vmselect/promql: remove metric name after applying clamp_min and clamp_max functions in order to be consistent with Prometheus" 
This reverts commit bb61a4769b.

Reason for revert: the previous behavior for VictoriaMetrics is easier to understand and use by users -
functions, which don't change the meaning of the time series shouldn't drop metric name.

Now the following functions do not drop metric name:

* clamp_min
* clamp_max

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/674
 2020-10-12 11:38:27 +03:00
Aliaksandr Valialkin
f7d28bddbf Revert "app/vmselect/promql: remove metric name from results of certain rollup functions in order to be consistent with Prometheus" 
This reverts commit e5202a4eae.

Reason for revert: the previous behavior for VictoriaMetrics is easier to understand and use by users -
functions, which don't change the meaning of the time series shouldn't drop metric name.

Now the following functions do not drop metric name:

* max_over_time
* min_over_time
* avg_over_time
* quantile_over_time
* geomean_over_time
* mode_over_time
* holt_winters
* predict_linear

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/674
 2020-10-12 11:35:18 +03:00
Aliaksandr Valialkin
e7f2907138 Merge branch 'public-single-node' into pmm-6401-read-prometheus-data-files 2020-10-07 21:40:45 +03:00
Aliaksandr Valialkin
e9f2e2cbc9 app/vmselect/promql: add missing label filters to binary operands before query execution 
This implements the optimization described at https://utcc.utoronto.ca/~cks/space/blog/sysadmin/PrometheusLabelNonOptimization

See also https://github.com/cortexproject/cortex/issues/3253
 2020-10-07 21:15:09 +03:00
Aliaksandr Valialkin
59c26feefa app/vmselect/promql: fill gaps on graphs for range_* and running_* functions 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/806
 2020-10-02 13:59:45 +03:00
Aliaksandr Valialkin
9fbb84d5c2 app/vmselect/promql: fill gaps on graphs for range_* and running_* functions 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/806
 2020-10-02 13:58:19 +03:00
Aliaksandr Valialkin
8df5a3c5f6 Merge branch 'public-single-node' into pmm-6401-read-prometheus-data-files 2020-09-30 08:55:57 +03:00
Aliaksandr Valialkin
f7636b0342 app/vmselect/graphite: do not substitute path and path. with path.. in /metrics/find/?format=completer output 2020-09-29 18:03:26 +03:00
Aliaksandr Valialkin
96ee276e6e app/vmselect/prometheus: check for errors returned from bufferedwriter.Write 
This makes `make errcheck` happy
 2020-09-29 11:37:01 +03:00
Aliaksandr Valialkin
6fdfc67620 app/vmselect/graphite: properly handle case when /metrics/find finds both leaf and node for the given query=prefix.* 
In this case only node must be returned with stripped dot in the end of id as carbonapi does
 2020-09-29 11:01:59 +03:00
Aliaksandr Valialkin
1b3efccb24 app/vmselect: stop /api/v1/export/* execution if client disconnects 2020-09-27 23:53:13 +03:00
Aliaksandr Valialkin
95688cbfc5 all: add native format for data export and import 
The data can be exported via [/api/v1/export/native](https://victoriametrics.github.io/#how-to-export-data-in-native-format) handler
and imported via [/api/v1/import/native](https://victoriametrics.github.io/#how-to-import-data-in-native-format) handler.
 2020-09-27 19:54:07 +03:00
Aliaksandr Valialkin
82973f8ae7 Revert "lib/storage: remove unused fetchData arg from BlockRef.MustReadBlock" 
This reverts commit bab6a15ae0.

Reason for revert: the `fetchData` arg is used in cluster branch.
Leaving this arg in master branch makes smaller the diff with cluster branch.
 2020-09-24 22:44:23 +03:00
Aliaksandr Valialkin
9d3eb3f4b8 Merge branch 'public-single-node' into pmm-6401-read-prometheus-data-files 2020-09-24 20:49:16 +03:00
Aliaksandr Valialkin
bab6a15ae0 lib/storage: remove unused fetchData arg from BlockRef.MustReadBlock 
This arg became unused after 23bdc1f107
 2020-09-24 20:48:40 +03:00
Aliaksandr Valialkin
23bdc1f107 app/vmselect/netstorage: do not spend CPU time on unpacking empty blocks during /api/v1/series calls 2020-09-24 20:18:10 +03:00
Aliaksandr Valialkin
8fc8874db4 Merge branch 'public-single-node' into pmm-6401-read-prometheus-data-files 2020-09-23 23:25:34 +03:00
Aliaksandr Valialkin
c584aece38 app/vmselect/promql: properly limit implicitly set rollup window to -search.maxStalenessInterval 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/784
 2020-09-23 23:23:59 +03:00
Aliaksandr Valialkin
ff1cbb524e app/vmselect: obtain labels and label values from Prometheus storage inside netstorage package 2020-09-23 22:43:01 +03:00
Aliaksandr Valialkin
a70df4bd83 PMM-6401 Initial implementaton for reading data from Prometheus files 2020-09-23 14:26:39 +03:00
Aliaksandr Valialkin
27500d7d4c app/vmselect/prometheus: code cleanup after 3ba507000c 2020-09-23 13:04:17 +03:00
Aliaksandr Valialkin
3ba507000c app/vmselect/prometheus: return timestamps from /api/v1/query, which match the time query arg 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/720
 2020-09-23 12:58:48 +03:00
Aliaksandr Valialkin
bed25e3c24 app/vmselect/netstorage: properly pre-allocate space for sbs 2020-09-22 23:49:55 +03:00
Aliaksandr Valialkin
09b0f7c202 app/vmselect/netstorage: release search resources on timeout errors 
Previously these resources weren't released, which could lead to resource leaks.
 2020-09-22 22:57:38 +03:00
Aliaksandr Valialkin
3b1e3a03e0 app/vmselect: make sure the request doesnt wait in pending queue more than the configured timeout 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/711
 2020-09-22 01:23:19 +03:00
Aliaksandr Valialkin
00b5145c69 app/vmselect/searchutils: fixed tests after 2eb72e09ab 2020-09-21 21:31:38 +03:00
Aliaksandr Valialkin
2eb72e09ab app/vmselect: use time value rounded to seconds if it isnt passed to /api/v1/query 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/720
 2020-09-21 21:24:40 +03:00
Aliaksandr Valialkin
98d1cd0971 app/vmselect/graphite: return proper results /metrics/find?query=foo.*.bar according to Graphite Metrics API 2020-09-18 11:00:00 +03:00
Aliaksandr Valialkin
9f79bcf64a app/vmselect: improve description for -search.maxQueryDuration 2020-09-16 21:15:41 +03:00
Aliaksandr Valialkin
1fec47a289 app/vmselect/netstorage: reduce memory usage when the time range from query touches big number of samples per each time series 2020-09-15 21:08:28 +03:00
Aliaksandr Valialkin
8c3d7c1a59 app/vmselect: typo fix in -search.maxStalenessInterval description 2020-09-15 14:24:27 +03:00
Aliaksandr Valialkin
0e533d1a9c app/vmselect/promql: support composite durations like Prometheus 2.21 does 
The following durations are supported now: `1h5m35s` or `1s543ms`

See https://github.com/prometheus/prometheus/releases/tag/v2.21.0
and https://github.com/prometheus/prometheus/pull/7713
 2020-09-11 23:39:13 +03:00
Aliaksandr Valialkin
a2f647d142 app/vmselect/prometheus: typo fix in the description for -search.latencyOffset command-line flag 2020-09-11 14:16:46 +03:00