Aliaksandr Valialkin
b796a0dc3f
app/vmselect/promql: optimize e1 op e2 when e1 returns an empty result
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3349
2022-11-21 16:09:10 +02:00
Aliaksandr Valialkin
cae0f37edd
app/vmselect/netstorage: remove superflouos map lookup at ProcessSearchQuery
...
This should reduce CPU usage a bit during querying
2022-11-18 13:40:04 +02:00
Yury Molodov
519bd2af7b
vmui: add trace analyzer ( #3310 )
...
* refactor: change structure project
* refactor: change structure project
* fix: add hooks for set query params
* refactor: add index for pages
* docs: add TESTCASES.md
* refactor: restructure components
* feat: add page with trace analyzer
* fix: change detect trace data
* Update app/vmui/packages/vmui/src/pages/TracePage/index.tsx
Co-authored-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
* Update app/vmui/packages/vmui/src/pages/TracePage/index.tsx
Co-authored-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
* fix: change descriptions on trace page
* Update app/vmui/packages/vmui/src/pages/TracePage/index.tsx
Co-authored-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
* feat: add base components
* feat: add reset styles
* docs: add description about trace analyzer
* feat: add styles for custom panel page
* feat: add styles for predefined panels
* feat: add style for TracingsView.tsx
* feat: add Alerts
* feat: add Tooltip.tsx
* fix: correct styles
* feat: add DatePicker.tsx
* feat: add tables
* feat: add theme provider
* fix: replace using callbacks as props to handlers
* fix: correct update time
* fix: change TimePicker.tsx
* fix: correct styles
* fix: update packages
* vmui: refactor code, remove material-ui
* feat: add paste json for trace analyzer
* vmui: update trace analyzer docs
* app/vmselect/vmui: `make vmui-update`
Co-authored-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-11-17 22:22:01 +02:00
Aliaksandr Valialkin
578bb58ea9
app/vmselect/vmui: make vmui-update after 51bfd1ab80
2022-11-17 18:54:55 +02:00
Aliaksandr Valialkin
a21c8e7b9a
app/vmselect/vmui: make vmui-update after bc8a782f74
2022-11-17 01:15:45 +02:00
Aliaksandr Valialkin
a260e2659e
app/vmselect/promql: add range_stdvar() and range_stddev() functions for calculating variance and deviation over time series on the selected time range
2022-11-17 01:03:40 +02:00
Aliaksandr Valialkin
c1a3192d8b
app/vmselect/promql: add range_linear_regression(q) function for calculating simple linear regression for the selected time series on the selected time range
2022-11-17 00:38:48 +02:00
Aliaksandr Valialkin
6c7361b1c5
app/vmselect/vmui: make vmui-update after 7130af7fd2
2022-11-09 16:43:18 +02:00
Aliaksandr Valialkin
9565dbed34
app/vmselect/vmui: make vmui-update after 54e1865d17
2022-10-28 14:51:23 +03:00
Aliaksandr Valialkin
3c66e45ef0
app/vmselect/vmui: make vmui-update after eae6063450
2022-10-26 02:50:46 +03:00
Aliaksandr Valialkin
c4265322f4
lib/fs: add canOverwrite arg to WriteFileAtomically when it is allowed to overwrite the file atomically if it already exists
2022-10-26 01:07:34 +03:00
Aliaksandr Valialkin
b7882dc9af
app/vmselect/vmui: make vmui-update after 274e235bf7
2022-10-24 21:29:13 +03:00
Aliaksandr Valialkin
a10647e0bf
app/vmselect/promql: expose missing metric vm_cache_size_max_bytes{type="promql/rollupResult"}
2022-10-23 12:13:47 +03:00
Aliaksandr Valialkin
3bb3893b2d
app/vmselect/vmui: make vmui-update after 080562030d
2022-10-18 10:50:35 +03:00
Yury Molodov
ff6151fa49
vmui: limit number of plotted series ( #3229 )
...
* feat: add maximum display series by tabs
* feat: add warning on PredefinedPanels.tsx
* docs/CHANGELOG.md: vmui limit number of plotted series
* docs/CHANGELOG.md: vmui limit number of plotted series
* wip
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-10-13 12:13:47 +03:00
Aliaksandr Valialkin
e6fd33044f
app/vmselect/promql: follow-up for 930f1ee153
...
Document the change at docs/CHANGELOG.md
Apply it to histogram_quantile() in the same way as to histogram_share()
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/3225
2022-10-13 12:03:08 +03:00
Siqi Liu
930f1ee153
BUGFIX: properly calculate histogram_quantile with the same value and different string le ( #3225 )
...
Co-authored-by: 647(siki.liu) <siki.liu@huolala.cn>
2022-10-13 11:57:16 +03:00
Aliaksandr Valialkin
b8da90b893
app/vmselect/promql: properly handle zero and negative values for -search.maxMemoryPerQuery
...
This is a follow-up for 04a05f161chttps://github.com/VictoriaMetrics/VictoriaMetrics/issues/3203
2022-10-12 09:25:17 +03:00
Aliaksandr Valialkin
04a05f161c
app/vmselect: return back the logic for limits the amounts of memory occupied by concurrently executed queries if -search.maxMemoryPerQuery isn't set
...
This is needed for preserving backwards compatibility with the previous releases of VictoriaMetrics.
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3203
2022-10-10 21:45:13 +03:00
Aliaksandr Valialkin
8e1ccecd97
docs: mention -search.maxMemoryPerQuery in the description to -search.maxConcurrentQueries command-line flag
...
This is a follow-up for 5138eaeea0https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3203
2022-10-09 13:56:59 +03:00
Aliaksandr Valialkin
5138eaeea0
app/vmselect: allow limiting per-query memory usage via -search.maxMemoryPerQuery command-line flag
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3203
2022-10-08 01:08:05 +03:00
Yury Molodov
27ed4b853e
vmui: auto-update chart after query field removed ( #3210 )
...
* feat: run query after query field removed
* app/vmselect/vmui: `make vmui-update`
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-10-07 23:02:41 +03:00
Aliaksandr Valialkin
8322760647
app/vmselect/vmui: make vmui-update after a54987f671
2022-10-07 03:31:17 +03:00
Aliaksandr Valialkin
f452c84579
app/vmselect/promql: properly calculate vm_rows_scanned_per_query histogram for rollup functions, which take into account only a few samples on the provided lookbehind window
2022-10-06 23:22:24 +03:00
Aliaksandr Valialkin
40e899fd67
app/vmselect/promql: properly calculate quantiles_over_time() over a single raw sample
2022-10-06 22:37:21 +03:00
Aliaksandr Valialkin
d9282027e6
app: follow-up after ec04fcac93
...
* Optimize fast path for /api/v1/import when importing numeric values
* Move the docs about the change from features to bugfixes at docs/CHANGELOG.md
* Update tests at lib/protoparser/vmimport
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3161
2022-10-06 14:52:02 +03:00
Dmytro Kozlov
ec04fcac93
Properly parse json when export import metric ( #3180 )
...
* app/vmselect: properly work when export import json from `api/v1/{export, import}` API
* app/vmselect: update convert function
* app/vmselect: export null if `math.IsNaN(v)`
* app/vmselect: get float from json
* lib/protoparser: add test
* docs: add change log
* lib/protoparser: make export import api compatible
2022-10-06 13:54:20 +03:00
Aliaksandr Valialkin
c53b7e66ef
app/vmselect: improve performance scalability on multi-CPU systems for /api/v1/export/... endpoints
2022-10-01 22:05:43 +03:00
Aliaksandr Valialkin
49311ae977
app/vmselect/prometheus: improve scalability of /federate endpoint on systems with many CPU cores
...
Minimize usage of global lock inside bufferedwriter.Write() when processing `/federate` data
on systems with many CPU cores
2022-10-01 20:13:24 +03:00
Aliaksandr Valialkin
fb1cc3cc94
app/vmselect/promql: increase scalability of incremental aggregate calculations on systems with many CPU cores
...
Use sync.Map instead of a global mutex there. This should lift scalability limits
on systems with many CPU cores.
2022-10-01 20:00:03 +03:00
Aliaksandr Valialkin
fcc7ab71b3
app/vmselect: do not export NaN values for stale metrics at /federate endpoint
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3185
2022-10-01 19:47:37 +03:00
Aliaksandr Valialkin
725dfb0ed6
lib/httpserver: use 302 redirects instead of 301 redirects
...
Incorrect 301 redirects can be cached by user agents such as web browsers.
This can complicate recovery procedure after the incorrect redirect is fixed,
e.g. web browser cache must be reset.
The related issue - https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1752
2022-10-01 16:53:35 +03:00
Aliaksandr Valialkin
b70f815dc4
app/vmselect/promql: remove empty series before applying aggregate function
...
Previously empty series (e.g. series with all NaN samples) were passed to aggregate functions.
Such series must be ingored by all the aggregate functions.
So it is better from consistency PoV filtering out empty series before applying aggregate functions.
2022-09-30 08:39:54 +03:00
Roman Khavronenko
b64b9b9fec
app/vmselect: ignore empty series for limit_offset ( #3178 )
...
* app/vmselect: ignore empty series for `limit_offset`
VictoriaMetrics doesn't return empty series (with all NaN values) to
the user. But such series are filtered after transform functions.
It means `limit_offset` will account for empty series as well.
For example, let's consider following data set:
```
time series:
foo{label="1"} NaN, NaN, NaN, NaN // empty series
foo{label="2"} 1, 2, 3, 4
foo{label="3"} 4, 3, 2, 1
```
When user requests all series for metric `foo` the empty series
will be filtered out:
```
/query=foo:
foo{label="v2"} 1, 2, 3, 4
foo{label="v3"} 4, 3, 2, 1
```
But `limit_offset(1, 1, foo)` is applied to original series, not filtered yet.
So it will return `foo{label="v2"}` (skips the first in list)
```
/query=limit_offset(1, 1, foo):
foo{label="v2"} 1, 2, 3, 4
```
Expected result would be to apply `limit_offset` to already filtered list,
so in result we receive `foo{label="v3"}`:
```
/query=limit_offset(1, 1, foo):
foo{label="v3"} 4, 3, 2, 1
```
The change does exactly that - filters empty series before applying `limit_offset`.
Signed-off-by: hagen1778 <roman@victoriametrics.com>
* app/vmselect: ignore empty series for `limit_offset`
Signed-off-by: hagen1778 <roman@victoriametrics.com>
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-09-30 08:20:34 +03:00
Roman Khavronenko
166d444159
vmselect/rollup: rm workaround for slow-changing counters ( #3163 )
...
The workaround was introduced to fix https://github.com/VictoriaMetrics/VictoriaMetrics/issues/962 .
However, it didn't prove itself useful. Instead, it is recommended using `increase_pure` function.
Removing the workaround makes VM to produce accurate results when calculating
`delta` or `increase` functions over slow-changing counters with vary intervals
between data points.
Signed-off-by: hagen1778 <roman@victoriametrics.com>
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-09-26 15:33:25 +03:00
Dmytro Kozlov
b75f1854c5
vmselect/promql: add alphanumeric sort by label (sort_by_label_numeric) ( #2982 )
...
* vmselect/promql: add alphanumeric sort by label (sort_by_label_numeric)
* vmselect/promql: fix tests, add documentation
* vmselect/promql: update test
* vmselect/promql: update for alphanumeric sorting, fix tests
* vmselect/promql: remove comments
* vmselect/promql: cleanup
* vmselect/promql: avoid memory allocations, update functions descriptions
* vmselect/promql: make linter happy (remove ineffectual assigment)
* vmselect/promql: add test case, fix behavior when strings are equal
* vendor: update github.com/VictoriaMetrics/metricsql from v0.44.1 to v0.45.0
this adds support for sort_by_label_numeric and sort_by_label_numeric_desc functions
* wip
* lib/promscrape: read response body into memory in stream parsing mode before parsing it
This reduces scrape duration for targets returning big responses.
The response body was already read into memory in stream parsing mode before this change,
so this commit shouldn't increase memory usage.
* wip
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-09-14 17:41:09 +03:00
Aliaksandr Valialkin
99bc18774c
app/vmselect/vmui: make vmui-update after 1304824201
2022-09-14 13:33:06 +03:00
Yury Molodov
1b41169415
vmui: fix data processing ( #3092 )
...
* fix: change data processing
* app/vmselect/vmui: `make vmui-update`
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-09-12 10:55:11 +03:00
Yury Molodov
9541ef2e9e
vmui: add lists of top queries ( #3065 )
...
* feat: add lists of top queries
* fix: change the field label
* refactor: add handlers for readability
* app/vmselect: `make vmui-update`
* docs: document `top queries` tab
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-09-08 21:43:37 +03:00
Dmytro Kozlov
4415c71a2b
vmselect/{promql, prometheus}: show flag names which user can update in error message ( #3049 )
...
* vmselect/{promql, prometheus}: show flag names which user can update in error message
* vmselect/{promql, prometheus}: fix typo
2022-09-06 13:25:59 +03:00
Aliaksandr Valialkin
ae31b2363f
app/vmselect/prometheus: follow-up after 50e2524bc2
...
- Add getCommonParamsWithDefaultDuration function and use it at /api/v1/series, /api/v1/labels and /api/v1/label/.../values
- Document the default behaviour for setting 5 minutes time range if start arg isn't passed to /api/v1/series, /api/v1/labels and /api/v1/label/.../values
- Document the change at docs/CHANGELOG.md
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/3052
2022-09-05 11:55:48 +03:00
匠心零度
50e2524bc2
api prometheus/api/v1/label/../values time not specified, ( #3052 )
...
modify default start values
2022-09-05 11:52:19 +03:00
Aliaksandr Valialkin
7dc632719d
app/vmselect/promql: consistently calculate rate_over_sum(m[d]) as sum_over_time(m[d])/d
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3045
2022-09-02 23:18:04 +03:00
Aliaksandr Valialkin
4076277cf0
app/vmselect/promql: evaluate union() args in parallel in order to increase query performance
...
Note that the parallel execution of `union()` args may take more memory and CPU time
than the sequential execution if args contain heavy queries, which may load all the available CPU,
disk and memory resources and vmselect and vmstorage levels.
2022-09-02 19:46:27 +03:00
匠心零度
6d81584d2a
reduce unnecessary vmstorage query ( #3031 )
...
* reduce unnecessary vmstorage query
* reduce unnecessary vmstorage query
* rollback limit logic /api/v1/label/*
2022-08-30 12:36:54 +03:00
Aliaksandr Valialkin
ad11b8d83d
app/vmselect/promql: follow-up after 2d71b4859c
...
- Use getScalar() function for obtaining the expected scalar from phi arg
- Reduce the error message returned to the user when incorrect phi is passed to histogram_quantiles
- Improve the description of this bugfix in the docs/CHANGELOG.md
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3026
2022-08-27 01:35:49 +03:00
Dmytro Kozlov
2d71b4859c
vmselect/promql: fix panic in histogram_quantiles function ( #3029 )
...
* vmselect/promql: fix panic in histogram_quantiles function
* Update docs/MetricsQL.md
Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
2022-08-27 01:33:56 +03:00
Dmytro Kozlov
463ea6897b
vmselect/promql: enable search.maxPointsSubqueryPerTimeseries for sub-queries ( #2963 )
...
* vmselect/promql: enable search.maxPointsPerTimeSeriesSubquery for sub-queries
* vmselect/promql: cleanup
* vmselect/promql: rename config flag
* vmselect/promql: add tests
* vmselect/promql: use test object instead of log
* vmselect/promql: fix posible panic is subquery has more points. add description
* vmselect/promql: update tests descriptions
* vmselect/promql: update doInternal validation
* vmselect/promql: fix linter
* vmselect/promql: fix linter
* vmselect/promql: update documentation and release notes
* wip
- Properly apply -search.maxPointsSubqueryPerTimeseries limit to subqueries.
Previously the -search.maxPointsPerTimeseries limit was unexpectedly applied to subqueries
if it was smaller than the -search.maxPointsSubqueryPerTimeseries .
- Clarify docs for -search.maxPointsSubqueryPerTimeseries command-line flag .
- Document -search.maxPointsPerTimeseries and -search.maxPointsSubqueryPerTimeseries flags at https://docs.victoriametrics.com/#resource-usage-limits .
- Update docs/CHANGELOG.md .
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2922
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-08-24 15:25:18 +03:00
Aliaksandr Valialkin
343241680b
all: remove the remaining bits of io/ioutil
...
The io/ioutil package is deprecated since Go1.16 - see https://tip.golang.org/doc/go1.16#ioutil
VictoriaMetrics requires at least Go1.18, so it is time to remove the io/ioutil from source code
This is a follow-up for 02ca2342ab
2022-08-22 00:20:58 +03:00
Aliaksandr Valialkin
9f94c295ab
all: use os.{Read|Write}File instead of ioutil.{Read|Write}File
...
The ioutil.{Read|Write}File is deprecated since Go1.16 -
see https://tip.golang.org/doc/go1.16#ioutil
VictoriaMetrics needs at least Go1.18, so it is safe to remove ioutil usage
from source code.
This is a follow-up for 02ca2342ab
2022-08-21 23:52:35 +03:00
Aliaksandr Valialkin
4ac79d29ad
app/vmselect: follow-up after 63e0f16062
...
* Explicitly store a pointer to UserReadableError in the error interface.
Previously Go automatically converted the value to a pointer before storing in the error interface.
* Add Unwrap() method to UserReadableError, so it can be used transparently with the other code,
which calls errors.Is() and errors.As().
* Document the change in docs/CHANGELOG.md
2022-08-15 13:50:16 +03:00
Roman Khavronenko
63e0f16062
vmselect: introduce UserReadableError type of error ( #2894 )
...
When read query fails, VM returns rich error message with
all the details. While these details might be useful
for debugging specific cases, they're usually too verbose
for users.
Introducing a new error type `UserReadableError` is supposed
to allow to return to user only the most important parts
of the error trace. This supposed to improve error readability
in web interfaces such as VMUI or Grafana.
The full error trace is still logged with the full context
and can be found in vmselect logs.
Signed-off-by: hagen1778 <roman@victoriametrics.com>
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-08-15 13:38:47 +03:00
Yury Molodov
1513866d51
vmui: shortcut keys legend ( #2971 )
...
* feat: add shortcut modal
* feat: add shortcut descriptions
* app/vmselect/vmui: `make vmui-update`
* docs/CHANGELOG.md: document the change
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-08-15 11:54:47 +03:00
Yury Molodov
3cb013aeb8
vmui: graph action on moush hold and move ( #2915 )
...
* fix: change event for graph panning
* fix: change detect key
* feat: add zoom in with mouse selection
* - document the change
- run `make vmui-update`
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-08-07 23:55:09 +03:00
Aliaksandr Valialkin
7478d423c5
app/vmselect/netstorage: cleanup after 92630c1ab4
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2896
2022-08-04 18:28:11 +03:00
Aliaksandr Valialkin
6b0550c023
app/{vmselect,vmalert}: properly generate http redirects if -http.pathPrefix command-line flag is set
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2918
2022-08-02 12:59:07 +03:00
Aliaksandr Valialkin
c2bd75926b
app/vmselect/netstorage: initialize tsw.rowsProcessed before calling tsw.f, since tsw.f can modify r.Timestamps and r.Values lengths
2022-07-30 00:39:36 +03:00
Aliaksandr Valialkin
19a0b4679a
app/vmselect/netstorage: re-use random generator used for series shuffle in Result.RunParallel
...
This should reduce CPU usage needed for rand.Rand initialization
2022-07-30 00:30:37 +03:00
Dmytro Kozlov
a927814e7b
vmselect/promql: add tests for vmrangeBucketsToLE ( #2907 )
...
* vmselect/promql: add tests for vmrangeBucketsToLE
* vmselect/promql: cleanup
* vmselect/promql: cleanup
* vmselect/promql: fix panic tests want result
* vmselect/promql: cleanup
* vmselect/promql: update test name
* vmselect/promql: fix linter error
* vmselect/promql: refactor testcases
* vmselect/promql: cleanup
* vmselect/promql: remove unused reassign to workers, fix typo
* wip
* wip
* wip
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-07-26 20:42:41 +03:00
Aliaksandr Valialkin
c888e6b9be
app/vmselect/promql: reduce the diff for f148cffc8a
...
This is a follow-up for c826f06366
2022-07-26 19:20:48 +03:00
Alan Liang
c826f06366
vmselect: fix vmrangeBucketsToLE func may panic when ts value equal zero ( #2902 )
...
Co-authored-by: alanwzliang <alanwzliang@tencent.com>
2022-07-25 10:55:13 +03:00
Aliaksandr Valialkin
92630c1ab4
app/vmselect/netstorage: improve the speed of queries over big number of time series on multi-CPU system
...
Reduce inter-CPU communications when processing the query over big number of time series.
This should improve performance for queries over big number of time series
on systems with many CPU cores.
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2896
Based on b596ac3745
2022-07-25 09:18:44 +03:00
Aliaksandr Valialkin
79d967d35a
app/vmselect/vmui: make vmui-update after edecd2493c
2022-07-21 20:59:52 +03:00
Roman Khavronenko
9ccf695d57
vmselect: return correct error for second part of expression ( #2893 )
...
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-07-20 16:44:28 +02:00
Aliaksandr Valialkin
cc7d499bbd
app/vmselect/promql: execute q1 and q2 from q1 op q2 in parallel if labels pushdown cannot be applied
...
This should improve query performance if VictoriaMetrics has enough resources for processing `q1` and `q2` in parallel.
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2886
2022-07-19 14:27:48 +03:00
Aliaksandr Valialkin
b5da47bfaf
app/vmselect/promql: properly return q1 series from q1 ifnot q2 when q2 returns nothing
2022-07-18 14:24:54 +03:00
Aliaksandr Valialkin
0792c4ca90
app/vmselect/promql/transform.go: reuse evalNumber() function for constructing timezone_offset() results
2022-07-18 14:24:53 +03:00
Yury Molodov
17c33132df
vmui: optimize table view ( #2867 )
...
* feat: optimize table view
* fix: add column display setting
* app/vmselect: `make vmui-update`
Also document the change at docs/CHANGELOG.md
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-07-14 00:15:43 +03:00
Aliaksandr Valialkin
29e53b9f55
app/vmselect/promql: consistency update after 93fbd0c54b
2022-07-13 12:33:14 +03:00
Roman Khavronenko
93fbd0c54b
promql: return step as scrapeInterval when it can't be calculated ( #2865 )
...
The change allows to specify default value for `getScrapeInterval`
function when actual interval can't be calculated.
Before the change, function were returning `maxSilenceInterval` (5m)
in such cases, which may be not correct for instant queries processing.
The specific scenario where using `maxSilenceInterval` caused issues
is the following:
1. Series becomes stale;
2. Client (in this case vmalert) continues to request series every 15s;
3. Database returns empty results as expected;
4. But at some specific moment of time database returns datapoints from `now()-5m`,
because lookback window was extended to `maxSilenceInterval`.
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-07-13 12:27:38 +03:00
Aliaksandr Valialkin
c2197ad139
app/vmselect/promql: validate function name before evaluating its arguments
...
This avoids unneeded evaluation of args for unknown functions
2022-07-12 19:48:26 +03:00
Aliaksandr Valialkin
159c2e15e3
app/vmselect/netstorage: optimize mergeSortBlocks() for the worst case when blocks contain interleaved samples
2022-07-12 12:31:38 +03:00
Aliaksandr Valialkin
8429d4af5a
app/vmselect/netstorage: add mergeSortBlocks benchmark for the worstcase
2022-07-12 12:31:36 +03:00
Aliaksandr Valialkin
cad471037a
app/vmselect/prometheus: follow-up after 3efe33b917
2022-07-11 20:35:28 +03:00
Dmytro Kozlov
3efe33b917
vmselect/prometeus: Add limit param to api/v1/series api endpoint ( #2851 )
...
* issue-2841: Add limit param to api/v1/series api endpoint
* issue-2841: add change log
* issue-2841: update logic
* issue-2841: simplify logic
* issue-2841: simplify logic, add information to documentation
2022-07-11 20:18:30 +03:00
Aliaksandr Valialkin
ce68e76d62
app/vmselect: follow-up after 8667307d73
2022-07-11 20:14:34 +03:00
Roman Khavronenko
8667307d73
vmselect: cover special cases for vmalert's routing in single-node version ( #2845 )
...
* vmselect: cover special cases for vmalert's routing in single-node version
* remove trailing `/` from requests
* redirect to vmalert's home page when `/vmalert` is requested.
Signed-off-by: hagen1778 <roman@victoriametrics.com>
* vmalert: fix review comments
Signed-off-by: hagen1778 <roman@victoriametrics.com>
* Update app/vmselect/main.go
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-07-11 19:52:22 +03:00
Aliaksandr Valialkin
5c8eee26bf
all: make fmt via the upcoming Go1.19
2022-07-11 19:22:15 +03:00
Aliaksandr Valialkin
cd09f583fe
app/vmselect/netstorage: add benchmarks for mergeSortBlocks
...
This is a follow-up for 743ff84863
2022-07-11 12:54:48 +03:00
Aliaksandr Valialkin
743ff84863
app/vmselect/netstorage: optimize mergeSortBlocks function
...
- Use binary search instead of linear scan when locating the run of smallest timestamps
in blocks with intersected time ranges. This should improve performance
when merging blocks with big number of samples
- Skip samples with duplicate timestamps. This should increase query performance
in cluster version of VictoriaMetrics with the enabled replication.
2022-07-09 00:34:42 +03:00
Roman Khavronenko
e1a41cfab5
metricsql: properly evaluate timezone_offset over time interval ( #2842 )
...
* metricsql: properly evaluate `timezone_offset` over time interval
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2771
Signed-off-by: hagen1778 <roman@victoriametrics.com>
* Update docs/CHANGELOG.md
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-07-08 14:03:56 +03:00
Aliaksandr Valialkin
2e9ae40d56
app/vmselect/vmui: follow-up after 0bf6841140
...
* Document the bugfix in docs/CHANGELOG.md
* Run `make vmui-update` for updating static js files for vmui, which are included into vmselect
2022-07-08 13:14:17 +03:00
Aliaksandr Valialkin
a6bd442ef9
app/vmui: tune visual presentation of trace view
2022-07-06 14:07:12 +03:00
Aliaksandr Valialkin
bd5b20445e
app/vmselect: add ability to query vmselect from another vmselect
2022-07-06 13:30:12 +03:00
Aliaksandr Valialkin
17de8a41c2
all: follow-up after ed89106274
2022-07-06 12:44:46 +03:00
Aliaksandr Valialkin
77cbbacfdb
lib/vmselectapi: pass storage.SearchQuery to API calls instead of []*storage.TagFilters + storage.TimeRange + maxMetrics
...
This reduces the number of args to vmselectapi calls
2022-07-06 12:37:54 +03:00
Aliaksandr Valialkin
f435924ab3
lib/vmselectapi: pass maxSuffixes arg to tagValueSuffixes RPC call
2022-07-06 12:37:54 +03:00
Aliaksandr Valialkin
e1b8059086
lib/vmselectapi: rename deleteMetrics to more correct deleteSeries
2022-07-06 12:37:54 +03:00
Aliaksandr Valialkin
a60e03b3a7
lib/vmselectapi: use string type for tagKey and tagValuePrefix args at TagValueSuffixes()
...
This improves the API consistency
2022-07-06 12:37:53 +03:00
Roman Khavronenko
ed89106274
vmselect: allow proxying requests to vmalert from single-node ( #2834 )
...
The change allows to proxy requests with prefix `/vmalert`
to the vmalert component if `-vmalert.proxyURL` is set.
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2825
and https://github.com/VictoriaMetrics/VictoriaMetrics/pull/2831
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-07-06 10:47:26 +02:00
Aliaksandr Valialkin
ae80cf76e0
app/vmselect: make fmt after f3ece83e67
2022-07-05 14:35:24 +03:00
Aliaksandr Valialkin
f3ece83e67
app/vmselect/promql: properly calculate histogram_quantile over unexpected le buckets
...
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2819
2022-07-05 13:19:24 +03:00
Aliaksandr Valialkin
84e373e5c7
app/vmselect/promql: properly handle partial counter resets in rate(), irate(), increase() and remove_resets() functions
2022-06-30 22:39:38 +03:00
Aliaksandr Valialkin
3e2dd85f7d
all: readability improvements for query traces
...
- show dates in human-readable format, e.g. 2022-05-07, instead of a numeric value
- limit the maximum length of queries and filters shown in trace messages
2022-06-30 18:20:33 +03:00
Dmytro Kozlov
4d9715f5a8
vmui: update render logic for nested component ( #2795 )
...
* vmui: update render logic for nested component, avoid rerender, remove local storage usage for tracing flag
* docs/url-examples.md: fix various documentation issues there
* docs: add Troubleshooting doc
This doc contains troubleshooting guides for typical problems with VictoriaMetrics.
* docs/Troubleshooting.md: add troubleshooting guide for cluster instability
* wip
* wip
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-06-30 15:47:12 +03:00
Aliaksandr Valialkin
a14188dd8e
app/vmselect: expose additional histograms at /metrics page, which may help get more insights for the query workload
...
This commit is based on https://github.com/VictoriaMetrics/VictoriaMetrics/pull/2792
2022-06-28 20:18:13 +03:00
Aliaksandr Valialkin
a43f2d0bc5
app/vmselect/promql: show the number of scanned samples in the query trace
2022-06-28 19:26:17 +03:00
Aliaksandr Valialkin
a5181703b1
app/vmselect/prometheus: reduce the default value for -search.maxSeries from 100k to 30k
...
Production experience shows that 100k is too big for /api/v1/series .
It leads to increased CPU usage when Grafana queries /api/v1/series over VictoriaMetrics
with big number of time series during auto-completion and when modifying template variables.
2022-06-28 18:22:30 +03:00
Aliaksandr Valialkin
a350d1e81c
lib/storage: return marshaled metric names from SearchMetricNames
...
Previously SearchMetricNames was returning unmarshaled metric names.
This wasn't great for vmstorage, which should spend additional CPU time
for marshaling the metric names before sending them to vmselect.
While at it, remove possible duplicate metric names, which could occur when
multiple samples for new time series are ingested via concurrent requests.
Also sort the metric names before returning them to the client.
This simplifies debugging of the returned metric names across repeated requests to /api/v1/series
2022-06-28 18:17:15 +03:00
Aliaksandr Valialkin
e578549b8a
app/vmselect: optimize /api/v1/series a bit for time ranges smaller than one day
2022-06-28 13:02:47 +03:00
Aliaksandr Valialkin
a963b2a0aa
all: show timeRange in traces in human-readable format instead of timestamps in milliseconds
2022-06-27 13:45:51 +03:00
Aliaksandr Valialkin
ba514284f1
lib/storage: add querytracer to more contexts
...
querytracer has been added to the following storage.Storage methods:
- RegisterMetricNames
- DeleteMetrics
- SearchTagValueSuffixes
- SearchGraphitePaths
2022-06-27 13:45:51 +03:00
Aliaksandr Valialkin
9a314106ca
app/vmselect/netstorage: remove Get prefix from netstorage functions
...
This makes these function names more consistent with the server side
2022-06-27 00:45:05 +03:00
Dmytro Kozlov
bb7f31541f
vmui: added query tracing ( #2748 )
...
* vmui: added query tracing
* vmui: updated ui
* vmui: update tracing logic, fix bugs, disable tracing by default
* vmui: use empty message as props
* vmui: fixed ui, added delete for each tacing data, show query in header
* vmui: added timelines
* vmui: speedup render
* vmui: use memo for sorting
* vmui: use Trace model, remove unused functions, simplify part of code
* vmui: update recursive logic
* vmui: fix set query to header
* vmui: code cleanup, remove unused code
* vmui: remove unused type, rename component
* wip
* wip
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-06-23 22:59:20 +03:00
Aliaksandr Valialkin
51362f9333
app/vmselect: add -search.setLookbackToStep command-line flag for making the gap filling algorithm similar to InfluxDB data model
...
This option should override `-search.maxStalenessInterval` for most cases when users migrate from InfluxDB to VictoriaMetrics
2022-06-22 14:19:30 +03:00
Aliaksandr Valialkin
7bf75c7e61
app/vmselect: typo fix in the exported metric name: vm_http_request_total -> vm_http_requests_total
2022-06-22 13:15:31 +03:00
Aliaksandr Valialkin
e6ed92529b
all: remove explicit "xxhash" name when importing github.com/cespare/xxhash/v2 package
...
This package already has the same name, so there is no need in explicit name
2022-06-21 20:23:32 +03:00
Dmytro Kozlov
10454d1735
vmui: added focusLabel, enable cardinality app configuratior ( #2736 )
...
* vmui: added focusLabel, enable app configuratior
* vmui: set focusLabel if {labelName!=""}
* wip
* docs/CHANGELOG.md: mention about focusLabel feature in cardinality explorer
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2730
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-06-17 13:03:02 +03:00
Roman Khavronenko
723d90536c
vmselect: limit end param max value by 2d in future ( #2729 )
...
* vmselect: limit `end` param max value by 2d in future
The change is applied only to service handlers like `/labels` or `/series`
and limits the `end` param by max value <= now() + 2 days. The same limit
is applied for the ingested data, so no reason to allow to request data
in future far than that.
The change is also needed for corner cases like https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2669
where too high `end` value triggers inefficient global index search.
Signed-off-by: hagen1778 <roman@victoriametrics.com>
* docs/CHANGELOG.md: document the bugfix
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-06-16 20:46:31 +03:00
Aliaksandr Valialkin
ec7963208d
app/vmselect: accept focusLabel query arg at /api/v1/status/tsdb
...
This allows filling the seriesCountByFocusLabelValue list in the /api/v1/status/tsdb response
with label values for the specified focusLabel, which contain the highest number of time series.
TODO: add this to Cardinality explorer at VMUI - https://docs.victoriametrics.com/#cardinality-explorer
2022-06-14 18:36:54 +03:00
Aliaksandr Valialkin
b6c1ca12b7
lib/storage: show top labels with the highest number of series in cardinality explorer
2022-06-14 16:32:38 +03:00
Dmytro Kozlov
af3dc91a51
vmui: refactor Cardinality panel ( #2726 )
...
* vmui: refactor Cardinality panel
* vmui: change width of the search panel
* vmui: code cleanup
* vmui: code cleanup
* vmui: fixed vulnerability (npm audit fix)
* wip
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-06-14 14:39:47 +03:00
Yury Molodov
879670418f
vmui: enhancements ( #2638 ) ( #2717 )
...
* feat: make datepicker to be set to last 30 min by default
* fix: correct spinner while loading data
* feat: change legend style
* app/vmselect: `make vmui-update`
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-06-13 09:43:37 +03:00
Aliaksandr Valialkin
374beb350e
app/vmselect: optimize /api/v1/labels and /api/v1/label/.../values handlers when match[] query arg is passed to them
2022-06-12 04:32:13 +03:00
Aliaksandr Valialkin
89b778902b
app/vmselect: add optional limit query arg to /api/v1/labels and /api/v1/label_values endpoints
...
This arg allows limiting the number of sample values returned from these APIs
2022-06-10 09:50:33 +03:00
Aliaksandr Valialkin
483b402bb2
app/vmselect/prometheus: extract common code for obtaining common query args into getCommonParams() function
2022-06-09 20:34:18 +03:00
Aliaksandr Valialkin
2bcb960f17
all: improve query tracing coverage for indexdb search
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1403
2022-06-09 20:07:07 +03:00
Aliaksandr Valialkin
a30333a79e
app/vmselect/graphite: remove additional redundant Request.ParseForm() calls after 38c785b851
2022-06-09 13:28:57 +03:00
Aliaksandr Valialkin
38c785b851
app/vmselect: remove redundant calls to Request.ParseForm()
...
Request.ParseForm() is implicitly called by the first call to Request.FormValue()
2022-06-09 13:11:26 +03:00
Aliaksandr Valialkin
12ac255dae
lib/querytracer: make it easier to use by passing trace context message to New and NewChild
...
The context message can be extended by calling Donef.
If there is no need to extend the message, then just call Done.
2022-06-08 21:06:52 +03:00
Dmytro Kozlov
018d2303c4
Cardinality explorer ( #2625 )
...
* Cardinality explorer
* vmui, vmselect: updated field name, added description to spinner
* make vmui-update
* updated const name, make vmui-update
* lib/storage: changes calculation for totalSeries values
* added static files
* wip
* wip
* wip
* wip
* docs/CHANGELOG.md: document cardinality explorer feature
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2233
Co-authored-by: f41gh7 <nik@victoriametrics.com>
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-06-08 18:43:05 +03:00
Aliaksandr Valialkin
8608dd093c
all: follow-up after 8edb390e21
...
- Remove unused js bloatware from /targets page. This strips down binary size by more than 100Kb
- Add /service-discovery page for API compatibility with Prometheus
- Properly load bootstrap.min.css from /prometheus/targets
- Serve static contents for /targets page from app/vminsert instead of app/vmselect, because /targets page is served from there
2022-06-07 00:57:09 +03:00
Dmytro Kozlov
8edb390e21
lib/promscrape: adds service discovery visualization for /targets page( #2675 )
...
* lib/promscrape: updated template
* lib/promscrape: fixed click on unhealthy and all btns
* app/vmselect: jquery scripts into static folder
Co-authored-by: f41gh7 <nik@victoriametrics.com>
2022-06-03 15:38:45 +02:00
Aliaksandr Valialkin
41958ed5dd
all: add initial support for query tracing
...
See https://docs.victoriametrics.com/Single-server-VictoriaMetrics.html#query-tracing
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1403
2022-06-01 02:29:23 +03:00
Dmytro Kozlov
11f91532c5
issue-2594: use embedded for static files ( #2650 )
...
embed static js and css files from CDN into vmalert, vmagent and vmsingle binaries.
Co-authored-by: f41gh7 <nik@victoriametrics.com>
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2594
2022-05-31 01:55:28 +02:00
Aliaksandr Valialkin
69b9cf7161
app/vmselect/vmui: make vmui-update after 492a615a88
2022-05-26 09:40:57 +03:00
Yury Molodov
492a615a88
vmui: import dashboards ( #2642 )
...
* fix: switch dashboards import to fetch
* make vmui-update
2022-05-26 09:39:10 +03:00
Aliaksandr Valialkin
c6d543e2f9
app/vmselect/vmui: make vmui-update
2022-05-20 14:53:38 +03:00
Yury Molodov
ff74472621
vmui: setup predefined dashboards without build ( #2541 )
...
vmui: support predefined dashboards in json format
See https://github.com/VictoriaMetrics/VictoriaMetrics/tree/master/app/vmselect/vmui/dashboards
2022-05-16 09:42:37 +02:00
Aliaksandr Valialkin
a7f18f8cb2
app/vmselect/promql: do not return values from label_value() if the original time series has no values at the selected timestamps
2022-05-09 17:57:39 +03:00
Aliaksandr Valialkin
ca4ca4630b
app/vmselect/vmui: make vmui-update after 450d879eaa
2022-05-05 21:26:01 +03:00
Aliaksandr Valialkin
fc7c7237e3
app/vmselect: follow-up after 8639e79d38
2022-05-04 23:35:57 +03:00
Nikolay
8639e79d38
app/vmselect: adds proxy for rules API ( #2516 )
...
* app/vmselect: adds proxy for rules and alerts API
It allows to visualization for rules at grafana
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1739
* Update app/vmselect/main.go
Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
2022-05-03 20:55:15 +03:00
Dima Lazerka
1e26dd1f82
Code cleanup ( #343 )
...
* Small code cleanup: remove Request from params
* Extract common params to all export handlers
* Renamed ExportParams -> exportParams
* wip
Co-authored-by: Dzmitry Lazerka <dlazerka@gmail.com>
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-05-03 16:04:16 +03:00
Aliaksandr Valialkin
bca4737fcf
app/vmui: fix up/down arrow keys on multi-line query after a186434b50
2022-05-03 10:50:16 +03:00
Aliaksandr Valialkin
a186434b50
app/vmui: execute query by pressing enter in the same way as Prometheus does
...
Multi-line query can be entered via `shift-enter` in the query input field
2022-05-02 19:49:29 +03:00
Yury Molodov
87693754d5
vmui: support node v.18 ( #2529 )
...
* fix: add support vmui node18
* fix: remove @mui/styles (legacy styling solution)
* Update app/vmui/Dockerfile-build
* app/vmselect/vmui: `make vmui-update`
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-05-02 18:41:53 +03:00
Aliaksandr Valialkin
7aa5167996
app/vmselect/main.go: move the code /api/v1/status/buildinfo handler to the same location as in the cluster branch
2022-04-29 13:02:14 +03:00
Aliaksandr Valialkin
464325a24b
app/vmselect/vmui: make vmui-update after da04e9d1de
2022-04-29 12:54:10 +03:00
Dima Lazerka
ed8e88af11
Export "null" in jsonl instead of NaN ( #2518 )
...
* Export "null" in jsonl instead of NaN
The NaN appeared because of staleness markers that were added for compatibility. I think it's better to use json `null`, implemented here.
Also maybe it also makes sense to add a flag like `?skip-staleness-markers=true` to `/export`, to skip nulls at all?
* Update app/vmselect/prometheus/export.qtpl
* app/vmselect/prometheus/export.qtpl.go: `make quicktemplate-gen`
* docs/CHANGELOG.md: document the change
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-04-29 12:49:47 +03:00
Nikolay
c7aad8d441
app/vmselect: adds API /api/v1/status/buildinfo ( #2515 )
...
* app/vmselect: adds API /api/v1/status/buildinfo
it should fix an compability error with grafana 8.5 prometheus datasource
https://github.com/grafana/grafana/pull/46771
* Update main.go
2022-04-29 11:36:28 +03:00
Aliaksandr Valialkin
cce1b6d7f9
app/vmselect/promql: add tlast_change_over_time(m[d]) function, which returns the timestamp for the last change of m on the given lookbehind window d
2022-04-27 10:59:03 +03:00
Yury Molodov
c7693e8bc1
vmui: expression alias ( #2495 )
...
* feat: add alias for queries
* docs: update docs for predefined dashboards
* app/vmselect: `make vmui-update`
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-04-26 15:59:37 +03:00
Yury Molodov
9b4bff67e0
vmui: add support relative time ( #2504 )
...
* feat: add support relative time
* app/vmselect: `make vmui-update`
* docs/CHANGELOG.md: document the change
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-04-26 15:46:06 +03:00
Aliaksandr Valialkin
25fe83577d
app/vmselect/promql: properly handle scalar default vector, scalar if vector and scalar ifnot vector queries
...
Previously `vector` time series could be unexpectedly returned from such queries
2022-04-21 15:34:36 +03:00
Aliaksandr Valialkin
d1a9fac894
app/vmselect/promql: fix comparison to nan
...
The comparison to nan has been broken in d335cc886chttps://github.com/VictoriaMetrics/VictoriaMetrics/issues/150
2022-04-21 14:55:37 +03:00
Aliaksandr Valialkin
de892239a9
app/vmselect/promql: add drop_common_labels() function
2022-04-21 14:20:20 +03:00
Aliaksandr Valialkin
7a44ba1234
app/vmselect/promql: fix q default b where b may have empty time series
2022-04-21 11:42:42 +03:00
Aliaksandr Valialkin
d335cc886c
app/vmselect/promql: fix duplicate time series error on joins against time series filtered by values
...
This should prevent from `duplicate time series` errors when executing the following query:
kube_pod_container_resource_requests{resource="cpu"} * on (namespace,pod) group_left() (kube_pod_status_phase{phase=~"Pending|Running"}==1)
where `kube_pod_status_phase{phase=~"Pending|Running"}==1` filters out diplicate time series
2022-04-20 22:18:44 +03:00
Aliaksandr Valialkin
ed97908ca9
app/vmselect/promql: rename removeNaNs() to more clear removeEmptySeries()
2022-04-20 19:53:46 +03:00
Yury Molodov
514e3660e2
fix: prevent graph hiding without data ( #2456 )
...
* fix: prevent graph hiding without data
* fix: add yaxis labels default
* app/vmselect: `make vmui-update`
* docs/CHANGELOG.md: document the change
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-04-16 17:15:31 +03:00
Aliaksandr Valialkin
ef66b048c9
app/vmui: further improvements for number display on graphs
...
This is a follow-up for c4d2cd8336https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2409
2022-04-12 16:01:27 +03:00
Aliaksandr Valialkin
7f83dc06c4
app/vmselect: make vmui-update
2022-04-12 14:35:19 +03:00
Aliaksandr Valialkin
61c7f6beae
app/vmselect/promql: allow calling InitRollupResultCache+StopRollupResultCache multiple times during tests
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2406
2022-04-11 12:34:43 +03:00
Aliaksandr Valialkin
19ecc4b2c3
app/vmselect: make vmui-update
2022-04-01 12:55:21 +03:00
Yury Molodov
f166f80f15
vmui: grid support for predefined panels ( #2386 )
...
* update packages
* feat: add setting width for predefined panels
* docs: update doc by predefined dashboards
* app/vmselect: `make vmui-update`
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-04-01 12:48:17 +03:00
Yury Molodov
c8d29ed78e
vmui: predefined panels ( #2243 )
...
* feat: add basic components for predefined dashboards
* fix: change display alert
* feat: add autosize and unit for axes
* feat: add component for CircularProgress
* feat: change layout for predefined dashboards
* feat: add override step for predefined panels
* feat: add override step for predefined panels
* feat: change yaxis limits for predefined panels
* fix: rename flag for hide legend
* feat: add formatted panel description
* feat: add README.md for dashboard setup
* feat: validate dashboard settings
* feat: add unit for y-ticks
* fix: correct display error for dashboards
* fix: disable auto refresh after route change
* update package-lock.json
* fix: add basename for BrowserRouter
* fix: add dynamic basename for routing
* update packages
* feat: add a pre-defined dashboard "per-job resource usage"
* feat: display unit in the hover-tooltip
* fix: change routing and home layout
* fix: change axis width calc
* updated packages
* app/vmselect: `make vmui-update`
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-03-26 13:03:11 +02:00
Aliaksandr Valialkin
6e364e19ef
app/vmselect: add fine-grained limits for the number of returned/scanned time series for various APIs
2022-03-26 11:29:49 +02:00
Aliaksandr Valialkin
e78f3ac8ac
app/vmselect/bufferedwriter: suppress trivial network errors, which can be generated by remote side
...
These errors include `broken pipe` and `reset by peer`.
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2334
2022-03-18 19:28:02 +02:00
Aliaksandr Valialkin
5d9b9b88b9
app/vmselect/vmui: make vmui-update
2022-03-18 13:18:26 +02:00
Yury Molodov
227d5182af
vmui: update packages ( #2264 )
...
* update package.json
* update package-lock.json
* app/vmselect: `make vmui-update`
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-03-03 18:49:13 +02:00
Aliaksandr Valialkin
0d47c23a03
app/vmselect/promql: reduce the maximum number of label values, which can be propagated from one side of the binary operation to another side of the binary operation from 10K to 1K
...
There are user reports that 10K unique values in a single label filter may lead to performance and memory usage issues
2022-02-24 04:05:18 +02:00
Aliaksandr Valialkin
62b46007c5
lib/workingsetcache: reduce the default cache rotation period from hour to 20 minutes
...
This should reduce memory usage under high time series churn rate
2022-02-23 13:41:45 +02:00
Aliaksandr Valialkin
205d34eae6
app/vmselect: make vmui-update
2022-02-22 21:02:37 +02:00
Yury Molodov
5808774e06
vmui: change logic autocomplete ( #2196 )
...
* fix: change autocomplete display logic
* fix: change z-index for labels of input fields
* fix: change autocomplete display logic
* wip
* wip
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-02-21 15:43:55 +02:00
Aliaksandr Valialkin
1d7c877b7b
app/vmui: small fixes
...
* Remove unneeded dependency on `numeral` package
* Properly parse numbers obtained from /api/v1/query_range according to
https://prometheus.io/docs/prometheus/latest/querying/api/#expression-query-result-formats
* Optimize updating processing the received data from /api/v1/query_range
* Make smoother zoom on `ctrl+scroll`
* Reduce the number of points received from /api/v1/query_range by 2x in order to reduce load on backend
2022-02-14 16:25:46 +02:00
Aliaksandr Valialkin
b1f94f7f0e
app/vmselect/promql: return at most one time series from absent_over_time() in the same way as Prometheus does
...
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2130
2022-02-12 15:45:09 +02:00
Roman Khavronenko
e3adcbec6e
lib/promscrape: support prometheus-like duration in scrape configs ( #2169 )
...
* lib/promscrape: support prometheus-like duration in scrape configs
The change allows to specify duration values like `1d`, `1w`
for fields `scrape_interval`, `scrape_timeout`, etc.
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/817#issuecomment-1033384766
Signed-off-by: hagen1778 <roman@victoriametrics.com>
* lib/blockcache: make linter happy
Signed-off-by: hagen1778 <roman@victoriametrics.com>
* lib/promscrape: support prometheus-like duration in scrape configs
* add support for extra fields `scrape_align_interval` and `scrape_offset`;
* support Prometheus duration parsing for `__scrape_interval__`
and `__scrape_duration__` labels;
Signed-off-by: hagen1778 <roman@victoriametrics.com>
* wip
* wip
* docs/CHANGELOG.md: document the feature
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-02-11 16:17:00 +02:00
Aliaksandr Valialkin
af77f449da
app/vmselect/vmui: make vmui-update
2022-02-08 17:26:50 +02:00
Aliaksandr Valialkin
96b7de6736
app/vmselect/promql: clarify comments on why the right side of if and and operators are executed at first
2022-02-03 00:26:14 +02:00
Aliaksandr Valialkin
4b850c2a59
app/vmselect/promql: do not push down filters, which enumerate more than 10k unique values
...
Such filters may slow down time series search, so just skip them.
This is a follow-up for e7f1ceeb84https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1827
2022-02-02 23:40:02 +02:00
Aliaksandr Valialkin
2016a2c899
app/vmselect/promql: properly handle foo or bar queries
...
Such queries may miss `bar` results after the commit e7f1ceeb84
2022-02-01 17:40:51 +02:00
Aliaksandr Valialkin
4bdd10ab90
lib/bytesutil: split Resize* funcs to MayOverallocate and NoOverallocate for more fine-grained control over memory allocations
...
Follow-up for f4989edd96https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2007
2022-02-01 00:18:42 +02:00
Aliaksandr Valialkin
a8d22e1223
app/vmselect/promql: check for binary operation in case-insensitive manner when deciding which side of the operation to perform the first
...
PromQL and MetricsQL operators are case-insensitive
2022-01-31 22:06:15 +02:00
Aliaksandr Valialkin
e7f1ceeb84
app/vmselect/promql: optimize queries, which join on _info metrics.
...
Automatically add common filters from one side of binary operation
to the other side before sending the query to storage subsystem.
See https://grafana.com/blog/2021/08/04/how-to-use-promql-joins-for-more-effective-queries-of-prometheus-metrics-at-scale/
and https://www.robustperception.io/exposing-the-software-version-to-prometheus
2022-01-31 19:32:36 +02:00
Aliaksandr Valialkin
7edf8be3bc
app/vmselect/vmui: make vmui-update
2022-01-31 12:36:59 +02:00
Aliaksandr Valialkin
74a2297dcc
app/vmselect/vmui: make vmui-update after e3995572bb
2022-01-25 17:51:46 +02:00
Aliaksandr Valialkin
f4989edd96
lib/bytesutil: split Resize() into ResizeNoCopy() and ResizeWithCopy() functions
...
Previously bytesutil.Resize() was copying the original byte slice contents to a newly allocated slice.
This wasted CPU cycles and memory bandwidth in some places, where the original slice contents wasn't needed
after slize resizing. Switch such places to bytesutil.ResizeNoCopy().
Rename the original bytesutil.Resize() function to bytesutil.ResizeWithCopy() for the sake of improved readability.
Additionally, allocate new slice with `make()` instead of `append()`. This guarantees that the capacity of the allocated slice
exactly matches the requested size. The `append()` could return a slice with bigger capacity as an optimization for further `append()` calls.
This could result in excess memory usage when the returned byte slice was cached (for instance, in lib/blockcache).
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2007
2022-01-25 15:24:44 +02:00
Aliaksandr Valialkin
285bb2bbec
app/vmselect/vmui: make vmui-update
2022-01-24 12:19:49 +02:00
Yury Molodov
ad5059f2d3
vmui: fixed display type switching ( #2088 )
...
* fix: correct switch display type
* docs/CHANGELOG.md: document the bugfix
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-01-21 16:56:22 +02:00
Yury Molodov
adbb821eac
vmui: fix time range selector ( #2085 )
...
* fix: add date validate for time range
* app/vmselect/vmui: `make vmui-update`
* docs/CHANGELOG.md: document the bugfix
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-01-21 12:02:38 +02:00
Aliaksandr Valialkin
d79a915583
app/vmui: fix last 6 months time range picker
...
Previously it was incorrectly selecting 6 minutes instead of 6 months.
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1960
2022-01-18 23:28:10 +02:00
Aliaksandr Valialkin
672fcba223
app/vmui: properly calculate graph range for y axis
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2037
2022-01-18 22:22:24 +02:00
Aliaksandr Valialkin
5a77c86e97
app/vmui: reduce the refresh interval during graph scrolling/zooming from 1 second to 300 milliseconds
...
One second feels too laggy, so let's reduce the refresh interval to 300 milliseconds.
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2064
2022-01-18 21:48:12 +02:00
Yury Molodov
8bdc45ba00
fix: remove buffer period ( #2078 )
...
* fix: remove buffer period
* app/vmselect/vmui: `make vmui-update`
* docs/CHANGELOG.md: document the implemented feature
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2064
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-01-18 21:42:56 +02:00
Yury Molodov
70737ea4ac
vmui: correct url encoding ( #2067 )
...
* fix: correct encode multi-line queries
* fix: change autocomplete for correct arrows work
* app/vmselect/vmui: `make vmui-update`
* docs/CHANGELOG.md: document the bugfix for https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2039
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-01-18 21:31:46 +02:00
Yury Molodov
8e3f9c1fbb
vmui: correct calc axes limits ( #2058 )
...
* fix: correct calc axes limits
* app/vmselect/vmui: `make vmui-update`
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-01-18 15:43:41 +02:00
Yury Molodov
fcd33fc409
vmui: change layout ( #2054 )
...
* fix: change query reset
* feat: replace @codemirror to text field
* feat: switch to Preact from React
* fix: optimize mui imports
* feat: move time selector to Header
* checkout
* fix: remove unused vars
* update package-lock.json
* fix: correct styles
* app/vmselect/vmui: `make vmui-update`
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-01-18 12:44:22 +02:00
Aliaksandr Valialkin
dc7b63a793
app/vmselect/promql: properly keep metric names when optimized path is used for aggregate function calculations
...
For example, `sum(rate(...) keep_metric_names) by (__name__)` didn't leave the original metric name because of this issue.
This is a follup-up for 1bdc71d917https://github.com/VictoriaMetrics/VictoriaMetrics/issues/949
2022-01-17 15:30:30 +02:00
Aliaksandr Valialkin
1bdc71d917
app/vmselect/promql: implement keep_metric_names modifier for transform and rollup functions
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/949
2022-01-14 04:14:59 +02:00
Aliaksandr Valialkin
f41846d002
app/vmselect/promql: add stale_samples_over_time() function
2022-01-14 01:48:04 +02:00
Aliaksandr Valialkin
c883c15878
app/vmselect/promql: add support for @ modifier
...
Add support for `@` modifier in MetricsQL according to https://prometheus.io/docs/prometheus/latest/querying/basics/#modifier
Extend the support with the following features:
* Allow using `@` modifier everywhere in the query. For example, `sum(foo) @ end()`
* Allow using arbitrary expression as `@` modifier. For example, `foo @ (end() - 1h)`
returns `foo` value at `end - 1 hour` timestamp on the selected time range `[start ... end]`
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1348
2022-01-13 22:12:06 +02:00
Aliaksandr Valialkin
9469696e46
app/vmselect/promql: fix limit_offset() test
...
The test has been broken in addae7fc6a
2022-01-13 16:59:28 +02:00
Yury Molodov
7d5ed49d23
vmui: switching to Preact ( #2053 )
...
* feat: replace @codemirror to text field
* feat: switch to Preact from React
* fix: optimize mui imports
* fix: remove unused vars
* update package-lock.json
* app/vmselect/vmui: `make vmui-update`
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-01-11 10:32:17 +02:00
Aliaksandr Valialkin
ceade70d4e
app/vmselect/vmui: run make vmui-update after 89ff7b2465
2022-01-03 21:03:37 +02:00
Yury Molodov
681a800086
vmui: legend fixes ( #1995 )
...
* feat: add a reset query by clicking the logo
* feat: add sequence number for query fields
* feat: invert behavior on the graph's legend
* app/vmselect/vmui: `make vmui-update`
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2021-12-23 12:14:16 +02:00
Roman Khavronenko
543bd0ea0c
vmselect: update /query_exemplars placeholder ( #2000 )
...
Grafana expects `data` in response to be a slice and logs an err
if it is not:
```
err="[]v1.ExemplarQueryResult: decode slice: expect [ or n, but found , error found in #0 byte of ...||..., bigger context ...||..."
```
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1999
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2021-12-23 11:53:50 +02:00
Yury Molodov
4b40acd964
vmui: add custom start range ( #1989 )
...
* feat: add custom start range
* app/vmselect/vmui: `make vmui-update`
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2021-12-21 20:19:33 +02:00
Yury Molodov
718c352946
vmui: graph fixes ( #1982 )
...
* fix: remove disabling custom step when zooming
* feat: add a dynamic calc of the width of the graph
* fix: add validate y-axis limits
* fix: correct axis limits for value 0
* fix: change logic create time series
* fix: change types for tooltip
* fix: correct points on the line
* fix: change the logic for set graph width
* fix: stop checking the period when auto-refresh is enabled
* app/vmselect/vmui: `make vmui-update`
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2021-12-20 17:37:02 +02:00
Aliaksandr Valialkin
6d77cc9b08
app/vmselect/vmui: make vmui-update
2021-12-20 13:51:02 +02:00
Aliaksandr Valialkin
974d9c0eee
app/vmselect/promql: follow-up after 177e345d8a
...
* Document changes_prometheus(), increase_prometheus() and delta_prometheus() functions.
* Simplify their implementation
* Mention these functions in docs/CHANGELOG.md
2021-12-20 13:19:44 +02:00
匠心零度
177e345d8a
add Prometheus semantics function :changes_prometheus、delta_prometheus、increase_prometheus ( #1972 )
...
Co-authored-by: lirenzuo <lirenzuo@shein.com>
2021-12-20 12:32:43 +02:00
Aliaksandr Valialkin
e6d4641bf0
app/vmselect/vmui: make vmui-update
2021-12-17 11:01:09 +02:00
Aliaksandr Valialkin
193331d522
app/vmselect: de-duplicate data exported via /api/v1/export/csv by default
...
Previously the exported data wasn't de-duplicated.
Now it is possible to export the raw data without deduplication
by passing reduce_mem_usage=1 query arg to /api/v1/export/csv
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1837
2021-12-17 10:57:39 +02:00
Aliaksandr Valialkin
5efe377a26
app/vmselect/promql: add timestamp_with_name(m[d]) function
...
This function works the same as `timestamp()`, but doesn't remove source time series names.
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/949#issuecomment-995222388
2021-12-15 23:37:07 +02:00
Yury Molodov
9baad51004
vmui: introduce application mode ( #1949 )
...
* feat: add a label for the Query field
* fix: change zoom position
* fix: add description and error code to alerts
* fix: correct logic query history
* fix: correct update query history
* feat: add custom step
* update package-lock.json
* feat: introduce application mode
* build vmui
* Revert "build vmui"
This reverts commit c0e2415550
2021-12-15 21:33:25 +02:00
Aliaksandr Valialkin
92070cbb67
lib/storage: return dedup interval in milliseconds from GetDedupInterval()
...
This removes duplicate .Milliseconds() calls after GetDedupInterval() calls.
2021-12-15 13:26:38 +02:00
Aliaksandr Valialkin
1d20a19c7d
lib/storage: explicitly pass dedupInterval to DeduplicateSamples() and deduplicateSamplesDuringMerge()
...
This improves the code readability and debuggability, since the output of these functions
stops depending on global state.
2021-12-14 20:49:12 +02:00
Yury Molodov
49e6a921df
vmui: custom step ( #1942 )
...
* feat: add a label for the Query field
* fix: change zoom position
* fix: add description and error code to alerts
* fix: correct logic query history
* fix: correct update query history
* feat: add custom step
* update package-lock.json
* docs: document that VMUI now supports overriding of `step` query arg, which is passed to `/api/v1/query_range`
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2021-12-14 14:51:45 +02:00
Yury Molodov
b5b701d590
vmui: minor fixes ( #1936 )
...
* feat: add a label for the Query field
* fix: change zoom position
* fix: add description and error code to alerts
* fix: correct logic query history
* fix: correct update query history
* app/vmselect/vmui: `make vmui-update`
* docs/CHANGELOG.md: document bugfixes
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2021-12-13 13:42:37 +02:00
Aliaksandr Valialkin
ce80a0ce5e
app/vmselect/vmui: make vmui-update
2021-12-13 13:32:54 +02:00
Yury Molodov
7c3b6365f0
vmui: add a label for the Query field ( #1923 )
...
* feat: add a label for the Query field
* app/vmselect/vmui: `make vmui-update`
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2021-12-12 19:06:39 +02:00
Aliaksandr Valialkin
d1f8915ed1
app/vmselect/promql: preserve the order of time series passed to limit_offset() function
...
Previously time series passed to `limit_offset()` were shuffled according to hash for their labels.
This was unexpected behaviour for most users.
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1920 and https://github.com/VictoriaMetrics/VictoriaMetrics/issues/951
2021-12-12 18:04:58 +02:00
Yury Molodov
c1fd93e8a0
vmui: multiple queries ( #1916 )
...
* feat: change duration by "enter"
* fix: optimize data processing for chart
* feat: set minimum step to 1ms
* update dependencies
* feat: remove save the last query to local storage
* fix: handle an error in a table with subqueries
* feat: store display type in URL
* Revert "feat: store display type in URL"
This reverts commit ccc242c69a
2021-12-08 16:40:15 +02:00
Aliaksandr Valialkin
ff15a752c1
app/vmselect: accept optional extra_filters[] query args for all the supported Prometheus querying APIs
...
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1863
2021-12-06 17:07:09 +02:00
Aliaksandr Valialkin
732a0cd3e1
app/vmselect/vmui: make vmui-update
2021-12-06 10:19:09 +02:00
Aliaksandr Valialkin
f67427ae61
app/vmselect/vmui: make vmui-update
2021-11-30 01:38:24 +02:00
Yury Molodov
1b58d126c0
vmui: optimize render ( #1852 )
...
* feat: change duration by "enter"
* fix: optimize data processing for chart
* feat: set minimum step to 1ms
* update dependencies
* update package-lock
* app/vmselect/vmui: `make vmui-update`
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2021-11-30 00:56:48 +02:00
Aliaksandr Valialkin
91243ad5cd
app/vmselect/vmui: make vmui-update
2021-11-29 21:55:08 +02:00
Aliaksandr Valialkin
ce31e837eb
app/vmselect/vmui: make vmui-update
2021-11-29 12:22:59 +02:00
Michael Fuller
cf8c171f85
vmselect: in promql evaluation, return bytes requested when rollup memory limiter is unable to satisfy the request ( #1838 )
...
Co-authored-by: Michael Fuller <mfuller@digitalocean.com>
2021-11-22 13:20:42 +03:00
Aliaksandr Valialkin
9bee043ff2
app/vmselect/promql: consistently return zero from deriv(const)
2021-11-17 18:02:05 +02:00
Aliaksandr Valialkin
b900560b83
app/vmselect/promql: add now() function, which returns the current timestamp as a floating-point value in seconds
2021-11-17 16:35:30 +02:00
Aliaksandr Valialkin
f43586c63c
app/vmselect/promql: arrange function names in the code in alphabetical order
...
This should simplify code maintenance in the future
2021-11-14 13:55:06 +02:00
Aliaksandr Valialkin
9fa098d8e3
app/vmselect/promql: prevent from incorrect calculations for deriv() over multiple samples with identical timestamps
2021-11-12 13:50:43 +02:00
Aliaksandr Valialkin
e5ac9d8e57
all: consistently return application/json content-type without charset=utf-8
...
The `application/json` content-type has utf-8 encoding by default.
See https://stackoverflow.com/questions/9254891/what-does-content-type-application-json-charset-utf-8-really-mean
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/897
2021-11-09 18:04:44 +02:00
Aliaksandr Valialkin
840ac283ef
app/vmselect/promql: properly return durations smaller than one second from duration_over_time() function
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1780
2021-11-09 11:41:56 +02:00
Yury Molodov
b78ab88a1c
vmui: migration MUI Core v4 to v5 ( #1795 )
...
* migration MUI Core v4 to v5
* app/vmui: `make vmui-update`
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2021-11-08 15:45:59 +02:00
Aliaksandr Valialkin
3419ac1d36
app/vmselect/promql: add duration_over_time(m[d], max_interval) function
...
This function calculates the actual lifetime of the time series on the given lookbehdind window `d`
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1780
2021-11-08 13:14:09 +02:00
Yury Molodov
6d1d558c4f
vmui: fix graph reset ( #1788 )
...
* feat: add query history
* fix: change detect keyUp for nav query history
* feat: set default query history
* feat: change graph legend
* update dependencies
* update codemirror version
* fix: correct update period time after zoom/pan
* fix: optimize data processing for the graph
* fix: eliminate memory leaks related to mouse events
* fix: correct display of straight line
* Merge branch 'master' into vmui-fix-reset-graph
* app/vmselect: `make vmui-update`
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2021-11-05 20:56:57 +02:00
Aliaksandr Valialkin
2178335618
app/vmselect: make vmui-update
2021-11-04 12:13:12 +02:00
Aliaksandr Valialkin
27044b84d2
app/vmselect/promql: add limit_offset(limit, offset, q) function, which can be used for paging over big number of time series
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1778
2021-11-03 16:02:27 +02:00
Aliaksandr Valialkin
43a58bd618
app/vmselect/promql: add label_graphite_group() function for extracting groups from Graphite metric names
2021-11-03 13:19:08 +02:00
Yury Molodov
47ee3744f2
vmui: correct migration material-ui ( #1758 )
...
* migration material-ui
* fix: rollback popover
* app/vmselect: `make vmui-update`
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2021-10-27 20:41:26 +03:00
Aliaksandr Valialkin
1952ab99aa
all: fix build issues and tests for Apple M1
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1653
2021-10-27 15:06:34 +03:00
Yury Molodov
956cf83e7b
vmui: update dependencies ( #1754 )
...
* update dependencies
* update codemirror version
* app/vmui: `make vmui-update`
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2021-10-26 19:31:20 +03:00
Aliaksandr Valialkin
df8f967040
app/vmselect/promql: reduce the precision from 15 significant digits to 13 significant digits when comparing float64 results in tests
...
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1738
2021-10-24 13:31:14 +03:00
Yury Molodov
2b266cb87e
vmui: query history ( #1732 )
...
* feat: add query history
* fix: change detect keyUp for nav query history
* feat: set default query history
* app/vmselect/vmui: `make vmui-update`
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2021-10-22 12:21:22 +03:00
Yury Molodov
a3e09a57c2
vmui: features ( #1711 )
...
* feat: initial uPlot graph
* feat: add zoom/pan for graph
* fix: add zoom by ctrl/mac
* fix: remove unused code
* feat: add toggle cache for fetch
* feat: add fix y-axis limits
* fix: stop point events while panning
* fix: change getting cursor position when scaling
* feat: add cursor tooltip to graph
* fix: uninstall chart.js
* fix: change link for create an issue
* fix: set default cache value to true
* app/vmalert: follow-up after 0e2486df565416e18007
2021-10-18 15:16:57 +03:00
Aliaksandr Valialkin
da97e58979
app/vmselect/promql: randomize the static selection of time series returned from limitk()
...
Sort series by a hash calculated from the series labels. This should guarantee "random" selection of the returned time series.
Previously the selection could be biased, since time series were sorted alphabetically by label names and label values.
2021-10-16 21:16:49 +03:00
Aliaksandr Valialkin
cae174b11c
app/vmselect/promql: typo fix in comment: didsn't -> didn't
2021-10-16 13:00:34 +03:00
Aliaksandr Valialkin
a5001b9c20
app/vmselect/promql: add atan2 binary operator, which is going to be added in Prometheus 2.31
...
See https://github.com/prometheus/prometheus/pull/9248
2021-10-11 21:15:53 +03:00
Aliaksandr Valialkin
81c6720392
app/vmselect/promql: add missing trigonometric functions, which are going to be added in Prometheus 2.31
...
See https://github.com/prometheus/prometheus/issues/9233
2021-10-11 21:01:33 +03:00
Aliaksandr Valialkin
92b92d4d2c
app/vmselect/promql: consistently return the same set of time series from limitk() function
...
This is the expected behaviour by most users.
2021-10-08 19:53:52 +03:00
Aliaksandr Valialkin
0ff8fcac6a
app/vmui: follow-up after 7bfb44113e
...
* Run `vmui-update`
* Document the changes in README.md and CHANGELOG.md
2021-10-08 15:09:29 +03:00
Aliaksandr Valialkin
2748255c8b
app/vmselect/promql: substitute rollupFuncsCannotAdjustWindow with rollupFuncsCanAdjustWindow
...
The list of functions, which can adjust lookbehind window is more limited than the rest of functions,
so it is better from maintainability and readability PoV using the allowlist instead of blocklist.
2021-10-07 13:18:42 +03:00
Aliaksandr Valialkin
c45210a6f9
app/vmselect/promql: return back the behaviour for deriv() function when the lookbehind window doesnt contain enough points
...
It is expected that the `deriv(m[d])` returns non-empty value if the lookbehind window `d`
contains less than 2 samples in the same way as `rate()` does.
This is a follow-up after 3e084be06b
2021-10-07 12:52:27 +03:00
Roman Khavronenko
3e084be06b
app/vmselect: make predict_linear and deriv compatible with Prometheus ( #1681 )
...
Previously, `predict_linear` returned slightly different results comparing
to Prometheus. The change makes linear regression algorithm compatible
with Prometheus.
`deriv` was excluded from the list of functions which can adjust the time
window for the same reasons.
2021-10-07 12:50:49 +03:00
Aliaksandr Valialkin
0e3de5a0cc
app/vmselect/promql: add topk_last and bottomk_last functions
2021-09-30 13:22:52 +03:00
Roman Khavronenko
a31407006c
app/vmselect: fix binary comparison func ( #1667 )
...
The fix makes the binary comparison func to check for NaNs
before executing the actual comparison. This prevents VM
to return values for non-existing samples for expressions
which contain bool comparisons. Please see added test
for example.
2021-09-30 12:24:17 +03:00
Roman Khavronenko
344490d89b
app/vmselect: fix testRowsEqual func NaN checks ( #1666 )
...
It appeared, that `testRowsEqual` NaN comparison was incorrect.
The fix caused some tests to fail. Please see the change and
tests updated.
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2021-09-30 12:08:47 +03:00
Yury Molodov
893af0a92c
vmui: fixed bug with time range (time zone) ( #1661 )
...
* fix: set date in query string in utc format
* app/vmselect: `make vmui-update`
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2021-09-29 01:00:44 +03:00
Roman Khavronenko
5dc84bf210
app/vmselect: disable time-window adjustment for min/max_over_time funcs ( #1658 )
...
Adjustment results into discrepancy between Prometheus and VM on time windows
smaller than scrape interval.
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2021-09-29 00:43:21 +03:00
Roman Khavronenko
de810031bf
app/vmselect: always return zero for stddev func if there is only one value ( #1659 )
...
The fix will always return zero if received set of items consists of one
element only, which also means no deviation.
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2021-09-29 00:38:55 +03:00
Roman Khavronenko
dd536b475c
app/vmselect: return NaN instead of 0 for empty value sets ( #1660 )
...
The change affects `count/stddev/stdvar_over_time` funcs and makes
them to return NaN instead of zero when there is no datapoints
in a time window.
This is needed for improving compatibility with Prometheus.
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2021-09-29 00:37:04 +03:00
Roman Khavronenko
03cd93bf1a
app/vmselect: rm quantile_over_time fast-path optimisations ( #1662 )
...
The removed fast path optimisations weren't consistent with
`quantile` function behavior and results into discrepancy.
Specifically, results didn't match in cases when:
* 0 < phi > 1;
* values contain only one element.
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2021-09-29 00:35:14 +03:00
Yury Molodov
a64155d91e
vmui: use Chart.js as default engine for graph ( #1634 )
...
* feat: add Plotly as default engine for graph
* fix: remove unused components
* feat: use Chart.js as default engine graph
* fix: correct styles for loader
* feat: add zoom/pan for chart
* feat: add height for chart
* fix: remove unused code
* fix: remove empty units from duration
* fix: change debounce for pan to 500ms
* fix: add utility for plugins register globally
* fix: optimize render graph
* feat: add buffer data for zoom
* fix: add limits for zoom in/out
* fix: change update data while zooming
* app/vmselect: `make vmui-update`
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2021-09-27 22:26:14 +03:00
Aliaksandr Valialkin
2efe0acfc9
app/vmselect/promql: add rollup_scrape_interval(m[d]) function
...
It calculates the min, max and avg scrape intervals for m over the given lookbehind window d
2021-09-27 19:21:24 +03:00
Aliaksandr Valialkin
c4c77aa2dd
app/vmselect/promql: follow-up after 526dd93b32
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1625
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1612
2021-09-27 18:55:39 +03:00
Roman Khavronenko
526dd93b32
app/vmselect: quantile func compatiblity with Prometheus ( #1646 )
...
* app/vmselect: `quantile` func compatiblity with Prometheus
The `quantile` func was previously calculated by https://github.com/valyala/histogram
package. The result of such calculation was always the closest real value to
requested quantile. While in Prometheus implementation interpolation is used.
Such difference may result into discrepancy in output between Prometheus and
VictoriaMetrics.
This commit adds a Prometheus-like `quantile` function. It also used by other
functions which depend on it, such as `quantiles`, `quantile_over_time`, `median` etc.
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1625
Signed-off-by: hagen1778 <roman@victoriametrics.com>
* app/vmselect: `quantile` review fixes
* quantile functions were split into multiple to provide
different API for already sorted data;
* float64sPool is used for reducing allocations. Items in pool may have
different sizes, but defining a new pool was complicates due to name collisions;
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2021-09-27 18:02:41 +03:00
Aliaksandr Valialkin
8ed95e82c6
app/vmselect/promql: follow-up after 57b3320478
2021-09-24 01:24:18 +03:00
Roman Khavronenko
57b3320478
app/vmselect: make sorting for query result similar to Prometheus ( #1647 )
...
* app/vmselect: make sorting for query result similar to Prometheus
Updated sorting allows to get the order of series in result similar or equal
to what Prometheus returns.
The change is needed for compatibility reasons.
* Update app/vmselect/promql/exec_test.go
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2021-09-24 01:03:12 +03:00
Aliaksandr Valialkin
e564411a62
app/vmselect/promql: align the behavior of or, and and unless operators with on (labels) modifier to Prometheus
...
Previously VictoriaMetrics could return unexpected result of the right-hand side operand
had multiple time series with the given set of labels mentioned in `on(labels)`.
See https://github.com/VictoriaMetrics/VictoriaMetrics/pull/1643
2021-09-24 00:46:25 +03:00
Aliaksandr Valialkin
a22aa0608b
app/vmselect: fix accessing /graphite/* endpoints
2021-09-21 13:56:35 +03:00
Aliaksandr Valialkin
9a3d0c43b5
app/vmselect/promql: add quantiles_over_time("phiLabel", phi1, ..., phiN, m[d]) function for calculating multiple quantiles at once
2021-09-17 23:35:10 +03:00
Aliaksandr Valialkin
2951dd0a57
app/vmselect/promql: add histogram_quantiles("phiLabel", phi1, ..., phiN, buckets) function
...
This function calculates multiple quantiles over the given buckets at once
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1573
2021-09-17 13:32:39 +03:00
Aliaksandr Valialkin
5a44be0e52
app/vmselect/promql: optimize quantiles() calculation
...
Calculate quantiles in one go instead of calculating each quantile individually
2021-09-17 12:33:42 +03:00
f41gh7
9375b60c5f
adds stub for functions api
2021-09-16 13:49:52 +03:00
Aliaksandr Valialkin
e60dfc96ff
app/vmselect/promql: add mad(q) and outliers_mad(tolerance, q) functions to MetricsQL
2021-09-16 13:33:53 +03:00
Aliaksandr Valialkin
eca75cc650
app/vmselect/prometheus: make more clear log messages for errors during sending data to remote clients
2021-09-16 12:56:58 +03:00
Aliaksandr Valialkin
2f86d4cf38
app/vmui: follow-up after 777ff75874
...
The commit contains the following changes:
- Show vmui when requesting /graph page in order to be compatible with Prometheus datasource in Grafana.
- Properly encode query args at vmui url.
- Set the number of points on the graph to the number of horizontal pixels divided by 2. Previously it was hardcoded to 30.
- Do not save server url to persistent storage at browser, since it should be always obtained from the url.
- Run `make vmui-update` for updating vmui embedded into VictoriaMetrics.
2021-09-15 17:40:48 +03:00
Aliaksandr Valialkin
d1a16e0891
app/vmselect/promql: use Prometheus-compatible label value formatting for count_values function
2021-09-13 13:48:06 +03:00
Aliaksandr Valialkin
f4dead529f
lib/storage: properly search series by multiple tag filters matching empty labels such as foo{bar=~"baz|",x=~"y|"}
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1601
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/395
2021-09-09 21:09:21 +03:00
Aliaksandr Valialkin
ea943911bc
app/vmselect/promql: keep metric name in rollup_candlestick results, since they don't change the original series meaning
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1600
2021-09-09 19:21:18 +03:00
Aliaksandr Valialkin
5ea689d61b
app/vmselect/promql: add quantile("phiLabel", phi1, ..., phiN, q) aggregate function to MetricsQL
...
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1573
2021-08-27 18:37:20 +03:00
Aliaksandr Valialkin
bec18e4fe9
app/vmselect: add -search.disableAutoCacheReset command-line option for disabling automatic cache reset when a sample with old timestamp outside -search.cacheTimestampOffset is inserted
...
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1570
2021-08-27 17:15:31 +03:00
Aliaksandr Valialkin
6c5760db9c
app/vmselect/promql: make fmt after 0078486ea7
2021-08-23 23:06:00 +03:00
Aliaksandr Valialkin
0078486ea7
app/vmselect/promql: rename sign() function to sgn() in order to be consistent with Prometheus
...
See https://github.com/prometheus/prometheus/pull/8457 for details.
2021-08-23 11:45:51 +03:00
Aliaksandr Valialkin
4f3a5742eb
app/vmselect/prometheus: do not extend [d] to the detected interval between samples for first_over_time(m[d])
...
This is for the sake of consistency with similar change for the last_over_time(m[d]) at a724229b5d
2021-08-21 19:56:14 +03:00
Roman Khavronenko
b5d6a0e499
vmselect: update vm_request_duration_seconds value when request fails ( #1537 )
...
Before, metric `vm_request_duration_seconds` was update only on successful
attempts which could be misleading. For example, timeout errors on netstorage
request may be not accounted in the metric and won't be visible on dashboards.
Using `defer` statement to update the metric after query arguments validation
may improve the situation.
2021-08-19 13:58:54 +03:00
Aliaksandr Valialkin
a724229b5d
app/vmselect/promql: do not override [d] at last_over_time(m[d]) if [d] is smaller than scrape_interval
...
Since most users do not expect the overriding of explicitly set `[d]`.
2021-08-19 10:31:48 +03:00
Aliaksandr Valialkin
cdc372bb98
app/vmselect: add -search.noStaleMarkers command-line flag for disabling stale markers handling in queries
...
This option allows reducing CPU usage a bit when VictoriaMetrics is used
for collecting and processing non-Prometheus data. For example, InfluxDB line protocol, Graphite, OpenTSDB, CSV, etc.
2021-08-18 13:59:02 +03:00
Aliaksandr Valialkin
db1e62495b
app/vmselect/promql: add bitmap_and(), bitmap_or() and bitmap_xor() functions to MetricsQL
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1541
2021-08-17 13:21:21 +03:00
Aliaksandr Valialkin
bd14b0887e
app/vmselect/promql: move common condition to dropStaleNaNs in order to improve code maintainability
2021-08-17 11:01:16 +03:00
Aliaksandr Valialkin
113f0a8a07
app/vmselect/promql: drop staleness marks before calling rollupConfig.Do
...
This allows dropping staleness marks only once and then calculate multiple rollup functions on the result.
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1526
2021-08-15 13:21:10 +03:00
Aliaksandr Valialkin
25997a70f1
Revert "app/vmselect/promql: properly handle Prometheus staleness marks in removeCounterResets functions"
...
This reverts commit 94dfcb6747a3b29a11d14e71bea21a2312bb6346.
It is better to remove staleness marks (decimal.StaleNaN) before calling rollupConfig.Do, e.g. in preFunc
2021-08-15 13:19:16 +03:00
Aliaksandr Valialkin
73d7b568da
app/vmselect/promql: properly handle Prometheus staleness marks in removeCounterResets functions
...
Prometheus stalenss marks shouldn't be changed in removeCounterResets. Otherwise they will be converted to an ordinary NaN values,
which couldn't be removed in dropStaleNaNs() function later. This may result in incorrect calculations for rollup functions.
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1526
2021-08-14 12:45:57 +03:00
Aliaksandr Valialkin
4401464c22
all: add support for Prometheus staleness markers
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1526
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/748
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1509
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1530
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/845
2021-08-13 12:10:17 +03:00
Aliaksandr Valialkin
95dd5a48bb
app/vmselect: make vmui-update after the commit 4ae14df864a7e327955f44941295a286175423b3
2021-08-11 13:41:41 +03:00
Aliaksandr Valialkin
e92fde7945
app/vmselect/promql: add present_over_time(m[d]) function, which will be available starting from Prometheus 2.29.0
...
See https://github.com/prometheus/prometheus/releases/tag/v2.29.0-rc.0 and https://github.com/prometheus/prometheus/pull/9097
2021-08-03 16:11:49 +03:00
Nikolay
7bbff7fb86
adds /rules and /alerts api for grafana ( #1504 )
...
Co-authored-by: Aliaksandr Valialkin <valyala@gmail.com>
2021-08-02 17:28:09 +03:00
Aliaksandr Valialkin
a1911e1330
app/vmselect/netstorage: unpack time series data in mostly local big chunks
...
This should improve performance on multi-CPU systems for queries selecting time series with big number of raw samples
2021-07-30 12:03:17 +03:00
Aliaksandr Valialkin
74ffaa45d9
app/vmselect/netstorage: do not query Go maps with unsafe string keys, since this breaks in Go 1.17
2021-07-30 09:57:53 +03:00
Aliaksandr Valialkin
192dfbfd90
app/vmselect: follow-up for ed95bc9531
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1493
2021-07-29 09:53:28 +03:00
arnoldyahad
00af4ff5a4
Add case prometheus/rules for grafana 8 ( #1502 )
2021-07-29 09:53:27 +03:00
Aliaksandr Valialkin
8ee8660ac4
app/vmselect: follow-up for 626073bca8
...
* Rename -search.maxMetricsPointSearch to -search.maxSamplesPerQuery, so it is more consistent with the existing -search.maxSamplesPerSeries
* Move the -search.maxSamplesPerQuery from vmstorage to vmselect, so it could effectively limit the number of raw samples obtained from all the vmstorage nodes
* Document the -search.maxSamplesPerQuery in docs/CHANGELOG.md
2021-07-28 18:00:23 +03:00
Aliaksandr Valialkin
8b7917cd81
all: add go:build lines for Go1.17
...
See https://tip.golang.org/doc/go1.17#gofmt for more details
2021-07-26 15:48:21 +03:00
Aliaksandr Valialkin
95aff47330
app/vmselect: prevent from possible deadlock when f callback blocks inside RunParallel
2021-07-26 15:47:30 +03:00
Aliaksandr Valialkin
fcaf152480
app/vmselect: make vmui-update after a91d41f12a
2021-07-26 10:31:11 +03:00
Aliaksandr Valialkin
05672ffc32
app/vmselect/promql: properly handle (a op b) default N if (a op b) returns NaN series
...
The result should be a series with `N` values and `a op b` labels. Previously such series has been removed from the result.
2021-07-16 01:44:58 +03:00
Aliaksandr Valialkin
ed10141ff8
app/vmselect/netstorage: use more scalable algorithm for ditributing the work among among multiple channels on systems with big number of CPU cores
2021-07-16 00:35:23 +03:00
Aliaksandr Valialkin
ca75432e66
app/vmselect: do not track queries with less than 1ms execution time at /api/v1/status/top_queries
...
This should improve the readability and usefullness of the /api/v1/status/top_queries when debugging slow queries
or queries that take too much cpu time.
2021-07-15 16:44:28 +03:00
Aliaksandr Valialkin
f4e81aef7e
app/vmselect/netstorage: add -search.maxSamplesPerSeries command-line option for limiting the number of samples a query can process per each series
...
This should prevent from out of memory crashes like in https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1067
2021-07-15 16:03:28 +03:00
Aliaksandr Valialkin
e6ef97a5ee
app/vmselect/netstorage: improve scalability of series unpacking on multi-CPU systems
2021-07-15 15:41:58 +03:00
Aliaksandr Valialkin
a925d5a3e1
app/vmselect/promql: duration handling improvements in MetricsQL queries
...
- Support durations anywhere in MetricsQL queries. E.g. sum_over_time(m[1h])/1h is equivalent to sum_over_time(m[1h])/3600
- Support durations without suffix. E.g. rate(m[300]) is equivalent to rate(m[5m])
2021-07-12 17:16:41 +03:00
Aliaksandr Valialkin
8c764e88f0
app/vmui: move source code from https://github.com/VictoriaMetrics/vmui to app/vmui
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1413
2021-07-09 17:15:23 +03:00
Aliaksandr Valialkin
c5f0b454f0
app/vmselect: follow-up after aa11ef6d3b
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1413
2021-07-07 17:43:35 +03:00
tony
e9e35a7d6a
add vmui for vmselect component ( #1431 )
...
Co-authored-by: Aliaksandr Valialkin <valyala@gmail.com>
2021-07-07 17:33:02 +03:00
Aliaksandr Valialkin
ed944313b0
app/{vminsert,vmselect}: export vminsert_request_duration_seconds and vmselect_request_duration_seconds histograms
2021-07-07 13:25:21 +03:00
Aliaksandr Valialkin
766edbc421
lib/httpserver: print full requestURI in httpserver.Errorf
...
This should simplify debugging.
2021-07-07 13:09:40 +03:00
Aliaksandr Valialkin
cdfae0117a
app/vmselect/promql: return the last timestamp for the max / min value from tmax_over_time() and tmin_over_time() function as most users expect
2021-06-23 14:19:00 +03:00
Aliaksandr Valialkin
83a4db813e
app/vmselect: log slow requests to all the /api/v1/* handlers if their execution time exceeds -search.logSlowQueryDuration
2021-06-18 19:04:42 +03:00
Roman Khavronenko
fb4f758715
promql: fix increase_pure calculation for cases with stale series ( #1381 )
...
Due to staleness handling, increase_pure were using incorrect previous value
during calculation in cases where series disappears for period longer
than staleness period and then returns back. The fix suppose to account
for a real datapoint value before staleness takes place. The fix should
remove unexpected spikes while using `increase_pure` for staled series.
2021-06-15 17:37:19 +03:00
John Belmonte
67b17cdd68
spelling fix: synonym ( #1363 )
2021-06-10 08:32:52 +03:00
Aliaksandr Valialkin
d0dca62026
app/vmselect/promql: typo fix in the comment
2021-06-09 18:34:31 +03:00
Aliaksandr Valialkin
0842bb9294
app/vmselect/promql: add timezone_offset(tz) function
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1306
2021-05-20 11:53:09 +03:00
Aliaksandr Valialkin
832651c6c2
app/vmselect: follow up after 8a0678678b
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1168
2021-05-12 17:18:30 +03:00
Nikolay
8a0678678b
Adds tsdb match filters ( #1282 )
...
* init work on filters
* init propose for status filters
* fixes tsdb status
adds test
* fix bug
* removes checks from test
2021-05-12 15:18:45 +03:00
Aliaksandr Valialkin
6bc52fe41a
all: rename https://victoriametrics.github.io to https://docs.victoriametrics.com
2021-04-20 20:16:17 +03:00
Aliaksandr Valialkin
544821b719
app/vmselect/promql: fix tests after d3fa0ccabd
2021-04-08 00:18:01 +03:00
Aliaksandr Valialkin
d3fa0ccabd
app/vmselect/promql: properly detect aggregate topk* and bottomk* aggregate functions in order to disable duplicate sorting
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1189
2021-04-08 00:09:40 +03:00
Aliaksandr Valialkin
cb12a8f0a8
app/vmselect: return data:null instead of data:[] from /api/v1/query_exemplars, since Grafana throws an error otherwise
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1186
2021-04-07 23:34:06 +03:00
Aliaksandr Valialkin
1177dca3da
app/vmselect: do not sort series returned from topk* and bottomk* functions, since these series are already sorted in user-expected order
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1189
2021-04-07 14:16:08 +03:00
Aliaksandr Valialkin
7d23598b33
app/vmselect: return dumb response on /api/v1/query_exemplars request
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1186
2021-04-05 23:25:08 +03:00
Aliaksandr Valialkin
7a0b964e8d
app/vmselect/promql: do not delete dst_label if src_label is empty in label_copy(q, src_label, dst_label) and label_move(q, src_label, dst_label)
2021-04-03 22:05:06 +03:00
Aliaksandr Valialkin
3055ab0115
app/vmselect/promql: add ability to set label value additionally to label name for the remaining sum of time series returned from topk_* and bottomk_* functions in the form: topk_min(N, m, "label=value")
2021-04-02 23:55:54 +03:00
Aliaksandr Valialkin
c79e4a2f90
app/vmselect/promql: remove the limit on the number of time series that can be sorted, since it may confuse users
...
Always sort time series returned from `/api/v1/query` and `/api/v1/query_range` unless `sort_*` function is used at top level of the query.
2021-04-02 15:02:08 +03:00
Aliaksandr Valialkin
e7fdea5953
app/vmselect: add -search.maxStatusRequestDuration command-line flag for limiting the duration of requests to /api/v1/status/* and /api/v1/series/count
2021-03-30 21:41:35 +03:00
Aliaksandr Valialkin
7962cf1af8
app/vmselect: prevent from possible incomplete query results after timed out query
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/711
2021-03-30 13:35:45 +03:00
Aliaksandr Valialkin
aa81039b42
app/vmselect: log the metric which trigger rollup result cache reset
...
This should help finding the source of stale metrics
2021-03-25 21:31:39 +02:00
Aliaksandr Valialkin
0b2a66db30
app/vmselect/promql: do not merge time series during requests to /api/v1/query
...
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1141
2021-03-25 13:56:07 +02:00
Aliaksandr Valialkin
331a6a2015
app/vmselect/graphite: accept and enforce extra_label in all the Graphite APIs
2021-03-23 15:29:16 +02:00
Aliaksandr Valialkin
b521d1d4f2
app/vmselect: move getEnforcedTagFiltersFromRequest to searchtuils, since it will be used in Graphite functions soon
2021-03-23 14:16:29 +02:00
Aliaksandr Valialkin
71b72304ae
app/vmselect: improve description for -search.maxPointsPerTimeseries command-line flag
2021-03-22 16:45:34 +02:00
Aliaksandr Valialkin
cef010d5f7
app/vmselect/promql: increment key prefix for faster reset for rollup result cache
2021-03-22 11:59:07 +02:00
Aliaksandr Valialkin
b0c956a178
app/vmselect/graphite: follow-up after 529d7be26b
2021-03-18 16:30:20 +02:00
Nikolay
529d7be26b
changes metricsFind api ( #1137 )
...
it should be able mitigate crash if label value contains *,[ or { symbols
2021-03-18 16:12:02 +02:00
Aliaksandr Valialkin
b1713e3fcd
app/vmselect/promql: typo fix after 9666834045
2021-03-17 15:12:11 +02:00
Aliaksandr Valialkin
9666834045
app/vmselect/promql: merge adjancent buckets with the smallest summary number of hits in buckets_limit() function
...
This should improve accuracy for the returned buckets
2021-03-17 14:31:41 +02:00
Aliaksandr Valialkin
7bc728bf53
app/vmselect: add vm_index_search_duration_seconds histogram for monitoring the performance of index search
2021-03-17 01:17:41 +02:00
Aliaksandr Valialkin
0ce557951f
app/vmselect/netstorage: reduce mutex contention when unpacking data on a system with high number of CPU cores
2021-03-16 21:51:31 +02:00
Aliaksandr Valialkin
f0a4157f89
app/vmselect/promql: do not crash if histogram_over_time() function name contains uppercase letters such as Histogram_over_time()
2021-03-16 12:24:21 +02:00
Aliaksandr Valialkin
85a95bf60c
all: various fixes in command-line flag descriptions
2021-03-15 21:59:25 +02:00
Aliaksandr Valialkin
923cdb0552
app/vmselect/promql: reduce overhead on scrape interval estimation
...
It should be enough to use the first 20 datapoints instead of 100 datapoints for scrape interval estimation.
2021-03-15 20:31:33 +02:00
Aliaksandr Valialkin
82aab87446
app/vmselect/promql: fix tests after 2dae0a2c47
2021-03-15 20:18:59 +02:00
Aliaksandr Valialkin
2dae0a2c47
app/vmselect: add round_digits query arg to /api/v1/query and /api/v1/query_range handlers for limiting the number of decimal digits after the point
2021-03-15 12:36:33 +02:00
Aliaksandr Valialkin
2096c6e464
app/vmselect/prometheus: typo fix after 7c002023d7
2021-03-12 12:19:36 +02:00
Aliaksandr Valialkin
7c002023d7
app/vmselect/prometheus: do not include datapoints with timestamps matching t-d when returning results from /api/v1/query?query=m[d]&time=t as Prometheus does
2021-03-12 12:16:50 +02:00
Aliaksandr Valialkin
fe8b12fbad
app/vmselect/promql: follow up for 433fff0006
2021-03-09 12:48:44 +02:00
Nikolay
433fff0006
duplicate timeseries fix for prometheus_buckets function ( #1119 )
...
* try fix for prometheus_buckets
* merge possible end of the bucket collision
2021-03-09 12:26:23 +02:00
Aliaksandr Valialkin
a14053ffa0
app/vmselect/promql: add histogram_avg(), histogram_stddev() and histogram_stdvar() functions to MetricsQL
2021-03-04 14:12:07 +02:00
Aliaksandr Valialkin
2b53add6b2
app/vmselect/querystats: show the number of matching queries in the top by average duration and in the top by summary duration
...
This should help debugging slow queries.
2021-02-28 19:40:19 +02:00
Nikolay
186c078fac
adds enforced tag filters into cache key ( #1095 )
2021-02-27 00:15:53 +02:00
Aliaksandr Valialkin
d86e9b49c4
app/vmselect/promql: increase accuracy for buckets_limit() function for small limits by skipping the first and the last buckets during merge
...
The first and the last buckets are usually `[0 ... leMin]` and `(leMax ... +Inf)`. If they are merged with adjancent buckets,
then the resulting accuracy can suffer.
2021-02-26 22:56:36 +02:00
Aliaksandr Valialkin
98854e5f2b
app/vmselect: add sign(q) and clamp(q, min, max) functions, which will be added in the upcoming Prometheus release
...
See https://twitter.com/roidelapluie/status/1363428376162295811
The `last_over_time(m[d])` function already exists in MetricsQL.
2021-02-24 17:24:56 +02:00
Aliaksandr Valialkin
f4135b0d14
app/vmselect/promql: properly calculate histogram_quantile() over zero buckets and only a single non-zero le="+Inf"` bucket like Prometheus does
2021-02-24 00:42:22 +02:00
Aliaksandr Valialkin
fa03e0d210
app/vmselect/promql: add increase_pure() function to MetricsQL
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/962
2021-02-22 19:14:15 +02:00
Aliaksandr Valialkin
9c70c1f21f
app/vmselect/promql: reduce the probability of duplicate time series errors when querying Kubernetes metrics
2021-02-18 22:07:29 +02:00
Aliaksandr Valialkin
c4756f94da
app/vmselect/netstorage: reuse timeseriesWork objects in order to reduce memory allocations
2021-02-16 16:08:53 +02:00
Aliaksandr Valialkin
5a401225c7
app/vmselect/netstorage: use unsafe string as a key for a map when the map already contains the given key
...
This should prevent from a memory allocation and a string copy.
2021-02-16 15:43:10 +02:00
Aliaksandr Valialkin
4e39bf148c
vendor: update github.com/VictoriaMetrics/metrics from v1.13.1 to v1.14.0
...
The new version switches from log-linear histograms to log-based histograms,
which provide up to 3.6 times better accuracy.
2021-02-15 15:12:29 +02:00
Aliaksandr Valialkin
acdb401585
app/vmselect/prometheus: treat match query arg in the same way as match[] query arg
2021-02-11 15:02:21 +02:00
Aliaksandr Valialkin
2d33230793
app/vmselect/promql: properly make copies of EvalConfig
2021-02-11 12:41:15 +02:00
Aliaksandr Valialkin
7a3a9421f3
app/vmselect/promql: make a copy of EvalConfig when executing q1 and q2 in parallel for q1 binary_op q2
...
This should prevent from data races if the underlying functions modify EvalConfig contents.
2021-02-10 23:05:55 +02:00
Aliaksandr Valialkin
04faea8b45
app/vmselect: parallelize q1 <binary_op> q2 queries by running q1 and q2 in parallel
...
This should reduce query execution times.
2021-02-10 22:59:39 +02:00
Aliaksandr Valialkin
553016ea99
lib/storage: disable composite index usage when querying old data
2021-02-10 14:57:50 +02:00
Aliaksandr Valialkin
c7dccebaef
lib/storage: optimize search by label filters matching big number of time series
2021-02-10 00:44:54 +02:00
Aliaksandr Valialkin
803a00102a
app/{vminsert,vmselect}: accept requests to paths with /graphite and /prometheus prefixes
...
This should improve compatibility with path prefixes from VictoriaMetrics cluster.
See https://victoriametrics.github.io/Cluster-VictoriaMetrics.html#url-format
2021-02-04 20:01:18 +02:00
Aliaksandr Valialkin
4d43ab0875
app/vmselect: typo fix when stripping url path prefixes
2021-02-04 19:29:25 +02:00
Aliaksandr Valialkin
8629fd8a72
app/vmselect: deprecate -search.treatDotsAsIsInRegexps in favor to {__graphite__="foo.*.bar"} syntax
2021-02-03 20:36:01 +02:00
Aliaksandr Valialkin
d16f22f3a1
app/vmselect,lib/storage: properly parse Graphite selectors with inner wildcards
...
Example: foo{bar{x,yz},a[b-c],*de}
2021-02-03 20:14:22 +02:00
Aliaksandr Valialkin
157c02622b
app/vmselect: add ability to set Graphite-compatible filter via {__graphite__="foo.*.bar"} syntax
2021-02-03 01:21:54 +02:00
Aliaksandr Valialkin
b43b498fd8
app/vmselect: add ability to pass extra_label=<label>=<value> query arg to Prometheus Querying API
...
This enforced `{label="value"}` label filter to the query.
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1021
2021-02-01 18:04:17 +02:00
Nikolay
195341a7cf
Graphite vmalert wip ( #112 )
...
* init implementation for graphite alerts
* adds graphite support for vmalert
* small fix
* changes vmalert graphite api with type
* updates tests
* small fix
* fixes graphite parse
* Fixes graphite from time
2021-02-01 15:05:32 +02:00
Aliaksandr Valialkin
c164a8d231
app/vmselect/promql: improve documentation for -search.maxPointsPertimeseries command-line flag
...
This should reduce incorrect usage and assumptions for this flag.
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1020
2021-01-22 13:00:10 +02:00
Aliaksandr Valialkin
8749c2dd92
app/vmselect: add -search.maxStepForPointsAdjustment command-line flag, which can be used for disabling adjustment for points returned from /api/v1/query_range handler if they have timestamps closer than -search.latencyOffset to the current time
2021-01-19 22:56:32 +02:00
Aliaksandr Valialkin
011c5da785
app/vmselect/graphite: extract getCanonicalPath() function from loop body inside getCanonicalPaths()
2021-01-18 17:30:26 +02:00
Aliaksandr Valialkin
1051d8aa2d
app/vmselect/promql: add ability to pass multiple labels to sort_by_label and sort_by_label_desc functions
...
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/992
2021-01-13 12:44:51 +02:00
Aliaksandr Valialkin
acf1a2c72b
app/vmselect/promql: properly parse escaped multibyte utf8 code sequences in metric names and labels names
...
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/990
2021-01-13 10:59:42 +02:00
Aliaksandr Valialkin
1837f2f7d3
app/vmselect/promql: add tfirst_over_time(m[d]) and tlast_over_time(m[d]) MetricsQL functions for returning timestamps for the first and the last samples in m over d
2021-01-12 16:12:12 +02:00
Aliaksandr Valialkin
c86286ec1d
app/vmselect/promql: do not ajdust offset value provided in the query
...
Previously it could be modified in order to improve response cache hit ratio.
This is unneeded, since cache hit ratio should remain good because the query time range
should be already aligned to multiple of `step` values.
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/976
2020-12-27 14:09:25 +02:00
Aliaksandr Valialkin
4b7105a65b
app/vmselect: sync query stats handling with cluster version
2020-12-27 13:00:29 +02:00
Aliaksandr Valialkin
df0309eae0
app/vmselect/promql: simplify defer call for querystats.RegisterQuery
2020-12-27 12:06:04 +02:00
Aliaksandr Valialkin
ad4e6a9283
app/vmselect/querystats: reduce the default number of last queries to track from 100K to 20K
...
This should reduce memory usage in constrained environments
2020-12-25 17:40:47 +02:00
Aliaksandr Valialkin
59183f66d0
app/vmselect: refactor /api/v1/stats/top_queries
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/907
2020-12-25 16:44:29 +02:00
Nikolay
86630350bf
Adds query stats handler ( #945 )
...
* Adds query stat handler,
for query and query_range api, victoriametrics tracks query execution time,
stats are expored at /api/v1/status/queries endpoint with topN param
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/907
* fixed query stats bugs
* improves queryStats tracker
* improves query stat
* small fix
* fix tests
* added more tests
* fixes 386 tests
* naming fixes
* adds drop for outdated records
2020-12-25 16:42:05 +02:00
Aliaksandr Valialkin
88ac4dfc07
app/vmselect: properly parse negative combined offsets such as -1h2m3s
...
Previously such offsets were parsed as `-1h + 2m + 3s`. Now they are parsed as `-(1h + 2m + 3s)`.
2020-12-19 01:23:46 +02:00
Aliaksandr Valialkin
11fa458e39
app/vmselect/promql: return expected increase() result for the first point on the graph with value not exceeding 100
2020-12-15 13:40:46 +02:00
Aliaksandr Valialkin
5ebfc275e6
app/victoria-metrics: automatically reset response cache when samples with too timestamps older than now - search.cacheTimestampOffset are ingested
2020-12-14 13:08:28 +02:00
Aliaksandr Valialkin
ba8ac08739
app/vmselect/graphite: properly handle wildcards and charsets inside curly braces
...
For example, `foo{bar*,[a-f]a*b}` should match `foobar`, `foobar123`, `foofab`, etc.
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/952
2020-12-11 17:24:12 +02:00
Aliaksandr Valialkin
4146fc4668
all: properly handle CPU limits set on the host system/container
...
This can reduce memory usage on systems with enabled CPU limits.
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/946
2020-12-08 21:07:29 +02:00
Aliaksandr Valialkin
1906f841c9
app/vmselect/promql: do not reduce lookbehind window for any_rollup_func(m) to -search.maxStalenessInterval. It should equal to step value passed to /api/v1/query_range as most users expect
2020-12-08 15:16:17 +02:00
Aliaksandr Valialkin
007dbf273d
app/vmselect/graphite: remove duplicate name tag from /tags/autoComplete/tags handler
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/942
2020-12-07 01:08:52 +02:00
Aliaksandr Valialkin
e2e8ef86d9
app/vmselect/promql: add count_eq_over_time(m[d], N) and count_ne_over_time(m[d], N) for calculating the number of samples in m over d that are equal / not equal to N
2020-12-05 12:30:46 +02:00
Aliaksandr Valialkin
66379cc69f
app/vmselect/promql: add label_uppercase(q, label1, ... labelN) and label_lowercase(q, label1, ... labelN) functions
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/936
2020-12-03 21:47:36 +02:00
Aliaksandr Valialkin
44a34a0f5f
app/vmselect/promql: make fmt
2020-12-02 21:33:35 +02:00
Aliaksandr Valialkin
1982505c2b
app/vmselect/promql: return nan from minute(m) when m equals to nan
...
This aligns VictoriaMetrics behaviour with Prometheus behaviour.
The issue has been spotted in https://promlabs.com/promql-compliance-test-results/2020-12-01/victoriametrics/
2020-12-02 20:16:58 +02:00
Aliaksandr Valialkin
9d87496b50
app/vmselect/promql: do not return 0 value from sum_over_time(m[d]) when there are no samples on the given d window.
...
This aligns the behaviour of `sum_over_time()` with other `_over_time()` functions and with Prometheus behavior.
2020-12-02 13:12:50 +02:00
Aliaksandr Valialkin
91a4c279cc
app/vmselect: return metric values from time() cmp_op metric query when cmp_op comparison is true
...
This aligns MetricsQL behavior to Prometheus' one.
The issue has been identified at https://promlabs.com/promql-compliance-test-results/2020-12-01/victoriametrics/
2020-12-02 12:09:34 +02:00
Aliaksandr Valialkin
700bda8e2e
app/vmselect/promql: return nan from a >bool b if a is nan in the same way as Prometheus does
2020-12-02 00:28:26 +02:00
