github-mirrors/VictoriaMetrics
1
0
Fork 0
mirror of https://github.com/VictoriaMetrics/VictoriaMetrics.git synced 2024-11-21 14:44:00 +00:00
Code Issues Projects Releases Packages Wiki Activity
5544 commits 150 branches 748 tags 347 MiB
Commit graph

832 commits

Author SHA1 Message Date
Dmytro Kozlov
463ea6897b
vmselect/promql: enable search.maxPointsSubqueryPerTimeseries for sub-queries (#2963) 
* vmselect/promql: enable search.maxPointsPerTimeSeriesSubquery for sub-queries

* vmselect/promql: cleanup

* vmselect/promql: rename config flag

* vmselect/promql: add tests

* vmselect/promql: use test object instead of log

* vmselect/promql: fix posible panic is subquery has more points. add description

* vmselect/promql: update tests descriptions

* vmselect/promql: update doInternal validation

* vmselect/promql: fix linter

* vmselect/promql: fix linter

* vmselect/promql: update documentation and release notes

* wip

- Properly apply -search.maxPointsSubqueryPerTimeseries limit to subqueries.
  Previously the -search.maxPointsPerTimeseries limit was unexpectedly applied to subqueries
  if it was smaller than the -search.maxPointsSubqueryPerTimeseries .
- Clarify docs for -search.maxPointsSubqueryPerTimeseries command-line flag .
- Document -search.maxPointsPerTimeseries and -search.maxPointsSubqueryPerTimeseries flags at https://docs.victoriametrics.com/#resource-usage-limits .
- Update docs/CHANGELOG.md .

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2922

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2022-08-24 15:25:18 +03:00
Aliaksandr Valialkin
343241680b
all: remove the remaining bits of io/ioutil 
The io/ioutil package is deprecated since Go1.16 - see https://tip.golang.org/doc/go1.16#ioutil

VictoriaMetrics requires at least Go1.18, so it is time to remove the io/ioutil from source code

This is a follow-up for 02ca2342ab
 2022-08-22 00:20:58 +03:00
Aliaksandr Valialkin
9f94c295ab
all: use os.{Read|Write}File instead of ioutil.{Read|Write}File 
The ioutil.{Read|Write}File is deprecated since Go1.16 -
see https://tip.golang.org/doc/go1.16#ioutil

VictoriaMetrics needs at least Go1.18, so it is safe to remove ioutil usage
from source code.

This is a follow-up for 02ca2342ab
 2022-08-21 23:52:35 +03:00
Aliaksandr Valialkin
4ac79d29ad
app/vmselect: follow-up after 63e0f16062 
* Explicitly store a pointer to UserReadableError in the error interface.
  Previously Go automatically converted the value to a pointer before storing in the error interface.

* Add Unwrap() method to UserReadableError, so it can be used transparently with the other code,
  which calls errors.Is() and errors.As().

* Document the change in docs/CHANGELOG.md
 2022-08-15 13:50:16 +03:00
Roman Khavronenko
63e0f16062
vmselect: introduce UserReadableError type of error (#2894) 
When read query fails, VM returns rich error message with
all the details. While these details might be useful
for debugging specific cases, they're usually too verbose
for users.
Introducing a new error type `UserReadableError` is supposed
to allow to return to user only the most important parts
of the error trace. This supposed to improve error readability
in web interfaces such as VMUI or Grafana.

The full error trace is still logged with the full context
and can be found in vmselect logs.

Signed-off-by: hagen1778 <roman@victoriametrics.com>

Signed-off-by: hagen1778 <roman@victoriametrics.com>
 2022-08-15 13:38:47 +03:00
Yury Molodov
1513866d51
vmui: shortcut keys legend (#2971) 
* feat: add shortcut modal

* feat: add shortcut descriptions

* app/vmselect/vmui: `make vmui-update`

* docs/CHANGELOG.md: document the change

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2022-08-15 11:54:47 +03:00
Yury Molodov
3cb013aeb8
vmui: graph action on moush hold and move (#2915) 
* fix: change event for graph panning

* fix: change detect key

* feat: add zoom in with mouse selection

* - document the change
- run `make vmui-update`

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2022-08-07 23:55:09 +03:00
Aliaksandr Valialkin
7478d423c5
app/vmselect/netstorage: cleanup after 92630c1ab4 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2896
 2022-08-04 18:28:11 +03:00
Aliaksandr Valialkin
6b0550c023
app/{vmselect,vmalert}: properly generate http redirects if -http.pathPrefix command-line flag is set 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2918
 2022-08-02 12:59:07 +03:00
Aliaksandr Valialkin
c2bd75926b
app/vmselect/netstorage: initialize tsw.rowsProcessed before calling tsw.f, since tsw.f can modify r.Timestamps and r.Values lengths 2022-07-30 00:39:36 +03:00
Aliaksandr Valialkin
19a0b4679a
app/vmselect/netstorage: re-use random generator used for series shuffle in Result.RunParallel 
This should reduce CPU usage needed for rand.Rand initialization
 2022-07-30 00:30:37 +03:00
Dmytro Kozlov
a927814e7b
vmselect/promql: add tests for vmrangeBucketsToLE (#2907) 
* vmselect/promql: add tests for vmrangeBucketsToLE

* vmselect/promql: cleanup

* vmselect/promql: cleanup

* vmselect/promql: fix panic tests want result

* vmselect/promql: cleanup

* vmselect/promql: update test name

* vmselect/promql: fix linter error

* vmselect/promql: refactor testcases

* vmselect/promql: cleanup

* vmselect/promql: remove unused reassign to workers, fix typo

* wip

* wip

* wip

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2022-07-26 20:42:41 +03:00
Aliaksandr Valialkin
c888e6b9be
app/vmselect/promql: reduce the diff for f148cffc8a 
This is a follow-up for c826f06366
 2022-07-26 19:20:48 +03:00
Alan Liang
c826f06366
vmselect: fix vmrangeBucketsToLE func may panic when ts value equal zero (#2902) 
Co-authored-by: alanwzliang <alanwzliang@tencent.com>
 2022-07-25 10:55:13 +03:00
Aliaksandr Valialkin
92630c1ab4
app/vmselect/netstorage: improve the speed of queries over big number of time series on multi-CPU system 
Reduce inter-CPU communications when processing the query over big number of time series.
This should improve performance for queries over big number of time series
on systems with many CPU cores.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2896

Based on b596ac3745
Thanks to @zqyzyq for the idea.
 2022-07-25 09:18:44 +03:00
Aliaksandr Valialkin
79d967d35a
app/vmselect/vmui: make vmui-update after edecd2493c 2022-07-21 20:59:52 +03:00
Roman Khavronenko
9ccf695d57
vmselect: return correct error for second part of expression (#2893) 
Signed-off-by: hagen1778 <roman@victoriametrics.com>
 2022-07-20 16:44:28 +02:00
Aliaksandr Valialkin
cc7d499bbd
app/vmselect/promql: execute q1 and q2 from q1 op q2 in parallel if labels pushdown cannot be applied 
This should improve query performance if VictoriaMetrics has enough resources for processing `q1` and `q2` in parallel.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2886
 2022-07-19 14:27:48 +03:00
Aliaksandr Valialkin
b5da47bfaf
app/vmselect/promql: properly return q1 series from q1 ifnot q2 when q2 returns nothing 2022-07-18 14:24:54 +03:00
Aliaksandr Valialkin
0792c4ca90
app/vmselect/promql/transform.go: reuse evalNumber() function for constructing timezone_offset() results 2022-07-18 14:24:53 +03:00
Yury Molodov
17c33132df
vmui: optimize table view (#2867) 
* feat: optimize table view

* fix: add column display setting

* app/vmselect: `make vmui-update`

Also document the change at docs/CHANGELOG.md

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2022-07-14 00:15:43 +03:00
Aliaksandr Valialkin
29e53b9f55
app/vmselect/promql: consistency update after 93fbd0c54b 2022-07-13 12:33:14 +03:00
Roman Khavronenko
93fbd0c54b
promql: return step as scrapeInterval when it can't be calculated (#2865) 
The change allows to specify default value for `getScrapeInterval`
function when actual interval can't be calculated.

Before the change, function were returning `maxSilenceInterval` (5m)
in such cases, which may be not correct for instant queries processing.
The specific scenario where using `maxSilenceInterval` caused issues
is the following:
1. Series becomes stale;
2. Client (in this case vmalert) continues to request series every 15s;
3. Database returns empty results as expected;
4. But at some specific moment of time database returns datapoints from `now()-5m`,
because lookback window was extended to `maxSilenceInterval`.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
 2022-07-13 12:27:38 +03:00
Aliaksandr Valialkin
c2197ad139
app/vmselect/promql: validate function name before evaluating its arguments 
This avoids unneeded evaluation of args for unknown functions
 2022-07-12 19:48:26 +03:00
Aliaksandr Valialkin
159c2e15e3
app/vmselect/netstorage: optimize mergeSortBlocks() for the worst case when blocks contain interleaved samples 2022-07-12 12:31:38 +03:00
Aliaksandr Valialkin
8429d4af5a
app/vmselect/netstorage: add mergeSortBlocks benchmark for the worstcase 2022-07-12 12:31:36 +03:00
Aliaksandr Valialkin
cad471037a
app/vmselect/prometheus: follow-up after 3efe33b917 2022-07-11 20:35:28 +03:00
Dmytro Kozlov
3efe33b917
vmselect/prometeus: Add limit param to api/v1/series api endpoint (#2851) 
* issue-2841: Add limit param to api/v1/series api endpoint

* issue-2841: add change log

* issue-2841: update logic

* issue-2841: simplify logic

* issue-2841: simplify logic, add information to documentation
 2022-07-11 20:18:30 +03:00
Aliaksandr Valialkin
ce68e76d62
app/vmselect: follow-up after 8667307d73 2022-07-11 20:14:34 +03:00
Roman Khavronenko
8667307d73
vmselect: cover special cases for vmalert's routing in single-node version (#2845) 
* vmselect: cover special cases for vmalert's routing in single-node version

* remove trailing `/` from requests
* redirect to vmalert's home page when `/vmalert` is requested.

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* vmalert: fix review comments

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* Update app/vmselect/main.go

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2022-07-11 19:52:22 +03:00
Aliaksandr Valialkin
5c8eee26bf
all: make fmt via the upcoming Go1.19 2022-07-11 19:22:15 +03:00
Aliaksandr Valialkin
cd09f583fe
app/vmselect/netstorage: add benchmarks for mergeSortBlocks 
This is a follow-up for 743ff84863
 2022-07-11 12:54:48 +03:00
Aliaksandr Valialkin
743ff84863
app/vmselect/netstorage: optimize mergeSortBlocks function 
- Use binary search instead of linear scan when locating the run of smallest timestamps
  in blocks with intersected time ranges. This should improve performance
  when merging blocks with big number of samples

- Skip samples with duplicate timestamps. This should increase query performance
  in cluster version of VictoriaMetrics with the enabled replication.
 2022-07-09 00:34:42 +03:00
Roman Khavronenko
e1a41cfab5
metricsql: properly evaluate timezone_offset over time interval (#2842) 
* metricsql: properly evaluate `timezone_offset` over time interval

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2771
Signed-off-by: hagen1778 <roman@victoriametrics.com>

* Update docs/CHANGELOG.md

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2022-07-08 14:03:56 +03:00
Aliaksandr Valialkin
2e9ae40d56
app/vmselect/vmui: follow-up after 0bf6841140 
* Document the bugfix in docs/CHANGELOG.md
* Run `make vmui-update` for updating static js files for vmui, which are included into vmselect
 2022-07-08 13:14:17 +03:00
Aliaksandr Valialkin
a6bd442ef9
app/vmui: tune visual presentation of trace view 2022-07-06 14:07:12 +03:00
Aliaksandr Valialkin
bd5b20445e
app/vmselect: add ability to query vmselect from another vmselect 2022-07-06 13:30:12 +03:00
Aliaksandr Valialkin
17de8a41c2
all: follow-up after ed89106274 2022-07-06 12:44:46 +03:00
Aliaksandr Valialkin
77cbbacfdb
lib/vmselectapi: pass storage.SearchQuery to API calls instead of []*storage.TagFilters + storage.TimeRange + maxMetrics 
This reduces the number of args to vmselectapi calls
 2022-07-06 12:37:54 +03:00
Aliaksandr Valialkin
f435924ab3
lib/vmselectapi: pass maxSuffixes arg to tagValueSuffixes RPC call 2022-07-06 12:37:54 +03:00
Aliaksandr Valialkin
e1b8059086
lib/vmselectapi: rename deleteMetrics to more correct deleteSeries 2022-07-06 12:37:54 +03:00
Aliaksandr Valialkin
a60e03b3a7
lib/vmselectapi: use string type for tagKey and tagValuePrefix args at TagValueSuffixes() 
This improves the API consistency
 2022-07-06 12:37:53 +03:00
Roman Khavronenko
ed89106274
vmselect: allow proxying requests to vmalert from single-node (#2834) 
The change allows to proxy requests with prefix `/vmalert`
to the vmalert component if `-vmalert.proxyURL` is set.

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2825
and https://github.com/VictoriaMetrics/VictoriaMetrics/pull/2831

Signed-off-by: hagen1778 <roman@victoriametrics.com>
 2022-07-06 10:47:26 +02:00
Aliaksandr Valialkin
ae80cf76e0
app/vmselect: make fmt after f3ece83e67 2022-07-05 14:35:24 +03:00
Aliaksandr Valialkin
f3ece83e67
app/vmselect/promql: properly calculate histogram_quantile over unexpected le buckets 
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2819
 2022-07-05 13:19:24 +03:00
Aliaksandr Valialkin
84e373e5c7
app/vmselect/promql: properly handle partial counter resets in rate(), irate(), increase() and remove_resets() functions 2022-06-30 22:39:38 +03:00
Aliaksandr Valialkin
3e2dd85f7d
all: readability improvements for query traces 
- show dates in human-readable format, e.g. 2022-05-07, instead of a numeric value
- limit the maximum length of queries and filters shown in trace messages
 2022-06-30 18:20:33 +03:00
Dmytro Kozlov
4d9715f5a8
vmui: update render logic for nested component (#2795) 
* vmui: update render logic for nested component, avoid rerender, remove local storage usage for tracing flag

* docs/url-examples.md: fix various documentation issues there

* docs: add Troubleshooting doc

This doc contains troubleshooting guides for typical problems with VictoriaMetrics.

* docs/Troubleshooting.md: add troubleshooting guide for cluster instability

* wip

* wip

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2022-06-30 15:47:12 +03:00
Aliaksandr Valialkin
a14188dd8e
app/vmselect: expose additional histograms at /metrics page, which may help get more insights for the query workload 
This commit is based on https://github.com/VictoriaMetrics/VictoriaMetrics/pull/2792
 2022-06-28 20:18:13 +03:00
Aliaksandr Valialkin
a43f2d0bc5
app/vmselect/promql: show the number of scanned samples in the query trace 2022-06-28 19:26:17 +03:00
Aliaksandr Valialkin
a5181703b1
app/vmselect/prometheus: reduce the default value for -search.maxSeries from 100k to 30k 
Production experience shows that 100k is too big for /api/v1/series .
It leads to increased CPU usage when Grafana queries /api/v1/series over VictoriaMetrics
with big number of time series during auto-completion and when modifying template variables.
 2022-06-28 18:22:30 +03:00
Aliaksandr Valialkin
a350d1e81c
lib/storage: return marshaled metric names from SearchMetricNames 
Previously SearchMetricNames was returning unmarshaled metric names.
This wasn't great for vmstorage, which should spend additional CPU time
for marshaling the metric names before sending them to vmselect.

While at it, remove possible duplicate metric names, which could occur when
multiple samples for new time series are ingested via concurrent requests.

Also sort the metric names before returning them to the client.
This simplifies debugging of the returned metric names across repeated requests to /api/v1/series
 2022-06-28 18:17:15 +03:00
Aliaksandr Valialkin
e578549b8a
app/vmselect: optimize /api/v1/series a bit for time ranges smaller than one day 2022-06-28 13:02:47 +03:00
Aliaksandr Valialkin
a963b2a0aa
all: show timeRange in traces in human-readable format instead of timestamps in milliseconds 2022-06-27 13:45:51 +03:00
Aliaksandr Valialkin
ba514284f1
lib/storage: add querytracer to more contexts 
querytracer has been added to the following storage.Storage methods:
- RegisterMetricNames
- DeleteMetrics
- SearchTagValueSuffixes
- SearchGraphitePaths
 2022-06-27 13:45:51 +03:00
Aliaksandr Valialkin
9a314106ca
app/vmselect/netstorage: remove Get prefix from netstorage functions 
This makes these function names more consistent with the server side
 2022-06-27 00:45:05 +03:00
Dmytro Kozlov
bb7f31541f
vmui: added query tracing (#2748) 
* vmui: added query tracing

* vmui: updated ui

* vmui: update tracing logic, fix bugs, disable tracing by default

* vmui: use empty message as props

* vmui: fixed ui, added delete for each tacing data, show query in header

* vmui: added timelines

* vmui: speedup render

* vmui: use memo for sorting

* vmui: use Trace model, remove unused functions, simplify part of code

* vmui: update recursive logic

* vmui: fix set query to header

* vmui: code cleanup, remove unused code

* vmui: remove unused type, rename component

* wip

* wip

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2022-06-23 22:59:20 +03:00
Aliaksandr Valialkin
51362f9333
app/vmselect: add -search.setLookbackToStep command-line flag for making the gap filling algorithm similar to InfluxDB data model 
This option should override `-search.maxStalenessInterval` for most cases when users migrate from InfluxDB to VictoriaMetrics
 2022-06-22 14:19:30 +03:00
Aliaksandr Valialkin
7bf75c7e61
app/vmselect: typo fix in the exported metric name: vm_http_request_total -> vm_http_requests_total 2022-06-22 13:15:31 +03:00
Aliaksandr Valialkin
e6ed92529b
all: remove explicit "xxhash" name when importing github.com/cespare/xxhash/v2 package 
This package already has the same name, so there is no need in explicit name
 2022-06-21 20:23:32 +03:00
Dmytro Kozlov
10454d1735
vmui: added focusLabel, enable cardinality app configuratior (#2736) 
* vmui: added focusLabel, enable app configuratior

* vmui: set focusLabel if {labelName!=""}

* wip

* docs/CHANGELOG.md: mention about focusLabel feature in cardinality explorer

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2730

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2022-06-17 13:03:02 +03:00
Roman Khavronenko
723d90536c
vmselect: limit end param max value by 2d in future (#2729) 
* vmselect: limit `end` param max value by 2d in future

The change is applied only to service handlers like `/labels` or `/series`
and limits the `end` param by max value <= now() + 2 days. The same limit
is applied for the ingested data, so no reason to allow to request data
in future far than that.

The change is also needed for corner cases like https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2669
where too high `end` value triggers inefficient global index search.

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* docs/CHANGELOG.md: document the bugfix

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2022-06-16 20:46:31 +03:00
Aliaksandr Valialkin
ec7963208d
app/vmselect: accept focusLabel query arg at /api/v1/status/tsdb 
This allows filling the seriesCountByFocusLabelValue list in the /api/v1/status/tsdb response
with label values for the specified focusLabel, which contain the highest number of time series.

TODO: add this to Cardinality explorer at VMUI - https://docs.victoriametrics.com/#cardinality-explorer
 2022-06-14 18:36:54 +03:00
Aliaksandr Valialkin
b6c1ca12b7
lib/storage: show top labels with the highest number of series in cardinality explorer 2022-06-14 16:32:38 +03:00
Dmytro Kozlov
af3dc91a51
vmui: refactor Cardinality panel (#2726) 
* vmui: refactor Cardinality panel

* vmui: change width of the search panel

* vmui: code cleanup

* vmui: code cleanup

* vmui: fixed vulnerability (npm audit fix)

* wip

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2022-06-14 14:39:47 +03:00
Yury Molodov
879670418f
vmui: enhancements (#2638) (#2717) 
* feat: make datepicker to be set to last 30 min by default

* fix: correct spinner while loading data

* feat: change legend style

* app/vmselect: `make vmui-update`

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2022-06-13 09:43:37 +03:00
Aliaksandr Valialkin
374beb350e
app/vmselect: optimize /api/v1/labels and /api/v1/label/.../values handlers when match[] query arg is passed to them 2022-06-12 04:32:13 +03:00
Aliaksandr Valialkin
89b778902b
app/vmselect: add optional limit query arg to /api/v1/labels and /api/v1/label_values endpoints 
This arg allows limiting the number of sample values returned from these APIs
 2022-06-10 09:50:33 +03:00
Aliaksandr Valialkin
483b402bb2
app/vmselect/prometheus: extract common code for obtaining common query args into getCommonParams() function 2022-06-09 20:34:18 +03:00
Aliaksandr Valialkin
2bcb960f17
all: improve query tracing coverage for indexdb search 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1403
 2022-06-09 20:07:07 +03:00
Aliaksandr Valialkin
a30333a79e
app/vmselect/graphite: remove additional redundant Request.ParseForm() calls after 38c785b851 2022-06-09 13:28:57 +03:00
Aliaksandr Valialkin
38c785b851
app/vmselect: remove redundant calls to Request.ParseForm() 
Request.ParseForm() is implicitly called by the first call to Request.FormValue()
 2022-06-09 13:11:26 +03:00
Aliaksandr Valialkin
12ac255dae
lib/querytracer: make it easier to use by passing trace context message to New and NewChild 
The context message can be extended by calling Donef.
If there is no need to extend the message, then just call Done.
 2022-06-08 21:06:52 +03:00
Dmytro Kozlov
018d2303c4
Cardinality explorer (#2625) 
* Cardinality explorer

* vmui, vmselect: updated field name, added description to spinner

* make vmui-update

* updated const name, make vmui-update

* lib/storage: changes calculation for totalSeries values

* added static files

* wip

* wip

* wip

* wip

* docs/CHANGELOG.md: document cardinality explorer feature

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2233

Co-authored-by: f41gh7 <nik@victoriametrics.com>
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2022-06-08 18:43:05 +03:00
Aliaksandr Valialkin
8608dd093c
all: follow-up after 8edb390e21 
- Remove unused js bloatware from /targets page. This strips down binary size by more than 100Kb
- Add /service-discovery page for API compatibility with Prometheus
- Properly load bootstrap.min.css from /prometheus/targets
- Serve static contents for /targets page from app/vminsert instead of app/vmselect, because /targets page is served from there
 2022-06-07 00:57:09 +03:00
Dmytro Kozlov
8edb390e21
lib/promscrape: adds service discovery visualization for /targets page(#2675) 
* lib/promscrape: updated template

* lib/promscrape: fixed click on unhealthy and all btns

* app/vmselect: jquery scripts into static folder

Co-authored-by: f41gh7 <nik@victoriametrics.com>
 2022-06-03 15:38:45 +02:00
Aliaksandr Valialkin
41958ed5dd
all: add initial support for query tracing 
See https://docs.victoriametrics.com/Single-server-VictoriaMetrics.html#query-tracing

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1403
 2022-06-01 02:29:23 +03:00
Dmytro Kozlov
11f91532c5
issue-2594: use embedded for static files (#2650) 
embed static js and css files from CDN into vmalert, vmagent and vmsingle binaries.

Co-authored-by: f41gh7 <nik@victoriametrics.com>

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2594
 2022-05-31 01:55:28 +02:00
Aliaksandr Valialkin
69b9cf7161
app/vmselect/vmui: make vmui-update after 492a615a88 2022-05-26 09:40:57 +03:00
Yury Molodov
492a615a88
vmui: import dashboards (#2642) 
* fix: switch dashboards import to fetch

* make vmui-update
 2022-05-26 09:39:10 +03:00
Aliaksandr Valialkin
c6d543e2f9
app/vmselect/vmui: make vmui-update 2022-05-20 14:53:38 +03:00
Yury Molodov
ff74472621
vmui: setup predefined dashboards without build (#2541) 
vmui: support predefined dashboards in json format

See https://github.com/VictoriaMetrics/VictoriaMetrics/tree/master/app/vmselect/vmui/dashboards
 2022-05-16 09:42:37 +02:00
Aliaksandr Valialkin
a7f18f8cb2
app/vmselect/promql: do not return values from label_value() if the original time series has no values at the selected timestamps 2022-05-09 17:57:39 +03:00
Aliaksandr Valialkin
ca4ca4630b
app/vmselect/vmui: make vmui-update after 450d879eaa 2022-05-05 21:26:01 +03:00
Aliaksandr Valialkin
fc7c7237e3
app/vmselect: follow-up after 8639e79d38 2022-05-04 23:35:57 +03:00
Nikolay
8639e79d38
app/vmselect: adds proxy for rules API (#2516) 
* app/vmselect: adds proxy for rules and alerts API
It allows to visualization for rules at grafana
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1739

* Update app/vmselect/main.go

Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
 2022-05-03 20:55:15 +03:00
Dima Lazerka
1e26dd1f82
Code cleanup (#343) 
* Small code cleanup: remove Request from params

* Extract common params to all export handlers

* Renamed ExportParams -> exportParams

* wip

Co-authored-by: Dzmitry Lazerka <dlazerka@gmail.com>
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2022-05-03 16:04:16 +03:00
Aliaksandr Valialkin
bca4737fcf
app/vmui: fix up/down arrow keys on multi-line query after a186434b50 2022-05-03 10:50:16 +03:00
Aliaksandr Valialkin
a186434b50
app/vmui: execute query by pressing enter in the same way as Prometheus does 
Multi-line query can be entered via `shift-enter` in the query input field
 2022-05-02 19:49:29 +03:00
Yury Molodov
87693754d5
vmui: support node v.18 (#2529) 
* fix: add support vmui node18

* fix: remove @mui/styles (legacy styling solution)

* Update app/vmui/Dockerfile-build

* app/vmselect/vmui: `make vmui-update`

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2022-05-02 18:41:53 +03:00
Aliaksandr Valialkin
7aa5167996
app/vmselect/main.go: move the code /api/v1/status/buildinfo handler to the same location as in the cluster branch 2022-04-29 13:02:14 +03:00
Aliaksandr Valialkin
464325a24b
app/vmselect/vmui: make vmui-update after da04e9d1de 2022-04-29 12:54:10 +03:00
Dima Lazerka
ed8e88af11
Export "null" in jsonl instead of NaN (#2518) 
* Export "null" in jsonl instead of NaN

The NaN appeared because of staleness markers that were added for compatibility. I think it's better to use json `null`, implemented here.

Also maybe it also makes sense to add a flag like `?skip-staleness-markers=true` to `/export`, to skip nulls at all?

* Update app/vmselect/prometheus/export.qtpl

* app/vmselect/prometheus/export.qtpl.go: `make quicktemplate-gen`

* docs/CHANGELOG.md: document the change

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2022-04-29 12:49:47 +03:00
Nikolay
c7aad8d441
app/vmselect: adds API /api/v1/status/buildinfo (#2515) 
* app/vmselect: adds API /api/v1/status/buildinfo
it should fix an compability error with grafana 8.5 prometheus datasource
https://github.com/grafana/grafana/pull/46771

* Update main.go
 2022-04-29 11:36:28 +03:00
Aliaksandr Valialkin
cce1b6d7f9
app/vmselect/promql: add tlast_change_over_time(m[d]) function, which returns the timestamp for the last change of m on the given lookbehind window d 2022-04-27 10:59:03 +03:00
Yury Molodov
c7693e8bc1
vmui: expression alias (#2495) 
* feat: add alias for queries

* docs: update docs for predefined dashboards

* app/vmselect: `make vmui-update`

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2022-04-26 15:59:37 +03:00
Yury Molodov
9b4bff67e0
vmui: add support relative time (#2504) 
* feat: add support relative time

* app/vmselect: `make vmui-update`

* docs/CHANGELOG.md: document the change

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2022-04-26 15:46:06 +03:00
Aliaksandr Valialkin
25fe83577d
app/vmselect/promql: properly handle scalar default vector, scalar if vector and scalar ifnot vector queries 
Previously `vector` time series could be unexpectedly returned from such queries
 2022-04-21 15:34:36 +03:00
Aliaksandr Valialkin
d1a9fac894
app/vmselect/promql: fix comparison to nan 
The comparison to nan has been broken in d335cc886c

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/150
 2022-04-21 14:55:37 +03:00
Aliaksandr Valialkin
de892239a9
app/vmselect/promql: add drop_common_labels() function 2022-04-21 14:20:20 +03:00
Aliaksandr Valialkin
7a44ba1234
app/vmselect/promql: fix q default b where b may have empty time series 2022-04-21 11:42:42 +03:00
Aliaksandr Valialkin
d335cc886c
app/vmselect/promql: fix duplicate time series error on joins against time series filtered by values 
This should prevent from `duplicate time series` errors when executing the following query:

kube_pod_container_resource_requests{resource="cpu"} * on (namespace,pod) group_left() (kube_pod_status_phase{phase=~"Pending|Running"}==1)

where `kube_pod_status_phase{phase=~"Pending|Running"}==1` filters out diplicate time series
 2022-04-20 22:18:44 +03:00
Aliaksandr Valialkin
ed97908ca9
app/vmselect/promql: rename removeNaNs() to more clear removeEmptySeries() 2022-04-20 19:53:46 +03:00
Yury Molodov
514e3660e2
fix: prevent graph hiding without data (#2456) 
* fix: prevent graph hiding without data

* fix: add yaxis labels default

* app/vmselect: `make vmui-update`

* docs/CHANGELOG.md: document the change

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2022-04-16 17:15:31 +03:00
Aliaksandr Valialkin
ef66b048c9
app/vmui: further improvements for number display on graphs 
This is a follow-up for c4d2cd8336

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2409
 2022-04-12 16:01:27 +03:00
Aliaksandr Valialkin
7f83dc06c4
app/vmselect: make vmui-update 2022-04-12 14:35:19 +03:00
Aliaksandr Valialkin
61c7f6beae
app/vmselect/promql: allow calling InitRollupResultCache+StopRollupResultCache multiple times during tests 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2406
 2022-04-11 12:34:43 +03:00
Aliaksandr Valialkin
19ecc4b2c3
app/vmselect: make vmui-update 2022-04-01 12:55:21 +03:00
Yury Molodov
f166f80f15
vmui: grid support for predefined panels (#2386) 
* update packages

* feat: add setting width for predefined panels

* docs: update doc by predefined dashboards

* app/vmselect: `make vmui-update`

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2022-04-01 12:48:17 +03:00
Yury Molodov
c8d29ed78e
vmui: predefined panels (#2243) 
* feat: add basic components for predefined dashboards

* fix: change display alert

* feat: add autosize and unit for axes

* feat: add component for CircularProgress

* feat: change layout for predefined dashboards

* feat: add override step for predefined panels

* feat: add override step for predefined panels

* feat: change yaxis limits for predefined panels

* fix: rename flag for hide legend

* feat: add formatted panel description

* feat: add README.md for dashboard setup

* feat: validate dashboard settings

* feat: add unit for y-ticks

* fix: correct display error for dashboards

* fix: disable auto refresh after route change

* update package-lock.json

* fix: add basename for BrowserRouter

* fix: add dynamic basename for routing

* update packages

* feat: add a pre-defined dashboard "per-job resource usage"

* feat: display unit in the hover-tooltip

* fix: change routing and home layout

* fix: change axis width calc

* updated packages

* app/vmselect: `make vmui-update`

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2022-03-26 13:03:11 +02:00
Aliaksandr Valialkin
6e364e19ef
app/vmselect: add fine-grained limits for the number of returned/scanned time series for various APIs 2022-03-26 11:29:49 +02:00
Aliaksandr Valialkin
e78f3ac8ac
app/vmselect/bufferedwriter: suppress trivial network errors, which can be generated by remote side 
These errors include `broken pipe` and `reset by peer`.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2334
 2022-03-18 19:28:02 +02:00
Aliaksandr Valialkin
5d9b9b88b9
app/vmselect/vmui: make vmui-update 2022-03-18 13:18:26 +02:00
Yury Molodov
227d5182af
vmui: update packages (#2264) 
* update package.json

* update package-lock.json

* app/vmselect: `make vmui-update`

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2022-03-03 18:49:13 +02:00
Aliaksandr Valialkin
0d47c23a03
app/vmselect/promql: reduce the maximum number of label values, which can be propagated from one side of the binary operation to another side of the binary operation from 10K to 1K 
There are user reports that 10K unique values in a single label filter may lead to performance and memory usage issues
 2022-02-24 04:05:18 +02:00
Aliaksandr Valialkin
62b46007c5
lib/workingsetcache: reduce the default cache rotation period from hour to 20 minutes 
This should reduce memory usage under high time series churn rate
 2022-02-23 13:41:45 +02:00
Aliaksandr Valialkin
205d34eae6
app/vmselect: make vmui-update 2022-02-22 21:02:37 +02:00
Yury Molodov
5808774e06
vmui: change logic autocomplete (#2196) 
* fix: change autocomplete display logic

* fix: change z-index for labels of input fields

* fix: change autocomplete display logic

* wip

* wip

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2022-02-21 15:43:55 +02:00
Aliaksandr Valialkin
1d7c877b7b
app/vmui: small fixes 
* Remove unneeded dependency on `numeral` package
* Properly parse numbers obtained from /api/v1/query_range according to
  https://prometheus.io/docs/prometheus/latest/querying/api/#expression-query-result-formats
* Optimize updating processing the received data from /api/v1/query_range
* Make smoother zoom on `ctrl+scroll`
* Reduce the number of points received from /api/v1/query_range by 2x in order to reduce load on backend
 2022-02-14 16:25:46 +02:00
Aliaksandr Valialkin
b1f94f7f0e
app/vmselect/promql: return at most one time series from absent_over_time() in the same way as Prometheus does 
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2130
 2022-02-12 15:45:09 +02:00
Roman Khavronenko
e3adcbec6e
lib/promscrape: support prometheus-like duration in scrape configs (#2169) 
* lib/promscrape: support prometheus-like duration in scrape configs

The change allows to specify duration values like `1d`, `1w`
for fields `scrape_interval`, `scrape_timeout`, etc.

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/817#issuecomment-1033384766
Signed-off-by: hagen1778 <roman@victoriametrics.com>

* lib/blockcache: make linter happy

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* lib/promscrape: support prometheus-like duration in scrape configs

* add support for extra fields `scrape_align_interval` and `scrape_offset`;
* support Prometheus duration parsing for `__scrape_interval__`
and `__scrape_duration__` labels;

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* wip

* wip

* docs/CHANGELOG.md: document the feature

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2022-02-11 16:17:00 +02:00
Aliaksandr Valialkin
af77f449da
app/vmselect/vmui: make vmui-update 2022-02-08 17:26:50 +02:00
Aliaksandr Valialkin
96b7de6736
app/vmselect/promql: clarify comments on why the right side of if and and operators are executed at first 2022-02-03 00:26:14 +02:00
Aliaksandr Valialkin
4b850c2a59
app/vmselect/promql: do not push down filters, which enumerate more than 10k unique values 
Such filters may slow down time series search, so just skip them.

This is a follow-up for e7f1ceeb84

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1827
 2022-02-02 23:40:02 +02:00
Aliaksandr Valialkin
2016a2c899
app/vmselect/promql: properly handle foo or bar queries 
Such queries may miss `bar` results after the commit e7f1ceeb84
because common label filters from `foo` could be mistakenly applied to `bar`.
 2022-02-01 17:40:51 +02:00
Aliaksandr Valialkin
4bdd10ab90
lib/bytesutil: split Resize* funcs to MayOverallocate and NoOverallocate for more fine-grained control over memory allocations 
Follow-up for f4989edd96

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2007
 2022-02-01 00:18:42 +02:00
Aliaksandr Valialkin
a8d22e1223
app/vmselect/promql: check for binary operation in case-insensitive manner when deciding which side of the operation to perform the first 
PromQL and MetricsQL operators are case-insensitive
 2022-01-31 22:06:15 +02:00
Aliaksandr Valialkin
e7f1ceeb84
app/vmselect/promql: optimize queries, which join on _info metrics. 
Automatically add common filters from one side of binary operation
to the other side before sending the query to storage subsystem.

See https://grafana.com/blog/2021/08/04/how-to-use-promql-joins-for-more-effective-queries-of-prometheus-metrics-at-scale/
and https://www.robustperception.io/exposing-the-software-version-to-prometheus
 2022-01-31 19:32:36 +02:00
Aliaksandr Valialkin
7edf8be3bc
app/vmselect/vmui: make vmui-update 2022-01-31 12:36:59 +02:00
Aliaksandr Valialkin
74a2297dcc
app/vmselect/vmui: make vmui-update after e3995572bb 2022-01-25 17:51:46 +02:00
Aliaksandr Valialkin
f4989edd96
lib/bytesutil: split Resize() into ResizeNoCopy() and ResizeWithCopy() functions 
Previously bytesutil.Resize() was copying the original byte slice contents to a newly allocated slice.
This wasted CPU cycles and memory bandwidth in some places, where the original slice contents wasn't needed
after slize resizing. Switch such places to bytesutil.ResizeNoCopy().

Rename the original bytesutil.Resize() function to bytesutil.ResizeWithCopy() for the sake of improved readability.

Additionally, allocate new slice with `make()` instead of `append()`. This guarantees that the capacity of the allocated slice
exactly matches the requested size. The `append()` could return a slice with bigger capacity as an optimization for further `append()` calls.
This could result in excess memory usage when the returned byte slice was cached (for instance, in lib/blockcache).

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2007
 2022-01-25 15:24:44 +02:00
Aliaksandr Valialkin
285bb2bbec
app/vmselect/vmui: make vmui-update 2022-01-24 12:19:49 +02:00
Yury Molodov
ad5059f2d3
vmui: fixed display type switching (#2088) 
* fix: correct switch display type

* docs/CHANGELOG.md: document the bugfix

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2022-01-21 16:56:22 +02:00
Yury Molodov
adbb821eac
vmui: fix time range selector (#2085) 
* fix: add date validate for time range

* app/vmselect/vmui: `make vmui-update`

* docs/CHANGELOG.md: document the bugfix

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2022-01-21 12:02:38 +02:00
Aliaksandr Valialkin
d79a915583
app/vmui: fix last 6 months time range picker 
Previously it was incorrectly selecting 6 minutes instead of 6 months.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1960
 2022-01-18 23:28:10 +02:00
Aliaksandr Valialkin
672fcba223
app/vmui: properly calculate graph range for y axis 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2037
 2022-01-18 22:22:24 +02:00
Aliaksandr Valialkin
5a77c86e97
app/vmui: reduce the refresh interval during graph scrolling/zooming from 1 second to 300 milliseconds 
One second feels too laggy, so let's reduce the refresh interval to 300 milliseconds.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2064
 2022-01-18 21:48:12 +02:00
Yury Molodov
8bdc45ba00
fix: remove buffer period (#2078) 
* fix: remove buffer period

* app/vmselect/vmui: `make vmui-update`

* docs/CHANGELOG.md: document the implemented feature

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2064

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2022-01-18 21:42:56 +02:00
Yury Molodov
70737ea4ac
vmui: correct url encoding (#2067) 
* fix: correct encode multi-line queries

* fix: change autocomplete for correct arrows work

* app/vmselect/vmui: `make vmui-update`

* docs/CHANGELOG.md: document the bugfix for https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2039

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2022-01-18 21:31:46 +02:00
Yury Molodov
8e3f9c1fbb
vmui: correct calc axes limits (#2058) 
* fix: correct calc axes limits

* app/vmselect/vmui: `make vmui-update`

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2022-01-18 15:43:41 +02:00
Yury Molodov
fcd33fc409
vmui: change layout (#2054) 
* fix: change query reset

* feat: replace @codemirror to text field

* feat: switch to Preact from React

* fix: optimize mui imports

* feat: move time selector to Header

* checkout

* fix: remove unused vars

* update package-lock.json

* fix: correct styles

* app/vmselect/vmui: `make vmui-update`

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2022-01-18 12:44:22 +02:00
Aliaksandr Valialkin
dc7b63a793
app/vmselect/promql: properly keep metric names when optimized path is used for aggregate function calculations 
For example, `sum(rate(...) keep_metric_names) by (__name__)` didn't leave the original metric name because of this issue.

This is a follup-up for 1bdc71d917

Udates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/949
 2022-01-17 15:30:30 +02:00
Aliaksandr Valialkin
1bdc71d917
app/vmselect/promql: implement keep_metric_names modifier for transform and rollup functions 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/949
 2022-01-14 04:14:59 +02:00
Aliaksandr Valialkin
f41846d002
app/vmselect/promql: add stale_samples_over_time() function 2022-01-14 01:48:04 +02:00
Aliaksandr Valialkin
c883c15878
app/vmselect/promql: add support for @ modifier 
Add support for `@` modifier in MetricsQL according to https://prometheus.io/docs/prometheus/latest/querying/basics/#modifier

Extend the support with the following features:
* Allow using `@` modifier everywhere in the query. For example, `sum(foo) @ end()`
* Allow using arbitrary expression as `@` modifier. For example, `foo @ (end() - 1h)`
  returns `foo` value at `end - 1 hour` timestamp on the selected time range `[start ... end]`

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1348
 2022-01-13 22:12:06 +02:00
Aliaksandr Valialkin
9469696e46
app/vmselect/promql: fix limit_offset() test 
The test has been broken in addae7fc6a
 2022-01-13 16:59:28 +02:00
Yury Molodov
7d5ed49d23
vmui: switching to Preact (#2053) 
* feat: replace @codemirror to text field

* feat: switch to Preact from React

* fix: optimize mui imports

* fix: remove unused vars

* update package-lock.json

* app/vmselect/vmui: `make vmui-update`

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2022-01-11 10:32:17 +02:00
Aliaksandr Valialkin
ceade70d4e
app/vmselect/vmui: run make vmui-update after 89ff7b2465 2022-01-03 21:03:37 +02:00
Yury Molodov
681a800086
vmui: legend fixes (#1995) 
* feat: add a reset query by clicking the logo

* feat: add sequence number for query fields

* feat: invert behavior on the graph's legend

* app/vmselect/vmui: `make vmui-update`

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2021-12-23 12:14:16 +02:00
Roman Khavronenko
543bd0ea0c
vmselect: update /query_exemplars placeholder (#2000) 
Grafana expects `data` in response to be a slice and logs an err
if it is not:
```
err="[]v1.ExemplarQueryResult: decode slice: expect [ or n, but found , error found in #0 byte of ...||..., bigger context ...||..."
```

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1999
Signed-off-by: hagen1778 <roman@victoriametrics.com>
 2021-12-23 11:53:50 +02:00