github-mirrors/VictoriaMetrics
1
0
Fork 0
mirror of https://github.com/VictoriaMetrics/VictoriaMetrics.git synced 2024-11-21 14:44:00 +00:00
Code Issues Projects Releases Packages Wiki Activity
5206 commits 150 branches 748 tags 347 MiB
Commit graph

871 commits

Author SHA1 Message Date
Dmytro Kozlov
6734d77d97
vmselect/prometeus: Add limit param to api/v1/series api endpoint (#2851) 
* issue-2841: Add limit param to api/v1/series api endpoint

* issue-2841: add change log

* issue-2841: update logic

* issue-2841: simplify logic

* issue-2841: simplify logic, add information to documentation
 2022-07-11 20:37:20 +03:00
Aliaksandr Valialkin
ed2126aec3
app/vmselect: follow-up after 6dda254e01 2022-07-11 19:48:18 +03:00
Roman Khavronenko
6dda254e01
vmselect: cover special cases for vmalert's routing in cluster version (#2844) 
* vmselect: cover special cases for vmalert's routing

* remove trailing `/` from requests
* redirect to vmalert's home page when `/vmalert` is requested.

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* vmselect: fix review comments

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* Update app/vmselect/main.go

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2022-07-11 19:33:33 +03:00
Aliaksandr Valialkin
eab8ebbe11
all: make fmt via the upcoming Go1.19 2022-07-11 19:23:25 +03:00
Aliaksandr Valialkin
c0af52228a
app/vmselect/netstorage: add benchmarks for mergeSortBlocks 
This is a follow-up for 743ff84863
 2022-07-11 12:53:46 +03:00
Aliaksandr Valialkin
d442ee4610
app/vmselect/netstorage: optimize mergeSortBlocks function 
- Use binary search instead of linear scan when locating the run of smallest timestamps
  in blocks with intersected time ranges. This should improve performance
  when merging blocks with big number of samples

- Skip samples with duplicate timestamps. This should increase query performance
  in cluster version of VictoriaMetrics with the enabled replication.
 2022-07-09 00:35:38 +03:00
Roman Khavronenko
78da074ad3
metricsql: properly evaluate timezone_offset over time interval (#2842) 
* metricsql: properly evaluate `timezone_offset` over time interval

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2771
Signed-off-by: hagen1778 <roman@victoriametrics.com>

* Update docs/CHANGELOG.md

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2022-07-08 14:04:40 +03:00
Aliaksandr Valialkin
decbf84e55
app/vmselect/vmui: follow-up after 0bf6841140 
* Document the bugfix in docs/CHANGELOG.md
* Run `make vmui-update` for updating static js files for vmui, which are included into vmselect
 2022-07-08 13:22:57 +03:00
Aliaksandr Valialkin
6bb4bcd30c
app/vmui: tune visual presentation of trace view 2022-07-06 14:16:24 +03:00
Aliaksandr Valialkin
eee384c9c9
app/vmselect: add -clusternative.tls* options for mTLS setup in multi-level clusters 2022-07-06 13:48:21 +03:00
Aliaksandr Valialkin
195dccf678
app/vmselect: add ability to query vmselect from another vmselect 2022-07-06 13:19:45 +03:00
Aliaksandr Valialkin
714958deab
all: follow-up after ed89106274 2022-07-06 12:48:52 +03:00
Aliaksandr Valialkin
cdd89d9cc2
app/vmselect: properly generate response for /api/v1/series 
The response has been broken in 7d5d33fd71
 2022-07-06 12:46:23 +03:00
Aliaksandr Valialkin
270e555f47
lib/vmselectapi: pass maxSuffixes arg to tagValueSuffixes RPC call 2022-07-06 12:46:22 +03:00
Aliaksandr Valialkin
f4df43f7cc
app/vmselect/netstorage: remove unused auth.Token arg 2022-07-06 12:46:21 +03:00
Aliaksandr Valialkin
78eeca6f0d
lib/vmselectapi: rename deleteMetrics to more correct deleteSeries 2022-07-06 12:46:21 +03:00
Roman Khavronenko
f51bc07d97
vmselect: allow proxying requests to vmalert (#2833) 
The change allows to proxy requests with prefix `/vmalert`
to the vmalert component if `-vmalert.proxyURL` is set.

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2825
and https://github.com/VictoriaMetrics/VictoriaMetrics/pull/2831

Signed-off-by: hagen1778 <roman@victoriametrics.com>
 2022-07-06 10:47:51 +02:00
Aliaksandr Valialkin
10c63f5853
app/vmselect: make fmt after f3ece83e67 2022-07-05 14:56:53 +03:00
Aliaksandr Valialkin
8516670582
app/vmselect/promql: properly calculate histogram_quantile over unexpected le buckets 
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2819
 2022-07-05 13:20:41 +03:00
Aliaksandr Valialkin
fa08220d27
app/vmselect/promql: properly handle partial counter resets in rate(), irate(), increase() and remove_resets() functions 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2787
 2022-06-30 22:40:26 +03:00
Aliaksandr Valialkin
4fb0f15322
all: readability improvements for query traces 
- show dates in human-readable format, e.g. 2022-05-07, instead of a numeric value
- limit the maximum length of queries and filters shown in trace messages
 2022-06-30 18:19:43 +03:00
Dmytro Kozlov
67753cc6f0
vmui: update render logic for nested component (#2795) 
* vmui: update render logic for nested component, avoid rerender, remove local storage usage for tracing flag

* docs/url-examples.md: fix various documentation issues there

* docs: add Troubleshooting doc

This doc contains troubleshooting guides for typical problems with VictoriaMetrics.

* docs/Troubleshooting.md: add troubleshooting guide for cluster instability

* wip

* wip

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2022-06-30 15:48:22 +03:00
Aliaksandr Valialkin
daefb64f38
app/vmselect: expose additional histograms at /metrics page, which may help get more insights for the query workload 
This commit is based on https://github.com/VictoriaMetrics/VictoriaMetrics/pull/2792
 2022-06-28 20:18:31 +03:00
Aliaksandr Valialkin
9adff0b686
app/vmselect/promql: show the number of scanned samples in the query trace 2022-06-28 19:32:03 +03:00
Aliaksandr Valialkin
c21f5a508f
app/vmselect/prometheus: reduce the default value for -search.maxSeries from 100k to 30k 
Production experience shows that 100k is too big for /api/v1/series .
It leads to increased CPU usage when Grafana queries /api/v1/series over VictoriaMetrics
with big number of time series during auto-completion and when modifying template variables.
 2022-06-28 18:22:24 +03:00
Aliaksandr Valialkin
7d5d33fd71
lib/storage: return marshaled metric names from SearchMetricNames 
Previously SearchMetricNames was returning unmarshaled metric names.
This wasn't great for vmstorage, which should spend additional CPU time
for marshaling the metric names before sending them to vmselect.

While at it, remove possible duplicate metric names, which could occur when
multiple samples for new time series are ingested via concurrent requests.

Also sort the metric names before returning them to the client.
This simplifies debugging of the returned metric names across repeated requests to /api/v1/series
 2022-06-28 18:16:32 +03:00
Aliaksandr Valialkin
399d4c36ae
app/vmselect: optimize /api/v1/series a bit for time ranges smaller than one day 2022-06-28 12:55:20 +03:00
Aliaksandr Valialkin
a667d339be
app/vmselect/netstorage/netstorage.go: group metrics in order to improve readability a bit 2022-06-27 14:00:24 +03:00
Aliaksandr Valialkin
6386f117c8
all: show timeRange in traces in human-readable format instead of timestamps in milliseconds 2022-06-27 13:42:57 +03:00
Aliaksandr Valialkin
08de733924
app/vmselect/netstorage: assume the response is full if up to -replicationFactor-1 vmstorage nodes are unavailable 
This is a follow-up for ee5c502446

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1767
 2022-06-27 12:21:26 +03:00
Aliaksandr Valialkin
bc9d704ef4
app/vmselect/netstorage: remove Get prefix from netstorage functions 
This makes these function names more consistent with the server side
 2022-06-27 00:37:49 +03:00
hagen1778
e40d015e9a vmselect: make vm_partial_results_total consistent 
Metrics `vm_partial_results_total` and `vm_requests_total` serving
the similar purpose, but contain inconsistent set of labels.
This change updates `vm_partial_results_total` labels to be consistent
with `vm_requests_total`.

The change breaks backward compatibility with assumption that
`vm_partial_results_total` wasn't widely used, since it is
not documented and absent in the alerts and dashboards.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
 2022-06-24 13:50:26 +02:00
Dmytro Kozlov
f28cbcc7b5
vmui: added query tracing (#2748) 
* vmui: added query tracing

* vmui: updated ui

* vmui: update tracing logic, fix bugs, disable tracing by default

* vmui: use empty message as props

* vmui: fixed ui, added delete for each tacing data, show query in header

* vmui: added timelines

* vmui: speedup render

* vmui: use memo for sorting

* vmui: use Trace model, remove unused functions, simplify part of code

* vmui: update recursive logic

* vmui: fix set query to header

* vmui: code cleanup, remove unused code

* vmui: remove unused type, rename component

* wip

* wip

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2022-06-23 23:00:45 +03:00
Nikolay
ee5c502446
app/vmselect: fixes partial response with replicationFactor (#2777) 
* app/vmselect: fixes partial response with replicationFactor
Allow partial response if it meets replicationFactor configured at vmselect
https://t.me/VictoriaMetrics_ru1/38490

* docs/CHANGELOG.md: document this change

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2022-06-23 20:17:24 +03:00
Aliaksandr Valialkin
7429dfbe9e
app/vmselect: add -search.setLookbackToStep command-line flag for making the gap filling algorithm similar to InfluxDB data model 
This option should override `-search.maxStalenessInterval` for most cases when users migrate from InfluxDB to VictoriaMetrics
 2022-06-22 14:20:02 +03:00
Aliaksandr Valialkin
4f64da874d
app/vmselect: typo fix in the exported metric name: vm_http_request_total -> vm_http_requests_total 2022-06-22 13:18:03 +03:00
Aliaksandr Valialkin
dceca7e864
all: remove explicit "xxhash" name when importing github.com/cespare/xxhash/v2 package 
This is a follow-up for fe2269b999
 2022-06-21 20:27:30 +03:00
Aliaksandr Valialkin
fe2269b999
all: remove explicit "xxhash" name when importing github.com/cespare/xxhash/v2 package 
This package already has the same name, so there is no need in explicit name
 2022-06-21 20:24:28 +03:00
Aliaksandr Valialkin
b28c6febf9
app/{vminsert,vmselect}: add -vmstorageDialTimeout command-line flag for tuning the maximum time needed for establishing connections to vmstorage 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/711
 2022-06-20 15:17:34 +03:00
Dmytro Kozlov
5bc13e2fe8
vmui: added focusLabel, enable cardinality app configuratior (#2736) 
* vmui: added focusLabel, enable app configuratior

* vmui: set focusLabel if {labelName!=""}

* wip

* docs/CHANGELOG.md: mention about focusLabel feature in cardinality explorer

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2730

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2022-06-17 13:04:09 +03:00
Roman Khavronenko
cbe39bbb8d
vmselect: limit end param max value by 2d in future (#2729) 
* vmselect: limit `end` param max value by 2d in future

The change is applied only to service handlers like `/labels` or `/series`
and limits the `end` param by max value <= now() + 2 days. The same limit
is applied for the ingested data, so no reason to allow to request data
in future far than that.

The change is also needed for corner cases like https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2669
where too high `end` value triggers inefficient global index search.

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* docs/CHANGELOG.md: document the bugfix

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2022-06-16 20:51:40 +03:00
Aliaksandr Valialkin
da1d1e83df
app/{vmselect,vmstorage}: properly pass seriesCountByLabelName and seriesCountByFocusLabelValue entries from vmstorage to vmselect 2022-06-16 10:44:29 +03:00
Aliaksandr Valialkin
ee9954082f
app/vmselect/netstorage: properly aggregate seriesCountByLabelName and seriesCountByFocusLabelValue obtained from multiple vmselect nodes at /api/v1/status/tsdb 2022-06-15 16:48:40 +03:00
Aliaksandr Valialkin
45fa9d798d
app/vmselect: accept focusLabel query arg at /api/v1/status/tsdb 2022-06-14 18:39:00 +03:00
Aliaksandr Valialkin
fb77843639
lib/storage: show top labels with the highest number of series in cardinality explorer 2022-06-14 16:34:13 +03:00
Dmytro Kozlov
ed9b4c6f6d
vmui: refactor Cardinality panel (#2726) 
* vmui: refactor Cardinality panel

* vmui: change width of the search panel

* vmui: code cleanup

* vmui: code cleanup

* vmui: fixed vulnerability (npm audit fix)

* wip

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2022-06-14 14:40:44 +03:00
Yury Molodov
65107a4585
vmui: enhancements (#2638) (#2717) 
* feat: make datepicker to be set to last 30 min by default

* fix: correct spinner while loading data

* feat: change legend style

* app/vmselect: `make vmui-update`

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2022-06-13 09:49:58 +03:00
Aliaksandr Valialkin
61e03f172b
app/vmselect: optimize /api/v1/labels and /api/v1/label/.../values handlers when match[] query arg is passed to them 2022-06-12 14:06:24 +03:00
Aliaksandr Valialkin
4a94cd81ce
app/vmselect: add optional limit query arg to /api/v1/labels and /api/v1/label_values endpoints 
This arg allows limiting the number of sample values returned from these APIs
 2022-06-10 10:24:07 +03:00
Aliaksandr Valialkin
1335698ba7
app/vmselect/prometheus: extract common code for obtaining common query args into getCommonParams() function 2022-06-09 20:33:36 +03:00
Aliaksandr Valialkin
cb39eada77
all: improve query tracing coverage for indexdb search 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1403
 2022-06-09 20:04:02 +03:00
Aliaksandr Valialkin
ad44eadd94
app/vmselect/graphite: remove additional redundant Request.ParseForm() calls after 38c785b851 2022-06-09 13:22:23 +03:00
Aliaksandr Valialkin
7fe60dad10
app/vmselect: remove redundant calls to Request.ParseForm() 
Request.ParseForm() is implicitly called by the first call to Request.FormValue()
 2022-06-09 13:09:52 +03:00
Aliaksandr Valialkin
a9ea3fee38
lib/querytracer: make it easier to use by passing trace context message to New and NewChild 
The context message can be extended by calling Donef.
If there is no need to extend the message, then just call Done.
 2022-06-08 21:16:12 +03:00
Aliaksandr Valialkin
2b343d8bd0
app: properly collect and merge /api/v1/status/tsdb info from vmstorage nodes 
The collection has been broken in f2754c3e90

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2233
 2022-06-08 19:26:09 +03:00
Dmytro Kozlov
f2754c3e90
Cardinality explorer (#2625) 
* Cardinality explorer

* vmui, vmselect: updated field name, added description to spinner

* make vmui-update

* updated const name, make vmui-update

* lib/storage: changes calculation for totalSeries values

* added static files

* wip

* wip

* wip

* wip

* docs/CHANGELOG.md: document cardinality explorer feature

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2233

Co-authored-by: f41gh7 <nik@victoriametrics.com>
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2022-06-08 18:54:27 +03:00
Aliaksandr Valialkin
68b6ddfb14
all: follow-up after 8edb390e21 
- Remove unused js bloatware from /targets page. This strips down binary size by more than 100Kb
- Add /service-discovery page for API compatibility with Prometheus
- Properly load bootstrap.min.css from /prometheus/targets
- Serve static contents for /targets page from app/vminsert instead of app/vmselect, because /targets page is served from there
 2022-06-07 01:05:53 +03:00
Dmytro Kozlov
ce8aade80e
lib/promscrape: adds service discovery visualization for /targets page(#2675) 
* lib/promscrape: updated template

* lib/promscrape: fixed click on unhealthy and all btns

* app/vmselect: jquery scripts into static folder

Co-authored-by: f41gh7 <nik@victoriametrics.com>
 2022-06-04 01:11:23 +03:00
Aliaksandr Valialkin
c92bc5394f
app/vmselect/netstorage: properly read trace from vmstorage when it returns error message to vmselect 2022-06-01 14:35:00 +03:00
Aliaksandr Valialkin
afced37c0b
all: add initial support for query tracing 
See https://docs.victoriametrics.com/Single-server-VictoriaMetrics.html#query-tracing

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1403
 2022-06-01 02:31:44 +03:00
Aliaksandr Valialkin
f0ab5ca61f
app/vmselect/vmui: make vmui-update after 492a615a88 2022-05-26 09:43:30 +03:00
Yury Molodov
b9ef9ddbfd
vmui: import dashboards (#2642) 
* fix: switch dashboards import to fetch

* make vmui-update
 2022-05-26 09:43:29 +03:00
Aliaksandr Valialkin
4409b9f376
app/vmselect/vmui: make vmui-update 2022-05-20 14:56:25 +03:00
Yury Molodov
d13437afd0
vmui: setup predefined dashboards without build (#2541) 
vmui: support predefined dashboards in json format

See https://github.com/VictoriaMetrics/VictoriaMetrics/tree/master/app/vmselect/vmui/dashboards
 2022-05-20 13:46:19 +03:00
Aliaksandr Valialkin
68bfa05437
app/vmselect/promql: do not return values from label_value() if the original time series has no values at the selected timestamps 2022-05-09 17:58:39 +03:00
Aliaksandr Valialkin
8babb4aebc
app/vmselect/vmui: make vmui-update after 450d879eaa 2022-05-05 22:11:41 +03:00
Aliaksandr Valialkin
3fee369997
app/vmselect: follow-up after 8639e79d38 2022-05-04 23:41:00 +03:00
Nikolay
48041305a2
app/vmselect: adds proxy for rules API (#2516) 
* app/vmselect: adds proxy for rules and alerts API
It allows to visualization for rules at grafana
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1739

* Update app/vmselect/main.go

Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
 2022-05-04 10:12:15 +03:00
Dima Lazerka
b5fce4190c
Code cleanup (#343) 
* Small code cleanup: remove Request from params

* Extract common params to all export handlers

* Renamed ExportParams -> exportParams

* wip

Co-authored-by: Dzmitry Lazerka <dlazerka@gmail.com>
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2022-05-03 15:53:58 +03:00
Aliaksandr Valialkin
4fa3cd701c
docs/Cluster-VictoriaMetrics.md: typo fix: serparated -> separated 2022-05-03 15:20:11 +03:00
Aliaksandr Valialkin
75bd7f1539
app/vmui: fix up/down arrow keys on multi-line query after a186434b50 2022-05-03 10:51:08 +03:00
Aliaksandr Valialkin
361b08c30e
lib/storage: leave the last sample per each discrete interval during the deduplicaton 
This aligns better with staleness logic in Prometheus - https://prometheus.io/docs/prometheus/latest/querying/basics/#staleness
 2022-05-02 21:59:31 +03:00
Aliaksandr Valialkin
7ca32c21c8
app/vmui: execute query by pressing enter in the same way as Prometheus does 
Multi-line query can be entered via `shift-enter` in the query input field
 2022-05-02 20:24:43 +03:00
Yury Molodov
6292185abf
vmui: support node v.18 (#2529) 
* fix: add support vmui node18

* fix: remove @mui/styles (legacy styling solution)

* Update app/vmui/Dockerfile-build

* app/vmselect/vmui: `make vmui-update`

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2022-05-02 20:24:42 +03:00
Nikolay
022b2875ad
app/vmselect: adds API /api/v1/status/buildinfo (#2515) 
* app/vmselect: adds API /api/v1/status/buildinfo
it should fix an compability error with grafana 8.5 prometheus datasource
https://github.com/grafana/grafana/pull/46771

* Update main.go
 2022-04-29 13:02:43 +03:00
Aliaksandr Valialkin
90f9ed260e
app/vmselect/vmui: make vmui-update after da04e9d1de 2022-04-29 13:02:43 +03:00
Dima Lazerka
e6ee235707
Export "null" in jsonl instead of NaN (#2518) 
* Export "null" in jsonl instead of NaN

The NaN appeared because of staleness markers that were added for compatibility. I think it's better to use json `null`, implemented here.

Also maybe it also makes sense to add a flag like `?skip-staleness-markers=true` to `/export`, to skip nulls at all?

* Update app/vmselect/prometheus/export.qtpl

* app/vmselect/prometheus/export.qtpl.go: `make quicktemplate-gen`

* docs/CHANGELOG.md: document the change

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2022-04-29 12:51:03 +03:00
Yury Molodov
dac203320a
vmui: expression alias (#2495) 
* feat: add alias for queries

* docs: update docs for predefined dashboards

* app/vmselect: `make vmui-update`

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2022-04-27 10:59:51 +03:00
Aliaksandr Valialkin
6eb1580158
app/vmselect/promql: add tlast_change_over_time(m[d]) function, which returns the timestamp for the last change of m on the given lookbehind window d 2022-04-27 10:58:40 +03:00
Yury Molodov
eae6f68be2
vmui: add support relative time (#2504) 
* feat: add support relative time

* app/vmselect: `make vmui-update`

* docs/CHANGELOG.md: document the change

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2022-04-26 15:46:47 +03:00
Aliaksandr Valialkin
dac24aa342
app/vmselect/promql: properly handle scalar default vector, scalar if vector and scalar ifnot vector queries 
Previously `vector` time series could be unexpectedly returned from such queries
 2022-04-21 15:34:14 +03:00
Aliaksandr Valialkin
bc54ae9608
app/vmselect/promql: fix comparison to nan 
The comparison to nan has been broken in d335cc886c

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/150
 2022-04-21 14:56:41 +03:00
Aliaksandr Valialkin
a5cfe5d13e
app/vmselect/promql: add drop_common_labels() function 2022-04-21 14:20:36 +03:00
Aliaksandr Valialkin
7ca08945fd
app/vmselect/promql: fix q default b where b may have empty time series 2022-04-21 11:43:09 +03:00
Aliaksandr Valialkin
25b841c6ed
app/vmselect/promql: fix duplicate time series error on joins against time series filtered by values 
This should prevent from `duplicate time series` errors when executing the following query:

kube_pod_container_resource_requests{resource="cpu"} * on (namespace,pod) group_left() (kube_pod_status_phase{phase=~"Pending|Running"}==1)

where `kube_pod_status_phase{phase=~"Pending|Running"}==1` filters out diplicate time series
 2022-04-20 22:21:20 +03:00
Aliaksandr Valialkin
0ef7a05fc0
app/vmselect/promql: rename removeNaNs() to more clear removeEmptySeries() 2022-04-20 19:53:24 +03:00
Yury Molodov
60c4e0022a
fix: prevent graph hiding without data (#2456) 
* fix: prevent graph hiding without data

* fix: add yaxis labels default

* app/vmselect: `make vmui-update`

* docs/CHANGELOG.md: document the change

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2022-04-16 17:16:17 +03:00
Aliaksandr Valialkin
3e2a4c07cd
app/vmui: further improvements for number display on graphs 
This is a follow-up for c4d2cd8336

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2409
 2022-04-12 16:02:37 +03:00
Aliaksandr Valialkin
4197c61ef8
app/vmselect: make vmui-update 2022-04-12 14:35:37 +03:00
Aliaksandr Valialkin
3605e1743b
app/vmselect/promql: allow calling InitRollupResultCache+StopRollupResultCache multiple times during tests 2022-04-11 12:32:23 +03:00
Aliaksandr Valialkin
a4a15a462b
app/vmselect/netstorage: bump RPC API versions for vmselect->vmstorage communications 
This is a follow-up after b843f0e229
 2022-04-08 12:36:04 +03:00
Aliaksandr Valialkin
feff0ef286
app/vmselect: make vmui-update 2022-04-01 12:55:53 +03:00
Yury Molodov
b2f0acaca4
vmui: grid support for predefined panels (#2386) 
* update packages

* feat: add setting width for predefined panels

* docs: update doc by predefined dashboards

* app/vmselect: `make vmui-update`

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2022-04-01 12:49:24 +03:00
Yury Molodov
7cbf19812d
vmui: predefined panels (#2243) 
* feat: add basic components for predefined dashboards

* fix: change display alert

* feat: add autosize and unit for axes

* feat: add component for CircularProgress

* feat: change layout for predefined dashboards

* feat: add override step for predefined panels

* feat: add override step for predefined panels

* feat: change yaxis limits for predefined panels

* fix: rename flag for hide legend

* feat: add formatted panel description

* feat: add README.md for dashboard setup

* feat: validate dashboard settings

* feat: add unit for y-ticks

* fix: correct display error for dashboards

* fix: disable auto refresh after route change

* update package-lock.json

* fix: add basename for BrowserRouter

* fix: add dynamic basename for routing

* update packages

* feat: add a pre-defined dashboard "per-job resource usage"

* feat: display unit in the hover-tooltip

* fix: change routing and home layout

* fix: change axis width calc

* updated packages

* app/vmselect: `make vmui-update`

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2022-03-26 13:05:21 +02:00
Aliaksandr Valialkin
b843f0e229
app/vmselect: add fine-grained limits for the number of returned/scanned time series for various APIs 2022-03-26 11:28:14 +02:00
Aliaksandr Valialkin
84b3234f3d
docs/CHANGELOG.md: document a1e17e91f8 2022-03-21 15:37:03 +02:00
Aliaksandr Valialkin
85695faa94
app/vmselect/bufferedwriter: suppress trivial network errors, which can be generated by remote side 
These errors include `broken pipe` and `reset by peer`.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2334
 2022-03-18 19:27:33 +02:00
Aliaksandr Valialkin
95da5e52c4
app/vmselect/vmui: make vmui-update 2022-03-18 13:18:57 +02:00
Yury Molodov
235ef947ed
vmui: update packages (#2264) 
* update package.json

* update package-lock.json

* app/vmselect: `make vmui-update`

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2022-03-03 18:50:44 +02:00
Aliaksandr Valialkin
99ee1c13c0
app/vmselect/promql: reduce the maximum number of label values, which can be propagated from one side of the binary operation to another side of the binary operation from 10K to 1K 
There are user reports that 10K unique values in a single label filter may lead to performance and memory usage issues
 2022-02-24 04:03:28 +02:00
Aliaksandr Valialkin
244c23ea2c
lib/workingsetcache: reduce the default cache rotation period from hour to 20 minutes 
This should reduce memory usage under high time series churn rate
 2022-02-23 13:42:27 +02:00
Aliaksandr Valialkin
701eb0495b
app/vmselect: make vmui-update 2022-02-22 21:02:20 +02:00
Aliaksandr Valialkin
89ead3daca
app/vmselect/netstorage: report vmstorage errors to vmselect clients even if partial responses are allowed 
If a vmstorage is reachable and returns an application-level error to vmselect,
then such error must be returned to the caller even if partial responses are allowed,
since it usually means cluster mis-configuration.

Partial responses may be returned only if some vmstorage nodes are temporarily unavailable.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1941
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/678
 2022-02-21 21:17:05 +02:00
Yury Molodov
cdd632985f
vmui: change logic autocomplete (#2196) 
* fix: change autocomplete display logic

* fix: change z-index for labels of input fields

* fix: change autocomplete display logic

* wip

* wip

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2022-02-21 15:44:54 +02:00
Aliaksandr Valialkin
c723ac6a8f
app/vmui: small fixes 
* Remove unneeded dependency on `numeral` package
* Properly parse numbers obtained from /api/v1/query_range according to
  https://prometheus.io/docs/prometheus/latest/querying/api/#expression-query-result-formats
* Optimize updating processing the received data from /api/v1/query_range
* Make smoother zoom on `ctrl+scroll`
* Reduce the number of points received from /api/v1/query_range by 2x in order to reduce load on backend
 2022-02-14 16:31:31 +02:00
Aliaksandr Valialkin
989668beba
app/vmselect/promql: return at most one time series from absent_over_time() in the same way as Prometheus does 
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2130
 2022-02-12 15:46:43 +02:00
Roman Khavronenko
791cad8c2e
lib/promscrape: support prometheus-like duration in scrape configs (#2169) 
* lib/promscrape: support prometheus-like duration in scrape configs

The change allows to specify duration values like `1d`, `1w`
for fields `scrape_interval`, `scrape_timeout`, etc.

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/817#issuecomment-1033384766
Signed-off-by: hagen1778 <roman@victoriametrics.com>

* lib/blockcache: make linter happy

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* lib/promscrape: support prometheus-like duration in scrape configs

* add support for extra fields `scrape_align_interval` and `scrape_offset`;
* support Prometheus duration parsing for `__scrape_interval__`
and `__scrape_duration__` labels;

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* wip

* wip

* docs/CHANGELOG.md: document the feature

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2022-02-11 16:17:51 +02:00
Aliaksandr Valialkin
d1c17fe385
app/vmselect/vmui: make vmui-update 2022-02-08 17:29:35 +02:00
Aliaksandr Valialkin
1173964d8d
app/vmselect/promql: clarify comments on why the right side of if and and operators are executed at first 2022-02-03 00:26:42 +02:00
Aliaksandr Valialkin
10367d7e4c
app/vmselect/promql: do not push down filters, which enumerate more than 10k unique values 
Such filters may slow down time series search, so just skip them.

This is a follow-up for e7f1ceeb84

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1827
 2022-02-02 23:42:25 +02:00
Aliaksandr Valialkin
5f266370c5
all: follow-up after 4bdd10ab90 
Properly use new bytesutil.Resize* functions
 2022-02-01 17:49:28 +02:00
Aliaksandr Valialkin
13c2692ca1
app/vmselect/promql: properly handle foo or bar queries 
Such queries may miss `bar` results after the commit e7f1ceeb84
because common label filters from `foo` could be mistakenly applied to `bar`.
 2022-02-01 17:43:10 +02:00
Aliaksandr Valialkin
02b2bfcff3
lib/bytesutil: split Resize* funcs to MayOverallocate and NoOverallocate for more fine-grained control over memory allocations 
Follow-up for f4989edd96

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2007
 2022-02-01 11:20:20 +02:00
Aliaksandr Valialkin
784cdcb080
app/vmselect/promql: check for binary operation in case-insensitive manner when deciding which side of the operation to perform the first 
PromQL and MetricsQL operators are case-insensitive
 2022-01-31 22:07:35 +02:00
Aliaksandr Valialkin
04d6596298
app/vmselect/promql: optimize queries, which join on _info metrics. 
Automatically add common filters from one side of binary operation
to the other side before sending the query to storage subsystem.

See https://grafana.com/blog/2021/08/04/how-to-use-promql-joins-for-more-effective-queries-of-prometheus-metrics-at-scale/
and https://www.robustperception.io/exposing-the-software-version-to-prometheus
 2022-01-31 20:25:15 +02:00
Aliaksandr Valialkin
d95d945046
app/vmselect/vmui: make vmui-update 2022-01-31 12:42:21 +02:00
Aliaksandr Valialkin
a3eafd2e7f
app/vmselect/vmui: make vmui-update after e3995572bb 2022-01-25 17:52:37 +02:00
Aliaksandr Valialkin
6232eaa938
lib/bytesutil: split Resize() into ResizeNoCopy() and ResizeWithCopy() functions 
Previously bytesutil.Resize() was copying the original byte slice contents to a newly allocated slice.
This wasted CPU cycles and memory bandwidth in some places, where the original slice contents wasn't needed
after slize resizing. Switch such places to bytesutil.ResizeNoCopy().

Rename the original bytesutil.Resize() function to bytesutil.ResizeWithCopy() for the sake of improved readability.

Additionally, allocate new slice with `make()` instead of `append()`. This guarantees that the capacity of the allocated slice
exactly matches the requested size. The `append()` could return a slice with bigger capacity as an optimization for further `append()` calls.
This could result in excess memory usage when the returned byte slice was cached (for instance, in lib/blockcache).

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2007
 2022-01-25 15:28:42 +02:00
Aliaksandr Valialkin
b5abea0e36
app/vmselect/vmui: make vmui-update 2022-01-24 12:25:00 +02:00
Yury Molodov
196bef8348
vmui: fixed display type switching (#2088) 
* fix: correct switch display type

* docs/CHANGELOG.md: document the bugfix

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2022-01-21 16:57:14 +02:00
Yury Molodov
034012c80f
vmui: fix time range selector (#2085) 
* fix: add date validate for time range

* app/vmselect/vmui: `make vmui-update`

* docs/CHANGELOG.md: document the bugfix

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2022-01-21 12:03:41 +02:00
Aliaksandr Valialkin
e8900c90a8
app/vmui: fix last 6 months time range picker 
Previously it was incorrectly selecting 6 minutes instead of 6 months.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1960
 2022-01-18 23:28:49 +02:00
Aliaksandr Valialkin
f4a9ae28f3
app/vmui: properly calculate graph range for y axis 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2037
 2022-01-18 22:23:40 +02:00
Aliaksandr Valialkin
f31776d0e4
app/vmui: reduce the refresh interval during graph scrolling/zooming from 1 second to 300 milliseconds 
One second feels too laggy, so let's reduce the refresh interval to 300 milliseconds.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2064
 2022-01-18 22:23:40 +02:00
Yury Molodov
1d19303f35
fix: remove buffer period (#2078) 
* fix: remove buffer period

* app/vmselect/vmui: `make vmui-update`

* docs/CHANGELOG.md: document the implemented feature

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2064

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2022-01-18 22:23:40 +02:00
Yury Molodov
5b286a4728
vmui: correct url encoding (#2067) 
* fix: correct encode multi-line queries

* fix: change autocomplete for correct arrows work

* app/vmselect/vmui: `make vmui-update`

* docs/CHANGELOG.md: document the bugfix for https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2039

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2022-01-18 21:32:30 +02:00
Yury Molodov
d4ccd48faa
vmui: correct calc axes limits (#2058) 
* fix: correct calc axes limits

* app/vmselect/vmui: `make vmui-update`

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2022-01-18 21:27:26 +02:00
Yury Molodov
e548fc9021
vmui: change layout (#2054) 
* fix: change query reset

* feat: replace @codemirror to text field

* feat: switch to Preact from React

* fix: optimize mui imports

* feat: move time selector to Header

* checkout

* fix: remove unused vars

* update package-lock.json

* fix: correct styles

* app/vmselect/vmui: `make vmui-update`

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2022-01-18 12:47:01 +02:00
Aliaksandr Valialkin
cc76174387
app/vmselect/promql: properly keep metric names when optimized path is used for aggregate function calculations 
For example, `sum(rate(...) keep_metric_names) by (__name__)` didn't leave the original metric name because of this issue.

This is a follup-up for 1bdc71d917

Udates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/949
 2022-01-17 15:33:39 +02:00
Aliaksandr Valialkin
5a1e4a140f
app/vmselect/promql: implement keep_metric_names modifier for transform and rollup functions 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/949
 2022-01-14 04:15:35 +02:00
Aliaksandr Valialkin
53c9a81f2b
app/vmselect/promql: add stale_samples_over_time() function 2022-01-14 01:49:24 +02:00
Aliaksandr Valialkin
0580a58feb
app/vmselect/promql: add support for @ modifier 
Add support for `@` modifier in MetricsQL according to https://prometheus.io/docs/prometheus/latest/querying/basics/#modifier

Extend the support with the following features:
* Allow using `@` modifier everywhere in the query. For example, `sum(foo) @ end()`
* Allow using arbitrary expression as `@` modifier. For example, `foo @ (end() - 1h)`
  returns `foo` value at `end - 1 hour` timestamp on the selected time range `[start ... end]`

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1348
 2022-01-13 22:15:19 +02:00
Aliaksandr Valialkin
7d635a2311
app/vmselect/promql: fix limit_offset() test 
The test has been broken in addae7fc6a
 2022-01-13 16:58:47 +02:00
Yury Molodov
7ee09286ef
vmui: switching to Preact (#2053) 
* feat: replace @codemirror to text field

* feat: switch to Preact from React

* fix: optimize mui imports

* fix: remove unused vars

* update package-lock.json

* app/vmselect/vmui: `make vmui-update`

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2022-01-11 10:34:01 +02:00
Aliaksandr Valialkin
cf5553d18e
app/vmselect/vmui: run make vmui-update after 89ff7b2465 2022-01-03 21:06:34 +02:00
Yury Molodov
85dc41007b
vmui: legend fixes (#1995) 
* feat: add a reset query by clicking the logo

* feat: add sequence number for query fields

* feat: invert behavior on the graph's legend

* app/vmselect/vmui: `make vmui-update`

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2021-12-23 12:15:21 +02:00
Roman Khavronenko
d1c474c5b3
vmselect: update /query_exemplars placeholder (#2000) 
Grafana expects `data` in response to be a slice and logs an err
if it is not:
```
err="[]v1.ExemplarQueryResult: decode slice: expect [ or n, but found , error found in #0 byte of ...||..., bigger context ...||..."
```

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1999
Signed-off-by: hagen1778 <roman@victoriametrics.com>
 2021-12-23 12:08:40 +02:00
Yury Molodov
0952caca24
vmui: add custom start range (#1989) 
* feat: add custom start range

* app/vmselect/vmui: `make vmui-update`

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2021-12-21 20:22:48 +02:00
Yury Molodov
4c9e563c89
vmui: graph fixes (#1982) 
* fix: remove disabling custom step when zooming

* feat: add a dynamic calc of the width of the graph

* fix: add validate y-axis limits

* fix: correct axis limits for value 0

* fix: change logic create time series

* fix: change types for tooltip

* fix: correct points on the line

* fix: change the logic for set graph width

* fix: stop checking the period when auto-refresh is enabled

* app/vmselect/vmui: `make vmui-update`

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2021-12-20 17:37:45 +02:00
Aliaksandr Valialkin
b393a16a25
app/vmselect/vmui: make vmui-update 2021-12-20 13:50:55 +02:00
Aliaksandr Valialkin
3396f08874
app/vmselect/promql: follow-up after 177e345d8a 
* Document changes_prometheus(), increase_prometheus() and delta_prometheus() functions.
* Simplify their implementation
* Mention these functions in docs/CHANGELOG.md
 2021-12-20 13:15:06 +02:00
匠心零度
cc4725843b
add Prometheus semantics function :changes_prometheus、delta_prometheus、increase_prometheus (#1972) 
Co-authored-by: lirenzuo <lirenzuo@shein.com>
 2021-12-20 13:15:05 +02:00
Aliaksandr Valialkin
c2b0a6109b
app/vmselect/prometheus: typo fix after bf0e3ad2171219b7883b149999c4dc0ac15b9797 2021-12-17 11:52:42 +02:00
Aliaksandr Valialkin
7c3239966e
app/vmselect/vmui: make vmui-update 2021-12-17 11:52:42 +02:00
Aliaksandr Valialkin
d9f075622b
app/vmselect: de-duplicate data exported via /api/v1/export/csv by default 
Previously the exported data wasn't de-duplicated.
Now it is possible to export the raw data without deduplication
by passing reduce_mem_usage=1 query arg to /api/v1/export/csv

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1837
 2021-12-17 11:52:42 +02:00
Aliaksandr Valialkin
c170841951
app/vmselect/promql: add timestamp_with_name(m[d]) function 
This function works the same as `timestamp()`, but doesn't remove source time series names.

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/949#issuecomment-995222388
 2021-12-15 23:38:09 +02:00
Yury Molodov
603be150c4
vmui: introduce application mode (#1949) 
* feat: add a label for the Query field

* fix: change zoom position

* fix: add description and error code to alerts

* fix: correct logic query history

* fix: correct update query history

* feat: add custom step

* update package-lock.json

* feat: introduce application mode

* build vmui

* Revert "build vmui"

This reverts commit c0e2415550.

* app/vmselect/vmui: `make vmui-update`

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2021-12-15 21:34:04 +02:00
Aliaksandr Valialkin
bc3923111b
lib/storage: return dedup interval in milliseconds from GetDedupInterval() 
This removes duplicate .Milliseconds() calls after GetDedupInterval() calls.
 2021-12-15 13:27:27 +02:00
Aliaksandr Valialkin
cdfe854c9b
lib/storage: explicitly pass dedupInterval to DeduplicateSamples() and deduplicateSamplesDuringMerge() 
This improves the code readability and debuggability, since the output of these functions
stops depending on global state.
 2021-12-14 20:52:29 +02:00
Yury Molodov
b3389ce26b
vmui: custom step (#1942) 
* feat: add a label for the Query field

* fix: change zoom position

* fix: add description and error code to alerts

* fix: correct logic query history

* fix: correct update query history

* feat: add custom step

* update package-lock.json

* docs: document that VMUI now supports overriding of `step` query arg, which is passed to `/api/v1/query_range`

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2021-12-14 14:52:44 +02:00