Commit graph

828 commits

Author SHA1 Message Date
Nikolay
195341a7cf Graphite vmalert wip (#112)
* init implementation for graphite alerts

* adds graphite support for vmalert

* small fix

* changes vmalert graphite api with type

* updates tests

* small fix

* fixes graphite parse

* Fixes graphite from time
2021-02-01 15:05:32 +02:00
Aliaksandr Valialkin
a4ae945a79 app/victoria-metrics: fix tests after 8749c2dd92 2021-02-01 14:34:11 +02:00
Aliaksandr Valialkin
b2aa80e74b app/vmagent: add -remoteWrite.roundDigits command-line option for limiting the number of digits after the point for stored values
This commit also adds --vm-round-digits command-line option to vmctl tool.
2021-02-01 14:27:09 +02:00
Aliaksandr Valialkin
29a7067827 app/vmctl: fix make check-all warnings 2021-02-01 01:31:25 +02:00
Aliaksandr Valialkin
d5c180e680 app/vmctl: move vmctl code from github.com/VictoriaMetrics/vmctl
It is better developing vmctl tool in VictoriaMetrics repository, so it could be released
together with the rest of vmutils tools such as vmalert, vmagent, vmbackup, vmrestore and vmauth.
2021-02-01 01:10:20 +02:00
Aliaksandr Valialkin
de3c662e8a all: consistently use timers from timerpool 2021-01-27 00:39:26 +02:00
Aliaksandr Valialkin
419ad74269 app/vmagent: add -remoteWrite.rateLimit command-line flag for limiting data rate to remote storage
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1035
2021-01-27 00:39:26 +02:00
weng zhao
cc3e69e963
vmalert: add option datasource.queryStep to allow user to address the inconsistency between grafana dashboards(query_range with step 15s usually) and ALERTS (#1027)
Co-authored-by: zhao.weng <zhao.weng@shopee.com>
2021-01-26 08:12:04 +00:00
Aliaksandr Valialkin
c164a8d231 app/vmselect/promql: improve documentation for -search.maxPointsPertimeseries command-line flag
This should reduce incorrect usage and assumptions for this flag.

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1020
2021-01-22 13:00:10 +02:00
Aliaksandr Valialkin
8749c2dd92 app/vmselect: add -search.maxStepForPointsAdjustment command-line flag, which can be used for disabling adjustment for points returned from /api/v1/query_range handler if they have timestamps closer than -search.latencyOffset to the current time 2021-01-19 22:56:32 +02:00
Aliaksandr Valialkin
011c5da785 app/vmselect/graphite: extract getCanonicalPath() function from loop body inside getCanonicalPaths() 2021-01-18 17:30:26 +02:00
Aliaksandr Valialkin
75f7c51cab docs/vmagent.md: follow-up for 184a659c5f 2021-01-13 13:53:14 +02:00
Aliaksandr Valialkin
1051d8aa2d app/vmselect/promql: add ability to pass multiple labels to sort_by_label and sort_by_label_desc functions
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/992
2021-01-13 12:44:51 +02:00
Aliaksandr Valialkin
acf1a2c72b app/vmselect/promql: properly parse escaped multibyte utf8 code sequences in metric names and labels names
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/990
2021-01-13 10:59:42 +02:00
Nikolay
dc9d7aedd5
adds extra_label to all import apis (#1007)
* adds extra_label to all import apis,
changes priority for extra_label - now it has priority over original labels

* Update README.md

Co-authored-by: Aliaksandr Valialkin <valyala@gmail.com>

* Update README.md

Co-authored-by: Aliaksandr Valialkin <valyala@gmail.com>

* adds extra labels to vmagent  import api
changes order for adding labels, now its added after user values

* adds tests for extra_label

* import fix

Co-authored-by: Aliaksandr Valialkin <valyala@gmail.com>
2021-01-13 00:52:50 +02:00
Aliaksandr Valialkin
1837f2f7d3 app/vmselect/promql: add tfirst_over_time(m[d]) and tlast_over_time(m[d]) MetricsQL functions for returning timestamps for the first and the last samples in m over d 2021-01-12 16:12:12 +02:00
Nikolay
7976c22797
Fixes error handling for promscrape.streamParse (#1009)
properly return error if client cannot read data,
properly suppress scraper errors
2021-01-12 13:31:47 +02:00
Aliaksandr Valialkin
24ffad74c1 all: use net.Dial instead of fasthttp.Dial, because fasthttp.Dial limits the number of concurrent dials to 1000 2021-01-11 12:53:30 +02:00
Roman Khavronenko
2e2e4f7e21
vmalert-989: return non-empty result in template func query stub to pass validation (#1002)
On templates validation stage vmalert does not acutally send queries, so for complex
chained expression validation may fail. To avoid this, we add a blank sample in response
so validation can pass successfully. Later, during the rule execution, stub will be replaced
with real `query` function.

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/989
2021-01-10 02:56:11 +03:00
Aliaksandr Valialkin
9dcb18e03d app/vmstorage: disable final merge by default, since it may result in high disk IO and CPU usage without measurable benefits such as increased query performance and reduced disk space usage 2021-01-08 00:16:05 +02:00
Aliaksandr Valialkin
c86286ec1d app/vmselect/promql: do not ajdust offset value provided in the query
Previously it could be modified in order to improve response cache hit ratio.
This is unneeded, since cache hit ratio should remain good because the query time range
should be already aligned to multiple of `step` values.

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/976
2020-12-27 14:09:25 +02:00
Aliaksandr Valialkin
4b7105a65b app/vmselect: sync query stats handling with cluster version 2020-12-27 13:00:29 +02:00
Aliaksandr Valialkin
df0309eae0 app/vmselect/promql: simplify defer call for querystats.RegisterQuery 2020-12-27 12:06:04 +02:00
Aliaksandr Valialkin
ad4e6a9283 app/vmselect/querystats: reduce the default number of last queries to track from 100K to 20K
This should reduce memory usage in constrained environments
2020-12-25 17:40:47 +02:00
Aliaksandr Valialkin
59183f66d0 app/vmselect: refactor /api/v1/stats/top_queries
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/907
2020-12-25 16:44:29 +02:00
Aliaksandr Valialkin
fb338c50a3 app/victoria-metrics: show usage info when incorrect command-line flag is passed to executable 2020-12-25 16:42:21 +02:00
Nikolay
86630350bf
Adds query stats handler (#945)
* Adds query stat handler,
for query and query_range api, victoriametrics tracks query execution time,
stats are expored at /api/v1/status/queries endpoint with topN param
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/907

* fixed query stats bugs

* improves queryStats tracker

* improves query stat

* small fix

* fix tests

* added more tests

* fixes 386 tests

* naming fixes

* adds drop for outdated records
2020-12-25 16:42:05 +02:00
Nikolay
1de15ad490
adds escape for CRLF (#984)
at external.alert.source - \n and \r symbols was url encoded, instead of direct usage.
replace it from "\n" to `\n`  allows to skip url encoding.
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/890
2020-12-25 11:03:13 +02:00
Aliaksandr Valialkin
0326638c90 app/vmalert: typo fix in descriptions for notifier.basicAuth.username and notifier.basicAuth.password command-line flags 2020-12-24 12:48:59 +02:00
Aliaksandr Valialkin
9df60518bb docs: mention that it is possible to set multiple -notifier.tlsInsecureSkipVerify command-line flags for vmalert
See c3a92968343c2b3619f1ab935702d0e9b3a46733
2020-12-22 22:32:13 +02:00
Nikolay
c270f8f3e6
changes vmalert notifier flag, (#978)
fixes issue with notifier insecure setting, now its possible to use multiple notifier.tlsInsecureSkipVerify multiple time.
2020-12-22 23:23:04 +03:00
Roman Khavronenko
404cbd1522
vmalert-974: fix order for labels templating (#975)
The change fixes bug caused by 3adf8c5a6f.

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/974
2020-12-19 14:10:59 +02:00
Aliaksandr Valialkin
88ac4dfc07 app/vmselect: properly parse negative combined offsets such as -1h2m3s
Previously such offsets were parsed as `-1h + 2m + 3s`. Now they are parsed as `-(1h + 2m + 3s)`.
2020-12-19 01:23:46 +02:00
Aliaksandr Valialkin
aa56b9217e app/vmagent: add vmagent_remotewrite_blocks_sent_total and vmagent_remotewrite_bytes_sent_total metrics per each -remoteWrite.url 2020-12-15 20:39:57 +02:00
Aliaksandr Valialkin
b10ad44692 docs/vmagent.md: typo fix: pearsed->parsed 2020-12-15 19:03:01 +02:00
Aliaksandr Valialkin
1eabbc0e27 docs/vmagent.md: mention that sample_limit option has no sense when stream parsing is enabled 2020-12-15 18:43:41 +02:00
Aliaksandr Valialkin
11fa458e39 app/vmselect/promql: return expected increase() result for the first point on the graph with value not exceeding 100 2020-12-15 13:40:46 +02:00
Nikolay
95c9b630cc
adds new Array Flags (#965)
* adds ArrayDuration and ArrayBool flags,
makes sendTimeout and tlsInsecure configurable per remoteWrite url

* added backward compatibility testcases for ArrayDuration and ArrayBool

* fixes bool flag

* fixes test cases
2020-12-15 12:51:12 +02:00
Roman Khavronenko
6247884057
vmalert: add function "query", "first" and "value" to alert templates functions (#960)
The commit adds a support for template function `query`,
`first` and `value`. The function `query` executes
a MetricsQL query for active alerts. In vmalert we
update templates on every evaluation for active alerts
to keep them up to date. With `query` func it may become
a perf issue since it will fire a query on every execution.
We should keep it in mind for now.

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/539
2020-12-14 20:11:45 +02:00
Aliaksandr Valialkin
f8e7f433cf app/victoria-metrics: prettify / page output 2020-12-14 14:07:58 +02:00
Aliaksandr Valialkin
069c9ade52 app/{vmagent,vminsert}: follow-up for ce8c2dd1f1: return /targets page in HTML when requested via web browser 2020-12-14 14:06:00 +02:00
Nikolay
ce8c2dd1f1
Changes targets api (#961)
* changes /targets api
adds html response if requester accepts text/html,
adds quick template for /targets api,
fixes pathPrefix for / requests

* changes namings

* renamed targets file

* Update app/victoria-metrics/main.go

Co-authored-by: Aliaksandr Valialkin <valyala@gmail.com>

* adds trimspace to qtpl,
moves content-type for targets response closer to writer

* fixes bug with prefix

Co-authored-by: Aliaksandr Valialkin <valyala@gmail.com>
2020-12-14 13:36:48 +02:00
Aliaksandr Valialkin
5ebfc275e6 app/victoria-metrics: automatically reset response cache when samples with too timestamps older than now - search.cacheTimestampOffset are ingested 2020-12-14 13:08:28 +02:00
Aliaksandr Valialkin
9d42546a27 docs: consistently use links to https://victoriametrics.github.io for documentation references 2020-12-11 21:08:18 +02:00
Aliaksandr Valialkin
ba8ac08739 app/vmselect/graphite: properly handle wildcards and charsets inside curly braces
For example, `foo{bar*,[a-f]a*b}` should match `foobar`, `foobar123`, `foofab`, etc.

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/952
2020-12-11 17:24:12 +02:00
Aliaksandr Valialkin
4146fc4668 all: properly handle CPU limits set on the host system/container
This can reduce memory usage on systems with enabled CPU limits.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/946
2020-12-08 21:07:29 +02:00
Aliaksandr Valialkin
1906f841c9 app/vmselect/promql: do not reduce lookbehind window for any_rollup_func(m) to -search.maxStalenessInterval. It should equal to step value passed to /api/v1/query_range as most users expect 2020-12-08 15:16:17 +02:00
Aliaksandr Valialkin
b6b1b06d70 app/{vmalert,vmagent}: skip empty values in -remoteWrite.label and -label lists 2020-12-08 14:55:13 +02:00
Aliaksandr Valialkin
007dbf273d app/vmselect/graphite: remove duplicate name tag from /tags/autoComplete/tags handler
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/942
2020-12-07 01:08:52 +02:00
Aliaksandr Valialkin
e2e8ef86d9 app/vmselect/promql: add count_eq_over_time(m[d], N) and count_ne_over_time(m[d], N) for calculating the number of samples in m over d that are equal / not equal to N 2020-12-05 12:30:46 +02:00