Aliaksandr Valialkin
8d021b73b5
docs/vmbackup.md: clarify docs on vmagent
clustering
2021-02-28 22:00:15 +02:00
Aliaksandr Valialkin
2b53add6b2
app/vmselect/querystats: show the number of matching queries in the top by average duration and in the top by summary duration
...
This should help debugging slow queries.
2021-02-28 19:40:19 +02:00
Aliaksandr Valialkin
e5ca8ac0db
lib/promscrape: add ability to spread scrape targets among multiple vmagent instances
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1084
2021-02-28 18:41:08 +02:00
Nikolay
317b0cbed2
adds query params for vmalert ( #1094 )
...
remoteWrite.url now accepts query params at provided url
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1087
2021-02-27 10:04:58 +00:00
Aliaksandr Valialkin
4f8a72806a
app/vmbackupmanager: add docs; the vmbackupmanager is available as a part of enterprise subscription
2021-02-27 02:17:55 +02:00
Aliaksandr Valialkin
7cc3d96a41
lib/fs: follow-up after f3a03c4164
2021-02-27 01:01:47 +02:00
Nikolay
f3a03c4164
Adds windows build ( #1040 )
...
* fixes windows compilation,
adds signal impl for windows,
adds free space usage for windows,
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/70
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1036
NOTE victoria metrics database still CANNOT work under windows system,
only vmagent is supported.
To completly port victoria metrics, you have to fix issues with separators,
parsing and posix file removall
* rollback separator
* Adds windows setInformation api,
it must behave like unix, need to test it.
changes procutil
* check for invlaid param
* Fixes posix delete semantic
* refactored a bit
* fixes openbsd build
* removed windows api call
* Fixes code after windows add
* Update lib/procutil/signal_windows.go
Co-authored-by: Aliaksandr Valialkin <valyala@gmail.com>
2021-02-27 00:37:07 +02:00
Nikolay
186c078fac
adds enforced tag filters into cache key ( #1095 )
2021-02-27 00:15:53 +02:00
Aliaksandr Valialkin
d86e9b49c4
app/vmselect/promql: increase accuracy for buckets_limit()
function for small limits by skipping the first and the last buckets during merge
...
The first and the last buckets are usually `[0 ... leMin]` and `(leMax ... +Inf)`. If they are merged with adjancent buckets,
then the resulting accuracy can suffer.
2021-02-26 22:56:36 +02:00
Aliaksandr Valialkin
98854e5f2b
app/vmselect: add sign(q)
and clamp(q, min, max)
functions, which will be added in the upcoming Prometheus release
...
See https://twitter.com/roidelapluie/status/1363428376162295811
The `last_over_time(m[d])` function already exists in MetricsQL.
2021-02-24 17:24:56 +02:00
Aliaksandr Valialkin
f4135b0d14
app/vmselect/promql: properly calculate histogram_quantile() over zero buckets and only a single non-zero
le="+Inf"` bucket like Prometheus does
2021-02-24 00:42:22 +02:00
Aliaksandr Valialkin
fa03e0d210
app/vmselect/promql: add increase_pure()
function to MetricsQL
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/962
2021-02-22 19:14:15 +02:00
Aliaksandr Valialkin
d136081040
lib/promrelabel: add more optimizations for relabeling for common cases
2021-02-22 16:33:55 +02:00
Aliaksandr Valialkin
c2678754e4
app/vmagent: properly perform graceful shutdown, which was broken in the commit 1d1ba889fe
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1065
2021-02-19 00:31:34 +02:00
Aliaksandr Valialkin
49e36e8d9d
app/vmagent: fix scrape config example for scrape_align_interval option
2021-02-18 23:57:23 +02:00
Aliaksandr Valialkin
f26162ec99
lib/promscrape: add scrape_align_interval config option into scrape config
...
This option allows aligning scrapes to a particular intervals.
2021-02-18 23:53:44 +02:00
Aliaksandr Valialkin
9c70c1f21f
app/vmselect/promql: reduce the probability of duplicate time series
errors when querying Kubernetes metrics
2021-02-18 22:07:29 +02:00
Aliaksandr Valialkin
a537c4f602
lib/storage: properly handle queries containing a filter on metric name plus any number of negative filters and zero non-negative filters
...
Example: `node_cpu_seconds_total{mode!="idle"}`
2021-02-18 18:46:36 +02:00
Aliaksandr Valialkin
03ebc028f7
app/vmalert: add missing multiarch Dockerfile
2021-02-18 15:23:17 +02:00
Aliaksandr Valialkin
d7de4807e1
app/victoria-metrics/testdata: add a test for {__graphite__="foo.*.bar"} selector
2021-02-17 21:52:43 +02:00
Aliaksandr Valialkin
edcdc39eb3
app/vmagent/remotewrite: cleanup after 1d1ba889fe
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1065
2021-02-17 21:42:55 +02:00
Nikolay
1d1ba889fe
adds pushback for fastqueue, ( #1075 )
...
during shutdown currently sending block was lost,
now its pushed back to fast queue and will be flushed on disk,
it may lead to data duplication.
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1065
2021-02-17 21:23:38 +02:00
Aliaksandr Valialkin
d4849561ef
app/vmstorage: export vm_composite_filter_success_conversions_total and vm_composite_filter_missing_conversions_total metrics
2021-02-17 19:13:38 +02:00
Aliaksandr Valialkin
7f2a6c7b54
docs: rename vmbackuper to vmbackupmanager
2021-02-16 22:00:39 +02:00
Aliaksandr Valialkin
c4756f94da
app/vmselect/netstorage: reuse timeseriesWork objects in order to reduce memory allocations
2021-02-16 16:08:53 +02:00
Aliaksandr Valialkin
5a401225c7
app/vmselect/netstorage: use unsafe string as a key for a map when the map already contains the given key
...
This should prevent from a memory allocation and a string copy.
2021-02-16 15:43:10 +02:00
Aliaksandr Valialkin
4e39bf148c
vendor: update github.com/VictoriaMetrics/metrics from v1.13.1 to v1.14.0
...
The new version switches from log-linear histograms to log-based histograms,
which provide up to 3.6 times better accuracy.
2021-02-15 15:12:29 +02:00
Aliaksandr Valialkin
802fabf0d7
deployment/docker: embed tzdata into prod Go app instead of installing it into base docker image
...
While this increases app size by 700Kb, this allows using -loggerTimezone in a scratch base image
See https://github.com/golang/go/issues/38017
2021-02-12 04:54:27 +02:00
Aliaksandr Valialkin
f9902b3372
deployment/docker: use docker buildx
for creating multiarch builds
...
See https://github.com/docker/buildx/
2021-02-12 04:31:22 +02:00
Aliaksandr Valialkin
acdb401585
app/vmselect/prometheus: treat match
query arg in the same way as match[]
query arg
2021-02-11 15:02:21 +02:00
Aliaksandr Valialkin
1e38ad6d20
app/vmauth: add ability to route requests from a single users to multiple targets depending on the requested path
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1064
2021-02-11 12:41:16 +02:00
Aliaksandr Valialkin
2d33230793
app/vmselect/promql: properly make copies of EvalConfig
2021-02-11 12:41:15 +02:00
Aliaksandr Valialkin
7a3a9421f3
app/vmselect/promql: make a copy of EvalConfig when executing q1
and q2
in parallel for q1 binary_op q2
...
This should prevent from data races if the underlying functions modify EvalConfig contents.
2021-02-10 23:05:55 +02:00
Aliaksandr Valialkin
04faea8b45
app/vmselect: parallelize q1 <binary_op> q2
queries by running q1
and q2
in parallel
...
This should reduce query execution times.
2021-02-10 22:59:39 +02:00
Aliaksandr Valialkin
cdecf83ce5
app/vmstorage: export vm_composite_index_min_timestamp metric
2021-02-10 17:14:08 +02:00
Aliaksandr Valialkin
553016ea99
lib/storage: disable composite index usage when querying old data
2021-02-10 14:57:50 +02:00
Aliaksandr Valialkin
c7dccebaef
lib/storage: optimize search by label filters matching big number of time series
2021-02-10 00:44:54 +02:00
Aliaksandr Valialkin
cb96a1865b
app/vmstorage: export missing vm_cache_size_bytes
metrics for indexdb and data caches
2021-02-09 00:47:00 +02:00
Aliaksandr Valialkin
9b9cb04511
docs/vmctl.md: fix title, so it is properly displayed in the header of https://victoriametrics.github.io/
2021-02-04 20:20:08 +02:00
Aliaksandr Valialkin
803a00102a
app/{vminsert,vmselect}: accept requests to paths with /graphite
and /prometheus
prefixes
...
This should improve compatibility with path prefixes from VictoriaMetrics cluster.
See https://victoriametrics.github.io/Cluster-VictoriaMetrics.html#url-format
2021-02-04 20:01:18 +02:00
Aliaksandr Valialkin
4d43ab0875
app/vmselect: typo fix when stripping url path prefixes
2021-02-04 19:29:25 +02:00
Roman Khavronenko
2ff038d841
vmalert: mention -datasource.appendTypePrefix
in README ( #1052 )
2021-02-03 23:44:37 +02:00
Dmitry Shevchuk
2baf98082b
Adds ability to query right vmselect endpoint based on the query type ( #1050 )
...
* Adds ability to query right vmselect endpoint based on the query type
Co-authored-by: Roman Khavronenko <hagen1778@gmail.com>
2021-02-03 21:26:30 +00:00
Aliaksandr Valialkin
a2344ef4b7
docs/vmalert.md: mention that type
option can be set at group level additionally to rule level
2021-02-03 21:13:13 +02:00
Aliaksandr Valialkin
8dc6095749
app/vmagent: add Advanced usage
section with the description for all the command-line flags
2021-02-03 21:02:59 +02:00
Aliaksandr Valialkin
8629fd8a72
app/vmselect: deprecate -search.treatDotsAsIsInRegexps
in favor to {__graphite__="foo.*.bar"}
syntax
2021-02-03 20:36:01 +02:00
Aliaksandr Valialkin
d16f22f3a1
app/vmselect,lib/storage: properly parse Graphite selectors with inner wildcards
...
Example: foo{bar{x,yz},a[b-c],*de}
2021-02-03 20:14:22 +02:00
Aliaksandr Valialkin
157c02622b
app/vmselect: add ability to set Graphite-compatible filter via {__graphite__="foo.*.bar"}
syntax
2021-02-03 01:21:54 +02:00
Aliaksandr Valialkin
b577cdd855
docs: increase heading sizes in vmagent, vmauth, vmbackup and vmrestore docs, so they match the heading sizes in VictoriaMetrics docs
2021-02-01 19:44:00 +02:00
Aliaksandr Valialkin
8164cd8932
docs/vmctl.md: update build instructions after the migration from github.com/VictoriaMetrics/vmctl to github.com/VictoriaMetrics/VictoriaMetrics/app/vmctl
2021-02-01 19:39:08 +02:00
Aliaksandr Valialkin
b43b498fd8
app/vmselect: add ability to pass extra_label=<label>=<value>
query arg to Prometheus Querying API
...
This enforced `{label="value"}` label filter to the query.
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1021
2021-02-01 18:04:17 +02:00
Aliaksandr Valialkin
5d87dbfd65
docs: document ability to query Graphite datasource from vmalert
2021-02-01 15:26:33 +02:00
Nikolay
195341a7cf
Graphite vmalert wip ( #112 )
...
* init implementation for graphite alerts
* adds graphite support for vmalert
* small fix
* changes vmalert graphite api with type
* updates tests
* small fix
* fixes graphite parse
* Fixes graphite from time
2021-02-01 15:05:32 +02:00
Aliaksandr Valialkin
a4ae945a79
app/victoria-metrics: fix tests after 8749c2dd92
2021-02-01 14:34:11 +02:00
Aliaksandr Valialkin
b2aa80e74b
app/vmagent: add -remoteWrite.roundDigits command-line option for limiting the number of digits after the point for stored values
...
This commit also adds --vm-round-digits command-line option to vmctl tool.
2021-02-01 14:27:09 +02:00
Aliaksandr Valialkin
29a7067827
app/vmctl: fix make check-all
warnings
2021-02-01 01:31:25 +02:00
Aliaksandr Valialkin
d5c180e680
app/vmctl: move vmctl code from github.com/VictoriaMetrics/vmctl
...
It is better developing vmctl tool in VictoriaMetrics repository, so it could be released
together with the rest of vmutils tools such as vmalert, vmagent, vmbackup, vmrestore and vmauth.
2021-02-01 01:10:20 +02:00
Aliaksandr Valialkin
de3c662e8a
all: consistently use timers from timerpool
2021-01-27 00:39:26 +02:00
Aliaksandr Valialkin
419ad74269
app/vmagent: add -remoteWrite.rateLimit command-line flag for limiting data rate to remote storage
...
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1035
2021-01-27 00:39:26 +02:00
weng zhao
cc3e69e963
vmalert: add option datasource.queryStep to allow user to address the inconsistency between grafana dashboards(query_range with step 15s usually) and ALERTS ( #1027 )
...
Co-authored-by: zhao.weng <zhao.weng@shopee.com>
2021-01-26 08:12:04 +00:00
Aliaksandr Valialkin
c164a8d231
app/vmselect/promql: improve documentation for -search.maxPointsPertimeseries
command-line flag
...
This should reduce incorrect usage and assumptions for this flag.
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1020
2021-01-22 13:00:10 +02:00
Aliaksandr Valialkin
8749c2dd92
app/vmselect: add -search.maxStepForPointsAdjustment
command-line flag, which can be used for disabling adjustment for points returned from /api/v1/query_range
handler if they have timestamps closer than -search.latencyOffset
to the current time
2021-01-19 22:56:32 +02:00
Aliaksandr Valialkin
011c5da785
app/vmselect/graphite: extract getCanonicalPath() function from loop body inside getCanonicalPaths()
2021-01-18 17:30:26 +02:00
Aliaksandr Valialkin
75f7c51cab
docs/vmagent.md: follow-up for 184a659c5f
2021-01-13 13:53:14 +02:00
Aliaksandr Valialkin
1051d8aa2d
app/vmselect/promql: add ability to pass multiple labels to sort_by_label
and sort_by_label_desc
functions
...
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/992
2021-01-13 12:44:51 +02:00
Aliaksandr Valialkin
acf1a2c72b
app/vmselect/promql: properly parse escaped multibyte utf8 code sequences in metric names and labels names
...
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/990
2021-01-13 10:59:42 +02:00
Nikolay
dc9d7aedd5
adds extra_label to all import apis ( #1007 )
...
* adds extra_label to all import apis,
changes priority for extra_label - now it has priority over original labels
* Update README.md
Co-authored-by: Aliaksandr Valialkin <valyala@gmail.com>
* Update README.md
Co-authored-by: Aliaksandr Valialkin <valyala@gmail.com>
* adds extra labels to vmagent import api
changes order for adding labels, now its added after user values
* adds tests for extra_label
* import fix
Co-authored-by: Aliaksandr Valialkin <valyala@gmail.com>
2021-01-13 00:52:50 +02:00
Aliaksandr Valialkin
1837f2f7d3
app/vmselect/promql: add tfirst_over_time(m[d])
and tlast_over_time(m[d])
MetricsQL functions for returning timestamps for the first and the last samples in m
over d
2021-01-12 16:12:12 +02:00
Nikolay
7976c22797
Fixes error handling for promscrape.streamParse ( #1009 )
...
properly return error if client cannot read data,
properly suppress scraper errors
2021-01-12 13:31:47 +02:00
Aliaksandr Valialkin
24ffad74c1
all: use net.Dial
instead of fasthttp.Dial
, because fasthttp.Dial
limits the number of concurrent dials to 1000
2021-01-11 12:53:30 +02:00
Roman Khavronenko
2e2e4f7e21
vmalert-989: return non-empty result in template func query
stub to pass validation ( #1002 )
...
On templates validation stage vmalert does not acutally send queries, so for complex
chained expression validation may fail. To avoid this, we add a blank sample in response
so validation can pass successfully. Later, during the rule execution, stub will be replaced
with real `query` function.
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/989
2021-01-10 02:56:11 +03:00
Aliaksandr Valialkin
9dcb18e03d
app/vmstorage: disable final merge by default, since it may result in high disk IO and CPU usage without measurable benefits such as increased query performance and reduced disk space usage
2021-01-08 00:16:05 +02:00
Aliaksandr Valialkin
c86286ec1d
app/vmselect/promql: do not ajdust offset
value provided in the query
...
Previously it could be modified in order to improve response cache hit ratio.
This is unneeded, since cache hit ratio should remain good because the query time range
should be already aligned to multiple of `step` values.
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/976
2020-12-27 14:09:25 +02:00
Aliaksandr Valialkin
4b7105a65b
app/vmselect: sync query stats handling with cluster version
2020-12-27 13:00:29 +02:00
Aliaksandr Valialkin
df0309eae0
app/vmselect/promql: simplify defer call for querystats.RegisterQuery
2020-12-27 12:06:04 +02:00
Aliaksandr Valialkin
ad4e6a9283
app/vmselect/querystats: reduce the default number of last queries to track from 100K to 20K
...
This should reduce memory usage in constrained environments
2020-12-25 17:40:47 +02:00
Aliaksandr Valialkin
59183f66d0
app/vmselect: refactor /api/v1/stats/top_queries
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/907
2020-12-25 16:44:29 +02:00
Aliaksandr Valialkin
fb338c50a3
app/victoria-metrics: show usage info when incorrect command-line flag is passed to executable
2020-12-25 16:42:21 +02:00
Nikolay
86630350bf
Adds query stats handler ( #945 )
...
* Adds query stat handler,
for query and query_range api, victoriametrics tracks query execution time,
stats are expored at /api/v1/status/queries endpoint with topN param
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/907
* fixed query stats bugs
* improves queryStats tracker
* improves query stat
* small fix
* fix tests
* added more tests
* fixes 386 tests
* naming fixes
* adds drop for outdated records
2020-12-25 16:42:05 +02:00
Nikolay
1de15ad490
adds escape for CRLF ( #984 )
...
at external.alert.source - \n and \r symbols was url encoded, instead of direct usage.
replace it from "\n" to `\n` allows to skip url encoding.
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/890
2020-12-25 11:03:13 +02:00
Aliaksandr Valialkin
0326638c90
app/vmalert: typo fix in descriptions for notifier.basicAuth.username and notifier.basicAuth.password command-line flags
2020-12-24 12:48:59 +02:00
Aliaksandr Valialkin
9df60518bb
docs: mention that it is possible to set multiple -notifier.tlsInsecureSkipVerify
command-line flags for vmalert
...
See c3a92968343c2b3619f1ab935702d0e9b3a46733
2020-12-22 22:32:13 +02:00
Nikolay
c270f8f3e6
changes vmalert notifier flag, ( #978 )
...
fixes issue with notifier insecure setting, now its possible to use multiple notifier.tlsInsecureSkipVerify multiple time.
2020-12-22 23:23:04 +03:00
Roman Khavronenko
404cbd1522
vmalert-974: fix order for labels templating ( #975 )
...
The change fixes bug caused by 3adf8c5a6f
.
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/974
2020-12-19 14:10:59 +02:00
Aliaksandr Valialkin
88ac4dfc07
app/vmselect: properly parse negative combined offsets such as -1h2m3s
...
Previously such offsets were parsed as `-1h + 2m + 3s`. Now they are parsed as `-(1h + 2m + 3s)`.
2020-12-19 01:23:46 +02:00
Aliaksandr Valialkin
aa56b9217e
app/vmagent: add vmagent_remotewrite_blocks_sent_total
and vmagent_remotewrite_bytes_sent_total
metrics per each -remoteWrite.url
2020-12-15 20:39:57 +02:00
Aliaksandr Valialkin
b10ad44692
docs/vmagent.md: typo fix: pearsed->parsed
2020-12-15 19:03:01 +02:00
Aliaksandr Valialkin
1eabbc0e27
docs/vmagent.md: mention that sample_limit
option has no sense when stream parsing is enabled
2020-12-15 18:43:41 +02:00
Aliaksandr Valialkin
11fa458e39
app/vmselect/promql: return expected increase()
result for the first point on the graph with value not exceeding 100
2020-12-15 13:40:46 +02:00
Nikolay
95c9b630cc
adds new Array Flags ( #965 )
...
* adds ArrayDuration and ArrayBool flags,
makes sendTimeout and tlsInsecure configurable per remoteWrite url
* added backward compatibility testcases for ArrayDuration and ArrayBool
* fixes bool flag
* fixes test cases
2020-12-15 12:51:12 +02:00
Roman Khavronenko
6247884057
vmalert: add function "query", "first" and "value" to alert templates functions ( #960 )
...
The commit adds a support for template function `query`,
`first` and `value`. The function `query` executes
a MetricsQL query for active alerts. In vmalert we
update templates on every evaluation for active alerts
to keep them up to date. With `query` func it may become
a perf issue since it will fire a query on every execution.
We should keep it in mind for now.
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/539
2020-12-14 20:11:45 +02:00
Aliaksandr Valialkin
f8e7f433cf
app/victoria-metrics: prettify /
page output
2020-12-14 14:07:58 +02:00
Aliaksandr Valialkin
069c9ade52
app/{vmagent,vminsert}: follow-up for ce8c2dd1f1
: return /targets
page in HTML when requested via web browser
2020-12-14 14:06:00 +02:00
Nikolay
ce8c2dd1f1
Changes targets api ( #961 )
...
* changes /targets api
adds html response if requester accepts text/html,
adds quick template for /targets api,
fixes pathPrefix for / requests
* changes namings
* renamed targets file
* Update app/victoria-metrics/main.go
Co-authored-by: Aliaksandr Valialkin <valyala@gmail.com>
* adds trimspace to qtpl,
moves content-type for targets response closer to writer
* fixes bug with prefix
Co-authored-by: Aliaksandr Valialkin <valyala@gmail.com>
2020-12-14 13:36:48 +02:00
Aliaksandr Valialkin
5ebfc275e6
app/victoria-metrics: automatically reset response cache when samples with too timestamps older than now - search.cacheTimestampOffset
are ingested
2020-12-14 13:08:28 +02:00
Aliaksandr Valialkin
9d42546a27
docs: consistently use links to https://victoriametrics.github.io for documentation references
2020-12-11 21:08:18 +02:00
Aliaksandr Valialkin
ba8ac08739
app/vmselect/graphite: properly handle wildcards and charsets inside curly braces
...
For example, `foo{bar*,[a-f]a*b}` should match `foobar`, `foobar123`, `foofab`, etc.
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/952
2020-12-11 17:24:12 +02:00
Aliaksandr Valialkin
4146fc4668
all: properly handle CPU limits set on the host system/container
...
This can reduce memory usage on systems with enabled CPU limits.
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/946
2020-12-08 21:07:29 +02:00
Aliaksandr Valialkin
1906f841c9
app/vmselect/promql: do not reduce lookbehind window for any_rollup_func(m)
to -search.maxStalenessInterval
. It should equal to step
value passed to /api/v1/query_range
as most users expect
2020-12-08 15:16:17 +02:00
Aliaksandr Valialkin
b6b1b06d70
app/{vmalert,vmagent}: skip empty values in -remoteWrite.label
and -label
lists
2020-12-08 14:55:13 +02:00