Commit graph

557 commits

Author SHA1 Message Date
Aliaksandr Valialkin
9e88ff3075 app/vmauth: add ability to route requests from a single users to multiple targets depending on the requested path
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1064
2021-02-11 12:41:38 +02:00
Aliaksandr Valialkin
985c3e301d app/vmselect: parallelize q1 <binary_op> q2 queries by running q1 and q2 in parallel
This should reduce query execution times.
2021-02-10 23:00:56 +02:00
Aliaksandr Valialkin
148422bcba lib/storage: disable composite index usage when querying old data 2021-02-10 14:57:58 +02:00
Aliaksandr Valialkin
fa0ef143b1 lib/storage: optimize search by label filters matching big number of time series 2021-02-10 00:46:17 +02:00
Aliaksandr Valialkin
7b7963a77f lib/mergeset: unconditionally cache indexdb blocks
Production workloads show that indexdb blocks must be cached unconditionally for reducing CPU usage.
This shouldn't increase memory usage too much, since unused blocks are removed from the cache every two minutes.
2021-02-09 00:49:59 +02:00
Aliaksandr Valialkin
62574c478a docs/CHANGELOG.md: mention about a bugfix for timezone data from df0cda3ab9 2021-02-08 16:01:26 +02:00
Aliaksandr Valialkin
2b36eb3d82 lib/cgroup: follow-up after b9bf3cbe3e 2021-02-08 16:01:26 +02:00
Aliaksandr Valialkin
35eb04b7dd docs/vmctl.md: fix title, so it is properly displayed in the header of https://victoriametrics.github.io/ 2021-02-04 20:20:38 +02:00
Aliaksandr Valialkin
191bc0bcf3 docs: sync with master branch 2021-02-04 20:02:24 +02:00
Aliaksandr Valialkin
8e5dad8483 docs/CHANGELOG.md: mention recently added changes 2021-02-04 16:42:33 +02:00
Aliaksandr Valialkin
b5eba70595 lib/httpserver: expose process_open_fds and process_max_fds metrics
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/402
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1037
2021-02-04 16:42:32 +02:00
Roman Khavronenko
2aa37b0450 vmalert: mention -datasource.appendTypePrefix in README (#1052) 2021-02-03 23:48:45 +02:00
Aliaksandr Valialkin
e802daa9ee docs/CHANGELOG.md: cut v1.53.1 2021-02-03 22:51:06 +02:00
Aliaksandr Valialkin
88ee836d0c docs/vmalert.md: mention that type option can be set at group level additionally to rule level 2021-02-03 21:12:39 +02:00
Aliaksandr Valialkin
a3aa9bdc9f app/vmagent: add Advanced usage section with the description for all the command-line flags 2021-02-03 21:03:52 +02:00
Aliaksandr Valialkin
e6bcef6514 docs/CHANGELOG.md: mention that {__graphite__="foo.*.bar"} syntax deprecates -search.treatDotsAsIsInRegexps command-line flag 2021-02-03 20:42:18 +02:00
Aliaksandr Valialkin
e4a013d5d7 docs/Single-server-VictoriaMetrics.md: remove misleading section about native format in How to export data in JSON line format chapter 2021-02-03 20:42:17 +02:00
Aliaksandr Valialkin
8de3a329ff app/vmselect: deprecate -search.treatDotsAsIsInRegexps in favor to {__graphite__="foo.*.bar"} syntax 2021-02-03 20:36:42 +02:00
Aliaksandr Valialkin
2976ec89b8 lib/storage: fix a bug, which breaks searching by Graphite wildcard filters 2021-02-03 20:15:50 +02:00
Aliaksandr Valialkin
785b770af3 docs/CHANGELOG.md: fix a link to Graphite Render API usage docs 2021-02-03 12:30:09 +02:00
Aliaksandr Valialkin
844f9991be docs: mention about Graphite render API implementation 2021-02-03 12:13:08 +02:00
Aliaksandr Valialkin
b0acea9d4f docs/Cluster-VictoriaMetrics.md: document Graphite Render API endpoint 2021-02-03 12:03:40 +02:00
Aliaksandr Valialkin
c69b52cf31 docs/CHANGELOG.md: cut v1.53.0 2021-02-03 03:43:14 +02:00
Aliaksandr Valialkin
52b218c07e docs/Single-server-VictoriaMetrics.md: sync with master branch 2021-02-03 01:37:55 +02:00
Aliaksandr Valialkin
4b930b9ffe app/vmselect: add ability to set Graphite-compatible filter via {__graphite__="foo.*.bar"} syntax 2021-02-03 01:17:19 +02:00
Aliaksandr Valialkin
755b0998ce lib/promscrape: add vm_promscrape_service_discovery_duration_seconds metric 2021-02-02 16:16:51 +02:00
Aliaksandr Valialkin
4c59dbc127 lib/promscrape: add vm_promscrape_scrape_retries_total, vm_promscrape_discovery_retries_total and vm_promscrape_discovery_requests_total metrics 2021-02-01 20:06:16 +02:00
Aliaksandr Valialkin
03da0b728c docs: increase heading sizes in vmagent, vmauth, vmbackup and vmrestore docs, so they match the heading sizes in VictoriaMetrics docs 2021-02-01 19:44:58 +02:00
Aliaksandr Valialkin
f4a4665857 docs/vmctl.md: update build instructions after the migration from github.com/VictoriaMetrics/vmctl to github.com/VictoriaMetrics/VictoriaMetrics/app/vmctl 2021-02-01 19:44:57 +02:00
Aliaksandr Valialkin
286625e4b3 docs: sync with master branch 2021-02-01 18:05:58 +02:00
Aliaksandr Valialkin
6811445b64 docs: document ability to query Graphite datasource from vmalert 2021-02-01 15:28:31 +02:00
Aliaksandr Valialkin
fdf9de98f8 app/vmagent: add -remoteWrite.roundDigits command-line option for limiting the number of digits after the point for stored values
This commit also adds --vm-round-digits command-line option to vmctl tool.
2021-02-01 14:42:15 +02:00
Aliaksandr Valialkin
7f4fb34182 app/vmctl: move vmctl code from github.com/VictoriaMetrics/vmctl
It is better developing vmctl tool in VictoriaMetrics repository, so it could be released
together with the rest of vmutils tools such as vmalert, vmagent, vmbackup, vmrestore and vmauth.
2021-02-01 01:18:39 +02:00
Aliaksandr Valialkin
ff526492eb docs/Cluster-VictoriaMetrics.md: mention about -search.denyPartialResponse command-line flag and deny_partial_response query arg 2021-01-27 14:06:14 +02:00
Aliaksandr Valialkin
db5b78f65c docs/CHANGELOG.md: typo fixes 2021-01-27 01:20:31 +02:00
Aliaksandr Valialkin
ffec5131ae lib/promscrape: export vm_promscrape_scrapes_failed_per_url_total and vm_promscrape_scrapes_skipped_by_sample_limit_per_url_total metrics
These metrics could be useful for determining imporperly working scrape targets.
Note that these metrics are exported only for failing scrape targets. They aren't exposed for normally working targets.
2021-01-27 00:40:39 +02:00
Aliaksandr Valialkin
29bf531f7d app/vmagent: add -remoteWrite.rateLimit command-line flag for limiting data rate to remote storage
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1035
2021-01-27 00:40:39 +02:00
Aliaksandr Valialkin
68a66be811 lib/logger: add -loggerTimezone command-line flag for adjusting timezone for timestamps in log messages 2021-01-26 22:53:25 +02:00
Aliaksandr Valialkin
9e175683b1 docs/CHANGELOG.md: mention about https://github.com/VictoriaMetrics/VictoriaMetrics/pull/1027 2021-01-26 16:38:22 +02:00
Aliaksandr Valialkin
fdced59278 lib/promscrape: retry scrape and service discovery requests when the remote server closes http keep-alive connection 2021-01-22 13:22:59 +02:00
Aliaksandr Valialkin
7308691865 docs/CHANGELOG.md: mention about the fix with too big HTTP reconnection rate to targets
This has been fixed in 0a45220b0a
2021-01-22 12:10:01 +02:00
Aliaksandr Valialkin
e55205220b app/vmselect: add -search.maxStepForPointsAdjustment command-line flag, which can be used for disabling adjustment for points returned from /api/v1/query_range handler if they have timestamps closer than -search.latencyOffset to the current time 2021-01-19 22:57:50 +02:00
Aliaksandr Valialkin
5640e6cbca docs/vmagent.md: follow-up for 184a659c5f 2021-01-13 13:54:28 +02:00
mancubus77
6f3560d256 Doco vmagent fix (#1014)
* Update section with remote_write.url for clustered version

* fix typo

Co-authored-by: mancubus <dont@write.me>
2021-01-13 13:54:27 +02:00
Aliaksandr Valialkin
d23d6d4bfa docs/CHANGELOG.md: cut v1.52.0 2021-01-13 12:59:08 +02:00
Aliaksandr Valialkin
c5bdab5a4c app/vmselect/promql: add ability to pass multiple labels to sort_by_label and sort_by_label_desc functions
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/992
2021-01-13 12:43:47 +02:00
Aliaksandr Valialkin
81789da731 lib/backup: increase backup chunk size from 128MB to 1GB
This should reduce costs for object storage API calls by 8x. See https://cloud.google.com/storage/pricing#operations-pricing
2021-01-13 12:16:39 +02:00
Aliaksandr Valialkin
8cae98aa78 app/vmselect/promql: properly parse escaped multibyte utf8 code sequences in metric names and labels names
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/990
2021-01-13 10:59:32 +02:00
Aliaksandr Valialkin
2d4006cb4a docs/CHANGELOG.md: document updated extra_label query arg behavior
Follow-up for dc9d7aedd5
2021-01-13 01:07:25 +02:00
Nikolay
821492bc0b adds extra_label to all import apis (#1007)
* adds extra_label to all import apis,
changes priority for extra_label - now it has priority over original labels

* Update README.md

Co-authored-by: Aliaksandr Valialkin <valyala@gmail.com>

* Update README.md

Co-authored-by: Aliaksandr Valialkin <valyala@gmail.com>

* adds extra labels to vmagent  import api
changes order for adding labels, now its added after user values

* adds tests for extra_label

* import fix

Co-authored-by: Aliaksandr Valialkin <valyala@gmail.com>
2021-01-13 01:07:24 +02:00
Aliaksandr Valialkin
bc8b38daca docs/CHANGELOG.md: mention that the minimum supported TLS version now is v1.2
Follow-up for 7bf5d48315
2021-01-13 00:45:08 +02:00
Aliaksandr Valialkin
b71748f1d9 docs/Single-server-VictoriaMetrics.md: typo fix 2021-01-12 22:03:46 +02:00
Aliaksandr Valialkin
8711860327 deployment/docker: upgrade base image for Docker packages from Alpine 3.13.1 to Alpine 3.12.3 in order to fix potential security issues
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1010
2021-01-12 21:57:07 +02:00
Aliaksandr Valialkin
df6e399f73 app/vmselect/promql: add tfirst_over_time(m[d]) and tlast_over_time(m[d]) MetricsQL functions for returning timestamps for the first and the last samples in m over d 2021-01-12 16:12:47 +02:00
Aliaksandr Valialkin
de04dd81cf docs/Articles.md: add https://cer6erus.medium.com/cloud-native-model-driven-telemetry-stack-on-openshift-80712621f5bc 2021-01-12 15:36:50 +02:00
Aliaksandr Valialkin
1dee98a331 docs/CHANGELOG.md: document big fixes from the commit 7976c22797 2021-01-12 13:44:10 +02:00
Aliaksandr Valialkin
c97681b45c lib/promscrape: properly show scrape duration on /targets page
Previously it has been shown as 0.000s for any scrape duration.
2021-01-11 21:15:50 +02:00
Aliaksandr Valialkin
bdf67a13cc docs/Single-server-VictoriaMetrics.md: mention about https://github.com/aorfanos/vmalert-cli in Integrations section 2021-01-11 18:52:56 +02:00
Aliaksandr Valialkin
4731780a38 docs/CHANGELOG.md: mention about a bugfix for https://github.com/VictoriaMetrics/VictoriaMetrics/issues/989 2021-01-11 13:11:53 +02:00
Aliaksandr Valialkin
4ee53c3961 all: use net.Dial instead of fasthttp.Dial, because fasthttp.Dial limits the number of concurrent dials to 1000 2021-01-11 12:52:51 +02:00
Aliaksandr Valialkin
d5a2b120e9 app/vmstorage: disable final merge by default, since it may result in high disk IO and CPU usage without measurable benefits such as increased query performance and reduced disk space usage 2021-01-08 00:12:12 +02:00
Aliaksandr Valialkin
fa176dacf1 docs/Single-server-VictoriaMetrics.md: sync with upstream 2021-01-07 23:39:21 +02:00
Aliaksandr Valialkin
5fb8cf00c3 docs/Cluster-VictoriaMetrics.md: typo fix 2021-01-07 23:39:12 +02:00
Nikolay
137efe6c71 Snap docs change (#986)
* adds snap docs,
adds release information for snap package,
adds docs notes about configuration management with snap package.

* adds release page mention

* version fix for snap, its awful

* revert version

* Apply suggestions from code review

Co-authored-by: Aliaksandr Valialkin <valyala@gmail.com>
2020-12-29 11:44:50 +02:00
Aliaksandr Valialkin
e2d8b9e091 README.md: mention about -search.queryStats.lastQueriesCount and -search.queryStats.minQueryDuration command-line flags in docs about query stats 2020-12-29 11:39:45 +02:00
Aliaksandr Valialkin
66a19e0079 docs/CHANGELOG.md: cut v1.51.0 2020-12-27 14:21:41 +02:00
Aliaksandr Valialkin
47872ada7e app/vmselect/promql: do not ajdust offset value provided in the query
Previously it could be modified in order to improve response cache hit ratio.
This is unneeded, since cache hit ratio should remain good because the query time range
should be already aligned to multiple of `step` values.

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/976
2020-12-27 14:10:15 +02:00
Aliaksandr Valialkin
5c8e4cf03e docs/Articles.md: add a link to https://www.percona.com/blog/2020/12/23/observations-on-better-resource-usage-with-percona-monitoring-and-management-v2-12-0/ 2020-12-27 13:02:08 +02:00
Aliaksandr Valialkin
5bbf200de2 app/vmselect: add per-tenant /api/v1/status/top_queries handler 2020-12-27 12:53:50 +02:00
Aliaksandr Valialkin
e6deb39064 app/vmselect: refactor /api/v1/stats/top_queries
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/907
2020-12-25 17:24:25 +02:00
Aliaksandr Valialkin
991262d53e docs/CHANGELOG.md: mention that vmalert now properly escapes multi-line queries when passing to Grafana
A follow-up for 1de15ad490

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/890
2020-12-25 11:12:57 +02:00
Nikolay
14915071d6 adds escape for CRLF (#984)
at external.alert.source - \n and \r symbols was url encoded, instead of direct usage.
replace it from "\n" to `\n`  allows to skip url encoding.
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/890
2020-12-25 11:06:47 +02:00
Aliaksandr Valialkin
b480585905 app/vmalert: typo fix in descriptions for notifier.basicAuth.username and notifier.basicAuth.password command-line flags 2020-12-24 12:49:40 +02:00
Aliaksandr Valialkin
2a8bd2b5cc docs/CHANGELOG.md: mention about adding missing __meta_kubernetes_service_* labels for endpoints and endpointslices roles in kubernetes_sd_config
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/982
2020-12-24 11:34:01 +02:00
Aliaksandr Valialkin
367fc17933 lib/promscrape: code prettifying for 8dd03ecf19 2020-12-24 10:57:20 +02:00
Aliaksandr Valialkin
d8511b6651 docs: mention that it is possible to set multiple -notifier.tlsInsecureSkipVerify command-line flags for vmalert
See c3a92968343c2b3619f1ab935702d0e9b3a46733
2020-12-22 22:32:56 +02:00
Aliaksandr Valialkin
fa3bcf220f lib/storage: remove stale parts as soon as they go outside the configured retention
Previously such parts could remain undeleted for long durations until they are merged with other parts.
This should help for `-retentionPeriod` values smaller than one month.
2020-12-22 19:55:07 +02:00
Artem Navoiev
f6b76b5c01 add linkedin to release announcement 2020-12-21 08:57:23 +02:00
Artem Navoiev
618ecd4708 Announcement guide 2020-12-20 00:45:02 +02:00
Aliaksandr Valialkin
787cbd2f7e docs/CHANGELOG.md: cut v1.50.2 2020-12-19 15:32:45 +02:00
Aliaksandr Valialkin
a5b17946fe docs/CHANGELOG.md: mention about bugfix for populating template variables in vmalert
See 404cbd1522
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/974
2020-12-19 14:21:28 +02:00
Aliaksandr Valialkin
262cf81757 app/vmselect: properly parse negative combined offsets such as -1h2m3s
Previously such offsets were parsed as `-1h + 2m + 3s`. Now they are parsed as `-(1h + 2m + 3s)`.
2020-12-19 01:25:03 +02:00
Aliaksandr Valialkin
5aa2b0de55 docs/CHANGELOG.md: there is no visible difference for CPU usage and disk IO usage on production workloads 2020-12-19 00:51:32 +02:00
Aliaksandr Valialkin
edbe35509e lib/{storage,mergeset}: tune background merge process in order to reduce CPU usage and disk IO usage 2020-12-18 20:01:20 +02:00
Aliaksandr Valialkin
c5b47df8a4 docs/CHANGELOG.md: mention that Docker images for vmagent, vmalert, vmauth, vmbackup and vmrestore with tags containing -cluster suffix are no longer published
See 441822c4cc for details
2020-12-18 20:01:18 +02:00
Aliaksandr Valialkin
39f7fa4743 docs: alphabetically sort links to case studies 2020-12-18 12:31:03 +02:00
Aliaksandr Valialkin
5b2fd08b27 docs/Articles.md: classify articles by themes 2020-12-18 12:05:39 +02:00
Aliaksandr Valialkin
6edd3330b4 add a link to https://www.percona.com/blog/2020/12/16/percona-monitoring-and-management-migration-from-prometheus-to-victoriametrics-faq/ 2020-12-18 11:44:31 +02:00
Aliaksandr Valialkin
4fd2973e7c lib/protoparser/prometheus: follow-up commit after 7d38627b9f6f212ae602aea6a72f469fe3c70ba2
Document the bugfix in docs/CHANGELOG.md and add a test for the bugfix.
2020-12-16 23:42:17 +02:00
Aliaksandr Valialkin
49b6c6f6ab docs/FAQ.md: add a link to https://valyala.medium.com/promscale-vs-victoriametrics-resource-usage-on-production-workload-91c8e3786c03 in the question about benchmarks 2020-12-16 23:15:49 +02:00
Aliaksandr Valialkin
2b5700b1c4 docs/Articles.md: add a link to https://valyala.medium.com/promscale-vs-victoriametrics-resource-usage-on-production-workload-91c8e3786c03 2020-12-16 14:27:01 +02:00
Aliaksandr Valialkin
f6b51889ce docs/Cluster-VictoriaMetrics.md: fix incorrect commands for profile collecting 2020-12-16 01:06:49 +02:00
Aliaksandr Valialkin
69b3ca37d0 docs/Cluster-VictoriaMetrics.md: add Profiling section 2020-12-16 00:59:53 +02:00
Aliaksandr Valialkin
34622f7f9b docs/CHANGELOG.md: cut v1.50.1 release 2020-12-15 21:11:53 +02:00
Aliaksandr Valialkin
1022505131 docs/Release-Guide.md: actualize release docs 2020-12-15 21:11:02 +02:00
Aliaksandr Valialkin
f89c3e505a docs/CHANGELOG.md: mention about bugfix, which properly removes unregistered targets from /targets page
See bugfix at a4c7fcb5e1
2020-12-15 21:07:28 +02:00
Aliaksandr Valialkin
49e800ba55 app/vmagent: add vmagent_remotewrite_blocks_sent_total and vmagent_remotewrite_bytes_sent_total metrics per each -remoteWrite.url 2020-12-15 20:41:08 +02:00
Aliaksandr Valialkin
9ab7ca1133 docs/vmagent.md: typo fix: pearsed->parsed 2020-12-15 19:03:35 +02:00
Aliaksandr Valialkin
11674a9b76 docs/vmagent.md: mention that sample_limit option has no sense when stream parsing is enabled 2020-12-15 18:44:19 +02:00
Aliaksandr Valialkin
dd47c8f084 docs/CHANGELOG.md: cut v1.50.0 release 2020-12-15 14:45:00 +02:00