github-mirrors/VictoriaMetrics
1
0
Fork 0
mirror of https://github.com/VictoriaMetrics/VictoriaMetrics.git synced 2024-11-21 14:44:00 +00:00
Code Issues Projects Releases Packages Wiki Activity
4618 commits 150 branches 748 tags 347 MiB
Commit graph

1701 commits

Author SHA1 Message Date
Aliaksandr Valialkin
f901788c7f
docs/CHANGELOG.md: document 8f4f5f1d68 2022-05-09 17:32:51 +03:00
Aliaksandr Valialkin
84326eacd6
docs/Release-Guide.md: typo fix: signle->single 2022-05-09 15:27:44 +03:00
Manuel Polo
58c1472394
docs: fix typo in quickstart 2022-05-08 19:43:42 +00:00
Artem Navoiev
3f78a609ac
docs: add flags list to vmbackupmanager (#2554) 
docs: add flags list to vmbackupmanager docs

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: tenmozes <artem@victoriametrics.com>
 2022-05-08 12:18:39 +00:00
Aliaksandr Valialkin
da9e96bc55
docs/CHANGELOG.md: cut v1.77.1 2022-05-07 01:46:27 +03:00
Aliaksandr Valialkin
f26daecc8d
app/vminsert/netstorage: re-route samples from readonly vmstorage nodes to healthy nodes if -dropSamplesOnOverload command-line flag is set 2022-05-07 01:41:00 +03:00
Aliaksandr Valialkin
d6ad8d090d
app/vmstorage: do not allow to set -retentionPeriod smaller than one day 
VictoriaMetrics doesn't support retention periods smaller than one day,
so do not allow to set it to small values.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2496
 2022-05-07 00:53:01 +03:00
Aliaksandr Valialkin
123aa4c79e
lib/promscrape: properly implement ScrapeConfig.clone() 
Previously ScrapeConfig.clone() was improperly copying promauth.Secret fields -
their contents was replaced with `<secret>` value.

This led to inability to use passwords and secrets in `-promscrape.config` file.
The bug has been introduced in v1.77.0 in the commit 67b10896d2

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2551
 2022-05-07 00:05:40 +03:00
Aliaksandr Valialkin
38629b53ef
docs/CHANGELOG.md: document e726340914 2022-05-06 18:10:40 +03:00
Dmytro Kozlov
9a63f6c1b8
vmbackup: Prevent save backups to the same folder where TSDB data is (#2547) 
* {vmbackup, vmbackup/snapshot}: validate snapshot name

* vmbackup/snapshot: added another checks

* backup/actions: added check that we ignore backup_complete.ignore file

* vmbackup: moved snapshot to lib directory

* lib/snapshot: added functions description

* lib/snapshot: fixed typo

* vmbackup: code cleanup

* wip

* vmbackup: Prevent save backups to the same folder where TSDB data is

* Apply suggestions from code review

* wip

* wip

* wip

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2022-05-06 18:04:09 +03:00
Aliaksandr Valialkin
80092f087e
docs/Cluster-VictoriaMetrics.md: typo fix: bandidth -> bandwidth 2022-05-06 16:35:36 +03:00
Aliaksandr Valialkin
8100c9a301
docs/Cluster-VictoriaMetrics.md: make the description for -rpc.disableCompression command-line flag more clear 2022-05-06 16:27:33 +03:00
Roman Khavronenko
20ccf0ba81
vmctl: add tip about safety flags during for native data export (#2540) 
Signed-off-by: hagen1778 <roman@victoriametrics.com>
 2022-05-06 15:12:29 +02:00
Aliaksandr Valialkin
8d81703635
app/vmagent: add missing _total suffix to vmagent_remotewrite_global_rows_pushed_before_relabel_total counter 
This is a follow up for c536139d0b
 2022-05-06 15:50:57 +03:00
Aliaksandr Valialkin
1dc4cc243b
lib/promscrape: rename promscrape_stale_samples_created_total metric to vm_promscrape_stale_samples_created_total, so its name is consistent with the rest of vm_promscrape_ metrics 2022-05-06 15:33:13 +03:00
Aliaksandr Valialkin
c536139d0b
app/vmagent: expose vmagent_remotewrite_global_rows_pushed_before_relabel and vmagent_remotewrite_rows_pushed_after_relabel_total metrics 2022-05-06 15:28:59 +03:00
Aliaksandr Valialkin
51e36fd533
app/vmagent: rename vmagent_remote_write_rate_limit_reached_total to vmagent_remotewrite_rate_limit_reached_total for the sake of consistency with other vmagent_remotewrite_ metrics 2022-05-06 15:01:54 +03:00
Aliaksandr Valialkin
d5b55fe22d
lib/promscrape/discovery/ec2: add ability to filter Availability Zones in ec2_sd_config via az_filters section 2022-05-06 12:43:29 +03:00
Aliaksandr Valialkin
ca4ca4630b
app/vmselect/vmui: make vmui-update after 450d879eaa 2022-05-05 21:26:01 +03:00
Denis Fondras
928728807c
Fix typo (#2538) 2022-05-05 21:16:03 +03:00
Aliaksandr Valialkin
6343b20943
docs/Cluster-VictoriaMetrics.md: update cluster scalability tips 2022-05-05 21:11:51 +03:00
Aliaksandr Valialkin
be76d49150
docs/CHANGELOG.md: document bf5e3774cc 2022-05-05 13:38:17 +03:00
Aliaksandr Valialkin
6bb32ab9de
docs/CHANGELOG.md: cut v1.77.0 2022-05-05 00:16:16 +03:00
Aliaksandr Valialkin
2c037ae0d3
docs/vmbackup.md: added missing -storageDataPath argument in the command for creating daily backups 2022-05-04 22:49:37 +03:00
Aliaksandr Valialkin
910f715ffe
docs/vmbackup.md: acutalize docs 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2503
 2022-05-04 22:44:15 +03:00
Aliaksandr Valialkin
e761d9449c
app/vmagent: rename -remoteWrite.useSigv4 command-line flag to -remoteWrite.aws.useSigv4, so its name is consistent with the other -remoteWrite.aws.* command-line flags 2022-05-04 20:41:17 +03:00
Aliaksandr Valialkin
381e2de59c
app/vmalert: run make quicktemplate-gen from the root directory after the commit f6dcfbcdd6 2022-05-04 20:27:36 +03:00
Nikolay
d289ecded1
{lib/promscrape,app/vmagent}: adds sigv4 support for vmagent remoteWrite (#2458) 
* {lib/promscrape,app/vmagent}: adds sigv4 support for vmagent remoteWrite
moves aws related code into separate lib from lib/promscrape
it allows to write data from vmagent to the AWS managed prometheus (cortex)

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1287

* Apply suggestions from code review

* wip

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2022-05-04 20:24:19 +03:00
Dmytro Kozlov
f6dcfbcdd6
vmalert/tpl: fixed truncating alerts expression in table (#2494) 
vmalert: improve `/groups` UI visual 

The change also fixes truncated rules expressions in UI
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2484
 2022-05-04 18:02:18 +02:00
Aliaksandr Valialkin
242856ce97
docs/guides/multi-regional-setup-dedicated-regions.md: clarify wording on vmagent configuration 2022-05-04 18:49:56 +03:00
Aliaksandr Valialkin
4e4ca1b6db
docs/Cluster-VictoriaMetrics.md: move environment variables entry closer to cluster setup section 2022-05-04 18:43:57 +03:00
Aliaksandr Valialkin
bab9670d69
docs/CHANGELOG.md: yet another typo fix: present -> pressed 2022-05-04 18:20:39 +03:00
Aliaksandr Valialkin
554008bb4e
docs/CHANGELOG.md: typo fixes 2022-05-04 18:18:37 +03:00
Aliaksandr Valialkin
2e6827ff04
docs/CHANGELOG.md: document 8639e79d38 2022-05-04 10:46:03 +03:00
Aliaksandr Valialkin
1c7a541247
docs/Cluster-VictoriaMetrics.md: typo fix: serparated -> separated 2022-05-03 15:19:15 +03:00
Aliaksandr Valialkin
2ced6746a7
docs/CHANGELOG.md: document 3575aabeaf 2022-05-03 14:01:15 +03:00
Aliaksandr Valialkin
b5fedfd3bb
all: add -cluster.tlsInsecureSkipVerify command-line option to vminsert, vmselect and vmstorage components in order to be able to disable TLS certificate verification in mTLS mode 
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2490
 2022-05-03 13:12:50 +03:00
Aliaksandr Valialkin
528b332fe0
docs/Cluster-VictoriaMetrics.md: refer to the doc on how to set up mTLS 2022-05-03 11:55:06 +03:00
Aliaksandr Valialkin
53cef612b0
docs/CHANGELOG.md: document 488c34f5e1 2022-05-03 11:01:02 +03:00
Aliaksandr Valialkin
aa02719d86
docs/CHANGELOG.md: document d0706c8c95 2022-05-02 22:24:45 +03:00
Aliaksandr Valialkin
0d86644d65
lib/storage: leave the last sample per each discrete interval during the deduplicaton 
This aligns better with staleness logic in Prometheus - https://prometheus.io/docs/prometheus/latest/querying/basics/#staleness
 2022-05-02 21:50:45 +03:00
Aliaksandr Valialkin
a186434b50
app/vmui: execute query by pressing enter in the same way as Prometheus does 
Multi-line query can be entered via `shift-enter` in the query input field
 2022-05-02 19:49:29 +03:00
Aliaksandr Valialkin
2c4565bb3d
docs/Cluster-VictoriaMetrics.md: move here the deduplication docs related to cluster version 2022-05-02 15:53:34 +03:00
Max Golionko
e79aa037b0
added details about deduplication (#2527) 
* added details about deduplication

* Update docs/README.md

Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>

* Apply suggestions from code review

* Update docs/README.md

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
 2022-05-02 15:45:57 +03:00
Aliaksandr Valialkin
bae7e8b16b
docs/CHANGELOG.md: document 3616337812 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2514
 2022-05-02 15:36:00 +03:00
Aliaksandr Valialkin
70d9e7346b
docs/CHANGELOG.md: document 32a6b67e6c 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1761
 2022-05-02 15:26:21 +03:00
Aliaksandr Valialkin
6039640a26
docs/CHANGELOG.md: document b2294d1cf1 2022-05-02 15:21:24 +03:00
Aliaksandr Valialkin
7bc6595b45
lib/netutil: close connections in ConnPool if they are idle for more than 30 seconds 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2508
 2022-05-02 15:14:05 +03:00
Artem Navoiev
37cf509c3a
lib/{storage,flagutil} - Add option for snapshot autoremoval (#2487) 
* lib/{storage,flagutil} - Add option for snapshot autoremoval

- add prometheus-like duration as command flag
- add option to delete stale snapshots
- update duration.go flag to re-use own code

* wip

* lib/flagutil: re-use Duration.Set() call in NewDuration

* wip

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2022-05-02 11:00:15 +03:00
Peter Dupej
8688ea8aa8
Update MetricsQL.md (#2519) 
remove typo from label_replace doc.
 2022-05-02 09:42:58 +03:00
Dima Lazerka
ed8e88af11
Export "null" in jsonl instead of NaN (#2518) 
* Export "null" in jsonl instead of NaN

The NaN appeared because of staleness markers that were added for compatibility. I think it's better to use json `null`, implemented here.

Also maybe it also makes sense to add a flag like `?skip-staleness-markers=true` to `/export`, to skip nulls at all?

* Update app/vmselect/prometheus/export.qtpl

* app/vmselect/prometheus/export.qtpl.go: `make quicktemplate-gen`

* docs/CHANGELOG.md: document the change

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2022-04-29 12:49:47 +03:00
Aliaksandr Valialkin
f5635d6920
docs/CHANGELOG.md: document c7aad8d441 2022-04-29 11:39:23 +03:00
Aliaksandr Valialkin
9bb35779d2
docs/MetricsQL.md: clarify keep_metric_names docs 2022-04-27 11:24:32 +03:00
Aliaksandr Valialkin
cce1b6d7f9
app/vmselect/promql: add tlast_change_over_time(m[d]) function, which returns the timestamp for the last change of m on the given lookbehind window d 2022-04-27 10:59:03 +03:00
Aliaksandr Valialkin
4176be38c4
app/vmagent: substitute hard-to-read 500000000 with 500MB in -remoteWrite.maxDiskUsagePerURL description 2022-04-26 15:48:20 +03:00
Yury Molodov
9b4bff67e0
vmui: add support relative time (#2504) 
* feat: add support relative time

* app/vmselect: `make vmui-update`

* docs/CHANGELOG.md: document the change

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2022-04-26 15:46:06 +03:00
Aliaksandr Valialkin
ce1190974c
docs/Cluster-VictoriaMetrics.md: remove incorrect and misleading instructions for passing -replicationFactor flag to vmselect nodes in multi-level setup. 
The `-replicationFactor` passed to top-level `vmselect` nodes mustn't exceed the `-replicationFactor` passed to top-level `vminsert` nodes
 2022-04-26 15:13:56 +03:00
Aliaksandr Valialkin
8594609385
docs/CHANGELOG.md: document 4c1fbcd6b0 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2368
 2022-04-26 15:10:00 +03:00
Aliaksandr Valialkin
5d87744ba2
docs/CHANGELOG.md: typo fix: may result -> could result 2022-04-23 00:31:22 +03:00
Aliaksandr Valialkin
eb5d7ad089
lib/promauth: add support for min_version option at tls_config section in the same way as Prometheus does 2022-04-23 00:16:39 +03:00
Aliaksandr Valialkin
174431e31b
lib/promauth: add support for proxy_url option at oauth2 section in the same way as Prometheus does 2022-04-23 00:00:44 +03:00
Aliaksandr Valialkin
18b14aad8e
lib/promauth: add support for tls_config section at oauth2 config in the same way as Prometheus does 2022-04-22 23:51:07 +03:00
Aliaksandr Valialkin
15190fcdae
lib/promscrape/discovery/kubernetes: allow attaching node-level labels and annotations to discovered pod targets in the same way as Prometheus 2.35 does 
See https://github.com/prometheus/prometheus/issues/9510
and https://github.com/prometheus/prometheus/pull/10080
 2022-04-22 20:15:41 +03:00
Aliaksandr Valialkin
1970b3db34
docs/Cluster-VictoriaMetrics.md: mention that enterprise binaries are available for evaluation 2022-04-21 15:58:03 +03:00
Aliaksandr Valialkin
912a5a6f28
docs/Single-server-VictoriaMetrics.md: refer to the docs on how to set up multiple vmagent instances for scraping the same set of targets 2022-04-21 15:51:52 +03:00
Aliaksandr Valialkin
25fe83577d
app/vmselect/promql: properly handle scalar default vector, scalar if vector and scalar ifnot vector queries 
Previously `vector` time series could be unexpectedly returned from such queries
 2022-04-21 15:34:36 +03:00
Aliaksandr Valialkin
de892239a9
app/vmselect/promql: add drop_common_labels() function 2022-04-21 14:20:20 +03:00
Aliaksandr Valialkin
98129d4a8e
app/vmstorage: expose vm_indexdb_items_added_total and vm_indexdb_items_added_size_bytes_total counters at /metrics page 
These counters can be used for monitoring the rate of addition of new entries in indexdb (aka inverted index).

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2471
 2022-04-21 13:18:39 +03:00
Aliaksandr Valialkin
d335cc886c
app/vmselect/promql: fix duplicate time series error on joins against time series filtered by values 
This should prevent from `duplicate time series` errors when executing the following query:

kube_pod_container_resource_requests{resource="cpu"} * on (namespace,pod) group_left() (kube_pod_status_phase{phase=~"Pending|Running"}==1)

where `kube_pod_status_phase{phase=~"Pending|Running"}==1` filters out diplicate time series
 2022-04-20 22:18:44 +03:00
Aliaksandr Valialkin
694887cea8
docs/CHANGELOG.md: document that the service discovery speed now scales with the number of CPU cores 2022-04-20 16:22:18 +03:00
Aliaksandr Valialkin
3d0549c982
lib/promscrape: optimize getScrapeWork() function 
Reduce the number of memory allocations in this function. This improves its performance by up to 50%.
This should improve service discovery speed when big number of potential targets with big number of meta-labels
are generated by service discovery.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2270
 2022-04-20 15:37:00 +03:00
Aliaksandr Valialkin
f6d0e5e74a
all: typo fix: Kuberntes -> Kubernetes 2022-04-20 10:50:49 +03:00
Dmytro Kozlov
a3ee275149
lib/promscrape: Enable filters for endpoint and labels (#2466) 
* lib/promscrape: Enable filters for endpoint and labels

* lib/promscrape: cleanup

* lib/promscrape: update template

* lib/promscrape: move logic filter logic to backend

* lib/promscrape: updated placeholder

* lib/promscrape: updated placeholder

* lib/promscrape: use two different fields for filters, updated form, added error on parsing queries

* lib/promscrape: rename functions

* lib/promscrape: removed unused values

* wip

* wip

* wip

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2022-04-19 18:26:21 +03:00
Nikolay
26b78ad707
lib/promscrape: adds job restart method (#2455) 
* lib/promscrape: adds job restart method
it must restart only ScrapeConfig with changed content
this change greatly reduce time, that needed for job restart
and it should decrease possible data loss when config frequently changed at kubernetes based deployments

Apply suggestions from code review

Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>

* wip

Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2022-04-16 20:28:46 +03:00
Yury Molodov
514e3660e2
fix: prevent graph hiding without data (#2456) 
* fix: prevent graph hiding without data

* fix: add yaxis labels default

* app/vmselect: `make vmui-update`

* docs/CHANGELOG.md: document the change

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2022-04-16 17:15:31 +03:00
Aliaksandr Valialkin
f62fc2f318
docs/Cluster-VictoriaMetrics.md: sync docs 2022-04-16 16:59:27 +03:00
Aliaksandr Valialkin
1097ebebe6
lib/httpserver: clarify that -tls flag enables TLS for http requests to -httpListenAddr 2022-04-16 16:59:26 +03:00
Aliaksandr Valialkin
cad488fe7e
app/vmstorage: add support for mTLS cipher suites via -cluster.tlsCipherSuites command-line flag 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2404
 2022-04-16 16:39:21 +03:00
Aliaksandr Valialkin
54bb8c2bc6
docs/Cluster-VictoriaMetrics.md: update docs after 26ae50ec26 2022-04-16 16:04:58 +03:00
Aliaksandr Valialkin
b49b8020d6
docs: sync docs with the latest changes 2022-04-16 15:59:53 +03:00
Aliaksandr Valialkin
7e4bdf31ba
lib/httpserver: follow up after def0032c7d 2022-04-16 15:27:21 +03:00
Aliaksandr Valialkin
ebaa1c7ad5
lib/promscrape: follow-up after baa1c24b36 2022-04-16 14:25:54 +03:00
Aliaksandr Valialkin
057c3a745f
docs/Cluster-VictoriaMetrics.md: clarify the issue with multi-level cluster, which may result in data gaps at some AZs 2022-04-16 13:04:29 +03:00
Aliaksandr Valialkin
4169b97af9
docs/Cluster-VictoriaMetrics.md: improve docs on cluster setup, resizing and scalability 2022-04-15 15:05:47 +03:00
Aliaksandr Valialkin
b11e9de386
docs/Cluster-VictoriaMetrics.md: mention about possible issues with multi-level cluster setup 2022-04-15 15:05:45 +03:00
Aliaksandr Valialkin
e6535a75f7
docs/CHANGELOG.md: document 45fcaa33e8 2022-04-13 14:12:17 +03:00
Aliaksandr Valialkin
77ffa4e447
docs/CHANGELOG.md: document f7e4c5a628 2022-04-13 14:09:11 +03:00
Roman Khavronenko
45fcaa33e8
vmalert: add DNS service discovery (#2465) 
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2460
Signed-off-by: hagen1778 <roman@victoriametrics.com>
 2022-04-13 11:50:26 +03:00
Anton Bystrov
9307fe3c04
Update CHANGELOG.md (#2463) 
May be mispint here?
 2022-04-13 11:18:42 +03:00
Aliaksandr Valialkin
f8de318bfc
docs/CHANGELOG.md: cut v1.76.1 2022-04-12 16:20:55 +03:00
Aliaksandr Valialkin
ef66b048c9
app/vmui: further improvements for number display on graphs 
This is a follow-up for c4d2cd8336

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2409
 2022-04-12 16:01:27 +03:00
Aliaksandr Valialkin
52cb80ed4f
docs/CHANGELOG.md: link to the bug related to improper handling of maxSeries limit passed from vmselect to vmstorage 2022-04-12 16:00:25 +03:00
Yurii Kravets
38383c0bec
Update Quick-Start (#2422) 
* Update Quick-Start

* Update docs/Quick-Start.md

Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>

* Update docs/Quick-Start.md

Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>

* Update docs/Quick-Start.md

Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>

* Update docs/Quick-Start.md

Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>

* update Quick-Start.md

added "Starting VM-Cluster via Docker" + Anchor fixes

* Apply suggestions from code review

Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2022-04-12 14:22:53 +03:00
Roman Khavronenko
dbbacc8847
docs: add managed vm documentation section (#2437) 
Signed-off-by: hagen1778 <roman@victoriametrics.com>
 2022-04-12 13:03:25 +03:00
Aliaksandr Valialkin
f3d4671bb6
lib/promscrape: follow-up after 7e79adfb55 2022-04-12 12:36:17 +03:00
Yurii Kravets
708a3ef276
url-examples.md (#2435) 
* url-examples.md

added 1 more example

* Update url-examples.md
 2022-04-12 11:27:17 +03:00
Aliaksandr Valialkin
54de0531a4
app/vmstorage: properly handle maxSeries limit passed from vmselect to vmstorage 2022-04-12 11:23:04 +03:00
Aliaksandr Valialkin
deaa8c1ffa
lib/protoparser/native: follow-up after fe01f4803d 2022-04-11 19:27:07 +03:00
Aliaksandr Valialkin
d7bf0a7348
vendor: update github.com/VictoriaMetrics/metricsql from v0.40.0 to v0.41.0 
This allows using built-in function names as with template names
 2022-04-11 18:31:44 +03:00
Aliaksandr Valialkin
e27dac25b9
docs/Single-server-VictoriaMetrics.md: clarify that ingestion protocol means data ingestion protocol 2022-04-11 12:57:26 +03:00
Aliaksandr Valialkin
b89e846ce3
docs/CHANGELOG.md: document ed364a42e3 2022-04-11 12:11:32 +03:00
Aliaksandr Valialkin
3121085e8f
docs/Single-server-VictoriaMetrics.md: mention that VictoriaMetrics supports data ingestion in Graphite protocol at Graphite API usage chapter 2022-04-10 16:20:41 +03:00
Aliaksandr Valialkin
f1ad5b6857
docs/Cluster-VictoriaMetrics.md: update docs after b843f0e229 2022-04-10 16:18:21 +03:00
Yurii Kravets
ed8c6f69e4
Update url-examples (#2410) 2022-04-08 13:05:46 +03:00
Aliaksandr Valialkin
a0e77744d4
docs/Cluster-VictoriaMetrics.md: clarify high availability docs 2022-04-08 12:51:57 +03:00
Aliaksandr Valialkin
fbd71f3083
docs/CHANGELOG.md: document backwards-incompatible changes in cluster version of v1.76.0 2022-04-08 12:05:45 +03:00
Dmytro Kozlov
66a03a7fa9
docs/guides: Multi-regional setup with VictoriaMetrics (#2416) 
* docs/guides: Multi-regional setup with VictoriaMetrics

* docs/guides: cleanup
 2022-04-08 11:39:40 +03:00
Aliaksandr Valialkin
dc60e99e94
docs/CHANGELOG.md: document the bugfix in hitCount function 2022-04-08 11:31:52 +03:00
Aliaksandr Valialkin
978f6d0f89
docs/CHANGELOG.md: typo fix 2022-04-07 17:19:59 +03:00
Aliaksandr Valialkin
ef690932ee
docs/CHANGELOG.md: cut v1.76.0 2022-04-07 15:33:55 +03:00
Aliaksandr Valialkin
7da20a4b3f
app/vmagent: reduce the probability of TLS handshake timeout when dialing the remote storage 
The following actions are taken:

- Increase the TLS hashdshake timeout from 5 seconds to 10 seconds
- Increase dial timeout from 5 seconds to 30 seconds
- Specify DialContext instead of Dial in http.Transport. This allows properly handling
  the Context arg during dialing the remote storage

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1699
 2022-04-06 12:34:25 +03:00
Aliaksandr Valialkin
cde1e2ec93
docs/Release-Guide.md: add missing steps 2022-04-06 11:41:09 +03:00
Aliaksandr Valialkin
7ecb72648d
docs/CHANGELOG.md: document 0c0efc7781 2022-04-05 19:21:49 +03:00
Nikolay
0c0efc7781
vmctl verify-blocks command (#2390) 
* lib/protoparser: changes ParseStream for native format
uses reader instead of http.Request
updates app/vmagent and app/vmagent method usage

* app/vmctl: add verify-block subcommand
it allows to check exported from VictoriaMetrics data block in native format
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2362

Update app/vmctl/README.md

Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
 2022-04-05 16:01:32 +02:00
Aliaksandr Valialkin
4ecb86c179
app/vminsert: reduce the max packet size, which vminsert can send to vmstorage 
This reduces the max memory usage for vminsert and vmstorage under heavy ingestion rate
by up to 50% on production workload
 2022-04-05 15:43:07 +03:00
Aliaksandr Valialkin
d011446f6f
docs/CHANGELOG.md: document 70bb0d2708 2022-04-04 13:02:27 +03:00
Roman Khavronenko
70bb0d2708
vmalert: add flag for disabling long-lived connections (keepalive) (#2395) 
The new flag `datasource.disableKeepAlive` allows disabling keepalive
connections. This may be useful if there are multiple datasource
replicas (e.g. vmselects) behind the HTTP balancer to avoid uneven
load spread because of long-lived connections.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
 2022-04-04 12:59:04 +03:00
Aliaksandr Valialkin
43df19a742
docs/CHANGELOG.md: document 173073364e1bb1e0259ddc873dbd96ce62b07543 2022-04-04 12:55:43 +03:00
Artem Navoiev
3d3b9e3b59
Update release process. Actualize helm-charts release process. Add re… (#2397) 
* Update release process. Actualize helm-charts release process. Add release guide for ansible playbooks

* add step to Ansible steps
 2022-04-04 12:14:27 +03:00
Ross Dougherty
0fd4c48568
fix object selectors link (#2391) 
* fix object selectors link

* update kustomize url
 2022-04-01 12:40:41 +03:00
Dima Lazerka
e2b1097545
Fix typo "vmanomapy" 2022-04-01 12:26:59 +03:00
Aliaksandr Valialkin
f977ca8eaf
docs/CHANGELOG.md: document a57e3807537914396ee3eb378648a464fa9e1b97 2022-04-01 12:24:49 +03:00
Aliaksandr Valialkin
1c38ff6f48
docs/CHANGELOG.md: document 0989649ad0 2022-04-01 12:01:34 +03:00
Yurii Kravets
a9b6cf53a2
url-examples (#2389) 2022-04-01 11:23:18 +03:00
Roman Khavronenko
56de8f0356
docs: fix typo in vmalert's API (#2380) 
The API handler was changed in 1.75 but docs
still contain the old address.

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2366
Signed-off-by: hagen1778 <roman@victoriametrics.com>
 2022-03-28 12:07:02 +02:00
Aliaksandr Valialkin
e210384f7e
docs/CHANGELOG.md: cut v1.75.1 2022-03-28 12:28:48 +03:00
Roman Khavronenko
cb878d50fc
docs: update increase_prometheus desc (#2381) 
Remove note that this func is supported by PromQL.
 2022-03-28 11:57:03 +03:00
Roman Khavronenko
3a2a60cb08
docs: escape chars for /label/values endpoint (#2379) 
Without escaping the part wrapped with `<` `>` chars
won't be rendered properly.
 2022-03-28 10:18:37 +02:00
Yury Molodov
c8d29ed78e
vmui: predefined panels (#2243) 
* feat: add basic components for predefined dashboards

* fix: change display alert

* feat: add autosize and unit for axes

* feat: add component for CircularProgress

* feat: change layout for predefined dashboards

* feat: add override step for predefined panels

* feat: add override step for predefined panels

* feat: change yaxis limits for predefined panels

* fix: rename flag for hide legend

* feat: add formatted panel description

* feat: add README.md for dashboard setup

* feat: validate dashboard settings

* feat: add unit for y-ticks

* fix: correct display error for dashboards

* fix: disable auto refresh after route change

* update package-lock.json

* fix: add basename for BrowserRouter

* fix: add dynamic basename for routing

* update packages

* feat: add a pre-defined dashboard "per-job resource usage"

* feat: display unit in the hover-tooltip

* fix: change routing and home layout

* fix: change axis width calc

* updated packages

* app/vmselect: `make vmui-update`

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2022-03-26 13:03:11 +02:00
Aliaksandr Valialkin
6e364e19ef
app/vmselect: add fine-grained limits for the number of returned/scanned time series for various APIs 2022-03-26 11:29:49 +02:00
Dima Lazerka
1fa0f3ec89
VMAnomaly docs fixes (#2361) 
* Added docs for vmanomaly

* Add example images

* Stylistic fixes

* Move images to root

* Update docs/vmanomaly.md

* Update docs/vmanomaly.md

Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>

* Squeeze vmanomaly after vmbackupmanager before Case Studies

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
 2022-03-25 12:08:17 +02:00
Dima Lazerka
7377163659
Add vmanomaly docs section (#2356) 
* Added docs for vmanomaly

* Add example images

* Stylistic fixes

* Move images to root

* Update docs/vmanomaly.md

* Update docs/vmanomaly.md

Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
 2022-03-24 11:56:23 +02:00
Yurii Kravets
c46d9be108
Update url-examples (#2358) 
* Update url-examples

Add federate example

* Update docs/url-examples.md

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2022-03-24 11:53:28 +02:00
Aliaksandr Valialkin
f8dfc22350
docs: a follow-up after 76a477c609 2022-03-23 19:39:08 +02:00
Yurii Kravets
76a477c609
Update Single-server-VictoriaMetrics.md (#2357) 
* Update Single-server-VictoriaMetrics.md

Adding /federate link

* Apply suggestions from code review

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2022-03-23 19:37:23 +02:00
Yurii Kravets
23d0fc220d
Update url-examples.md (#2297) 
* Update url-examples.md

+additional examples

* Update

* Update url-examples

Some changes requested by Roman.

* Update url-examples.md

* Update url-example

* Update url-examples

Additional info and marking for /labels part

* Update url-example

Added example with complex query which needs encoding:
 How to execute the query similar to - sum(increase(foo{status="bar"}[5m])) by (status)

* Update url-samples

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2022-03-22 15:11:43 +02:00
Aliaksandr Valialkin
b421a1f57b
docs/Cluster-VictoriaMetrics.md: clarify mTLS protection docs 2022-03-22 13:55:40 +02:00
Arash Hatami
a8de1ab000
A good change for MD files (#2353) 
* Lint YAML

* Remove extra comment

* Fix command problem

* Format MD files

* Format & fix problem of MD files for docs

* Another fix for MD files
 2022-03-22 13:40:55 +02:00
Roman Khavronenko
f367ff086c
docs: update release notes (#2349) 
Warn about memory issue introduced in releases 1.73 - 1.74
 2022-03-21 15:40:50 +02:00
Aliaksandr Valialkin
5ab6c350ec
docs/CHANGELOG.md: document a1e17e91f8 2022-03-21 15:34:49 +02:00
hagen1778
82659ab5b6 docs: add update note to v1.75.0 release note 
Signed-off-by: hagen1778 <roman@victoriametrics.com>
 2022-03-19 17:55:38 +01:00
Aliaksandr Valialkin
87d356348b
docs/CHANGELOG.md: cut v1.75.0 2022-03-18 19:53:45 +02:00
Aliaksandr Valialkin
e78f3ac8ac
app/vmselect/bufferedwriter: suppress trivial network errors, which can be generated by remote side 
These errors include `broken pipe` and `reset by peer`.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2334
 2022-03-18 19:28:02 +02:00
Aliaksandr Valialkin
ec03dec72d
app/vmagent/remotewrite: prevent from infinite recursion panic when pushing a time series with big number of samples to remote storage 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2335
 2022-03-18 19:06:22 +02:00
Aliaksandr Valialkin
620b605786
docs: document 20bb5e703c 2022-03-18 18:41:19 +02:00
Aliaksandr Valialkin
2ae3a9a8a3
lib/storage: reduce the interval for checking for free disk space from 30 seconds to 1 second 
This should reduce the probability of out of disk space panics when -storage.minFreeDiskSpaceBytes is set to low values.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2305
 2022-03-18 16:52:27 +02:00
Aliaksandr Valialkin
88605a7ea2
lib/blockcache: properly release memory occupied by deleted entries 
Proviously the deleted entries could remain referenced via lastAccessHeap for long time.
This could lead to increased memory usage for the following caches starting from v1.73.0:

* indexdb/indexBlocks
* indexdb/dataBlocks
* storage/indexBlocks

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2242
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2007
 2022-03-18 16:52:27 +02:00
Aliaksandr Valialkin
db781a9342
docs/CHANGELOG.md: document e5868b9c29 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/546
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2255
 2022-03-18 13:07:32 +02:00
Aliaksandr Valialkin
09c6c7350b
docs/vmalert.md: sync after 11ae1ae924 2022-03-17 20:17:40 +02:00
Aliaksandr Valialkin
e6acd16daf
docs/CHANGELOG.md: document 11ae1ae924 2022-03-17 20:08:05 +02:00
Aliaksandr Valialkin
11869a8307
docs: document the addition of mTLS communication between cluster components 2022-03-17 20:00:56 +02:00
Aliaksandr Valialkin
8ae9825bb4
docs/CHANGELOG.md: document c1d07e7c52f0a2ab892921b0639cd42677aa33a8 2022-03-16 14:25:44 +02:00
Aliaksandr Valialkin
54ec080bbc
docs/CHANGELOG.md: document changes from fb6eab03a2 2022-03-16 13:21:57 +02:00
Aliaksandr Valialkin
370024c7ed
docs/CHANGELOG.md: document 565bd08c43 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1824
 2022-03-16 12:50:49 +02:00
Aliaksandr Valialkin
e03e46af4e
docs: make docs-sync 2022-03-16 12:30:02 +02:00
hagen1778
894416b4ca docs: add update details for some releases 
Some of the releases could negatively affect performance for a limited
period of time due to some changes in core. Update details are meant to
warn users about expected changes in peformance after the update.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
 2022-03-15 07:06:48 +03:00
Roman Khavronenko
0fa7effc4b
docs: fix broken links (#2303) 
Signed-off-by: hagen1778 <roman@victoriametrics.com>
 2022-03-13 15:56:01 +02:00
Dmytro Kozlov
565bd08c43
Issue-1824: added flags and different auth types support (#2287) 
* vmalert/notifier: added flags and different auth types support

Co-authored-by: hagen1778 <roman@victoriametrics.com>
 2022-03-10 13:09:12 +02:00
Ted Robertson
ef1afeed6c
Clarify lack of support for InfluxQL (#2273) 2022-03-07 09:49:14 +02:00
Bastien Dronneau
8b21f40217
docs(vmalert): typo in path (#2278) 2022-03-05 22:35:10 +02:00
Aliaksandr Valialkin
74bb9ea734
docs/CHANGELOG.md: cut v1.74.0 2022-03-03 19:30:41 +02:00
Aliaksandr Valialkin
7717967d42
docs: make docs-sync after 702aa4948b 2022-03-03 18:34:06 +02:00
Dan Dascalescu
702aa4948b
docs: clarify Retention, improve English (#2266) 2022-03-03 18:28:46 +02:00
nemobis
df088dd78a
Fix typo, sentence flow in operator description (#2251) 2022-03-03 18:27:12 +02:00
Aliaksandr Valialkin
ce8d28f8f4
docs/CHANGELOG.md: document performance improvements when registering new time series 2022-03-03 17:11:30 +02:00
Aliaksandr Valialkin
e757ebc58b
app/vmselect/netstorage: report vmstorage errors to vmselect clients even if partial responses are allowed 
If a vmstorage is reachable and returns an application-level error to vmselect,
then such error must be returned to the caller even if partial responses are allowed,
since it usually means cluster mis-configuration.

Partial responses may be returned only if some vmstorage nodes are temporarily unavailable.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1941
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/678
 2022-02-25 13:56:42 +02:00
Nikolay
9fe2e4e2c2
fixes incorrect step for calculation for MovingWindow functions (#283) 
* fixes incorrect step for calculation for MovingWindow functions
https://victoriametrics.zendesk.com/agent/tickets/99

* wip

* wip

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2022-02-25 13:54:03 +02:00
Aliaksandr Valialkin
0d79c8cbef
docs/CHANGELOG.md: document cfc6c14dc48ae9dd35e65f1a6e5c7af8ccb9f029 2022-02-25 13:52:48 +02:00
Aliaksandr Valialkin
a16f1ae565
lib/storage: properly handle series selector matching multiple metric names plus a negative filter 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2238

This is a follow-up for 00cbb099b6
 2022-02-24 12:15:54 +02:00
Aliaksandr Valialkin
3f49bdaeff
lib/promrelabel: add support for conditional relabeling via if filter 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1998
 2022-02-24 02:27:26 +02:00
Nikolay
fbac1a9dad
fixes jwt token parse with correct base64Url decoding (#281) 
* fixes jwt token parse with correct base64Url decoding
it must be applied according to jwt RFC that requires token to be URL safe

added slow path for decoding tokens with std base64 decoding

adds error logging for vmgateway

* docs/CHANGELOG.md: document the bugfix

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2022-02-23 13:57:36 +02:00
Aliaksandr Valialkin
62b46007c5
lib/workingsetcache: reduce the default cache rotation period from hour to 20 minutes 
This should reduce memory usage under high time series churn rate
 2022-02-23 13:41:45 +02:00
Aliaksandr Valialkin
acbea6c1ee
docs/CHANGELOG.md: cut v1.73.1 2022-02-22 21:11:42 +02:00
Aliaksandr Valialkin
0a38542a45
docs/CHANGELOG.md: link to the feature request for X-Influxdb-Version response header 
Follow-up for 71ef3155c8

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2209
 2022-02-22 20:35:53 +02:00
Yurii Kravets
75f3db1f5c
Update URL examples (#2207) 
added div with copy function
 2022-02-22 20:08:35 +02:00
Denys Holius
1685e181ae
Added minimal supported version of AlertManager (#2237) 
* added minimal supported version of supported AlertManager

* docs: `make docs-sync`

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2022-02-22 20:07:04 +02:00
Roman Khavronenko
69d1893f4c
Consul SD - update services on the watcher's start (#2202) 
* lib/discovery/consul: update services on the watcher's start

Previously, watcher's start was only initing goroutines for discovery
but not waiting for the first iteration to end. It means first Consul
discovery wasn't returning discovered targets until the next iteration.

The change makes the watcher's start blocking until we get first discovery
iteration done and all registries updated.

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* vmalert: remove workarounds for consul SD

Now when consul SD lib properly updates services
on the first start, we don't need workarounds in vmalert.

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* lib/discovery/consul: update after review

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* wip

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2022-02-21 15:32:45 +02:00
Aliaksandr Valialkin
979d89f5dd
app/vmagent: add -influxDBLabel in the same way as for app/vminsert 
This is a follow-up for 3d19fa6932
 2022-02-21 14:09:20 +02:00
Roman Khavronenko
b6ed9afd6d
lib: allow to configure cache size by type (#2206) 
* lib: allow to configure cache size by type

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1940
Signed-off-by: hagen1778 <roman@victoriametrics.com>

* Apply suggestions from code review

* wip

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2022-02-21 13:50:34 +02:00
Pablo Ley
affaf373ea Added missing inlineScrapeConfig field to the VMAgentSpec docs 2022-02-18 19:46:17 +03:00
Aliaksandr Valialkin
debe75f51c
docs: make docs-sync 2022-02-18 12:24:49 +02:00
Roman Khavronenko
3ac3124eed
docs: make profiling commands more copy-friendly (#2213) 
* docs: make profiling commands more copy-friendly

The change adds `copy text` snippet to code examples
and replaces hostname placeholders with `0.0.0.0`.

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* Apply suggestions from code review

* Apply suggestions from code review

* Apply suggestions from code review

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2022-02-18 12:23:36 +02:00
Aliaksandr Valialkin
71ef3155c8
app/vminsert: add X-Influxdb-Version response header for InfluxDB API requests 
This is needed for some clients, which expect this header.
See https://github.com/ntop/ntopng/issues/5449#issuecomment-1005347597
 2022-02-17 12:47:43 +02:00
Aliaksandr Valialkin
3c3805865b
docs: document 3d19fa6932 2022-02-16 23:30:17 +02:00
Aliaksandr Valialkin
b71be42d90
lib/storage: use binary search instead of full scan for skipping artificial tags when searching for tag names or tag values 
This should improve performance for /api/v1/labels and /api/v1/label/<label_name>/values

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2200
 2022-02-16 18:15:41 +02:00
Aliaksandr Valialkin
21c92d7ef1
docs/Single-server-VictoriaMetrics.md: mention that VictoriaMetrics doesnt check DD_API_KEY when accepting data via DataDog protocol 2022-02-16 14:38:00 +02:00
匠心零度
88a2659f1a
Correct URL:How to delete time series (#2201) 
Co-authored-by: lirenzuo <lirenzuo@shein.com>
 2022-02-16 08:32:57 +02:00
Aliaksandr Valialkin
5748139aa4
docs/url-examples.md: formatting fixes 2022-02-15 21:24:49 +02:00
Aliaksandr Valialkin
424121c126
docs/CHANGELOG.md: document 2efa46a11c 2022-02-15 21:11:56 +02:00
Aliaksandr Valialkin
55facde841
docs/Single-server-VictoriaMetrics.md: clarify that the data is imported into VictoriaMetrics at how to import data section 2022-02-15 21:08:22 +02:00
Aliaksandr Valialkin
ee5da826e9
docs: update -help output for VictoriaMetrics components 2022-02-15 21:08:22 +02:00
Denys Holius
b3e1119592
Update revisions of dashboards (#2199) 
* update revision of vmcluster for operator guide

* update revision of vmcluster for vmcluster guide
 2022-02-15 17:21:42 +02:00
Yurii Kravets
64720d3c03
Create url-examples.md 2022-02-15 16:19:55 +02:00
Oct
6f685c8e43
docs: correct address in the docs (#2189) 
* docs/FAQ.md:fix 404 url

* app/vmui/packages/vmui/README.md:rm 404 url
 2022-02-15 12:58:48 +02:00
Aliaksandr Valialkin
0f9e107e36
docs/CHANGELOG.md: document ad6bdd78d0 2022-02-15 12:47:55 +02:00
Aliaksandr Valialkin
ac502785b6
docs/CHANGELOG.md: cut v1.73.0 2022-02-14 17:54:32 +02:00
Aliaksandr Valialkin
1215f51043
docs/CHANGELOG.md: document 3d890e89f1 2022-02-14 17:39:12 +02:00
Nikolay
75e84144c7
adds release build for macos darwin amd64 and arm64 (#2185) 
* adds release build for macos darwin amd64 and arm64

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1896
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1851

* wip

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2022-02-14 17:28:56 +02:00
Aliaksandr Valialkin
578a37aa14
docs/CHANGELOG.md: document c90c1c4d54 2022-02-14 13:09:12 +02:00
Aliaksandr Valialkin
f10c38b827
lib/promscrape: add expand all and collapse all buttons to /targets page 2022-02-12 18:41:29 +02:00
Aliaksandr Valialkin
b1f94f7f0e
app/vmselect/promql: return at most one time series from absent_over_time() in the same way as Prometheus does 
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2130
 2022-02-12 15:45:09 +02:00
Aliaksandr Valialkin
a56b77db5b
docs/vmagent.md: add unsupported Prometheus config sections chapter 2022-02-12 01:08:10 +02:00
Aliaksandr Valialkin
d8ffbf55a2
docs/CHANGELOG.md: document ea153e5f90 2022-02-12 00:48:06 +02:00
Roman Khavronenko
cf1a8bce6b
lib/index: reduce read/write load after indexDB rotation (#2177) 
* lib/index: reduce read/write load after indexDB rotation

IndexDB in VM is responsible for storing TSID - ID's used for identifying
time series. The index is stored on disk and used by both ingestion and read path.

IndexDB is stored separately to data parts and is global for all stored data.
It can't be deleted partially as VM deletes data parts. Instead, indexDB is
rotated once in `retention` interval.

The rotation procedure means that `current` indexDB becomes `previous`,
and new freshly created indexDB struct becomes `current`. So in any time,
VM holds indexDB for current and previous retention periods.
When time series is ingested or queried, VM checks if its TSID is present
in `current` indexDB. If it is missing, it checks the `previous` indexDB.
If TSID was found, it gets copied to the `current` indexDB. In this way
`current` indexDB stores only series which were active during the retention
period.

To improve indexDB lookups, VM uses a cache layer called `tsidCache`. Both
write and read path consult `tsidCache` and on miss the relad lookup happens.

When rotation happens, VM resets the `tsidCache`. This is needed for ingestion
path to trigger `current` indexDB re-population. Since index re-population
requires additional resources, every index rotation event may cause some extra
load on CPU and disk. While it may be unnoticeable for most of the cases,
for systems with very high number of unique series each rotation may lead
to performance degradation for some period of time.

This PR makes an attempt to smooth out resource usage after the rotation.
The changes are following:
1. `tsidCache` is no longer reset after the rotation;
2. Instead, each entry in `tsidCache` gains a notion of indexDB to which
they belong;
3. On ingestion path after the rotation we check if requested TSID was
found in `tsidCache`. Then we have 3 branches:
3.1 Fast path. It was found, and belongs to the `current` indexDB. Return TSID.
3.2 Slow path. It wasn't found, so we generate it from scratch,
add to `current` indexDB, add it to `tsidCache`.
3.3 Smooth path. It was found but does not belong to the `current` indexDB.
In this case, we add it to the `current` indexDB with some probability.
The probability is based on time passed since the last rotation with some threshold.
The more time has passed since rotation the higher is chance to re-populate `current` indexDB.
The default re-population interval in this PR is set to `1h`, during which entries from
`previous` index supposed to slowly re-populate `current` index.

The new metric `vm_timeseries_repopulated_total` was added to identify how many TSIDs
were moved from `previous` indexDB to the `current` indexDB. This metric supposed to
grow only during the first `1h` after the last rotation.

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1401

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* wip

* wip

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2022-02-12 00:30:08 +02:00
Roman Khavronenko
e3adcbec6e
lib/promscrape: support prometheus-like duration in scrape configs (#2169) 
* lib/promscrape: support prometheus-like duration in scrape configs

The change allows to specify duration values like `1d`, `1w`
for fields `scrape_interval`, `scrape_timeout`, etc.

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/817#issuecomment-1033384766
Signed-off-by: hagen1778 <roman@victoriametrics.com>

* lib/blockcache: make linter happy

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* lib/promscrape: support prometheus-like duration in scrape configs

* add support for extra fields `scrape_align_interval` and `scrape_offset`;
* support Prometheus duration parsing for `__scrape_interval__`
and `__scrape_duration__` labels;

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* wip

* wip

* docs/CHANGELOG.md: document the feature

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2022-02-11 16:17:00 +02:00
Aliaksandr Valialkin
3cb72ccc2a
lib/promscrape/discovery/kubernetes: add __meta_kubernetes_endpointslice_{label,annotation}* labels to be consistent with other role values for Kubernetes service discovery 2022-02-11 14:54:47 +02:00
Nikolay
4e7f7f3302
fixes service discovery for kubernetes (#2173) 
* fixes service discovery for kubernetes
now it must take in account all pods that belong to the discovered endpoint and endpointslice
adds simple test for endpoints
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2134

* wip

* docs/CHANGELOG.md: document the change

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2022-02-11 13:34:22 +02:00
Aliaksandr Valialkin
8f2d03fdc7
docs/CHANGELOG.md: document 4e722c459b 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2167
 2022-02-10 12:20:12 +02:00
Aliaksandr Valialkin
480e40b344
docs/Articles.md: add an article from Asserts.io, which mentions VictoriaMetrics components 2022-02-10 12:16:43 +02:00
Aliaksandr Valialkin
0028b2c6d1
docs/CHANGELOG.md: add instructions on how to build VictoriaMetrics components from source code in order to test tip changes 2022-02-08 16:44:08 +02:00
Nikolay
a8acad7453
adds CGO build for arm64 (#2102) 
* adds CGO build for arm64
it must improve performance for arm64 based deployments of vmstorage and
vmsingle for 15-20%

it depends on gozstd package update for correct musl gozstd vendoring

* typo fixes

* docs/CHANGELOG.md: document the change

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2022-02-08 16:25:59 +02:00
Roman Khavronenko
e855b202df
docs: add Storage section to docs (#2155) 
* docs: add Storage section to docs

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2151

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* Apply suggestions from code review

* docs: `make docs-sync`

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2022-02-08 15:59:17 +02:00
Aliaksandr Valialkin
3e783aa2a1
docs/Cluster-VictoriaMetrics.md: sync with cluster branch 2022-02-08 15:43:21 +02:00
Aliaksandr Valialkin
9bb60ab00f
lib/promscrape: set -promscrape.config.strictParse to true by default 
This allows detecting long-living silent errors in -promscrape.config
 2022-02-08 15:41:43 +02:00
Aliaksandr Valialkin
de26d1ff23
docs/Single-server-VictoriaMetrics.md: add a link to exlanation on how downsampling works 2022-02-08 15:10:41 +02:00
Aliaksandr Valialkin
8b36044c93
docs/CHANGELOG.md: add links to issues, which could benefit from improved re-routing algorithm 2022-02-07 16:55:28 +02:00
Aliaksandr Valialkin
baab622db6
docs: fix links to FAQ.md entries after they have been changed in ccf04239e6 2022-02-07 16:02:44 +02:00
Aliaksandr Valialkin
865f09ecbb
docs: sync with cluster branch 2022-02-07 14:54:16 +02:00
Aliaksandr Valialkin
ba1b3b8ef2
docs: cross-link downsampling docs from deduplication and vmalert docs 2022-02-04 11:57:05 +02:00
Aliaksandr Valialkin
b5b3c585b3
lib/promscrape: show the total number of scrapes and the total number of scrape errors per target at /targets page 
This information may be useful when debugging unreliable scrape targets
 2022-02-03 20:22:41 +02:00
Aliaksandr Valialkin
2968779f16
lib/promscrape: provide the ability to fetch target responses on behalf of vmagent or single-node VictoriaMetrics 
This feature may be useful when debugging metrics for the given target located in isolated environment
 2022-02-03 19:00:55 +02:00
Aliaksandr Valialkin
4b850c2a59
app/vmselect/promql: do not push down filters, which enumerate more than 10k unique values 
Such filters may slow down time series search, so just skip them.

This is a follow-up for e7f1ceeb84

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1827
 2022-02-02 23:40:02 +02:00
Aliaksandr Valialkin
4ef32df4fa
docs/CHANGELOG.md: document 55e3bbd4cc 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1567
 2022-02-02 23:32:35 +02:00
Aliaksandr Valialkin
6530bcedec
docs: updates after 5da71eb685 
* Mention about the ability to configure vmalert notifiers via files in docs/CHANGELOG.md
* Mention about the ability to use Consul service discovery for vmalert notifiers in docs/CHANGELOG.md
* Run `make docs-sync` in order to sync app/vmalert/README.md to docs/vmalert.md
 2022-02-02 23:26:18 +02:00
Pablo Ley
a6587ded51 Added missing links in EmbeddedHPA API docs 2022-02-02 18:59:42 +03:00
Aliaksandr Valialkin
ead66155ef
lib/cgroup: expose process_cpu_cores_available metric 
This metric shows the number of CPU cores available to the process.
This allows creating alerting rules on CPU saturation with the following query:

    rate(process_cpu_seconds_total[5m]) / process_cpu_cores_available > 0.9

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2107
 2022-01-31 20:24:41 +02:00
Aliaksandr Valialkin
e7f1ceeb84
app/vmselect/promql: optimize queries, which join on _info metrics. 
Automatically add common filters from one side of binary operation
to the other side before sending the query to storage subsystem.

See https://grafana.com/blog/2021/08/04/how-to-use-promql-joins-for-more-effective-queries-of-prometheus-metrics-at-scale/
and https://www.robustperception.io/exposing-the-software-version-to-prometheus
 2022-01-31 19:32:36 +02:00
Aliaksandr Valialkin
15475a9d1f
docs/Release-Guide.md: add instructions on how to create signed release tags 2022-01-31 19:18:48 +02:00
Aliaksandr Valialkin
3d8a4bf023
docs/CHANGELOG.md: document 6a519896db 2022-01-31 12:41:57 +02:00
Aliaksandr Valialkin
97947c5fcf
docs/vmauth.md: mention that backend services must be accessible only via vmauth 2022-01-28 12:06:45 +02:00
Denys Holius
527bee4b1e
FAQ update (#2111) 
* FAQ update: how downsampling and deduplication will work at the same time

* Update docs/FAQ.md

Co-authored-by: Aliaksandr Valialkin <valyala@gmail.com>
 2022-01-27 20:35:16 +02:00
jaysony
2e59b17108
correct VMAgentStatus description (#2109) 
correct VMAgentStatus description from VMAlert to VMAgent.
 2022-01-27 20:18:05 +02:00
Aliaksandr Valialkin
e02e0508da
vendor: update github.com/VictoriaMetrics/metricsql from v0.37.0 to v0.38.0 
This adds more optimization cases for https://utcc.utoronto.ca/~cks/space/blog/sysadmin/PrometheusLabelNonOptimization

For example:

* Multi-level transform functions. For example, abs(round(foo{a="b"})) + bar{x="y"}
  is now optimized to abs(round(foo{a="b",x="y"})) + bar{a="b",x="y"}
* Binary operations with `on()`, `without()`, `group_left()` and `group_right()` modifiers.
  For example, foo{a="b"} on (a) + bar is now optimized to foo{a="b"} on (a) + bar{a="b"}
* Multi-level binary operations. For example, foo{a="b"} + bar{x="y"} + baz{z="q"}
  is now optimized to foo{a="b",x="y",z="q"} + bar{a="b",x="y",z="q"} + baz{a="b",x="y",z="q"}
* Aggregate functions. For example, sum(foo{a="b"}) by (c) + bar{c="d"}
  is now optimized to sum(foo{a="b",c="d"}) by (c) + bar{c="d"}
 2022-01-27 19:03:54 +02:00
Aliaksandr Valialkin
a41c34705e
docs/operator: fix links to Operator API docs 2022-01-25 17:38:16 +02:00
Aliaksandr Valialkin
746ee191e8
lib/logger/throttler.go: show the original location of the error and warning message 
Previously the location inside LogThrottler implementation was shown. This could complicate debugging.
 2022-01-23 13:55:00 +02:00
f41gh7
ceb1376267
updates operator docs 2022-01-21 19:03:19 +03:00
Yury Molodov
ad5059f2d3
vmui: fixed display type switching (#2088) 
* fix: correct switch display type

* docs/CHANGELOG.md: document the bugfix

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2022-01-21 16:56:22 +02:00
Yurii Kravets
3ea054a52c
Created Operator folder in docs (#2034) 
* Created Operator folder in docs

Transferred Operator documentation

* Removed Contributing and Release

* Changed sort numbering

* Renamed folder

Operator -> operator

* 1

1

* Name change

Operator -> operator

* Removed colon symbol

* Useful links transformed to links style

* "updated at..." is no longer a header

* delete manager patch.yaml

* delete kustomization.yaml

* removed part with links

* community and contributions part removed

* Delete readme

* Docs navigation removed
 2022-01-21 12:05:58 +02:00
Yury Molodov
adbb821eac
vmui: fix time range selector (#2085) 
* fix: add date validate for time range

* app/vmselect/vmui: `make vmui-update`

* docs/CHANGELOG.md: document the bugfix

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2022-01-21 12:02:38 +02:00
Aliaksandr Valialkin
eb4bd92fac
all: update link to EULA from https://victoriametrics.com/assets/VM_EULA.pdf to https://victoriametrics.com/legal/eula/ 2022-01-20 22:23:27 +02:00
Aliaksandr Valialkin
145337792d
lib/{mergeset,storage}: properly limit cache sizes for indexdb 
Previously these caches could exceed limits set via `-memory.allowedPercent` and/or `-memory.allowedBytes`,
since limits were set independently per each data part. If the number of data parts was big, then limits could be exceeded,
which could result to out of memory errors.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2007
 2022-01-20 18:37:17 +02:00
Aliaksandr Valialkin
84f6b3014c
docs/CHANGELOG.md: document the bugfix for highestMax() function is Graphite render API 2022-01-20 12:16:18 +02:00
Aliaksandr Valialkin
56168d8565
docs/Release-Guide.md: update instructions for exporting snapcraft login to ~/.snap/login.json 2022-01-19 15:20:59 +02:00
Aliaksandr Valialkin
109363de49
docs/CHANGELOG.md: add missing parens in example for @ modifier 2022-01-19 13:04:51 +02:00
Oct
ccf04239e6
docs: Correct some urls in the docs (#2081) 
* docs:update Swift S3 api url

* docs/vmbackup.md: update Swift S3 api url

* docs: update KairosDB docs url

* docs: update thanos docs url
 2022-01-19 11:55:30 +02:00
Aliaksandr Valialkin
98edeac7b7
docs/CHANGELOG.md: fix incorrect link to the issue 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1911
 2022-01-19 00:06:44 +02:00
Aliaksandr Valialkin
8f5902dfcf
docs/CHANGELOG.md: cut v1.72.0 2022-01-18 22:43:18 +02:00
Aliaksandr Valialkin
919ee73153
docs/vmbackup.md: make docs-sync after ca11def2a5 2022-01-18 22:29:17 +02:00
Roman Khavronenko
0b0bf94c96
docs: update retention docs with additional details (#2060) 
Signed-off-by: hagen1778 <roman@victoriametrics.com>
 2022-01-18 22:25:48 +02:00
Yury Molodov
8bdc45ba00
fix: remove buffer period (#2078) 
* fix: remove buffer period

* app/vmselect/vmui: `make vmui-update`

* docs/CHANGELOG.md: document the implemented feature

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2064

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2022-01-18 21:42:56 +02:00