github-mirrors/VictoriaMetrics
1
0
Fork 0
mirror of https://github.com/VictoriaMetrics/VictoriaMetrics.git synced 2024-11-21 14:44:00 +00:00
Code Issues Projects Releases Packages Wiki Activity
3467 commits 150 branches 748 tags 347 MiB
Commit graph

1141 commits

Author SHA1 Message Date
Aliaksandr Valialkin
77168e3e94 docs/vmagent.md: sync with app/vmagent/README.md by running make docs-sync 2021-09-11 11:04:42 +03:00
Aliaksandr Valialkin
9286107e82 lib/promscrape: send stale markers for disappeared metrics like Prometheus does 2021-09-11 10:51:04 +03:00
Aliaksandr Valialkin
cfed015bb6 docs/vmalert.md: typo fix in Multitenancy chapter 2021-09-10 17:57:14 +03:00
Aliaksandr Valialkin
f4dead529f lib/storage: properly search series by multiple tag filters matching empty labels such as foo{bar=~"baz|",x=~"y|"} 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1601
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/395
 2021-09-09 21:09:21 +03:00
Aliaksandr Valialkin
ea943911bc app/vmselect/promql: keep metric name in rollup_candlestick results, since they don't change the original series meaning 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1600
 2021-09-09 19:21:18 +03:00
Aliaksandr Valialkin
f27980dcb3 docs/CHANGELOG.md: typo fix 2021-09-09 18:57:30 +03:00
Aliaksandr Valialkin
4aeb8db83f lib/promscrape: add ability to set series_limit and stream_parse options via relabeling 
This allows managing these options on a per-target basis.

Typical use case: to manage these options for pods via Kubernetes annotations.
 2021-09-09 18:49:39 +03:00
Aliaksandr Valialkin
d2708a1fb7 docs/vmagent.md: typo fix in Relabeling chapter 2021-09-09 16:39:40 +03:00
Aliaksandr Valialkin
d6bd956930 lib/promrelabel: add keep_metrics and drop_metrics actions to relabeling rules 
These actions simlify metrics filtering. For example,

- action: keep_metrics
  regex: 'foo|bar|baz'

would leave only metrics with `foo`, `bar` and `baz` names, while the rest of metrics will be deleted.

The commit also makes possible to split long regexps into multiple lines. For example, the following config is equivalent to the config above:

- action: keep_metrics
  regex:
  - foo
  - bar
  - baz
 2021-09-09 16:18:21 +03:00
Aliaksandr Valialkin
3a827b98cd docs/vmalert.md: make docs-sync after 21f022e5f0 2021-09-09 16:16:25 +03:00
Aliaksandr Valialkin
a8053d9fc6 docs/MetricsQL.md: add a link to VictoriaMetrics github 2021-09-08 00:14:59 +03:00
Aliaksandr Valialkin
e6c9869d86 app/vmalert: follow-up after 21f022e5f0 2021-09-07 22:43:37 +03:00
Aliaksandr Valialkin
6fbaf8f978 docs/CHANGELOG.md: document 42e07cfaea 2021-09-07 22:34:39 +03:00
Aliaksandr Valialkin
e34f7081d0 docs/vmagent.md: add API path for Prometheus text exposition format 2021-09-07 16:14:51 +03:00
Aliaksandr Valialkin
0166eae7c4 docs/Articles.md: add a link to case studies 2021-09-06 10:57:59 +03:00
Aliaksandr Valialkin
5e3ef376b5 docs/Articles.md: add an url to https://www.vultr.com/docs/install-and-configure-victoriametrics-on-debian 2021-09-03 11:49:37 +03:00
Aliaksandr Valialkin
ef27786e37 docs/Single-server-VictoriaMetrics.md: fix a link multitenancy docs for cluster version in VictoriaMetrics 2021-09-02 17:43:35 +03:00
Aliaksandr Valialkin
f529058d3a docs/CHANGELOG.md: cut v1.65.0 2021-09-01 17:12:32 +03:00
Aliaksandr Valialkin
bddd1c35e2 docs/FAQ.md: add questions on how to migrate data from various systems (Prometheus, InfluxDB, OpenTSDB, Graphite) to VictoriaMetrics 2021-09-01 16:47:30 +03:00
Aliaksandr Valialkin
ed818fceef docs: update -help output for victoria-metrics and vmagent after f77dde837a 2021-09-01 16:34:32 +03:00
Aliaksandr Valialkin
f77dde837a lib/promscrape: add the ability to limit the number of unique series per each scrape target 
The number of series per target can be limited with the following options:

* Global limit with `-promscrape.maxSeriesPerTarget` command-line option.
* Per-target limit with `max_series: N` option in `scrape_config` section.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1561
 2021-09-01 16:03:59 +03:00
Aliaksandr Valialkin
111ea89a7d docs: make docs-sync 2021-09-01 12:02:34 +03:00
Aliaksandr Valialkin
9e41b05401 docs/CHANGELOG.md: document eff940aa76 2021-09-01 12:00:02 +03:00
Aliaksandr Valialkin
fce87bfe8d docs/CHANGELOG.md: document 7c70dcbe3b 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1471
 2021-09-01 11:56:23 +03:00
Nikolay
7c70dcbe3b
adds external_labels per group for vmalert (#1485) 
* adds external_label per group for vmalert
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1471
 2021-08-31 14:52:34 +03:00
Aliaksandr Valialkin
6e085e6dac docs/Single-server-VictoriaMetrics.md: remove outdated link to VictoriaMetrics wiki 
VictoriaMetrics wiki became outdated after publishing all the docs at https://docs.victoriametrics.com
 2021-08-31 11:48:32 +03:00
Aliaksandr Valialkin
8b228a5873 docs/CHANGELOG.md: add a link to Prometheus staleness tracking 2021-08-31 11:48:32 +03:00
Aliaksandr Valialkin
5c63d69454 lib/promscrape/discovery/kubernetes: return back support role: endpointslices, since it is used by VictoriaMetrics operator 
This is a follow up commit after 31b42b30b6
 2021-08-29 12:37:03 +03:00
Aliaksandr Valialkin
db330232ac lib/protoparser/opentsdb: follow-up after 8ee75ca45a 2021-08-29 11:49:21 +03:00
Aliaksandr Valialkin
31b42b30b6 lib/promscrape/discovery/kubernetes: rename role: endpointslices to role: endpointslice to be consistent with Prometheus 
See 2ec6c7dbb8/discovery/kubernetes/kubernetes.go (L99)
 2021-08-29 11:23:08 +03:00
Aliaksandr Valialkin
2e001db4de lib/promscrape/discovery/kubernetes: use v1 API instead of v1beta1 API for role: ingress and role: endpointslices 
This should fix service discovery for these roles in Kubernetes v1.22 and newer versions.
See https://kubernetes.io/docs/reference/using-api/deprecation-guide/#ingress-v122

The corresponding change in Prometheus - https://github.com/prometheus/prometheus/pull/9205
 2021-08-29 11:16:59 +03:00
Aliaksandr Valialkin
189507d9d0 docs/Single-server-VictoriaMetrics.md: mention that downsampling doesnt improve query performance on high churn rate 2021-08-27 18:50:26 +03:00
Aliaksandr Valialkin
5ea689d61b app/vmselect/promql: add quantile("phiLabel", phi1, ..., phiN, q) aggregate function to MetricsQL 
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1573
 2021-08-27 18:37:20 +03:00
Aliaksandr Valialkin
bec18e4fe9 app/vmselect: add -search.disableAutoCacheReset command-line option for disabling automatic cache reset when a sample with old timestamp outside -search.cacheTimestampOffset is inserted 
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1570
 2021-08-27 17:15:31 +03:00
Aliaksandr Valialkin
67189be1cb docs/{vmgateway,vmbackupmanager}: mention that enterprise binaries are free for download and evaluation 2021-08-27 14:54:09 +03:00
Aliaksandr Valialkin
321da535fa docs: link to active time series, churn rate and high cardinality questions 2021-08-27 14:44:53 +03:00
Aliaksandr Valialkin
c8c153fb91 docs/CHANGELOG.md: document the bugfix for possible timeout error in vmbackupmanager when making snapshots 
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1571
 2021-08-27 13:04:06 +03:00
Aliaksandr Valialkin
2bc79042f6 docs: mention that enterprise binaries can be downloaded and evaluated for free 2021-08-27 12:48:14 +03:00
Aliaksandr Valialkin
2bef940add docs/vmagent.md: document the ability to load scrape configs from multiple files 
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1559
 2021-08-26 09:13:14 +03:00
Aliaksandr Valialkin
10f960fa0c lib/promscrape: add ability to load scrape configs from multiple files 
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1559
 2021-08-26 08:51:16 +03:00
Aliaksandr Valialkin
e24203cde8 docs/CHANGELOG.md: document 48f33d098b 2021-08-25 13:31:35 +03:00
Aliaksandr Valialkin
9fc9d76a7f docs/Cluster-VictoriaMetrics.md: mention that the -replicationFactor at vmselect is an optional parameter 2021-08-25 13:10:57 +03:00
Aliaksandr Valialkin
c27ee35c5c lib/promscrape: expose promscrape_discovery_http_errors_total metric for tracking errors per each http_sd config 2021-08-25 13:05:49 +03:00
Aliaksandr Valialkin
ffc0ab1774 lib/{mergeset,storage}: improve the detection of the needed free space for background merge 
This should prevent from possible out of disk space crashes during big merges.

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1560
 2021-08-25 09:35:44 +03:00
Aliaksandr Valialkin
a287a48634 docs/FAQ.md: add more entries for frequently asked questions 
The following topics are covered:

* Active time series
* High cardinality
* High churn rate
* Slow inserts
 2021-08-24 11:34:44 +03:00
Aliaksandr Valialkin
8358890e33 docs/MetricsQL.md: typo fix: histogram_qunatile -> histogram_quantile 2021-08-23 23:08:16 +03:00
Aliaksandr Valialkin
7c57745f40 docs/MetricsQL.md: fix the indentation for median function 2021-08-23 12:04:31 +03:00
Aliaksandr Valialkin
a78672f95a docs/MetricsQL.md: typo fix: convesions->conversions 2021-08-23 12:02:03 +03:00
Aliaksandr Valialkin
17a1241022 docs/MetricsQL.md: typo fixes 2021-08-23 11:59:12 +03:00
Aliaksandr Valialkin
60ac3e1e46 docs/MetricsQL.md: rehaul the documentation on MetricsQL 
* Document all the functions supported by MetricsQL, including PromQL functions
* Group functions by their type: rollup functions, transform functions, label manipulation functions and aggregate functions.
* Document implicit query transformations.
 2021-08-23 11:45:52 +03:00
Aliaksandr Valialkin
0078486ea7 app/vmselect/promql: rename sign() function to sgn() in order to be consistent with Prometheus 
See https://github.com/prometheus/prometheus/pull/8457 for details.
 2021-08-23 11:45:51 +03:00
Aliaksandr Valialkin
69c291353b deployment/docker: update Go builder from Go1.16.0 to Go1.17.0 
This improves data ingestion and query performance by up to 5% according to benchmarks.

See https://go.dev/blog/go1.17
 2021-08-21 22:20:49 +03:00
Aliaksandr Valialkin
d5622b32e2 lib/promscrape: reduce memory and CPU usage when Prometheus staleness tracking is enabled for metrics from deleted / disappeared scrape targets 
Store the scraped response body instead of storing the parsed and relabeld metrics.
This should reduce memory usage, since the response body takes less memory than the parsed and relabeled metrics.
This is especially true for Kubernetes service discovery, which adds many long labels for all the scraped metrics.

This should also reduce CPU usage, since the marshaling of the parsed
and relabeld metrics has been substituted by response body copying.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1526
 2021-08-21 21:17:26 +03:00
Aliaksandr Valialkin
2288e75f03 docs/vmalert.md: run make docs-sync after 9ee3d0378f 2021-08-21 20:24:56 +03:00
Aliaksandr Valialkin
6ffbb46aef docs/CHANGELOG.md: document 9ee3d0378f 2021-08-21 20:20:08 +03:00
Aliaksandr Valialkin
c14edc860b docs/CHANGELOG.md: document b5d6a0e499 2021-08-19 14:03:20 +03:00
Aliaksandr Valialkin
80ddade4ed docs/CHANGELOG.md: clarify the change, which adds -search.noStaleMarkers command-line flag 2021-08-19 13:56:04 +03:00
Aliaksandr Valialkin
a724229b5d app/vmselect/promql: do not override [d] at last_over_time(m[d]) if [d] is smaller than scrape_interval 
Since most users do not expect the overriding of explicitly set `[d]`.
 2021-08-19 10:31:48 +03:00
Aliaksandr Valialkin
ce0c270e75 docs/CHANGELOG.md: cut v1.64.1 
This is mostly bugfix release, which includes fixes for staleness handling and a security update for Alpine base image
 2021-08-18 22:06:05 +03:00
Aliaksandr Valialkin
c09446a9aa lib/promscrape: send stale markers for the previously scraped metrics on failed scrapes like Prometheus does 2021-08-18 21:59:03 +03:00
Aliaksandr Valialkin
04c3e9916d docs/CHANGELOG.md: document 06bf21c21b 2021-08-18 14:01:04 +03:00
Aliaksandr Valialkin
cdc372bb98 app/vmselect: add -search.noStaleMarkers command-line flag for disabling stale markers handling in queries 
This option allows reducing CPU usage a bit when VictoriaMetrics is used
for collecting and processing non-Prometheus data. For example, InfluxDB line protocol, Graphite, OpenTSDB, CSV, etc.
 2021-08-18 13:59:02 +03:00
Aliaksandr Valialkin
226143f31b lib/promscrape: add ability to disable sending Prometheus staleness markers with -promscrape.disableStaleMarkers command-line flag 
This option can be useful when vmagent consumes too much additional memory
for staleness markers functionality and when staleness markers aren't needed.
 2021-08-18 13:43:21 +03:00
Aliaksandr Valialkin
db1e62495b app/vmselect/promql: add bitmap_and(), bitmap_or() and bitmap_xor() functions to MetricsQL 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1541
 2021-08-17 13:21:21 +03:00
Aliaksandr Valialkin
277538d655 docs/Single-server-VictoriaMetrics.md: mention that vmctl can migrate data from OpenTSDB to VictoriaMetrics 2021-08-17 11:12:16 +03:00
Aliaksandr Valialkin
03c959f1df lib/promscrape: stop scrapers for the removed targets before starting scrapers for the added targets 
This should prevent from possible time series overlap when old target is substituted by new target (for example, during Kubernetes deployments).

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1526
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1530
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/748
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1509
 2021-08-17 00:55:51 +03:00
Aliaksandr Valialkin
90434ba25b app/vmalert: mention -remoteWrite.disablePathAppend in the description for -remoteWrite.url 2021-08-16 15:22:47 +03:00
Aliaksandr Valialkin
f37b963619 app/vmalert: follow-up for 2400f85761 2021-08-16 15:20:22 +03:00
Aliaksandr Valialkin
4547d4f692 docs/CHANGELOG.md: update urls to Prometheus 2.29 release 
Previously these urls were pointing to rc0 release
 2021-08-16 14:53:38 +03:00
Aliaksandr Valialkin
ae9f923449 docs/CHANGELOG.md: typo fix: satureated -> saturated 2021-08-16 14:53:38 +03:00
Alexander Rickardsson
2400f85761
vmalert: enable configuring explicit path (#1536) 
* vmalert: allow to disable automatically added path to remote write address via disablePathAppend flag
* docs: update docs to include remoteWrite.disablePathAppend
 2021-08-16 14:20:57 +03:00
Aliaksandr Valialkin
c518858145 docs/CHANGELOG.md: cut v1.64.0 2021-08-15 23:52:03 +03:00
Aliaksandr Valialkin
a0e18f06eb lib/promscrape: restore red highlighting for DOWN targets at /targets page 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1461
 2021-08-15 16:03:57 +03:00
Aliaksandr Valialkin
40a859a760 docs/CHANGELOG.md: mention the bugfix when more than 27 time series are selected at /vmui 2021-08-15 15:10:41 +03:00
Aliaksandr Valialkin
386ee5b82c docs/CHANGELOG.md: mention that VMUI automatically fills Server URL field 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1506
 2021-08-15 14:45:09 +03:00
Aliaksandr Valialkin
aefba16d5e app/vmagent/remotewrite: expose vmagent_remotewrite_send_duration_seconds_total metric 
This metric can be used for determining high saturation of every connection to remote storage with
an alerting query `rate(vmagent_remotewrite_send_duration_seconds_total) > 0.9s`.
This query triggers when a connection is satureated by more than 90%
 2021-08-15 13:34:12 +03:00
Aliaksandr Valialkin
4401464c22 all: add support for Prometheus staleness markers 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1526
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/748
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1509
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1530
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/845
 2021-08-13 12:10:17 +03:00
Aliaksandr Valialkin
9a8d1bcec5 docs/Cluster-VictoriaMetrics.md: meniton that vmagent can be used for replicating the data among multiple clusters 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1491
 2021-08-12 12:47:32 +03:00
Denys Holius
81e4d644dd
added guide for HA monitoring setup in K8s via VM Cluster (#1523) 
* added guide for HA monitoring setup in K8s via VM Cluster

* fixed missed divs

* fixed different typos

* Update docs/guides/k8s-ha-monitoring-via-vm-cluster.md

Co-authored-by: Roman Khavronenko <hagen1778@gmail.com>

* Update docs/guides/k8s-ha-monitoring-via-vm-cluster.md

Co-authored-by: Roman Khavronenko <hagen1778@gmail.com>

* Update docs/guides/k8s-ha-monitoring-via-vm-cluster.md

Co-authored-by: Roman Khavronenko <hagen1778@gmail.com>

* Update docs/guides/k8s-ha-monitoring-via-vm-cluster.md

Co-authored-by: Roman Khavronenko <hagen1778@gmail.com>

* Update docs/guides/k8s-ha-monitoring-via-vm-cluster.md

Co-authored-by: Roman Khavronenko <hagen1778@gmail.com>

* Update docs/guides/k8s-ha-monitoring-via-vm-cluster.md

Co-authored-by: Roman Khavronenko <hagen1778@gmail.com>

* Update docs/guides/k8s-ha-monitoring-via-vm-cluster.md

Co-authored-by: Roman Khavronenko <hagen1778@gmail.com>

* Update docs/guides/k8s-ha-monitoring-via-vm-cluster.md

Co-authored-by: Roman Khavronenko <hagen1778@gmail.com>

* Update docs/guides/k8s-ha-monitoring-via-vm-cluster.md

Co-authored-by: Roman Khavronenko <hagen1778@gmail.com>

* Update docs/guides/k8s-ha-monitoring-via-vm-cluster.md

Co-authored-by: Roman Khavronenko <hagen1778@gmail.com>

* Update docs/guides/k8s-ha-monitoring-via-vm-cluster.md

Co-authored-by: Roman Khavronenko <hagen1778@gmail.com>

Co-authored-by: Roman Khavronenko <hagen1778@gmail.com>
 2021-08-11 11:45:42 +03:00
Aliaksandr Valialkin
755f65f4bc app/vminsert: add vm_rpc_send_duration_seconds_total metric per each vminsert->vmstorage link 
This metric is useful for determining high link saturation with the following alerting rule:

rate(vm_rpc_send_duration_seconds_total) > 0.9s
 2021-08-11 11:44:40 +03:00
Aliaksandr Valialkin
869ff25392 docs/Cluster-VictoriaMetrics.md: update -help output for cluster components after the d375d9b878 2021-08-11 11:44:39 +03:00
Aliaksandr Valialkin
bcffd04e3a docs: make docs-sync after e0ee69797d 2021-08-11 10:53:49 +03:00
Aliaksandr Valialkin
d375d9b878 lib/envflag: add a link to docs for -envflag.enable 2021-08-11 10:29:33 +03:00
Aliaksandr Valialkin
5ce531027f docs/CHANGELOG.md: document new metrics added to vmalert at 7416fdaa8b 2021-08-05 10:13:08 +03:00
Aliaksandr Valialkin
c1185363ca app/vmagent: typo fix in the description for -remoteWrite.queues 2021-08-05 10:01:35 +03:00
Aliaksandr Valialkin
d826352688 app/vmagent: follow-up after fe445f753b 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1491
 2021-08-05 09:52:32 +03:00
Aliaksandr Valialkin
f07165977a docs/Articles.md: actualize links and re-order some links 2021-08-03 16:11:49 +03:00
Aliaksandr Valialkin
50663ba41f lib/promscrape/discovery/gce: add __meta_gce_interface_ipv4_<name> labels as in Prometheus 2.29 
See https://github.com/prometheus/prometheus/pull/8978
 2021-08-03 16:11:49 +03:00
Aliaksandr Valialkin
3cad8b4564 lib/promscrape/discovery/ec2: add __meta_ec2_availability_zone_id label as Prometheus 2.29 does 2021-08-03 16:11:49 +03:00
Aliaksandr Valialkin
e92fde7945 app/vmselect/promql: add present_over_time(m[d]) function, which will be available starting from Prometheus 2.29.0 
See https://github.com/prometheus/prometheus/releases/tag/v2.29.0-rc.0 and https://github.com/prometheus/prometheus/pull/9097
 2021-08-03 16:11:49 +03:00
Roman Khavronenko
370fe9fa2a
docs: add "Scaling to trillions of metric data points" to articles (#1517) 2021-08-03 11:07:45 +03:00
Roman Khavronenko
b385fa622b
docs: mention "Push Prometheus metrics to VictoriaMetrics or other exporters" article (#1511) 2021-08-02 17:23:10 +03:00
Roman Khavronenko
a641102ec2
docs: fix indentation for guide articles (#1512) 2021-08-02 17:16:58 +03:00
Aliaksandr Valialkin
66eb60f20d docs/CaseStudies.md: typo fix: hed->had 2021-07-30 18:32:12 +03:00
Aliaksandr Valialkin
8a3c13fd53 docs/CHANGELOG.md: typo fix 2021-07-30 12:35:57 +03:00
Aliaksandr Valialkin
a3b4fc0474 docs/CHANGELOG.md: document d05cac6c98 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1486
 2021-07-30 12:19:53 +03:00
Aliaksandr Valialkin
a1911e1330 app/vmselect/netstorage: unpack time series data in mostly local big chunks 
This should improve performance on multi-CPU systems for queries selecting time series with big number of raw samples
 2021-07-30 12:03:17 +03:00
Aliaksandr Valialkin
192dfbfd90 app/vmselect: follow-up for ed95bc9531 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1493
 2021-07-29 09:53:28 +03:00
Aliaksandr Valialkin
e20ec090b2 docs: remove SampleSizeCalculations.md, since it is outdated and no longer used 
There was a reference to this doc from the old victoriametrics.com site
 2021-07-28 19:25:16 +03:00
Aliaksandr Valialkin
8ee8660ac4 app/vmselect: follow-up for 626073bca8 
* Rename -search.maxMetricsPointSearch to -search.maxSamplesPerQuery, so it is more consistent with the existing -search.maxSamplesPerSeries
* Move the -search.maxSamplesPerQuery from vmstorage to vmselect, so it could effectively limit the number of raw samples obtained from all the vmstorage nodes
* Document the -search.maxSamplesPerQuery in docs/CHANGELOG.md
 2021-07-28 18:00:23 +03:00
Denys Holius
9ffd70a921
Added new guide for monitoring k8s via VictoriaMetrics cluster (#1476) 
* renamed and moved screenshots

* fixed cluster guide, updated helm chart versions, added values.yaml for vm single

* renamed guide files

* fixed typo

* add some fixes

* fixed typos,added guide k8s-monitoring-via-vm-cluster

* added fixes for yamls
 2021-07-27 18:01:12 +03:00
Aliaksandr Valialkin
8481f4f004 docs/CHANGELOG.md: document 9d45b46f4c 2021-07-27 12:38:31 +03:00
Aaron France
fec509fe2d fix: typo in metrics.md docs 2021-07-26 21:53:26 +03:00
Aliaksandr Valialkin
95aff47330 app/vmselect: prevent from possible deadlock when f callback blocks inside RunParallel 2021-07-26 15:47:30 +03:00
Aliaksandr Valialkin
bfb18438ec docs/Articles.md: add links to new articles 2021-07-23 21:06:58 +03:00
Aliaksandr Valialkin
05672ffc32 app/vmselect/promql: properly handle (a op b) default N if (a op b) returns NaN series 
The result should be a series with `N` values and `a op b` labels. Previously such series has been removed from the result.
 2021-07-16 01:44:58 +03:00
Aliaksandr Valialkin
ca75432e66 app/vmselect: do not track queries with less than 1ms execution time at /api/v1/status/top_queries 
This should improve the readability and usefullness of the /api/v1/status/top_queries when debugging slow queries
or queries that take too much cpu time.
 2021-07-15 16:44:28 +03:00
Aliaksandr Valialkin
f4e81aef7e app/vmselect/netstorage: add -search.maxSamplesPerSeries command-line option for limiting the number of samples a query can process per each series 
This should prevent from out of memory crashes like in https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1067
 2021-07-15 16:03:28 +03:00
Aliaksandr Valialkin
171d44acd8 docs/CHANGELOG.md: typo fix: suffxies->suffixes 2021-07-15 15:02:20 +03:00
Aliaksandr Valialkin
61cc13c16f docs/CHANGELOG.md: cut v1.63.0 2021-07-15 14:02:13 +03:00
Aliaksandr Valialkin
682662b2ae lib/storage: remove cache directory if it contains reset_cache_on_startup file 
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1447
 2021-07-13 17:58:51 +03:00
Aliaksandr Valialkin
8d99e94a52 docs: clarify why spare CPU and RAM resources are needed in capacity planning 2021-07-13 15:48:25 +03:00
Aliaksandr Valialkin
8d0ec47be9 docs/Cluster-VictoriaMetrics.md: clarify the docs about the needed values for -dedup.minScrapeInterval at vmselect during replication when the data is pushed from HA pair 2021-07-13 15:43:02 +03:00
Aliaksandr Valialkin
2df66dad7b lib/httpserver: add is_set label to flag metrics 
This label allows determining the set flags with the query `flag{is_set="true"}`
 2021-07-13 15:10:13 +03:00
Aliaksandr Valialkin
a925d5a3e1 app/vmselect/promql: duration handling improvements in MetricsQL queries 
- Support durations anywhere in MetricsQL queries. E.g. sum_over_time(m[1h])/1h is equivalent to sum_over_time(m[1h])/3600
- Support durations without suffix. E.g. rate(m[300]) is equivalent to rate(m[5m])
 2021-07-12 17:16:41 +03:00
Aliaksandr Valialkin
f3a5465ece docs/CHANGELOG.md: document the change from bfba4c28a4 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1444
 2021-07-12 12:44:06 +03:00
Aliaksandr Valialkin
a684408e27 docs: update http://slack.victoriametrics.com to https://slack.victoriametrics.com 2021-07-12 10:57:16 +03:00
Aliaksandr Valialkin
f539772ca6 docs: sync with the cluster branch 2021-07-10 12:45:38 +03:00
Aliaksandr Valialkin
8c764e88f0 app/vmui: move source code from https://github.com/VictoriaMetrics/vmui to app/vmui 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1413
 2021-07-09 17:15:23 +03:00
Aliaksandr Valialkin
2be340a4c9 docs: clarify what does "workload" mean in capacity planning docs 2021-07-09 12:49:53 +03:00
Aliaksandr Valialkin
c5f0b454f0 app/vmselect: follow-up after aa11ef6d3b 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1413
 2021-07-07 17:43:35 +03:00
Aliaksandr Valialkin
92b8380fdf docs/Cluster-VictoriaMetrics.md: improve capacity planning recommendations 2021-07-07 16:22:51 +03:00
Aliaksandr Valialkin
6e0553c92e lib/mergeset: cache indexBlock items only on the second request 
This should reduce the indexdb/indexBlocks cache size, since it won't contain one-time-wonders items.
 2021-07-07 15:23:06 +03:00
Aliaksandr Valialkin
ed944313b0 app/{vminsert,vmselect}: export vminsert_request_duration_seconds and vmselect_request_duration_seconds histograms 2021-07-07 13:25:21 +03:00
Aliaksandr Valialkin
766edbc421 lib/httpserver: print full requestURI in httpserver.Errorf 
This should simplify debugging.
 2021-07-07 13:09:40 +03:00
Aliaksandr Valialkin
c55a64ba08 docs: clarify capacity planning docs 2021-07-07 12:46:55 +03:00
Roman Khavronenko
75f12bfe78
add option to add Copy button for code snippets (#1433) 
To add a Copy button wrap code snippet with the following element:
```
<div class="with-copy" markdown="1">

<your-code-snippet>

</div>
```

See the changes to `Kubernetes monitoring with VictoriaMetrics Single` for details.
 2021-07-06 08:23:39 +03:00
Aliaksandr Valialkin
8f973e34fb lib/workingsetcache: properly switch to whole mode 
Previously the switch from `split` to `whole` mode had been performed too early,
e.g. when the current cache size became bigger than 1/4 of the allowed cache size.

Now it is performed when the current cache size becomes bigger than 1/2 of the allowed cache size.

This change can reduce memory usage for data ingestion path when big number of active time series are ingested.
 2021-07-05 15:16:11 +03:00
Roman Khavronenko
bd6b8f7e31
move github-pages docs to the main repo (#1432) 
* move github-pages docs to the main repo

* rm github actions for copying docs to VictoriaMetrics/VictoriaMetrics.github.io
 2021-07-05 14:34:10 +03:00
Aliaksandr Valialkin
888d62e40c docs/CHANGELOG.md: document the bugfix for vm_merge_need_free_disk_space metric at 9a83e9018d 2021-07-05 12:01:08 +03:00
Aliaksandr Valialkin
6bef227388 docs/Articles.md: add an url to https://medium.com/ibm-garage/monitoring-of-multiple-openshift-clusters-with-victoriametrics-d4f0979e2544 2021-07-05 11:51:37 +03:00
Aliaksandr Valialkin
f6bb130898 app/{vmselect,vmstorage}: clarify the description for -dedup.minScrapeInterval command-line flag 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1426
 2021-07-02 15:05:57 +03:00
Aliaksandr Valialkin
1e52f11e87 docs/Cluster-VictoriaMetrics.md: typo fix: siplify -> simplify 2021-07-02 10:50:55 +03:00
Aliaksandr Valialkin
5beb099ff9 docs/Cluster-VictoriaMetrics.md: add a chapter describing a toy cluster setup on a single host 
While at it, refer to available tools, which can simplify cluster setup
 2021-07-02 10:48:56 +03:00
Aliaksandr Valialkin
158c50c0ee lib/promauth: reload TLS certificates from disk on every mTLS connection as Prometheus does 
This allows updating client certificates without the need to restart vmagent and/or single-node VictoriaMetrics.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1420
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/470
 2021-07-01 15:27:47 +03:00
Aliaksandr Valialkin
02ed4b8b46 docs/CHANGELOG.md: document ae485c2bfd 2021-07-01 11:51:19 +03:00
Aliaksandr Valialkin
c25b839078 lib/workingsetcache: reset the cache mode when the cache is reset 
This should reduce memory usage if the working set is reduced after the cache reset.
 2021-07-01 11:50:11 +03:00
Aliaksandr Valialkin
1b0501a09e lib/promscrape: typo fix in /targets output 
The typo has been introduced in fb72a2133f

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1408
 2021-06-28 21:26:37 +03:00
Aliaksandr Valialkin
3af2162085 docs/vmagent.md: mention about docker_sd_config support 2021-06-25 20:52:15 +03:00
Aliaksandr Valialkin
008033a374 docs/CHANGELOG.md: cut v1.62.0 2021-06-25 13:29:38 +03:00
Aliaksandr Valialkin
cb5453953f lib/promscrape: split docker and dockerswarm service discovery code bases, since they have very little in common 
This is a follow up after c85a5b7fcb
 2021-06-25 13:20:20 +03:00
Aliaksandr Valialkin
4345c07777 docs: consistently put the link to articles and slides about VictoriaMetrics after the links to case studies 2021-06-24 15:37:38 +03:00
Aliaksandr Valialkin
b2fca1ab22 docs/CaseStudies.md: add a case study for DFKI 2021-06-24 15:24:39 +03:00
Aliaksandr Valialkin
906fca9e88 docs/CaseStudies.md: add Groove X case study 2021-06-24 15:04:49 +03:00
Aliaksandr Valialkin
d25a161579 docs/CaseStudies.md: add Sensedia case study 2021-06-24 14:36:13 +03:00
Aliaksandr Valialkin
ca42410afd docs/CHANGELOG.md: document the bugfix in increase_pure() function from the commit fb4f758715 2021-06-24 12:05:39 +03:00
Aliaksandr Valialkin
5d64ed73c5 lib/protoparser/clusternative: do not pool unmarshalWork structs, since they can occupy big amounts of memory (more than 100MB per each struct) 
This should reduce memory usage for vmstorage under high ingestion rate when the vmstorage runs on a system with big number of CPU cores
 2021-06-23 15:46:50 +03:00
Aliaksandr Valialkin
cdfae0117a app/vmselect/promql: return the last timestamp for the max / min value from tmax_over_time() and tmin_over_time() function as most users expect 2021-06-23 14:19:00 +03:00
Aliaksandr Valialkin
70e2852376 docs/CHANGELOG.md: document the bugfix for incorrect stats collection for concurrently executed tag filter 
Follow up for c22114c6f0
 2021-06-23 14:05:28 +03:00
Aliaksandr Valialkin
94f3e40ab3 app/vminsert/netstorage: sort the -storageNode list passed to vminsert nodes 
This should reduce resource usage (CPU, RAM, disk IO) at vmstorage nodes
if the addresses of vmstorage nodes are passed in random order to vminsert nodes.
 2021-06-23 14:01:41 +03:00
Aliaksandr Valialkin
ac54f34f9e lib/promscrape/discovery/http: follow up after e307bbb29a 2021-06-22 13:40:33 +03:00
Aliaksandr Valialkin
755040a171 lib/promscrape/discovery: support generic auth configs in Consul service discovery in the same way as Prometheus 2.28 does 2021-06-22 13:34:02 +03:00
Aliaksandr Valialkin
59e7755df9 docs/CHANGELOG.md: document the support for Consul namepsace 
See 58a2989fe7
 2021-06-22 13:34:02 +03:00
Nikolay
e307bbb29a
adds http_sd (#1399) 
* adds http_sd

* adds X-Prometheus-Refresh-Interval-Seconds header

* Update lib/promscrape/discovery/http/api.go

Co-authored-by: Aliaksandr Valialkin <valyala@gmail.com>
 2021-06-22 13:33:37 +03:00
Aliaksandr Valialkin
3bc83d3b17 docs/PerTenantStatistic.md: document that the per-tenant statistic is a part of cluster version of VictoriaMetrics 2021-06-22 12:43:01 +03:00
Aliaksandr Valialkin
a3262daac0 docs/CHANGELOG.md: typo fixes 2021-06-18 19:14:53 +03:00
Aliaksandr Valialkin
83a4db813e app/vmselect: log slow requests to all the /api/v1/* handlers if their execution time exceeds -search.logSlowQueryDuration 2021-06-18 19:04:42 +03:00
Aliaksandr Valialkin
4cfedc5931 docs/Single-server-VictoriaMetrics.md: mention that it is recommended to use a single scrape_interval across all the scrape targets 2021-06-18 15:39:33 +03:00
Aliaksandr Valialkin
570f36b344
app/vmctl: limit JSON line size by 10K samples (#1394) 
This should reduce the maximum memory usage at VictoriaMetrics when importing time series with big number of samples.
 2021-06-18 15:26:47 +03:00
Aliaksandr Valialkin
eb1af09a04 docs/Cluster-VictoriaMetrics.md: clarify docs about VictoriaMetrics cluster architecture 2021-06-18 14:36:39 +03:00
Aliaksandr Valialkin
cbd9159a22 docs/CHANGELOG.md: document the reduced disk write IO usage 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1338
 2021-06-18 14:02:59 +03:00
Aliaksandr Valialkin
fb72a2133f lib/promscrape: show jobs with empty scrape targets on /targets page 2021-06-18 10:53:52 +03:00
Aliaksandr Valialkin
d8ab409418 docs/{vmgateway,vmbackupmanager}: explicitly mention that these components are a part of an enterprise package 2021-06-17 17:19:49 +03:00
Aliaksandr Valialkin
9eb3fc346f docs/vmagent.md: sync with app/vmagent/README.md via make docs-sync 2021-06-16 12:36:49 +03:00
Aliaksandr Valialkin
6d17a4e12d docs/CHANGELOG.md: document the changed -remoteWrite.queues value 
This is a follow-up for 0a796f7c3a

See https://github.com/VictoriaMetrics/VictoriaMetrics/pull/1385
 2021-06-16 12:35:46 +03:00
Aliaksandr Valialkin
6a8369f0fc docs/Single-server-VictoriaMetrics.md: mention that VictoriaMetrics works great with APM workloads (aka Application Performance Monitoring) 2021-06-15 17:32:52 +03:00
Aliaksandr Valialkin
84fb59b0ba lib/storage: move deletedMetricIDs set from indexDB to Storage 
This makes consitent the list of deleted metricIDs when it is used from both the current indexDB and the previous indexDB (aka extDB).
This should fix the issue, which could lead to storing new samples under deleted metricIDs after indexDB rotation.
See more details at https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1347#issuecomment-861232136 .

Thanks to @tangqipengleoo for the initial analysis and the pull request - https://github.com/VictoriaMetrics/VictoriaMetrics/pull/1383 .

This commit resolves the issue in more generic way compared to https://github.com/VictoriaMetrics/VictoriaMetrics/pull/1383 .

The downside of the commit is the deletedMetricIDs set isn't cleaned from the metricIDs outside the retention. It needs app restart.
This should be OK in most cases.
 2021-06-15 15:04:30 +03:00
Aliaksandr Valialkin
36d55bff66 lib/promscrape: show the number of samples collected during the last scrape at /targets and /api/v1/targets pages 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1377
 2021-06-14 14:04:00 +03:00
Aliaksandr Valialkin
05bc9667c1 docs/CHANGELOG.md: document the addition of DigitalOcean service discovery 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1367
 2021-06-14 13:18:19 +03:00
Nikolay
729c4eeb9c
adds digital ocean sd (#1376) 
* adds digital ocean sd config

* adds digital ocean sd
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1367

* typo fix
 2021-06-14 13:15:04 +03:00
Aliaksandr Valialkin
3e5b6bae66 docs/Cluster-VictoriaMetrics.md: add lists for command-line flags for cluster components 2021-06-14 12:22:05 +03:00
Aliaksandr Valialkin
8a519f1518 docs/vmalert.md: follow-up after 6d5a8c28cd 2021-06-14 11:37:26 +03:00
Aliaksandr Valialkin
2ef2e3d6f8 docs/CHANGELOG.md: cut v1.61.1 2021-06-11 13:01:31 +03:00
Aliaksandr Valialkin
c4f3fbfa5d lib/storage: reset cache on disk during series deletion and during indexdb rotation 
This should prevent from inconsistent behavior (aka partially missing data for some time series) after unclean shutdown.

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1347
 2021-06-11 12:42:28 +03:00
Aliaksandr Valialkin
d979e14da2 docs/CHANGELOG.md: document the bugfix from 7adfe878e1 2021-06-11 11:28:10 +03:00
John Belmonte
67b17cdd68
spelling fix: synonym (#1363) 2021-06-10 08:32:52 +03:00
Aliaksandr Valialkin
b83a51366e docs/CHANGELOG.md: cut v1.61.0 2021-06-09 19:04:44 +03:00
Aliaksandr Valialkin
10d105ee25 docs/FAQ.md: add a chapter comparing VictoriaMetrics to QuestDB 2021-06-09 19:03:23 +03:00
Aliaksandr Valialkin
46d2c7d640 docs/Articles.md: update the broken link to https://nordicapis.com/api-monitoring-with-prometheus-grafana-alertmanager-and-victoriametrics/ 2021-06-09 16:39:56 +03:00
Aliaksandr Valialkin
2422b5091f app/vmauth: improve readability for a config with multiple src_paths 2021-06-09 15:39:32 +03:00
Aliaksandr Valialkin
329b6cd146 docs/CHANGELOG.md: document the enterprise bugfix for the target property in Graphite Render API 2021-06-09 13:50:52 +03:00
Aliaksandr Valialkin
db1c548cb4 docs/CHANGELOG.md: document improvements in re-routing handling in vminsert 
See the following commits:

* 1c09e71f5b
* 0d067eb112
* 2c6b917749
 2021-06-09 13:40:37 +03:00
Aliaksandr Valialkin
f93a31b490 docs/vmagent.md: mention that vmagent supports scrape targets sharding 2021-06-09 12:29:34 +03:00
Aliaksandr Valialkin
ab15bf8c90 docs: document rules replay feature for vmalert 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/836

This is a follow-up for 2a259ef5e7
 2021-06-09 12:27:34 +03:00
Aliaksandr Valialkin
c16edf8287 docs/CHANGELOG.md: document the bugfix, which prevents panics for aborted http requests in vmauth 
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1353

This is a follow-up for 6b29b955c0
 2021-06-09 12:11:53 +03:00
Aliaksandr Valialkin
78f83dc5ad app/{vmagent,vminsert}: follow-up after 2fe045e2a4 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1343
 2021-06-04 20:27:58 +03:00
jelmd
2fe045e2a4
new feature: debug relabeling (#1344) 
* new feature: relabel logging

Use scrape_configs[x].relabel_debug = true to log metric names inkl.
labels before and after relabeling. After relabeling related metrics
get dropped, i.e. not submitted to servers.

* vminsert wants relabel logging, too.
 2021-06-04 17:50:23 +03:00
Aliaksandr Valialkin
5ac25d2585 docs/CHANGELOG.md: document the bugfix from 6f19bb23a1 2021-06-04 11:53:00 +03:00
Aliaksandr Valialkin
f963f04d3d app/vminsert: add -disableRerouting command-line flag for disabling re-routing if some vmstorage nodes have lower performance than the others 2021-06-04 04:42:01 +03:00
Aliaksandr Valialkin
d2d746c4fc docs/CHANGELOG.md: document that it is possible to build VictoriaMetrics components for Solaris 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1322

This is a follow-up for ddc8022702
 2021-05-31 09:33:29 +03:00
Aliaksandr Valialkin
b22e380a34 app/vmauth: allow balancing the load among multiple backend nodes by specifying multiple urls in url_prefix config 2021-05-29 01:03:37 +03:00
Aliaksandr Valialkin
ffa4cd6fa5 docs/Articles.md: add a link to https://www.percona.com/blog/2021/05/26/compiling-a-percona-monitoring-and-management-v2-client-in-arm-raspberry-pi-3/ 2021-05-28 14:34:31 +03:00
Aliaksandr Valialkin
cecbdee00d docs/MetricsQL.md: add a link to technical details about rate() and increase() calculations in Prometheus and VictoriaMetrics 2021-05-28 13:13:50 +03:00
Aliaksandr Valialkin
634eeac804 docs/Single-server-VictoriaMetrics.md: remove misleading wording about querying Graphite metrics with MetricsQL 2021-05-28 02:39:14 +03:00
Aliaksandr Valialkin
60341722d5 docs: document f0c21b6300 2021-05-27 15:03:30 +03:00
Aliaksandr Valialkin
eda6ee40b6 docs: make docs-sync after 2bbb1cc7c1 2021-05-26 12:32:16 +03:00
Roman Khavronenko
2bbb1cc7c1
Docs review (#1330) 
* re-order components by prioritizing Cluster-VictoriaMetrics.md

* drop Home.md since it just duplicates other links
 2021-05-26 12:28:58 +03:00
Aliaksandr Valialkin
7f531e3a60 docs/CHANGELOG.md: document changes from 2233d6ed8a and d210958fd0 2021-05-26 12:23:22 +03:00
Dan Fredell
c78d7dde92
Fix quote difference on label_move example (#1321) 
Fix quote difference on label_move example
 2021-05-24 23:20:38 +03:00
Aliaksandr Valialkin
08234aa7a0 docs/CHANGELOG.md: cut v1.60.0 2021-05-24 15:55:08 +03:00