Commit graph

2377 commits

Author SHA1 Message Date
Aliaksandr Valialkin
2c0c87ee79
docs/Articles.md: add a link to https://victoriametrics.com/blog/mimir-benchmark/ 2022-09-12 10:38:10 +03:00
Yury Molodov
c8de98e03f
vmui: add lists of top queries (#3065)
* feat: add lists of top queries

* fix: change the field label

* refactor: add handlers for readability

* app/vmselect: `make vmui-update`

* docs: document `top queries` tab

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-09-08 21:44:43 +03:00
John Belmonte
eb0b2ef3d9
MetricsQL doc spellcheck (#3080) 2022-09-08 21:28:49 +03:00
Aliaksandr Valialkin
052b527b39
docs/CHANGELOG.md: document ec273eafef
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3076
2022-09-08 21:21:26 +03:00
Aliaksandr Valialkin
f6b3ad0c37
docs/Release-Guide.md: require manual push of the created release tags to public Github
The automated push of release tags to Github require specifying the remote repository name
when doing `git push <remote-repo-name> v1.xx.y`.
The remote repository name can differ in different environments,
so it cannot be put into Makefile rule.

TODO: create a Makefile rule, which generates standard remote names for public
and private repositories in Git, so `git push` for release tags could be automated then.
2022-09-08 15:00:34 +03:00
Aliaksandr Valialkin
f0eea5b02d
app/vmselect/netstorage: fix a typo, which leads to incorrect query results in VictoriaMetrics cluster
The typo has been introduced in the commit 1a254ea20c

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3067
2022-09-08 13:46:40 +03:00
Max Golionko
e07f23a1b9
moved cluster dashboard to master (#3074)
dashboards: move cluster dashboard to master branch

This change should simplify dashboards management.
2022-09-08 11:47:25 +03:00
Aliaksandr Valialkin
6ebe4f5392
docs/Single-server-VictoriaMetrics.md: add a note that cardinality explorer at cluster version of VictoriaMetrics may return lower than expected number of unique label values
See the corresponding comment in the code:

5a6e617b5e/app/vmselect/netstorage/netstorage.go (L1039-L1045)
2022-09-06 14:48:21 +03:00
Aliaksandr Valialkin
5a6e617b5e
docs/Articles.md: add https://tech.bedrockstreaming.com/2022/09/06/monitoring-at-scale-with-victoriametrics.html 2022-09-06 13:33:07 +03:00
Aliaksandr Valialkin
f6990871a3
docs/vmctl.md: make docs-sync after c5261d5f56
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2733
2022-09-06 13:20:16 +03:00
Zakhar Bessarab
ef01f01b6c
vmctl: implement support of chunking data for vm-native export (#3044)
vmctl: implement support of chunking data for vm-native export process

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2733
2022-09-06 13:14:48 +03:00
Craig Rodrigues
d02f09700f
docs: Clarify DataDog examples for VictoriaMetrics cluster (#3048)
Signed-off-by: Craig Rodrigues <rodrigc@crodrigues.org>
2022-09-06 13:12:25 +03:00
Yurii Kravets
b47adb3e3e
docs: Update keyConcepts (#3040)
docs: update keyConcepts

Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
2022-09-06 13:10:26 +03:00
Aliaksandr Valialkin
051e722112
lib/storage: verify that timestamps in block are in the range specified by blockHeader.{Min,Max}Timestamp when upacking the block
This should reduce chances of unnoticed on-disk data corruption.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2998
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3011

This change modifies the format for data exported via /api/v1/export/native -
now this data contains MaxTimestamp and PrecisionBits fields from blockHeader.

This is OK, since the native export format is undocumented.
2022-09-06 13:07:49 +03:00
Aliaksandr Valialkin
b7f3569522
app/vmselect/prometheus: follow-up after 50e2524bc2
- Add getCommonParamsWithDefaultDuration function and use it at /api/v1/series, /api/v1/labels and /api/v1/label/.../values
- Document the default behaviour for setting 5 minutes time range if start arg isn't passed to /api/v1/series, /api/v1/labels and /api/v1/label/.../values
- Document the change at docs/CHANGELOG.md

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/3052
2022-09-05 11:57:08 +03:00
Aliaksandr Valialkin
c6b74148cf
app/vmselect/promql: consistently calculate rate_over_sum(m[d]) as sum_over_time(m[d])/d
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3045
2022-09-02 23:19:05 +03:00
Aliaksandr Valialkin
569843c0dd
docs/CHANGELOG.md: cut v1.81.1 2022-09-02 22:00:34 +03:00
Aliaksandr Valialkin
9cca3a0a1b
app/vmselect/netstorage: fix potential panic under high load
The panic may trigger during data blocks' processing received
from vmstorage nodes when some of vmstorage nodes return an error
or when `-replicationFactor` is set to values higher than 2 at `vmselect`.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3058
2022-09-02 21:36:15 +03:00
Aliaksandr Valialkin
024e2f18da
app/vmselect/promql: evaluate union() args in parallel in order to increase query performance
Note that the parallel execution of `union()` args may take more memory and CPU time
than the sequential execution if args contain heavy queries, which may load all the available CPU,
disk and memory resources and vmselect and vmstorage levels.
2022-09-02 21:01:04 +03:00
Aliaksandr Valialkin
a580d7ac13
docs/Articles.md: add a link to https://aatarasoff.medium.com/optimizing-linkerd-metrics-in-prometheus-de607ec10f6b 2022-08-31 05:03:57 +03:00
Aliaksandr Valialkin
33b2a1df45
docs/FAQ.md: mention that single-node vm is easier to setup and operate than the cluster version of vm 2022-08-31 05:03:56 +03:00
Aliaksandr Valialkin
a11eae3a76
docs/CHANGELOG.md: cut v1.81.0 2022-08-31 02:33:36 +03:00
Max Golionko
0359fce052
simplify release process (#3012)
* simplify release process

* address comments

* address comments

* wip

* wip

* wip

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-08-31 02:28:45 +03:00
Aliaksandr Valialkin
30a1d0c622
docs/CHANGELOG.md: add missing link to vmagent 2022-08-31 02:22:52 +03:00
Craig Rodrigues
6e057faf1b
docs: Add DataDog to list of ingestion methods (#3047)
Signed-off-by: Craig Rodrigues <rodrigc@crodrigues.org>

Signed-off-by: Craig Rodrigues <rodrigc@crodrigues.org>
2022-08-31 00:23:07 +03:00
Aliaksandr Valialkin
cdb22c5ca3
docs/CHANGELOG.md: document v1.79.3 LTS release 2022-08-31 00:15:26 +03:00
Craig Rodrigues
ee988cc71c
docs: add https://github.com/VictoriaMetrics/ansible-playbooks to README.md (#3046)
Signed-off-by: Craig Rodrigues <rodrigc@crodrigues.org>
Co-authored-by: Roman Khavronenko <hagen1778@gmail.com>
2022-08-31 00:13:26 +03:00
Aliaksandr Valialkin
74efe00649
docs: clarify why cluster version is recommended to use for serving high loads 2022-08-30 13:21:43 +03:00
Aliaksandr Valialkin
be5299be2c
docs/FAQ.md: add chapters explaining main differences between single-node and cluster versions of VictoriaMetrics
- Which type of VictoriaMetrics is better to use in production?
- How to migrate data from single-node VictoriaMetrics to cluster?
2022-08-30 13:12:57 +03:00
Artem Navoiev
1f5efd2a4c
update multi-region guide, specify multi-level vmselect option (#3039)
* update multi-region guide, specify multi-level vmselect option

Signed-off-by: Artem Navoiev <tenmozes@gmail.com>

* add more references

* Apply suggestions from code review

Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-08-30 12:34:53 +03:00
Aliaksandr Valialkin
4d5803c16d
docs: fix links to DogStatsD
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3042
2022-08-30 12:19:30 +03:00
Aliaksandr Valialkin
529bcc0761
docs/CHANGELOG.md: document the 044d51b668 2022-08-30 09:42:51 +03:00
Aliaksandr Valialkin
8aaaf221cc
app/vmselect/promql: follow-up after 2d71b4859c
- Use getScalar() function for obtaining the expected scalar from phi arg
- Reduce the error message returned to the user when incorrect phi is passed to histogram_quantiles
- Improve the description of this bugfix in the docs/CHANGELOG.md

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3026
2022-08-27 01:38:17 +03:00
Dmytro Kozlov
2cf29cc4d1
vmselect/promql: fix panic in histogram_quantiles function (#3029)
* vmselect/promql: fix panic in histogram_quantiles function

* Update docs/MetricsQL.md

Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
2022-08-27 01:34:54 +03:00
Aliaksandr Valialkin
86394b4179
lib/promscrape: optimize discoveryutils.SanitizeLabelName()
Cache sanitized label names and return them next time.
This reduces the number of allocations and speeds up the SanitizeLabelName()
function for common case when the number of unique label names is smaller than 100k
2022-08-27 00:18:19 +03:00
Aliaksandr Valialkin
e1bd38fa97
lib/promrelabel: optimize matching for commonly used regex patterns in if option
The following regex patterns are optimized:

- literal string match, e.g. "foo"
- prefix match, e.g. "foo.*" and "foo.+"
- substring match, e.g. ".*foo.*" and ".+foo.+"
- alternate values match, e.g. "foo|bar|baz"
2022-08-26 14:55:13 +03:00
Aliaksandr Valialkin
d60654eb0a
lib/promrelabel: optimize action: {labeldrop,labelkeep,keep,drop} with regex containing alternate values
For example, the following relabeling rule must work much faster now:

- action: labeldrop
  regex: "foo|bar|baz"
2022-08-24 17:55:54 +03:00
Aliaksandr Valialkin
d4cf52f092
app/vmagent: follow-up after 2b22aa1537
- Document the change at docs/CHANGELOG.md
- Move auth token parsing from app/vmagent/opentsdbhttp/ to app/vmagent/main.go,
  since it must be parsed only when multitenancy support is enabled at vmagent side.
  See https://docs.victoriametrics.com/vmagent.html#multitenancy
2022-08-24 16:21:23 +03:00
Dmytro Kozlov
d32a6359b0
vmselect/promql: enable search.maxPointsSubqueryPerTimeseries for sub-queries (#2963)
* vmselect/promql: enable search.maxPointsPerTimeSeriesSubquery for sub-queries

* vmselect/promql: cleanup

* vmselect/promql: rename config flag

* vmselect/promql: add tests

* vmselect/promql: use test object instead of log

* vmselect/promql: fix posible panic is subquery has more points. add description

* vmselect/promql: update tests descriptions

* vmselect/promql: update doInternal validation

* vmselect/promql: fix linter

* vmselect/promql: fix linter

* vmselect/promql: update documentation and release notes

* wip

- Properly apply -search.maxPointsSubqueryPerTimeseries limit to subqueries.
  Previously the -search.maxPointsPerTimeseries limit was unexpectedly applied to subqueries
  if it was smaller than the -search.maxPointsSubqueryPerTimeseries .
- Clarify docs for -search.maxPointsSubqueryPerTimeseries command-line flag .
- Document -search.maxPointsPerTimeseries and -search.maxPointsSubqueryPerTimeseries flags at https://docs.victoriametrics.com/#resource-usage-limits .
- Update docs/CHANGELOG.md .

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2922

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-08-24 15:27:41 +03:00
Aliaksandr Valialkin
e2d8916935
docs: mention that it is safe sharing the collected profiles from security PoV
The collected profiles do not contain sensitive information
2022-08-24 14:08:30 +03:00
Aliaksandr Valialkin
7b9ba456ff
app/vmstorage: expose vm_{hourly,daily}_series_limit_{max,current}_series metrics if -storage.max{Hourly,Daily}Series limits are set
These metrics allow alerting when the number of unique series approach the limit.
For example, the following query alerts when the number of series reaches 90% of the configured limit:

    vm_hourly_series_limit_current_series / vm_hourly_series_limit_max_series > 0.9
2022-08-24 13:41:57 +03:00
Aliaksandr Valialkin
ad6652247c
all: update Google Analytics tracking code from Unversal Analytics to v4
This is needed because Google Analytics devs decided to force their users
to update all their tracking codes to GA v4.

See https://support.google.com/analytics/answer/9744165

The link to the new tracking property ( G-N9SVT8S3HK ) - https://analytics.google.com/analytics/web/?authuser=1#/a129683199p328513681/admin/streams/table/4004610168
2022-08-24 12:16:18 +03:00
Aliaksandr Valialkin
547d655d29
docs/vmagent.md: fix alerting query when scraped samples are dropped because of exceeded series limit
This is a follow-up after 7d26414b2e
2022-08-24 01:23:37 +03:00
Roman Khavronenko
555a202d80
docs: follow-up after 88425bb285 (#3007)
Signed-off-by: hagen1778 <roman@victoriametrics.com>

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-08-24 01:23:37 +03:00
laixintao
76a291a95b
vmalert: add $activeAt into template variables. (#3000)
vmalert: add `$activeAt` template variable for annotations

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2999
2022-08-24 01:23:37 +03:00
Aliaksandr Valialkin
c5325f2db5
docs/CHANGELOG.md: document d59d829cdb
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/2673
2022-08-21 23:37:15 +03:00
Ivan Dudin
c7ca124803
Update Cluster-VictoriaMetrics.md (#3005)
Flags `-search.maxTagKeys` and `-search.maxTagValues` are present at vmstorage and not at vmselect
```
  -search.maxTagKeys int
  -search.maxTagValues int
```
2022-08-21 23:30:15 +03:00
Aliaksandr Valialkin
7f46c17e4e
docs/Cluster-VictoriaMetrics.md: document the best strategies for cluster update / upgrade 2022-08-21 23:20:26 +03:00
Aliaksandr Valialkin
1c7f402598
app/vmagent: add ability to construct a label from multiple existing labels by referring them in the replacement field during relabeling
For example:

- target_label: composite-label
  replacement: {{source_label1}}-{{source_label2}}
2022-08-21 22:49:24 +03:00
Aliaksandr Valialkin
ecd8a03b28
docs: change links to Prometheus docs about instant and range queries to links to VictoriaMetrics docs 2022-08-21 19:02:46 +03:00
Aliaksandr Valialkin
3c597d1985
docs/keyConcepts.md: more fixes 2022-08-21 18:53:49 +03:00
Aliaksandr Valialkin
8e86d52c23
docs/FAQ.md: add a question on differences between Grafana Mimir and VictoriaMetrics 2022-08-21 17:00:49 +03:00
Aliaksandr Valialkin
7df5e19de5
docs/keyConcepts.md - clarify docs a bit 2022-08-21 12:00:53 +03:00
Aliaksandr Valialkin
34d764874f
docs/Cluster-VictoriaMetrics.md: clarify required conditions for cluster availability 2022-08-20 09:39:07 +03:00
Aliaksandr Valialkin
6d2354e7a4
app/vmalert/README.md: sync with docs/vmalert.md after a229182dbe 2022-08-20 08:54:55 +03:00
laixintao
107ac2a85e
doc: fix broken url of template function for vmalert. (#3002) 2022-08-20 08:54:54 +03:00
Aliaksandr Valialkin
76844e519b
docs/CHANGELOG.md: document 10402459d8 2022-08-19 11:44:29 +03:00
Roman Khavronenko
2256d51418
docs: fix docs formatting related to vmalert (#2994)
Signed-off-by: hagen1778 <roman@victoriametrics.com>

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-08-19 11:05:08 +03:00
Aliaksandr Valialkin
84c80b0481
docs/relabeling.md: fix heading for useful tips for target relabeling 2022-08-18 01:33:22 +03:00
Aliaksandr Valialkin
26212aa14a
docs/relabeling.md: typo fixes 2022-08-18 01:30:27 +03:00
Aliaksandr Valialkin
570ce15cb8
docs/relabeling.md: improve relabeling tips for scrape targets 2022-08-18 01:24:35 +03:00
Aliaksandr Valialkin
aed6e33cb4
docs/relabeling.md: add a link to VictoriaMetrics enhancements for relabeling 2022-08-18 01:24:33 +03:00
Aliaksandr Valialkin
a6970e3052
docs: fix ordering after adding the docs/relabeling.md 2022-08-18 01:17:55 +03:00
Aliaksandr Valialkin
29eb9d4602
docs/relabeling.md: add a cookbook for common relabeling tasks 2022-08-18 01:14:30 +03:00
Aliaksandr Valialkin
bbfa52bd75
docs: follow-up after 68e56b6fc5 2022-08-17 21:27:24 +03:00
Roman Khavronenko
cfcb5ab15b
vmalert: set alert's source link to UI instead of JSON source (#2986)
We switch default alert's source link to redirect user
to vmalert's UI instead of previous JSON object. While it breaks
compatibility, it also supposed to improve user's experience.
The old behavior can be achieved by updating `-external.alert.source`
command-line flag.

Signed-off-by: hagen1778 <roman@victoriametrics.com>

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-08-17 21:27:24 +03:00
Aliaksandr Valialkin
d62e9bc964
docs/sd_configs.md: clarify a bit honor_labels config option 2022-08-17 14:24:53 +03:00
Aliaksandr Valialkin
1812d33a2d
lib/promscrape: automatically generate additional per-target labels for targets with non-zero series limit
The following metrics are generated:

- scrape_series_limit
- scrape_series_current
- scrape_series_limit_samples_dropped

These metrics simplify alerting on targets, which expose too many time series

See https://docs.victoriametrics.com/vmagent.html#automatically-generated-metrics
and https://docs.victoriametrics.com/vmagent.html#cardinality-limiter for more details
2022-08-17 13:22:02 +03:00
Roman Khavronenko
0261118765
docs: add new article link (#2989)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-08-17 13:22:02 +03:00
Aliaksandr Valialkin
e5c61cac22
docs/CHANGELOG.md: group vmalert features closer to each other 2022-08-17 11:49:53 +03:00
Aliaksandr Valialkin
aa37e6b438
lib/promscrape: retry http requests if the server returns 429 status code
The 429 status code means that the server is overwhelmed with requests.
The client can retry the request after some wait time.
Implement this strategy for service discovery and scrape requests.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2940
2022-08-16 14:57:26 +03:00
Aliaksandr Valialkin
dc929e0d16
lib/storage: improve performance for /api/v1/labels and /api/v1/label/.../values endpoints when match[] filter matches small number of time series
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2978
2022-08-16 13:34:23 +03:00
Aliaksandr Valialkin
d39915ff8e
docs/MetricsQL.md: add the list of supported ... functions lines just before the corresponding lists
This improves the readability a bit
2022-08-16 12:10:09 +03:00
Roman Khavronenko
436b4d90af
docs: update vmalert docs (#2987)
* mention recently added `$alertID` and `$groupID` variables in the changelog
* properly escape template examples in the vmalert's README

Signed-off-by: hagen1778 <roman@victoriametrics.com>

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-08-16 12:10:08 +03:00
Roman Khavronenko
0d5c403b6e
vmalert: support $alertID and $groupID in template variables (#2983)
Support of these two variables allows building custom URLs with
alert's ID and group ID params.

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/517#issuecomment-1207141432

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-08-16 11:24:01 +03:00
Artem Navoiev
5ae251d074
fix typo in influx guide (#2984) 2022-08-16 11:23:59 +03:00
Aliaksandr Valialkin
7d7cf2b6fd
app/vmselect: follow-up after 63e0f16062
* Explicitly store a pointer to UserReadableError in the error interface.
  Previously Go automatically converted the value to a pointer before storing in the error interface.

* Add Unwrap() method to UserReadableError, so it can be used transparently with the other code,
  which calls errors.Is() and errors.As().

* Document the change in docs/CHANGELOG.md
2022-08-15 13:53:19 +03:00
Roman Khavronenko
79e396dac9
docs: mention default vmalert's behavior change in Update notes (#2981)
Signed-off-by: hagen1778 <roman@victoriametrics.com>

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-08-15 13:31:59 +03:00
Yury Molodov
dc52b283a3
vmui: shortcut keys legend (#2971)
* feat: add shortcut modal

* feat: add shortcut descriptions

* app/vmselect/vmui: `make vmui-update`

* docs/CHANGELOG.md: document the change

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-08-15 12:04:30 +03:00
Yurii Kravets
d7dc7c81e7
doc: add upgrade/downgrade without downtime to FAQ (#2973)
* doc: add upgrade/downgrade without downtime to FAQ

Added info on how to upgrade or downgrade VictoriaMetrics without downtime to FAQ
Based on reply https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2944#issuecomment-1207502038

* Udate pr

Fixing format
2022-08-15 11:31:00 +03:00
Aliaksandr Valialkin
c27bd63f6c
lib/promscrape: update links to sd_configs from Prometheus site to https://docs.victoriametrics.com/sd_configs.html 2022-08-15 01:40:48 +03:00
Aliaksandr Valialkin
1a00c9ef03
lib/promscrape/discovery/kubernetes: add __meta_kubernetes_pod_container_image label in the same way as Prometheus 2.38 does
See https://github.com/prometheus/prometheus/pull/11034
2022-08-15 01:18:57 +03:00
Aliaksandr Valialkin
2fb63dda83
lib/promscrape/discovery/kubernetes: add __meta_kubernetes_service_port_number label to role: service in the same way as Prometheus 2.38 does
See https://github.com/prometheus/prometheus/pull/11002
2022-08-15 01:07:19 +03:00
Aliaksandr Valialkin
198d8eeeaa
app/vmalert/templates: add toTime() template function in the same way as Prometheus 2.38 does
See https://github.com/prometheus/prometheus/pull/10993
2022-08-15 00:49:52 +03:00
Aliaksandr Valialkin
2b58bd9876
lib/promscrape/discovery/dns: add support for resolving MX records
See https://github.com/prometheus/prometheus/pull/10099
2022-08-15 00:33:06 +03:00
Aliaksandr Valialkin
78f584fb0b
docs/CHANGELOG.md: clarify the change at 28441711e6 2022-08-11 23:53:32 +03:00
Roman Khavronenko
36660bcfe2
vmalert: follow-up after 28441711e6 (#2972)
Signed-off-by: hagen1778 <roman@victoriametrics.com>

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-08-11 23:52:18 +03:00
Aliaksandr Valialkin
ec3df0b913
app/vmselect/netstorage: improve scalability of blocks processing on systems with multiple CPU cores
Previously a single syncwg.WaitGroup was used for tracking the lifetime of processBlock callbacks
across all the per-vmstorage goroutines. This could be slow on systems with many CPU cores
because of inter-CPU synchronization overhead.

Use a separate per-vmstorage sync.WaitGroup instead in order to reduce inter-CPU synchronization overhead.
This should imrpove performance for heavy queries over big number of blocks on multi-CPU systems.
2022-08-11 21:37:24 +03:00
Roman Khavronenko
f42853275f
lib/storage: prevent excessive loops when storage is in RO (#2962)
* lib/storage: prevent excessive loops when storage is in RO

Returning nil error when storage is in RO mode results
into excessive loops and function calls which could
result into CPU exhaustion. Returning an err instead
will trigger delays in the for loop and save some resources.

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* document the change

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-08-09 12:17:47 +03:00
Aliaksandr Valialkin
91d0b45537
docs/CHANGELOG.md: cut v1.80.0 2022-08-08 19:59:02 +03:00
Aliaksandr Valialkin
7e892d3322
docs/CHANGELOG.md: add changes for v1.79.2 2022-08-08 18:00:38 +03:00
Aliaksandr Valialkin
b8877da35b
docs/CHANGELOG.md: link to the issue related to vmselect panic in multi-level cluster setup
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2961
2022-08-08 15:09:39 +03:00
Aliaksandr Valialkin
ceb158c5bb
docs/vmagent.md: typo fix in __meta_kubernetes_annotation_prometheus_io_tenant label name
It must be __meta_kubernetes_pod_annotation_prometheus_io_tenant
2022-08-08 14:53:28 +03:00
Aliaksandr Valialkin
310779d8b5
lib/promscrape: follow-up after 2c553d5a2f
- fix broken tests
- cosmetic code cleanup
- document the change at https://docs.victoriametrics.com/vmagent.html#multitenancy
- document the change at https://docs.victoriametrics.com/CHANGELOG.html
2022-08-08 14:49:16 +03:00
Aliaksandr Valialkin
69d62d5736
docs: sync -help output with the latest changes 2022-08-08 14:05:36 +03:00
Aliaksandr Valialkin
221dd3a224
all: bump the minimum supported version of Go from 1.17 to 1.18
This is needed because some dependencies uses generics, which have been appeared in Go1.18

This is a follow-up for caf3dd4fa2
2022-08-08 13:45:39 +03:00
Roman Khavronenko
6af40f6275
vmalert: remove notions of vmalert being compatible with VM only (#2954)
vmalert can be successfully used with datasources
compatible with Prometheus HTTP API. So we remove comments or
notes in Readme which are saying opposite.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-08-08 13:44:15 +03:00
Aliaksandr Valialkin
1996e36cf0
app/vmselect/netstorage: prevent from calling processBlocks callback after the exit from ProcessBlocks function
This should prevent from panic at multi-level vmselect
when the top-level vmselect is configured with -replicationFactor > 1
2022-08-08 13:32:44 +03:00
Aliaksandr Valialkin
12f4c89ff7
docs/CHANGELOG.md: link to the issue with improper handling of enpoint option at ec2_sd_configs
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2917
2022-08-08 03:33:17 +03:00
Aliaksandr Valialkin
9039f23bd1
lib/promrelabel: do not split regex into multiple lines if it contains groups
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2928
2022-08-08 03:16:15 +03:00
Aliaksandr Valialkin
23b7fa897a
docs/CHANGELOG.md: link to the issue regarding the increased load on Consul
This is a follow-up for 68de1f4e4a

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2940
2022-08-08 02:23:01 +03:00
Aliaksandr Valialkin
b5a05f627c
docs/_includes/img.html: fix https://github.com/VictoriaMetrics/VictoriaMetrics/security/code-scanning/28 2022-08-08 01:04:20 +03:00
Yury Molodov
a2ecf311b8
vmui: graph action on moush hold and move (#2915)
* fix: change event for graph panning

* fix: change detect key

* feat: add zoom in with mouse selection

* - document the change
- run `make vmui-update`

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-08-07 23:56:08 +03:00
Aliaksandr Valialkin
7a1a95fd3e
docs/sd_configs.md: mention that http client configs can contain headers and proxy_headers options 2022-08-07 22:41:02 +03:00
Aliaksandr Valialkin
573f6c8d6c
Makefile: remove redundant -mod=vendor option when running Go tools
The `-mod=vendor` is automatically set when there is a `vendor` directory
starting from Go1.14 - see https://go.dev/doc/go1.14#go-command

Since the minimum supported Go version for VictoriaMetrics is Go1.17,
then the `-mod=vendor` option is no longer needed.
2022-08-07 20:40:41 +03:00
Aliaksandr Valialkin
ed891049d5
docs/sd_configs.md: mention that http client options can be specified in scrape_configs section 2022-08-07 00:20:28 +03:00
Aliaksandr Valialkin
5e43145031
docs: add docs for scrape_configs section 2022-08-07 00:04:55 +03:00
Aliaksandr Valialkin
0596bae703
docs/sd_configs.md: add docs for static_configs 2022-08-06 23:17:28 +03:00
Aliaksandr Valialkin
d4c3702afe
docs/sd_configs.md: add docs for openstack_sd_configs 2022-08-06 23:07:11 +03:00
Aliaksandr Valialkin
659f093d87
docs/sd_configs.md: document kubernetes_sd_configs 2022-08-06 22:39:16 +03:00
Aliaksandr Valialkin
77bd4e37cc
lib/promscrape/discovery/kubernetes: add missing __meta_kubernetes_ingress_class_name label for role: ingress
See 7e65ad3e43
and 7e1111ff14
2022-08-06 22:39:14 +03:00
Aliaksandr Valialkin
98deef04f5
docs/sd_configs.md: document http_sd_configs 2022-08-05 19:50:29 +03:00
Aliaksandr Valialkin
8d63e316a9
docs/sd_configs.md: document gce_sd_configs 2022-08-05 19:37:23 +03:00
Aliaksandr Valialkin
ecdd57e13b
docs/sd_configs.md: document file_sd_configs 2022-08-05 19:17:29 +03:00
Aliaksandr Valialkin
6d5c8f82cf
docs/sd_configs.md: document eureka_sd_configs 2022-08-05 19:05:33 +03:00
Aliaksandr Valialkin
091c654c92
docs/sd_configs.md: document ec2_sd_configs 2022-08-05 18:52:39 +03:00
Aliaksandr Valialkin
ecbe1ddf1b
lib/promscrape/discovery/ec2: properly handle custom endpoint option in ec2_sd_configs
This option was ignored since d289ecded1

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1287
2022-08-05 18:52:37 +03:00
Aliaksandr Valialkin
8d6ac974b2
docs/sd_configs.md: add docs for dockerswarm_sd_configs 2022-08-05 16:20:30 +03:00
Aliaksandr Valialkin
80ecfcf759
lib/promscrape/discovery/dockerswarm: properly set __meta_dockerswarm_container_label_* labels instead of __meta_dockerswarm_task_label_* labels
See https://github.com/prometheus/prometheus/issues/9187
2022-08-05 16:20:29 +03:00
Aliaksandr Valialkin
d02ecc71bb
docs/sd_configs.md: document docker_sd_configs 2022-08-05 15:36:52 +03:00
Aliaksandr Valialkin
a931f75d51
docs/sd_configs.md: document dns_sd_configs 2022-08-05 15:27:07 +03:00
Aliaksandr Valialkin
81dc3a34ed
docs/sd_configs.md: document digitalocean_sd_configs 2022-08-05 15:17:09 +03:00
Aliaksandr Valialkin
c4e6973bc8
docs/sd_configs.md: document consul_sd_configs 2022-08-05 15:04:06 +03:00
Aliaksandr Valialkin
85b04732ed
lib/promscrape/discovery/consul: allow stale responses from Consul service discovery by default
This aligns with Prometheus behaviour.

See `allow_stale` option description at https://prometheus.io/docs/prometheus/latest/configuration/configuration/#consul_sd_config
2022-08-05 15:04:05 +03:00
Aliaksandr Valialkin
2c2a5ff553
docs/sd_configs.md: document azure_sd_configs 2022-08-05 14:16:10 +03:00
Aliaksandr Valialkin
df34f8c5b8
docs: fixes after 83a4abda3f 2022-08-05 10:15:28 +03:00
Aliaksandr Valialkin
8ddad31eef
lib/promscrape/discovery/yandexcloud: follow-up after 6e5ac32fba
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1386
2022-08-04 22:28:21 +03:00
Igor Tiunov
0ba86fe87e
YC service discovery (#2923)
* YC service discovery

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1386

* Fixed linter suggestions

* fixed golint errors
2022-08-04 22:28:20 +03:00
Aliaksandr Valialkin
a07891cca6
docs: exporing -> exporting typo fix
This is a follow-up after ccb6cb6501
2022-08-04 20:31:59 +03:00
kevinflynn387
6d909c87f9
Docs: Fix typo from "HA paris" to "HA pairs" (#2935)
* Update README.md:  fix typo to "HA pairs"

Update README.md to fix typo from "HA paris" to "HA pairs"

* Update docs/README.md : Fix of typo from "HA paris"

Update docs/README.md : Fix of typo from "HA paris" to "HA pairs"

* Update of Single-server-VictoriaMetrics.md to fix typo "HA paris" to "HA pairs"

Update of Single-server-VictoriaMetrics.md to fix typo "HA paris" to "HA pairs"
2022-08-04 20:29:02 +03:00
Aliaksandr Valialkin
98caffcfc4
docs: fix the recommended url for -vmalert.proxyURL accroding to 8667307d73 2022-08-04 18:03:24 +03:00
Aliaksandr Valialkin
64183d4416
docs/CHANGELOG.md: document v1.79.1 security fix 2022-08-02 13:31:14 +03:00
Aliaksandr Valialkin
a1e49606ed
app/{vmselect,vmalert}: properly generate http redirects if -http.pathPrefix command-line flag is set
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2918
2022-08-02 13:01:13 +03:00
Aliaksandr Valialkin
f1a49fe779
docs/Release-Guide.md: document that feature docs must be updated with the release where the feature appeared 2022-07-30 00:16:34 +03:00
Aliaksandr Valialkin
962ed46583
lib/mergeset: optimize mergeInmemoryBlocks() function
Do not spend CPU time on converting inmemoryBlock structs to inmemoryPart structs.
Just merge inmemoryBlock structs directly.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2249
2022-07-28 00:05:45 +03:00
Aliaksandr Valialkin
8722342819
docs/CHANGELOG.md: clarify docs about the ability to configure multiple sections for vmauth with identical username and different password values 2022-07-26 19:39:00 +03:00
Aliaksandr Valialkin
86e0abe357
docs/Single-server-VictoriaMetrics.md: improve "Push metrics" docs 2022-07-26 19:34:29 +03:00
Aliaksandr Valialkin
9f1e558c58
all: rename -pushmetrics.extraLabels to -pushmetrics.extraLabel for the sake of consistency 2022-07-26 19:25:26 +03:00
Aliaksandr Valialkin
bf797ba4d9
all: push metrics to -pusmetrics.url in gzip-compressed form in order to reduce the needed network bandwidth 2022-07-25 10:54:54 +03:00
Aliaksandr Valialkin
3d4c312ba2
app/vmselect/netstorage: improve the speed of queries over big number of time series on multi-CPU system
Reduce inter-CPU communications when processing the query over big number of time series.
This should improve performance for queries over big number of time series
on systems with many CPU cores.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2896

Based on b596ac3745
Thanks to @zqyzyq for the idea.
2022-07-25 09:22:28 +03:00
Aliaksandr Valialkin
1b5799f894
lib/promscrape: set up=0 for partially failed scrape in stream parsing mode
This behaviour aligns with Prometheus behavior
2022-07-22 13:38:25 +03:00
Aliaksandr Valialkin
b1129fda0c
docs/vmgateway: typo fix in regexp filter: ~= is substituted with =~ 2022-07-21 22:37:42 +03:00
Aliaksandr Valialkin
cfeb6138ae
docs/BestPractices.md: do not mention NFS
Though VictoriaMetrics works OK on NFS-like filesystems, it is not a best practice
2022-07-21 21:31:14 +03:00
Aliaksandr Valialkin
aae522e501
docs/CHANGELOG.md: document bugfix for https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2874
This is a follow-up for edecd2493c
2022-07-21 21:00:34 +03:00
Aliaksandr Valialkin
3cbe232186
docs/CHANGELOG.md: follow-up after 88edb3f6cf 2022-07-21 20:53:24 +03:00
Roman Khavronenko
d0abdc2b5b
vmalert: allow configuring custom headers per group (#2901)
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2860

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-07-21 20:48:05 +03:00
Roman Khavronenko
356d8b99e0
vmalert: allow configuring custom headers for URLs (#2897)
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2860

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-07-21 20:45:12 +03:00
Aliaksandr Valialkin
db96221cee
docs/Cluster-VictoriaMetrics.md: update after fe68bb3ba7 2022-07-21 20:21:39 +03:00
Aliaksandr Valialkin
fe68bb3ba7
all: follow-up after 46f803fa7a
Add -pushmetrics.* command-line flags to all the VictoriaMetrics apps
2022-07-21 20:18:25 +03:00
Aliaksandr Valialkin
f00a6bf837
all: add ability to push internal metrics to remote storage system specified via -pushmetrics.url 2022-07-21 20:15:29 +03:00
Aliaksandr Valialkin
7a06ee11fd
docs/vmauth.md: mention that multiple recrods for the same username are supported
This is a follow-up for 88029c521c
2022-07-19 19:43:20 +03:00
Nikolay
0102eb3869
app/vmauth: allow duplicate usernames (#2888)
Usernames could be duplicate if it has uniq password.
vmauth makes routing based on auth token and username + password combination must be unique for this case.
2022-07-19 19:35:28 +03:00
Denys Holius
419f913bb9
docs: faq update (#2889)
* docs: faq update

add links for OpenBSD ports & pre built binaries

* Update docs/FAQ.md

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-07-19 17:51:04 +03:00
Aliaksandr Valialkin
f992f96a88
app/vmselect/promql: execute q1 and q2 from q1 op q2 in parallel if labels pushdown cannot be applied
This should improve query performance if VictoriaMetrics has enough resources for processing `q1` and `q2` in parallel.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2886
2022-07-19 14:29:41 +03:00
Aliaksandr Valialkin
69c1272d1b
docs/Articles.md: add a link to "How do We Keep Metrics for a Long Time in VictoriaMetrics" talk 2022-07-19 13:01:39 +03:00
Aliaksandr Valialkin
2d1366353c
lib/promscrape: reload all the scrape configs when the global section is changed inside -promscrape.config
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2884
2022-07-18 17:15:42 +03:00
Aliaksandr Valialkin
d920b0afec
app/vmselect/promql: properly return q1 series from q1 ifnot q2 when q2 returns nothing 2022-07-18 14:24:23 +03:00
Aliaksandr Valialkin
83fef7daf5
docs/CHANGELOG.md: document 2f9668eba5 2022-07-18 12:37:34 +03:00
Aliaksandr Valialkin
979444b4ed
all: fix other typos in the same way as 6f4d9b2a48 does 2022-07-18 12:10:41 +03:00
cui fliter
e0d30f6ec5
fix some typos (#2882)
Signed-off-by: cui fliter <imcusg@gmail.com>
2022-07-18 12:10:40 +03:00
Aliaksandr Valialkin
b2d9379bd0
docs/CHANGELOG.md: mention that v1.79.0 changes binary names to $(APP_NAME)-$(GOOS)-$(GOARCH)... naming 2022-07-18 12:00:45 +03:00
Aliaksandr Valialkin
7669d34931
docs/Single-server-VictoriaMetrics.md: add a link on docs how to send signals to processes in Linux 2022-07-18 10:59:28 +03:00
Aliaksandr Valialkin
fae54e2e56
docs/Troubleshooting.md: clarify some texts there 2022-07-15 12:23:39 +03:00
Aliaksandr Valialkin
0237d987c0
docs/Troubleshooting.md: mention about -search.setLookbackToStep command-line flag in the section explaining why /api/v1/query_range returns calculated data 2022-07-15 11:55:34 +03:00
Aliaksandr Valialkin
0d4676f313
docs/Single-server-VictoriaMetrics.md: mention that the delete API doesnt delete the associated entries from inverted index 2022-07-14 21:29:28 +03:00
Aliaksandr Valialkin
a87f143e92
docs/CHANGELOG.md: cut v1.79.0 2022-07-14 16:15:35 +03:00
Roman Khavronenko
80de6face8
vmalert: drop support of deprecated extra_filter_labels param (#2870)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-07-14 16:10:46 +03:00
Aliaksandr Valialkin
36062d27c5
docs/CHANGELOG.md: formatting fixes for update notes 2022-07-14 12:48:17 +03:00
Aliaksandr Valialkin
e6ee940daf
docs: update descriptions for command-line flags according to the latest changes 2022-07-14 00:57:44 +03:00
Yury Molodov
1ca20caa4b
vmui: optimize table view (#2867)
* feat: optimize table view

* fix: add column display setting

* app/vmselect: `make vmui-update`

Also document the change at docs/CHANGELOG.md

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-07-14 00:16:57 +03:00
Dmytro Kozlov
99402541fb
vmui: update time picker behavior (#2847)
* vmui: update time picker behavior

* docs/CHANGELOG.md: document the change

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-07-14 00:06:22 +03:00
Yury Molodov
23c3b5f8c7
vmui: change selection from autocomplete (#2862)
* fix: change selection from autocomplete

* update docs/CHANELOG.md

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-07-13 23:56:04 +03:00
Nikolay
c007b129cb
lib/promscrape: adds azure service discovery (#2743)
* lib/promscrape: adds azure service discovery
Adds azure service discovery mechanism
implements authorization with oauth and msi
lists virtual machines and virtual machines managed by scaleSet

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1364

* makes linter happy

* Apply suggestions from code review

Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>

* wip

Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-07-13 23:45:43 +03:00
Aliaksandr Valialkin
da6c85a2f6
all: follow-up for d99ba3481b 2022-07-13 17:17:08 +03:00
Dmytro Kozlov
4e4def9df8
Rename release packages (#2810)
* makefile: add os to each release file

* makefile: update vmutils arm64

* makefile: update victoria-metrics release process

* makefile: update publish with os

* makefile: update publish with os

* makefile: change tar library

* update release logic

* copy all releases

* sort command by GOOS

* rollback commands

* rollback OSARCH

* fix commands

* cleanup

* fix windows build

* sort build by GOOS, update README.md
2022-07-13 17:11:01 +03:00
Aliaksandr Valialkin
be5fb139d6
docs/CHANGELOG.md: document 91faa152a5 2022-07-13 12:41:25 +03:00
Aliaksandr Valialkin
205c2a468a
docs/CHANGELOG.md: mention that the communication protocol between vmselect and vmstorage nodes is updated in the new release 2022-07-13 12:16:59 +03:00
Aliaksandr Valialkin
03680b9334
docs: add links to https://docs.victoriametrics.com/CHANGELOG.html in relevant docs 2022-07-11 21:23:51 +03:00
Aliaksandr Valialkin
b95db7045d
docs/Articles.md: add a link to https://habr.com/ru/company/sravni/blog/672908/ 2022-07-11 21:10:59 +03:00
Aliaksandr Valialkin
dd2b1f9ec4
app/vmselect/prometheus: follow-up after 3efe33b917 2022-07-11 20:37:21 +03:00
Dmytro Kozlov
6734d77d97
vmselect/prometeus: Add limit param to api/v1/series api endpoint (#2851)
* issue-2841: Add limit param to api/v1/series api endpoint

* issue-2841: add change log

* issue-2841: update logic

* issue-2841: simplify logic

* issue-2841: simplify logic, add information to documentation
2022-07-11 20:37:20 +03:00
Aliaksandr Valialkin
3662ce8323
docs: make more clear the relation between replication and deduplication
This is based on https://github.com/VictoriaMetrics/VictoriaMetrics/pull/2852
2022-07-11 19:23:24 +03:00
Aliaksandr Valialkin
d442ee4610
app/vmselect/netstorage: optimize mergeSortBlocks function
- Use binary search instead of linear scan when locating the run of smallest timestamps
  in blocks with intersected time ranges. This should improve performance
  when merging blocks with big number of samples

- Skip samples with duplicate timestamps. This should increase query performance
  in cluster version of VictoriaMetrics with the enabled replication.
2022-07-09 00:35:38 +03:00
Roman Khavronenko
78da074ad3
metricsql: properly evaluate timezone_offset over time interval (#2842)
* metricsql: properly evaluate `timezone_offset` over time interval

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2771
Signed-off-by: hagen1778 <roman@victoriametrics.com>

* Update docs/CHANGELOG.md

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-07-08 14:04:40 +03:00
Aliaksandr Valialkin
decbf84e55
app/vmselect/vmui: follow-up after 0bf6841140
* Document the bugfix in docs/CHANGELOG.md
* Run `make vmui-update` for updating static js files for vmui, which are included into vmselect
2022-07-08 13:22:57 +03:00
Aliaksandr Valialkin
d0a943c487
docs/CHANGELOG.md: clarifications after e9b977859b 2022-07-08 13:05:11 +03:00
Aliaksandr Valialkin
a5588bc6f4
docs/CHANGELOG.md: recommend clearing caches after the upgrade from v1.78.0 to v1.78.1 2022-07-08 13:05:11 +03:00
Roman Khavronenko
b29fafa86b
vmalert: deprecate alert's status link (#2840)
* vmalert: deprecate alert's status link

Deprecate alert's status link `/api/v1/<groupID>/<alertID>/status` in favour of
`api/v1/alerts?group_id=<group_id>&alert_id=<alert_id>"`.

The change was needed for simplifying logic in vmselect for proxying vmalert's requests.

The old alert's status link will be still supported for a few versions but will be removed in the future.

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2825
Signed-off-by: hagen1778 <roman@victoriametrics.com>

* vmalert: fix review comments

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-07-08 13:05:11 +03:00
Aliaksandr Valialkin
9ccae66337
docs/CHANGELOG.md: link to the issue related to multi-level vmselect 2022-07-08 05:23:51 +03:00
Aliaksandr Valialkin
e855744871
docs/CHANGELOG.md: recommend upgrading from v1.78.0 to v1.78.1 because of the bug in v1.78.0 2022-07-08 03:45:06 +03:00
Aliaksandr Valialkin
5a5c8534d1
docs/CHANGELOG.md: cut v1.78.1 2022-07-08 01:03:04 +03:00
Aliaksandr Valialkin
69718a6c16
docs/vmagent.md: clarify that vmagent supports Prometheus-compatible service discovery 2022-07-07 21:22:02 +03:00
Aliaksandr Valialkin
2a53b8961e
docs/vmagent.md: typo fixes after ef2eeeb642 2022-07-07 21:17:13 +03:00
Aliaksandr Valialkin
372ecc5c0e
docs/vmagent.md: actualize docs 2022-07-07 20:37:20 +03:00
Aliaksandr Valialkin
1ff2b8ed0e
docs/Single-server-VictoriaMetrics.md: actualize vmui docs according to recent changes 2022-07-07 20:37:20 +03:00
Aliaksandr Valialkin
d8483f599e
docs/CHANGELOG.md: typo fixes 2022-07-07 20:37:20 +03:00
Aliaksandr Valialkin
3d0ce901cb
docs: sync after recent changes 2022-07-07 02:44:24 +03:00
Aliaksandr Valialkin
f03d4b2aa2
docs/CHANGELOG.md: link another bugreport related to the bug with per-day inverted index in v1.78.0 2022-07-07 02:35:35 +03:00
Aliaksandr Valialkin
5794886662
lib/promscrape: properly set Host header when sending requests via http proxy
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2794
2022-07-07 02:28:47 +03:00
Aliaksandr Valialkin
9dd5d3a431
docs/CHANGELOG.md: link to the issue about incorrect per-day index handling in v1.78.0 2022-07-07 02:00:58 +03:00
Aliaksandr Valialkin
95add1e8e4
app/{vmagent,vminsert}: follow-up after d19e46de55
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/2839
2022-07-07 01:32:11 +03:00
Roman Khavronenko
218dfe7956
docs: mention deduplication issue for HA vmalert topology (#2838)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-07-07 01:13:56 +03:00
Aliaksandr Valialkin
4d03ac90fc
lib/promscrape/discovery/kubernetes: properly populate service-level labels for role: endpointslice targets
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2823
2022-07-07 00:36:25 +03:00
Aliaksandr Valialkin
c4cc45d7f8
lib/promscrape/discovery/kubernetes: allow attaching node-level labels to role: endpoints and role: endpointlice targets in the same way as Prometheus does
See https://github.com/prometheus/prometheus/pull/10759
2022-07-07 00:36:24 +03:00
Aliaksandr Valialkin
eee384c9c9
app/vmselect: add -clusternative.tls* options for mTLS setup in multi-level clusters 2022-07-06 13:48:21 +03:00
Aliaksandr Valialkin
195dccf678
app/vmselect: add ability to query vmselect from another vmselect 2022-07-06 13:19:45 +03:00
Aliaksandr Valialkin
ea4ab0df36
docs: sync with single-node branch after ed89106274 2022-07-06 13:02:05 +03:00
Aliaksandr Valialkin
714958deab
all: follow-up after ed89106274 2022-07-06 12:48:52 +03:00
Aliaksandr Valialkin
498c6d6e72
lib/promscrape: push scrape_samples_limit metric to remote storage if sample_limit option is set in scrape_config for this target
See https://github.com/VictoriaMetrics/operator/issues/497
2022-07-06 12:46:23 +03:00
Roman Khavronenko
f51bc07d97
vmselect: allow proxying requests to vmalert (#2833)
The change allows to proxy requests with prefix `/vmalert`
to the vmalert component if `-vmalert.proxyURL` is set.

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2825
and https://github.com/VictoriaMetrics/VictoriaMetrics/pull/2831

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-07-06 10:47:51 +02:00
Aliaksandr Valialkin
78f9a8aafd
lib/storage: put the (date, metricID) entry in dateMetricIDCache just after the corresponding series is registered in the per-day inverted index
Previously the time series could be put into dateMetricIDCache without
registering in the per-day inverted index if GetOrCreateTSIDByName
finds TSID entry in the global index. This could lead to missing
series in query results.

The issue has been introduced in the commit 55e7afae3a,
which has been included in VictoriaMetrics v1.78.0
2022-07-05 14:56:55 +03:00
Aliaksandr Valialkin
8516670582
app/vmselect/promql: properly calculate histogram_quantile over unexpected le buckets
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2819
2022-07-05 13:20:41 +03:00
Artem Navoiev
82581ab728
Docs: Operator Additional Scrape Configuration - update docs (#2826)
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2022-07-05 13:20:41 +03:00
Aliaksandr Valialkin
24fc96dc1f
docs/Troubleshooting.md: add a link to Monitoring chapter added at 41d1834a99 2022-07-04 11:57:14 +03:00
Roman Khavronenko
51209c114a
docs: update Troubleshooting guide (#2809)
Follow-up after a4d9388ecb

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-07-04 11:57:12 +03:00
Roman Khavronenko
78da4b5f5e
docs: warn about potential issue with read queries for 1.78.0 (#2818)
* docs: warn about potential issue with read queries for 1.78.0

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* docs: warn about potential issue with read queries for 1.78.0

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-07-04 11:47:41 +03:00
Aliaksandr Valialkin
f3b905dde1
docs/Cluster-VictoriaMetrics.md: mention about -storage.maxDailySeries and -storage.maxHourlySeries options in resource usage limits chapter 2022-06-30 23:17:14 +03:00
Aliaksandr Valialkin
fa08220d27
app/vmselect/promql: properly handle partial counter resets in rate(), irate(), increase() and remove_resets() functions
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2787
2022-06-30 22:40:26 +03:00
Aliaksandr Valialkin
0f6c17ed06
app/vmagent/remotewrite: clarify descriptions for -remoteWrite.* options, which must be set per each -remoteWrite.url 2022-06-30 20:18:42 +03:00
Aliaksandr Valialkin
7fc03a1deb
app/vmagent/remotewrite: add -remoteWrite.header command-line flag for setting additional http headers to send to -remoteWrite.url
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2805
2022-06-30 20:00:59 +03:00
Dmytro Kozlov
67753cc6f0
vmui: update render logic for nested component (#2795)
* vmui: update render logic for nested component, avoid rerender, remove local storage usage for tracing flag

* docs/url-examples.md: fix various documentation issues there

* docs: add Troubleshooting doc

This doc contains troubleshooting guides for typical problems with VictoriaMetrics.

* docs/Troubleshooting.md: add troubleshooting guide for cluster instability

* wip

* wip

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-06-30 15:48:22 +03:00
Aliaksandr Valialkin
2a70a9296e
docs/Troubleshooting.md: refer to subqueries doc, since this is the most frequent source of improperly consutructed queries 2022-06-30 15:10:02 +03:00
Aliaksandr Valialkin
edf39f9281
docs/Troubleshooting.md: refer to capacity planning docs 2022-06-30 15:00:21 +03:00
Aliaksandr Valialkin
47f3c4b12b
docs/Troubleshooting.md: various typo fixes and clarifications 2022-06-30 14:57:05 +03:00
Aliaksandr Valialkin
7c54cc2123
docs/Troubleshooting.md: formatting fixes 2022-06-30 14:39:15 +03:00
Aliaksandr Valialkin
74338ef300
docs/Troubleshooting.md: add troubleshooting guide for cluster instability 2022-06-30 14:36:18 +03:00
Aliaksandr Valialkin
da3179667e
docs/CHANGELOG.md: document bdf9f4669a 2022-06-30 13:52:44 +03:00
Aliaksandr Valialkin
92e6c55b75
docs: add Troubleshooting doc
This doc contains troubleshooting guides for typical problems with VictoriaMetrics.
2022-06-30 13:52:44 +03:00
Aliaksandr Valialkin
0b76d07c21
docs/url-examples.md: fix various documentation issues there 2022-06-29 11:58:34 +03:00
Aliaksandr Valialkin
daefb64f38
app/vmselect: expose additional histograms at /metrics page, which may help get more insights for the query workload
This commit is based on https://github.com/VictoriaMetrics/VictoriaMetrics/pull/2792
2022-06-28 20:18:31 +03:00
Aliaksandr Valialkin
5e22e6046d
docs/Single-server-VictoriaMetrics.md: mention about -search.maxTagValueSuffixesPerSearch command-line flag in resource limits docs 2022-06-28 18:17:49 +03:00
Aliaksandr Valialkin
7d5d33fd71
lib/storage: return marshaled metric names from SearchMetricNames
Previously SearchMetricNames was returning unmarshaled metric names.
This wasn't great for vmstorage, which should spend additional CPU time
for marshaling the metric names before sending them to vmselect.

While at it, remove possible duplicate metric names, which could occur when
multiple samples for new time series are ingested via concurrent requests.

Also sort the metric names before returning them to the client.
This simplifies debugging of the returned metric names across repeated requests to /api/v1/series
2022-06-28 18:16:32 +03:00
Aliaksandr Valialkin
1af6efd737
docs/Single-server-VictoriaMetrics.md: mention about -search.maxTagValueSuffixesPerSearch command-line flag in resource limits docs 2022-06-27 14:02:01 +03:00
Aliaksandr Valialkin
a38568ddfe
docs/CHANGELOG.md: document 45f20ad1aa 2022-06-27 13:53:26 +03:00
Aliaksandr Valialkin
6386f117c8
all: show timeRange in traces in human-readable format instead of timestamps in milliseconds 2022-06-27 13:42:57 +03:00
Aliaksandr Valialkin
ea2aa617e5
docs/CHANGELOG.md: update after e40d015e9a 2022-06-24 18:03:56 +03:00
Aliaksandr Valialkin
94b81165d0
all: limit the maximum memory usage for regexp cache, which stores parsed regular expressions in MetricsQL queries
Previously the cache could store 10K unique regexps. When every regexp is huge (e.g. hundreds of kilobytes),
then the total cache size could grow to multiples of gigabytes. Now the cache size is limited by the total length
of all cached regexps. So huge regexps won't result in high memory usage for the cache.
2022-06-24 17:58:47 +03:00
hagen1778
e40d015e9a vmselect: make vm_partial_results_total consistent
Metrics `vm_partial_results_total` and `vm_requests_total` serving
the similar purpose, but contain inconsistent set of labels.
This change updates `vm_partial_results_total` labels to be consistent
with `vm_requests_total`.

The change breaks backward compatibility with assumption that
`vm_partial_results_total` wasn't widely used, since it is
not documented and absent in the alerts and dashboards.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-06-24 13:50:26 +02:00
Dmytro Kozlov
f28cbcc7b5
vmui: added query tracing (#2748)
* vmui: added query tracing

* vmui: updated ui

* vmui: update tracing logic, fix bugs, disable tracing by default

* vmui: use empty message as props

* vmui: fixed ui, added delete for each tacing data, show query in header

* vmui: added timelines

* vmui: speedup render

* vmui: use memo for sorting

* vmui: use Trace model, remove unused functions, simplify part of code

* vmui: update recursive logic

* vmui: fix set query to header

* vmui: code cleanup, remove unused code

* vmui: remove unused type, rename component

* wip

* wip

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-06-23 23:00:45 +03:00
Nikolay
ee5c502446
app/vmselect: fixes partial response with replicationFactor (#2777)
* app/vmselect: fixes partial response with replicationFactor
Allow partial response if it meets replicationFactor configured at vmselect
https://t.me/VictoriaMetrics_ru1/38490

* docs/CHANGELOG.md: document this change

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-06-23 20:17:24 +03:00
Yurii Kravets
f0c1edb175
docs: Update CHANGELOG Update notes (#2776)
* docs: Update CHANGELOG Update notes

Specified the reason why `vmselect` and `vmstorage` nodes may log communication errors.
2022-06-23 19:40:20 +03:00
Aliaksandr Valialkin
71b0dfdefa
lib/promscrape: always send stale markers with the real scrape timestamp
This guarantees that query won't return data just after the series is disappeared.
2022-06-23 11:49:13 +03:00
Denys Holius
d7c4b01472
Adds a list of supported architectures (#2769)
* add list of supported architectures

* Update docs/BestPractices.md

Co-authored-by: Aliaksandr Valialkin <valyala@gmail.com>
2022-06-22 22:01:29 +03:00
Aliaksandr Valialkin
3ae6300497
lib/promauth: add ability to send additional http headers in requests to scrape targets
This solves https://stackoverflow.com/questions/66032498/prometheus-scrape-metric-with-custom-header
2022-06-22 20:40:50 +03:00
Aliaksandr Valialkin
7429dfbe9e
app/vmselect: add -search.setLookbackToStep command-line flag for making the gap filling algorithm similar to InfluxDB data model
This option should override `-search.maxStalenessInterval` for most cases when users migrate from InfluxDB to VictoriaMetrics
2022-06-22 14:20:02 +03:00
Aliaksandr Valialkin
3e0f364489
docs/Cluster-VictoriaMetrics.md: small fixes 2022-06-22 13:42:07 +03:00
Roman Khavronenko
54f0f2d384
docs: follow-up for 197d3cdd74 (#2766)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-06-22 13:18:03 +03:00
云原生驿站
67e5833ced
docs: supplement vmalert downsampling docs (#2765)
Co-authored-by: 吴典秋 <muti_kube@163.com>
2022-06-22 13:18:03 +03:00
Denys Holius
432b261c13
url-examples: added curl output after deleting metrics (#2764)
docs: add more details to url-examples for series deleting
2022-06-21 17:56:00 +03:00
Aliaksandr Valialkin
9362da2e7f
docs: update -help output for vmbackup, vmbackupmanager, vmgateway and vmrestore components 2022-06-21 15:49:26 +03:00
Aliaksandr Valialkin
597bce4f55
docs: update docs after e4d6b750f6
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2753
2022-06-21 14:01:25 +03:00
Aliaksandr Valialkin
24097f2417
docs/CHANGELOG.md: cut v1.78.0 2022-06-20 18:11:04 +03:00
Aliaksandr Valialkin
a1629bd3be
lib/netutil.ConnPool: skip dialing remote address if the previous dial attempt was unsuccessful
If the previous dial attempt was unsuccessful, then all the new dial attempts are skipped
until the background goroutine determines that the given address can be successfully dialed.

This reduces query latency when some of vmstorage nodes are unavailable and dialing them is slow.

This should help with https://github.com/VictoriaMetrics/VictoriaMetrics/issues/711

This commit is based on ideas from the https://github.com/VictoriaMetrics/VictoriaMetrics/pull/2756

The main differences are:

- The check for healthy/unhealthy storage nodes is moved one level lower from app/vmselect/netstorage to lib/netutil.ConnPool.
  This makes possible re-using this feature everywhere lib/netutil.ConnPool is used.
- The check doesn't take into account handshake errors for already established connections.
  Handshake errors usually mean improperly configured VictoriaMetrics cluster, so they shouldn't be ignored.
2022-06-20 17:33:54 +03:00
Aliaksandr Valialkin
45e9732764
docs: follow-up after e4d6b750f6 2022-06-20 17:15:52 +03:00
Nikolay
15662c0f29
lib/httpserver: adds flagsAuthKey command-line flag (#2758)
* lib/httpserver: adds flagsAuthKey command-line flag
It protects /flags endpoint with authKey.

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2753O

* Apply suggestions from code review

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-06-20 17:15:51 +03:00
Aliaksandr Valialkin
0abf46d66a
docs/Articles.md: add a link to https://www.sobyte.net/post/2022-05/victoriametrics-bloomfilter/ 2022-06-20 14:44:11 +03:00
Roman Khavronenko
3ada676879
docs: reference links from key concepts (#2745)
Signed-off-by: hagen1778 <roman@victoriametrics.com>

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-06-19 23:14:30 +03:00
Artem Navoiev
42c69ae74e
docs: replace bash code block type with console (#2746) 2022-06-19 23:02:00 +03:00
Aliaksandr Valialkin
4ae1c5655f
docs/CHANGELOG.md: document ef7f52e0e6 2022-06-19 22:49:11 +03:00
Aliaksandr Valialkin
88e1221b35
lib/storage: do not register new series if -storage.maxHourlySeries or -storage.maxDailySeries limits are exceeded
Previously samples for new series weren't added as expected when series limits were reached,
but new series were still registered in indexdb.
2022-06-19 22:03:02 +03:00
Aliaksandr Valialkin
c5ac176153
lib/storage: reset metric id caches for the previous and the current hour
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2698
2022-06-19 22:02:51 +03:00
Aliaksandr Valialkin
149c5c9381
docs: update docs on how to add tags to metrics collected by DataDog agent
Follow-up for f16072c3c1
2022-06-17 13:12:21 +03:00
Dmytro Kozlov
5bc13e2fe8
vmui: added focusLabel, enable cardinality app configuratior (#2736)
* vmui: added focusLabel, enable app configuratior

* vmui: set focusLabel if {labelName!=""}

* wip

* docs/CHANGELOG.md: mention about focusLabel feature in cardinality explorer

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2730

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-06-17 13:04:09 +03:00
Dmytro Kozlov
82440e76c5
doc: added workaround for datadog agent (#2712)
* Added workaround for datadog agent

* docs: update datadog workaround

* doc: update doc description

* Apply suggestions from code review

* docs: `make docs-sync`

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-06-16 22:24:17 +03:00
Roman Khavronenko
287bb96758
docs: mention sandbox update in release procedure (#2724)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-06-16 20:51:41 +03:00
Roman Khavronenko
cbe39bbb8d
vmselect: limit end param max value by 2d in future (#2729)
* vmselect: limit `end` param max value by 2d in future

The change is applied only to service handlers like `/labels` or `/series`
and limits the `end` param by max value <= now() + 2 days. The same limit
is applied for the ingested data, so no reason to allow to request data
in future far than that.

The change is also needed for corner cases like https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2669
where too high `end` value triggers inefficient global index search.

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* docs/CHANGELOG.md: document the bugfix

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-06-16 20:51:40 +03:00
Aliaksandr Valialkin
cda3ce09e5
docs/vmagent.md: typo fix: configued -> configured 2022-06-16 20:51:40 +03:00
Aliaksandr Valialkin
450aa0ae5a
lib/promrelabel: support action: graphite relabeling
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2737
2022-06-16 20:25:49 +03:00
Roman Khavronenko
ba7ece02c4
docs: add multiple-remote-writes topology to vmalert (#2738)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-06-16 20:21:12 +03:00
Aliaksandr Valialkin
b4e75a0b89
docs/CHANGELOG.md: document dd327bfa9e2c69fe21ab1d92c14636733d7c5620 2022-06-15 18:41:26 +03:00
Aliaksandr Valialkin
14ce7b0e25
docs/CHANGELOG.md: document 00719e5779a3e4eeedb74cb3d25a9ecfe0e16063 2022-06-15 18:09:23 +03:00
Aliaksandr Valialkin
45fa9d798d
app/vmselect: accept focusLabel query arg at /api/v1/status/tsdb 2022-06-14 18:39:00 +03:00
Aliaksandr Valialkin
fb77843639
lib/storage: show top labels with the highest number of series in cardinality explorer 2022-06-14 16:34:13 +03:00
Aliaksandr Valialkin
ce953d5e95
docs: update command-line flags' descriptions according to recent changes 2022-06-14 13:28:23 +03:00
Roman Khavronenko
0065b13243
docs: reduce free disk recommendation from 30% to 20% (#2728)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-06-14 12:55:41 +03:00
Aliaksandr Valialkin
12618925fc
docs/guides/migrate-from-influx.md: suggest more real-world value for -search.maxStalenessInterval
Suggest `-search.maxStalenessInterval=10s` instead of `-search.maxStalenessInterval=1ms`,
since `1ms` would result in empty graphs in most cases, since the interval between data points
on the graph is usually much higher than 1ms. For example, if the graph shows time range of one hour
and it contains 1000 points, then the interval between points on the graph would equal to
3600s/1000=3.6 seconds.
2022-06-14 12:51:35 +03:00
Roman Khavronenko
9873001539
docs: migrating from influx (#2720)
Mention `-search.maxStalenessInterval` flag and its effect on query engine.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-06-14 12:51:33 +03:00
Aliaksandr Valialkin
b48ca34e05
docs/Single-server-VictoriaMetrics.md: recommend running all the VictoriaMetrics components behind auth proxy in Security chapter 2022-06-13 10:30:52 +03:00
Aliaksandr Valialkin
c846d0db01
docs/CHANGELOG.md: document 99dbe7f9d4 2022-06-13 10:07:19 +03:00
Roman Khavronenko
81b6efea9e
Readme cleanup (#2715)
* docs: minor styling and wording changes

Changes made after reading https://developers.google.com/tech-writing

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* docs: set proper types for code blocks

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* docs: add `copy` wrapper for some commands

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* docs: sync

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* docs: resolve conflicts

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-06-13 09:49:59 +03:00
Yury Molodov
65107a4585
vmui: enhancements (#2638) (#2717)
* feat: make datepicker to be set to last 30 min by default

* fix: correct spinner while loading data

* feat: change legend style

* app/vmselect: `make vmui-update`

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-06-13 09:49:58 +03:00
Aliaksandr Valialkin
9d8c37d280
docs/CHANGELOG.md: refer to the issue, which should be solved after the optimization to /api/v1/labels and /api/v1/label/.../values is added
The optimization has been added in 374beb350e

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1533
2022-06-12 14:32:57 +03:00
Aliaksandr Valialkin
61e03f172b
app/vmselect: optimize /api/v1/labels and /api/v1/label/.../values handlers when match[] query arg is passed to them 2022-06-12 14:06:24 +03:00
Aliaksandr Valialkin
4a94cd81ce
app/vmselect: add optional limit query arg to /api/v1/labels and /api/v1/label_values endpoints
This arg allows limiting the number of sample values returned from these APIs
2022-06-10 10:24:07 +03:00
Roman Khavronenko
3c09d25039
vmalert: followup for 76f05f8670 (#2706)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-06-09 13:15:35 +03:00
Howie
4afd7aa695
feat: rule limit (#2676)
vmalert: support `limit` param in groups definition

`limit` param limits number of time series samples produced by a single rule
during execution.
On reaching the limit rule will return an err.

Signed-off-by: lihaowei <haoweili35@gmail.com>
2022-06-09 13:15:33 +03:00
Aliaksandr Valialkin
a9ea3fee38
lib/querytracer: make it easier to use by passing trace context message to New and NewChild
The context message can be extended by calling Donef.
If there is no need to extend the message, then just call Done.
2022-06-08 21:16:12 +03:00
Aliaksandr Valialkin
5524d13a03
docs/Single-server-VictoriaMetrics.md: explain why free disk space shortage may negatively impact VictoriaMetrics performance 2022-06-08 20:04:30 +03:00
Aliaksandr Valialkin
2520b8efb8
docs: add a link to cardinality explorer playground 2022-06-08 19:40:12 +03:00
Aliaksandr Valialkin
8d98b8ae10
docs: refer to cardinality explorer
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2233
2022-06-08 19:18:31 +03:00
Dmytro Kozlov
f2754c3e90
Cardinality explorer (#2625)
* Cardinality explorer

* vmui, vmselect: updated field name, added description to spinner

* make vmui-update

* updated const name, make vmui-update

* lib/storage: changes calculation for totalSeries values

* added static files

* wip

* wip

* wip

* wip

* docs/CHANGELOG.md: document cardinality explorer feature

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2233

Co-authored-by: f41gh7 <nik@victoriametrics.com>
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-06-08 18:54:27 +03:00
Aliaksandr Valialkin
ba406ff28c
docs/CHANGELOG.md: document 63b538ecd1 2022-06-07 15:53:08 +03:00
Aliaksandr Valialkin
c07f99cd4f
docs/CHANGELOG.md: document https://github.com/VictoriaMetrics/VictoriaMetrics/pull/2685 2022-06-07 15:39:53 +03:00
Aliaksandr Valialkin
c7ff4505d2
docs/CHANGELOG.md: document backwards-incompatible changes in communication protocol between vmselect and vmstorage
The changes are related to the added query tracing in afced37c0b

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1403
2022-06-07 15:25:58 +03:00
Aliaksandr Valialkin
2b72d21840
docs/CHANGELOG.md: document e755d0ec3f 2022-06-07 15:21:10 +03:00
Aliaksandr Valialkin
0b4f767c53
docs/CHANGELOG.md: document 1ee1e986da 2022-06-07 15:02:57 +03:00
Aliaksandr Valialkin
33afaa49d9
docs: run make docs-sync after fda8da297e 2022-06-07 14:51:32 +03:00
Dmytro Kozlov
50393d0024
docs: fixed typos (#2680)
* docs: fixed typos

* Update README.md

* Update docs/README.md

* Update docs/Single-server-VictoriaMetrics.md

* docs: added examples with start and end params in request

* Apply suggestions from code review

Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-06-07 14:51:31 +03:00
Aliaksandr Valialkin
933f4ae148
docs/Cluster-VictoriaMetrics.md: run make docs-sync after 1c96dce367 2022-06-07 14:30:09 +03:00
Aliaksandr Valialkin
b4909351c5
docs: run make docs-sync after a439e887a3 2022-06-07 14:27:37 +03:00
Aliaksandr Valialkin
68b6ddfb14
all: follow-up after 8edb390e21
- Remove unused js bloatware from /targets page. This strips down binary size by more than 100Kb
- Add /service-discovery page for API compatibility with Prometheus
- Properly load bootstrap.min.css from /prometheus/targets
- Serve static contents for /targets page from app/vminsert instead of app/vmselect, because /targets page is served from there
2022-06-07 01:05:53 +03:00
Aliaksandr Valialkin
3dbb19d624
lib/promscrape/discovery/kubernetes: follow-up after 006b8c7534
- make more clear error logs
- simplify testing for newKubeConfig by passing only the path to kube_config file instead of SDConfig struct
2022-06-06 14:41:28 +03:00
Aliaksandr Valialkin
6d2832f413
docs/Articles.md: add a link to https://percona.community/blog/2022/06/02/long-time-keeping-metrics-victoriametrics/ 2022-06-06 13:15:42 +03:00
Aliaksandr Valialkin
6c2fb9d8c4
lib/promscrape: add -promscrape.cluster.name command-line flag
This flag is used for proper data de-duplication when the same target is scraped
from multiple vmagent clusters.

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2679
2022-06-04 01:11:23 +03:00
Dmytro Kozlov
ce8aade80e
lib/promscrape: adds service discovery visualization for /targets page(#2675)
* lib/promscrape: updated template

* lib/promscrape: fixed click on unhealthy and all btns

* app/vmselect: jquery scripts into static folder

Co-authored-by: f41gh7 <nik@victoriametrics.com>
2022-06-04 01:11:23 +03:00
Nikolay
72e43ef2fe
lib/promscrape/discovery/kubernetes: follow-up after 0b5c874911 (#2672) 2022-06-04 01:11:23 +03:00
Aliaksandr Valialkin
70bb84739d
docs/Single-server-VictoriaMetrics.md: small clarification about custom cache size tuning 2022-06-01 14:57:39 +03:00
Aliaksandr Valialkin
cc226e6ebe
docs/CHANGELOG.md: follow-up after 2177089f94 2022-06-01 14:57:39 +03:00
Roman Khavronenko
e9ee043879
lib/storage: make indexdb/tagFilters cache size configurable (#2667)
The default size of `indexdb/tagFilters` now can be overridden via
`storage.cacheSizeIndexDBTagFilters` flag.
Please, be careful with changing default size since it may
lead to inefficient work of the vmstorage or OOM exceptions.

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2663
Signed-off-by: hagen1778 <roman@victoriametrics.com>

Co-authored-by: Nikolay <nik@victoriametrics.com>
2022-06-01 14:57:39 +03:00
Roman Khavronenko
bca90d7148
promrelabel: add support of lowercase and uppercase relabeling actions (#2665)
* promrelabel: add support of `lowercase` and `uppercase` relabeling actions

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2664
Signed-off-by: hagen1778 <roman@victoriametrics.com>

* lib/storage: make golangci-lint happy

Signed-off-by: hagen1778 <roman@victoriametrics.com>

Co-authored-by: Nikolay <nik@victoriametrics.com>
2022-06-01 14:57:39 +03:00
Aliaksandr Valialkin
fedfc9e686
lib/storage: stop background merge when storage enters read-only mode
This should prevent from `no space left on device` errors when VictoriaMetrics
under-estimates the additional disk space needed for background merge.

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2603
2022-06-01 14:22:12 +03:00
Aliaksandr Valialkin
afced37c0b
all: add initial support for query tracing
See https://docs.victoriametrics.com/Single-server-VictoriaMetrics.html#query-tracing

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1403
2022-06-01 02:31:44 +03:00
Aliaksandr Valialkin
e83b96366f
docs/CHANGELOG.md: follow-up after 11f91532c5
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2594
2022-05-31 12:42:48 +03:00
Dmytro Kozlov
cd1fa2e4cd
issue-2594: use embedded for static files (#2650)
embed static js and css files from CDN into vmalert, vmagent and vmsingle binaries.

Co-authored-by: f41gh7 <nik@victoriametrics.com>

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2594
2022-05-31 12:42:48 +03:00
Howie
9c5f998438
fix: docs (#2658)
Signed-off-by: lihaowei <haoweili35@gmail.com>
2022-05-30 12:27:19 +03:00
Roman Khavronenko
4f84afb032
docs: add more details to the key concepts (#2648)
- data modification
- more applications for gauges
- recommendations for instrumenting app with metrics

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-05-30 12:26:37 +03:00
Dima Lazerka
3299865620
Change img urls to the same domain (#2649) 2022-05-27 20:13:49 +02:00
Aliaksandr Valialkin
f149d56ac2
lib/promscrape: add -promscrape.suppressScrapeErrorsDelay command-line flag
This flag can be used for reducing the amounts of logs when scraping unreliable scrape targets.

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2575

The patch is based on https://github.com/VictoriaMetrics/VictoriaMetrics/pull/2576 .
Thanks to @jelmd .
2022-05-25 23:00:30 +03:00
Roman Khavronenko
d367a65707
docs: update single version docs by adding extra links and formatting (#2643)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-05-25 23:00:28 +03:00
Aliaksandr Valialkin
38beb9fe04
lib/storage: add ability to change the indexdb rotation time offset with -retentionTimezoneOffset command-line flag
This is a follow-up for 0fbf59199a

See https://github.com/VictoriaMetrics/VictoriaMetrics/pull/2574
2022-05-25 16:07:14 +03:00
Aliaksandr Valialkin
05e47482a5
docs/CHANGELOG.md: document 9e343faa41 2022-05-25 15:05:42 +03:00
Roman Khavronenko
cd8e58d0ae
docs: mention guide for migrating from inlfux in vmctl readme (#2640)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-05-25 15:05:12 +03:00
Aliaksandr Valialkin
df9ffa6854
app: expose /api/v1/status/config endpoint in the same way as Prometheus does
This endpoint is needed for third-party tools.

See https://prometheus.io/docs/prometheus/latest/querying/api/#config
2022-05-25 10:00:00 +03:00
Aliaksandr Valialkin
090c331c12
docs/Single-server-VictoriaMetrics.md: mention Cardinality limiter docs at Resource usage limits section 2022-05-23 10:55:42 +03:00
Nikolay
754d3b1f93
Updates operator docs (#2622)
* docs/operator: adds information about VMAgent statefulMode
* docs/operator: adds description for alertmanager configuration
* docs/operator: adds description for configuration syncronization

https://github.com/VictoriaMetrics/operator/issues/124

Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
2022-05-23 10:55:42 +03:00
Roman Khavronenko
d597cdd06f
vmalert: mention how to build a custom image (#2626)
Thanks to @f41gh7

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-05-23 10:55:42 +03:00
Aliaksandr Valialkin
513f9537a9
docs/CHANGELOG.md: cut v1.77.2 2022-05-21 02:26:29 +03:00
Aliaksandr Valialkin
cdbee6f470
docs/CHANGELOG.md: group vmalert features together 2022-05-21 01:52:13 +03:00
Aliaksandr Valialkin
35ed63f32e
docs/CHANGELOG.md: document 2cf586da78 2022-05-21 01:17:49 +03:00
Roman Khavronenko
7406665fc3
lib/promscrape/discovery/kubernetes: fixes kubernetes service discovery (#2615)
* lib/promscrape/discovery/kubernetes: properly updates discovered scrape works
previously, added or updated scrapeworks may override previuosly
discovered.
it happens because swosByKey may contain small subset of kubernetes
objects with it's labels.
It happens for objectsUpdated and objectsAdded maps, which include only changed elements

* Properly calculate vm_promscrape_discovery_kubernetes_scrape_works

Co-authored-by: f41gh7 <nik@victoriametrics.com>
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-05-21 01:17:21 +03:00
Yurii Kravets
a723de6ae1
Update README.md (#2624) 2022-05-21 01:07:04 +03:00
Aliaksandr Valialkin
aba00b8cb2
docs: update the description for command-line flags according to recent changes 2022-05-20 15:11:37 +03:00
Aliaksandr Valialkin
4ca5d97459
docs/CHANGELOG.md: document 3df8caca15 2022-05-20 14:24:36 +03:00
Roman Khavronenko
e9165b041e
docs: add migration guide for influxdb (#2619)
docs: add migration guide for influxdb

Co-authored by @denisgolius 

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-05-20 14:24:35 +03:00
Aliaksandr Valialkin
386ef69124
docs/CHANGELOG.md: formatting fixes 2022-05-20 14:22:02 +03:00
Aliaksandr Valialkin
98123e95dd
docs/CHANGELOG.md: formatting fix for the issue url https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2607 2022-05-20 14:22:02 +03:00
Aliaksandr Valialkin
c5765dd6ad
docs/CHANGELOG.md: link to the feature request about reusable templates in vmalert
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2510
2022-05-20 14:21:35 +03:00
Roman Khavronenko
f71d15c79e
docs: follow-up after 4b3eb40658 (#2602)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-05-20 14:21:09 +03:00
Roman Khavronenko
dcd8387601
docs: fix a typo of mentioning scrape support for vmselect (#2617)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-05-20 14:20:37 +03:00
Roman Khavronenko
2aeb00f98f
vmalert: support scalar type in response (#2610)
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2607

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-05-20 14:08:19 +03:00
Roman Khavronenko
a07ddf9b65
vmalert: support strings in humanize.* templates (#2606)
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2569

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-05-20 14:05:53 +03:00
Yurii Kravets
40c614dc4e
Update vmalert.md (#2580)
docs: update vmalert/README.md

Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
2022-05-20 14:02:34 +03:00
Roman Khavronenko
a616e0b997
docs: quickstart update (#2572)
docs: update docs for beginners

QuickStart page was updated with more relevant information.
Key Concepts was added to cover basics for the VictoriaMetrics.
2022-05-20 14:00:43 +03:00
Roman Khavronenko
87e4e76537
vmalert: support /rules path for Grafana's ngalert requests (#2593)
Unexpectedly, Grafana makes an extra request to `/rules`
handler in addition to `/api/v1/rules` calls in alerts UI.
This happens only for Grafana versions older than 8.5.*.
Apparently, this is related to support of other monitoring
systems.
Prometheus responds with `text/html` content for UI page `/rules`
to such requests. Actually, returning just a blank page with
SC=200 works as well.

Returning actual response of `/api/v1/rules`
results in error in Grafana since it expects a `yaml` (?) in response.
So we add a placeholder to `vmalert`.

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2583
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-05-20 13:53:55 +03:00
Roman Khavronenko
ee1f8bae2e
docs: fix liquid syntax errors (#2592)
For liquid text processor double braces `{{` `}}`
are special chars for templating.
Since we use them in some of our docs with different purpose,
we must escape them to avoid syntax errors from liquid.

For escaping curly braces we use bult-in plugin which helps
to enclose sections of text via `{% raw %}` and `{% endraw %}`.
This approach prevents liquid syntax errors and makes render correct.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-05-20 13:42:47 +03:00
Roman Khavronenko
6e5ba1921d
vmalert: fix readme formatting (#2587)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-05-20 13:40:41 +03:00
Roman Khavronenko
0790ba5e26
vmalert: follow-up after 0ac1cdfff5 (#2586)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-05-20 12:25:13 +03:00
Andrii Chubatiuk
7789c47e41
added reusable templates support (#2532)
Signed-off-by: Andrii Chubatiuk <andrew.chubatiuk@gmail.com>
2022-05-20 12:25:11 +03:00
Aliaksandr Valialkin
83ff4c411d
app/vmalert: apply -remoteRead.disablePathAppend to -datasource.url in the same way as for the -remoteRead.url
This is a follow-up for 0e2486df56

The related pull requests:
- https://github.com/VictoriaMetrics/VictoriaMetrics/pull/1536
- https://github.com/VictoriaMetrics/VictoriaMetrics/pull/1712
2022-05-13 16:59:16 +03:00
Aliaksandr Valialkin
c1dd458f56
docs/CHANGELOG.md: document 3f0ecee128
See https://github.com/VictoriaMetrics/VictoriaMetrics/pull/2577
2022-05-13 16:59:16 +03:00
Yurii Kravets
d55791f59f
Update docs (#2566)
* deployment/docker: pass `-buildvs=false` to `go build` for production builds

This should resolve the `error obtaining VCS status: exit status 128` error
when the environment contains incorrect version of git or has incorrect access rights
to the directory with VictoriaMetrics source code.

See the following links for additional info:
- https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2508#issuecomment-1117126702 ,
- https://github.com/google/ko/issues/672
- https://github.com/golang/go/issues/49004

* lib/netutil: limit the number of concurrently established connections when calling ConnPool.Get()

This should reduce potential spikes in the number of established connections in the following cases:
- when the connection establishing procedure becomes temporarily slow
- after a temporary spike in the rate of ConnPool.Get() calls

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2552

* docs/CHANGELOG.md: document c8af625bcc

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1322#issuecomment-1120276146

* docs/Cluster-VictoriaMetrics.md: typo fix: `by by` -> `by`

* docs: add `resource usage limits` docs, which describe fine-grained tuning for various resource usage limits

* docs/Cluster-VictoriaMetrics.md: the `/api/v1/label/.../values` query can take CPU and ram at both vmstorage and vmselect

* Update root Readme and root vmagent readme

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-05-13 16:59:16 +03:00
Aliaksandr Valialkin
116c0b8f2e
docs/vmagent.md: typo fix in the description for -promscrape.cluster.replicationFactor command-line flag 2022-05-12 18:51:20 +03:00
Aliaksandr Valialkin
a47717f2ab
docs/Cluster-VictoriaMetrics.md: the /api/v1/label/.../values query can take CPU and ram at both vmstorage and vmselect 2022-05-11 20:00:36 +03:00
Aliaksandr Valialkin
7830448a36
docs: add resource usage limits docs, which describe fine-grained tuning for various resource usage limits 2022-05-11 19:53:19 +03:00
Aliaksandr Valialkin
6048e2e82c
docs/Cluster-VictoriaMetrics.md: typo fix: by by -> by 2022-05-11 18:05:05 +03:00
Aliaksandr Valialkin
e35e488220
docs/CHANGELOG.md: document c8af625bcc
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1322#issuecomment-1120276146
2022-05-11 14:33:06 +03:00
Denys Holius
f3ac421413
docs: fixed typos in CHANGELOG.md (#2565) 2022-05-11 14:30:45 +03:00
Aliaksandr Valialkin
d8a276fbe4
lib/netutil: limit the number of concurrently established connections when calling ConnPool.Get()
This should reduce potential spikes in the number of established connections in the following cases:
- when the connection establishing procedure becomes temporarily slow
- after a temporary spike in the rate of ConnPool.Get() calls

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2552
2022-05-11 14:11:06 +03:00
Aliaksandr Valialkin
68bfa05437
app/vmselect/promql: do not return values from label_value() if the original time series has no values at the selected timestamps 2022-05-09 17:58:39 +03:00
Aliaksandr Valialkin
d5efd3949b
docs/CHANGELOG.md: document 8f4f5f1d68 2022-05-09 17:33:04 +03:00
Aliaksandr Valialkin
86d344f6c0
docs/Release-Guide.md: typo fix: signle->single 2022-05-09 15:27:57 +03:00
Manuel Polo
2cc5486248
docs: fix typo in quickstart 2022-05-09 15:27:55 +03:00
Artem Navoiev
7197de0c8c
docs: add flags list to vmbackupmanager (#2554)
docs: add flags list to vmbackupmanager docs

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: tenmozes <artem@victoriametrics.com>
2022-05-09 15:26:24 +03:00
Aliaksandr Valialkin
e3819d3e8d
docs/CHANGELOG.md: cut v1.77.1 2022-05-07 01:49:21 +03:00
Aliaksandr Valialkin
e613ca9ba8
app/vminsert/netstorage: re-route samples from readonly vmstorage nodes to healthy nodes if -dropSamplesOnOverload command-line flag is set 2022-05-07 01:40:02 +03:00
Aliaksandr Valialkin
e961aec551
app/vmstorage: do not allow to set -retentionPeriod smaller than one day
VictoriaMetrics doesn't support retention periods smaller than one day,
so do not allow to set it to small values.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2496
2022-05-07 00:54:42 +03:00
Aliaksandr Valialkin
810dd74fb9
lib/promscrape: properly implement ScrapeConfig.clone()
Previously ScrapeConfig.clone() was improperly copying promauth.Secret fields -
their contents was replaced with `<secret>` value.

This led to inability to use passwords and secrets in `-promscrape.config` file.
The bug has been introduced in v1.77.0 in the commit 67b10896d2

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2551
2022-05-07 00:06:19 +03:00
Aliaksandr Valialkin
994c06cb84
docs/CHANGELOG.md: document e726340914 2022-05-06 18:11:16 +03:00
Dmytro Kozlov
8b5a819266
vmbackup: Prevent save backups to the same folder where TSDB data is (#2547)
* {vmbackup, vmbackup/snapshot}: validate snapshot name

* vmbackup/snapshot: added another checks

* backup/actions: added check that we ignore backup_complete.ignore file

* vmbackup: moved snapshot to lib directory

* lib/snapshot: added functions description

* lib/snapshot: fixed typo

* vmbackup: code cleanup

* wip

* vmbackup: Prevent save backups to the same folder where TSDB data is

* Apply suggestions from code review

* wip

* wip

* wip

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-05-06 18:04:58 +03:00
Aliaksandr Valialkin
7cd416f5e6
docs/Cluster-VictoriaMetrics.md: typo fix: bandidth -> bandwidth 2022-05-06 16:34:44 +03:00
Roman Khavronenko
fe697a9253
vmctl: add tip about safety flags during for native data export (#2540)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-05-06 16:28:24 +03:00
Aliaksandr Valialkin
925fa9a7de
docs/Cluster-VictoriaMetrics.md: make the description for -rpc.disableCompression command-line flag more clear 2022-05-06 16:24:56 +03:00
Aliaksandr Valialkin
983b465757
app/vmagent: add missing _total suffix to vmagent_remotewrite_global_rows_pushed_before_relabel_total counter
This is a follow up for c536139d0b
2022-05-06 15:51:34 +03:00
Aliaksandr Valialkin
af0da45d3e
lib/promscrape: rename promscrape_stale_samples_created_total metric to vm_promscrape_stale_samples_created_total, so its name is consistent with the rest of vm_promscrape_ metrics 2022-05-06 15:33:43 +03:00
Aliaksandr Valialkin
2baa9e8f48
app/vmagent: expose vmagent_remotewrite_global_rows_pushed_before_relabel and vmagent_remotewrite_rows_pushed_after_relabel_total metrics 2022-05-06 15:30:10 +03:00
Aliaksandr Valialkin
9c89ea1c90
app/vmagent: rename vmagent_remote_write_rate_limit_reached_total to vmagent_remotewrite_rate_limit_reached_total for the sake of consistency with other vmagent_remotewrite_ metrics 2022-05-06 15:02:18 +03:00
Aliaksandr Valialkin
9d40bb7137
lib/promscrape/discovery/ec2: add ability to filter Availability Zones in ec2_sd_config via az_filters section 2022-05-06 12:44:01 +03:00
Aliaksandr Valialkin
8babb4aebc
app/vmselect/vmui: make vmui-update after 450d879eaa 2022-05-05 22:11:41 +03:00
Denis Fondras
53b68eb254
Fix typo (#2538) 2022-05-05 21:16:39 +03:00
Aliaksandr Valialkin
c5e47929c3
docs/Cluster-VictoriaMetrics.md: update cluster scalability tips 2022-05-05 21:09:17 +03:00
Aliaksandr Valialkin
0634a894a9
docs/CHANGELOG.md: document bf5e3774cc 2022-05-05 13:38:44 +03:00
Aliaksandr Valialkin
0ad5b64930
docs/CHANGELOG.md: cut v1.77.0 2022-05-05 00:16:31 +03:00
Aliaksandr Valialkin
2213eef6b7
docs/vmbackup.md: added missing -storageDataPath argument in the command for creating daily backups 2022-05-04 22:50:18 +03:00
Aliaksandr Valialkin
4ae1a8514a
docs/vmbackup.md: acutalize docs
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2503
2022-05-04 22:44:40 +03:00
Aliaksandr Valialkin
9fea63d1c8
app/vmagent: rename -remoteWrite.useSigv4 command-line flag to -remoteWrite.aws.useSigv4, so its name is consistent with the other -remoteWrite.aws.* command-line flags 2022-05-04 20:41:45 +03:00
Aliaksandr Valialkin
358fa99af2
app/vmalert: run make quicktemplate-gen from the root directory after the commit f6dcfbcdd6 2022-05-04 20:28:37 +03:00
Nikolay
7e58cba6cf
{lib/promscrape,app/vmagent}: adds sigv4 support for vmagent remoteWrite (#2458)
* {lib/promscrape,app/vmagent}: adds sigv4 support for vmagent remoteWrite
moves aws related code into separate lib from lib/promscrape
it allows to write data from vmagent to the AWS managed prometheus (cortex)

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1287

* Apply suggestions from code review

* wip

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-05-04 20:28:37 +03:00
Dmytro Kozlov
0aeefeb5f1
vmalert/tpl: fixed truncating alerts expression in table (#2494)
vmalert: improve `/groups` UI visual 

The change also fixes truncated rules expressions in UI
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2484
2022-05-04 20:28:37 +03:00
Aliaksandr Valialkin
a297315ff1
docs/guides/multi-regional-setup-dedicated-regions.md: clarify wording on vmagent configuration 2022-05-04 18:50:37 +03:00
Aliaksandr Valialkin
4dfd84c505
docs/Cluster-VictoriaMetrics.md: move environment variables entry closer to cluster setup section 2022-05-04 18:41:44 +03:00
Aliaksandr Valialkin
9dc9e03a14
docs/CHANGELOG.md: yet another typo fix: present -> pressed 2022-05-04 18:20:58 +03:00
Aliaksandr Valialkin
c2610e4186
docs/CHANGELOG.md: typo fixes 2022-05-04 18:19:05 +03:00
Aliaksandr Valialkin
47010a9875
docs/CHANGELOG.md: document 8639e79d38 2022-05-04 10:46:32 +03:00
Aliaksandr Valialkin
4fa3cd701c
docs/Cluster-VictoriaMetrics.md: typo fix: serparated -> separated 2022-05-03 15:20:11 +03:00
Aliaksandr Valialkin
25266d2194
docs/CHANGELOG.md: document 3575aabeaf 2022-05-03 14:01:58 +03:00
Aliaksandr Valialkin
ec3a37896f
all: add -cluster.tlsInsecureSkipVerify command-line option to vminsert, vmselect and vmstorage components in order to be able to disable TLS certificate verification in mTLS mode
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2490
2022-05-03 13:13:43 +03:00
Aliaksandr Valialkin
7291c81f0d
docs/Cluster-VictoriaMetrics.md: refer to the doc on how to set up mTLS 2022-05-03 11:54:05 +03:00
Aliaksandr Valialkin
ccf44e810c
docs/CHANGELOG.md: document 488c34f5e1 2022-05-03 11:01:27 +03:00
Aliaksandr Valialkin
d384997657
docs/CHANGELOG.md: document d0706c8c95 2022-05-02 22:25:47 +03:00
Aliaksandr Valialkin
361b08c30e
lib/storage: leave the last sample per each discrete interval during the deduplicaton
This aligns better with staleness logic in Prometheus - https://prometheus.io/docs/prometheus/latest/querying/basics/#staleness
2022-05-02 21:59:31 +03:00
Aliaksandr Valialkin
7ca32c21c8
app/vmui: execute query by pressing enter in the same way as Prometheus does
Multi-line query can be entered via `shift-enter` in the query input field
2022-05-02 20:24:43 +03:00
Aliaksandr Valialkin
58e4e16588
docs/Cluster-VictoriaMetrics.md: move here the deduplication docs related to cluster version 2022-05-02 15:54:48 +03:00
Max Golionko
be81a0a327
added details about deduplication (#2527)
* added details about deduplication

* Update docs/README.md

Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>

* Apply suggestions from code review

* Update docs/README.md

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
2022-05-02 15:54:46 +03:00
Aliaksandr Valialkin
a70ac35ac7
docs/CHANGELOG.md: document 3616337812
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2514
2022-05-02 15:37:54 +03:00
Aliaksandr Valialkin
75f4adab40
docs/CHANGELOG.md: document 32a6b67e6c
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1761
2022-05-02 15:37:54 +03:00
Aliaksandr Valialkin
693e1838b3
docs/CHANGELOG.md: document b2294d1cf1 2022-05-02 15:37:54 +03:00
Aliaksandr Valialkin
190c8b463c
lib/netutil: close connections in ConnPool if they are idle for more than 30 seconds
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2508
2022-05-02 15:01:52 +03:00
Artem Navoiev
11db05a4ff
lib/{storage,flagutil} - Add option for snapshot autoremoval (#2487)
* lib/{storage,flagutil} - Add option for snapshot autoremoval

- add prometheus-like duration as command flag
- add option to delete stale snapshots
- update duration.go flag to re-use own code

* wip

* lib/flagutil: re-use Duration.Set() call in NewDuration

* wip

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-05-02 11:24:12 +03:00
Peter Dupej
9e193fb764
Update MetricsQL.md (#2519)
remove typo from label_replace doc.
2022-05-02 09:43:57 +03:00
Aliaksandr Valialkin
eaae544238
docs/CHANGELOG.md: document c7aad8d441 2022-04-29 13:02:43 +03:00
Dima Lazerka
e6ee235707
Export "null" in jsonl instead of NaN (#2518)
* Export "null" in jsonl instead of NaN

The NaN appeared because of staleness markers that were added for compatibility. I think it's better to use json `null`, implemented here.

Also maybe it also makes sense to add a flag like `?skip-staleness-markers=true` to `/export`, to skip nulls at all?

* Update app/vmselect/prometheus/export.qtpl

* app/vmselect/prometheus/export.qtpl.go: `make quicktemplate-gen`

* docs/CHANGELOG.md: document the change

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-04-29 12:51:03 +03:00
Aliaksandr Valialkin
83db416567
docs/MetricsQL.md: clarify keep_metric_names docs 2022-04-27 11:25:02 +03:00
Aliaksandr Valialkin
6eb1580158
app/vmselect/promql: add tlast_change_over_time(m[d]) function, which returns the timestamp for the last change of m on the given lookbehind window d 2022-04-27 10:58:40 +03:00
Aliaksandr Valialkin
917bb702ef
app/vmagent: substitute hard-to-read 500000000 with 500MB in -remoteWrite.maxDiskUsagePerURL description 2022-04-26 15:48:44 +03:00
Yury Molodov
eae6f68be2
vmui: add support relative time (#2504)
* feat: add support relative time

* app/vmselect: `make vmui-update`

* docs/CHANGELOG.md: document the change

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-04-26 15:46:47 +03:00
Aliaksandr Valialkin
7d9d790bfb
docs/Cluster-VictoriaMetrics.md: remove incorrect and misleading instructions for passing -replicationFactor flag to vmselect nodes in multi-level setup.
The `-replicationFactor` passed to top-level `vmselect` nodes mustn't exceed the `-replicationFactor` passed to top-level `vminsert` nodes
2022-04-26 15:13:08 +03:00
Aliaksandr Valialkin
cd4d1599cb
docs/CHANGELOG.md: document 4c1fbcd6b0
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2368
2022-04-26 15:09:57 +03:00
Aliaksandr Valialkin
374212333f
docs/CHANGELOG.md: typo fix: may result -> could result 2022-04-23 00:31:47 +03:00
Aliaksandr Valialkin
4c3cd96db5
lib/promauth: add support for min_version option at tls_config section in the same way as Prometheus does 2022-04-23 00:24:11 +03:00
Aliaksandr Valialkin
808a2f3b61
lib/promauth: add support for proxy_url option at oauth2 section in the same way as Prometheus does 2022-04-23 00:01:53 +03:00
Aliaksandr Valialkin
4ade8511e2
lib/promauth: add support for tls_config section at oauth2 config in the same way as Prometheus does 2022-04-23 00:01:52 +03:00
Aliaksandr Valialkin
a89e31b304
lib/promscrape/discovery/kubernetes: allow attaching node-level labels and annotations to discovered pod targets in the same way as Prometheus 2.35 does
See https://github.com/prometheus/prometheus/issues/9510
and https://github.com/prometheus/prometheus/pull/10080
2022-04-22 20:15:34 +03:00
Aliaksandr Valialkin
a69dabf709
docs/Cluster-VictoriaMetrics.md: mention that enterprise binaries are available for evaluation 2022-04-21 15:57:13 +03:00
Aliaksandr Valialkin
c341a46745
docs/Single-server-VictoriaMetrics.md: refer to the docs on how to set up multiple vmagent instances for scraping the same set of targets 2022-04-21 15:52:22 +03:00
Aliaksandr Valialkin
dac24aa342
app/vmselect/promql: properly handle scalar default vector, scalar if vector and scalar ifnot vector queries
Previously `vector` time series could be unexpectedly returned from such queries
2022-04-21 15:34:14 +03:00
Aliaksandr Valialkin
a5cfe5d13e
app/vmselect/promql: add drop_common_labels() function 2022-04-21 14:20:36 +03:00
Aliaksandr Valialkin
ed1b394a1a
app/vmstorage: expose vm_indexdb_items_added_total and vm_indexdb_items_added_size_bytes_total counters at /metrics page
These counters can be used for monitoring the rate of addition of new entries in indexdb (aka inverted index).

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2471
2022-04-21 13:19:42 +03:00
Aliaksandr Valialkin
25b841c6ed
app/vmselect/promql: fix duplicate time series error on joins against time series filtered by values
This should prevent from `duplicate time series` errors when executing the following query:

kube_pod_container_resource_requests{resource="cpu"} * on (namespace,pod) group_left() (kube_pod_status_phase{phase=~"Pending|Running"}==1)

where `kube_pod_status_phase{phase=~"Pending|Running"}==1` filters out diplicate time series
2022-04-20 22:21:20 +03:00
Aliaksandr Valialkin
90db923662
docs/CHANGELOG.md: document that the service discovery speed now scales with the number of CPU cores 2022-04-20 16:22:50 +03:00
Aliaksandr Valialkin
45385a5dc6
lib/promscrape: optimize getScrapeWork() function
Reduce the number of memory allocations in this function. This improves its performance by up to 50%.
This should improve service discovery speed when big number of potential targets with big number of meta-labels
are generated by service discovery.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2270
2022-04-20 15:34:18 +03:00
Aliaksandr Valialkin
d0bac8e224
all: typo fix: Kuberntes -> Kubernetes 2022-04-20 10:51:41 +03:00
Dmytro Kozlov
17552dba8b
lib/promscrape: Enable filters for endpoint and labels (#2466)
* lib/promscrape: Enable filters for endpoint and labels

* lib/promscrape: cleanup

* lib/promscrape: update template

* lib/promscrape: move logic filter logic to backend

* lib/promscrape: updated placeholder

* lib/promscrape: updated placeholder

* lib/promscrape: use two different fields for filters, updated form, added error on parsing queries

* lib/promscrape: rename functions

* lib/promscrape: removed unused values

* wip

* wip

* wip

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-04-19 18:27:44 +03:00
Nikolay
628905f080
lib/promscrape: adds job restart method (#2455)
* lib/promscrape: adds job restart method
it must restart only ScrapeConfig with changed content
this change greatly reduce time, that needed for job restart
and it should decrease possible data loss when config frequently changed at kubernetes based deployments

Apply suggestions from code review

Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>

* wip

Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-04-16 20:29:33 +03:00
Yury Molodov
60c4e0022a
fix: prevent graph hiding without data (#2456)
* fix: prevent graph hiding without data

* fix: add yaxis labels default

* app/vmselect: `make vmui-update`

* docs/CHANGELOG.md: document the change

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-04-16 17:16:17 +03:00
Aliaksandr Valialkin
944070eafa
docs/Cluster-VictoriaMetrics.md: sync docs 2022-04-16 16:59:42 +03:00
Aliaksandr Valialkin
7debf57ca6
lib/httpserver: clarify that -tls flag enables TLS for http requests to -httpListenAddr 2022-04-16 16:59:41 +03:00
Aliaksandr Valialkin
a7689e1b0c
app/vmstorage: add support for mTLS cipher suites via -cluster.tlsCipherSuites command-line flag
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2404
2022-04-16 16:36:38 +03:00
Aliaksandr Valialkin
0156a8b1e1
docs/Cluster-VictoriaMetrics.md: update docs after 26ae50ec26 2022-04-16 16:04:32 +03:00
Aliaksandr Valialkin
6bd032a6d3
docs: sync docs with the latest changes 2022-04-16 16:00:27 +03:00
Aliaksandr Valialkin
27e74f25d6
lib/httpserver: follow up after def0032c7d 2022-04-16 15:52:44 +03:00
Aliaksandr Valialkin
c50e48a74c
lib/promscrape: follow-up after baa1c24b36 2022-04-16 14:26:38 +03:00
Aliaksandr Valialkin
f6fed6637d
docs/Cluster-VictoriaMetrics.md: clarify the issue with multi-level cluster, which may result in data gaps at some AZs 2022-04-16 13:03:39 +03:00
Aliaksandr Valialkin
ef86cbabcc
docs/Cluster-VictoriaMetrics.md: improve docs on cluster setup, resizing and scalability 2022-04-15 15:04:10 +03:00
Aliaksandr Valialkin
437a12cb64
docs/Cluster-VictoriaMetrics.md: mention about possible issues with multi-level cluster setup 2022-04-15 14:26:01 +03:00
Aliaksandr Valialkin
564996da14
docs/CHANGELOG.md: document 45fcaa33e8 2022-04-13 14:14:25 +03:00
Aliaksandr Valialkin
951b2a0067
docs/CHANGELOG.md: document f7e4c5a628 2022-04-13 14:14:25 +03:00
Roman Khavronenko
56cd2b918a
vmalert: add DNS service discovery (#2465)
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2460
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-04-13 14:14:25 +03:00
Anton Bystrov
af96e6594c
Update CHANGELOG.md (#2463)
May be mispint here?
2022-04-13 14:14:25 +03:00
Aliaksandr Valialkin
4a6b56e5be
docs/CHANGELOG.md: cut v1.76.1 2022-04-12 16:21:06 +03:00
Aliaksandr Valialkin
3e2a4c07cd
app/vmui: further improvements for number display on graphs
This is a follow-up for c4d2cd8336

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2409
2022-04-12 16:02:37 +03:00
Aliaksandr Valialkin
50a6354a03
docs/CHANGELOG.md: link to the bug related to improper handling of maxSeries limit passed from vmselect to vmstorage 2022-04-12 16:02:37 +03:00
Yurii Kravets
488d300d8b
Update Quick-Start (#2422)
* Update Quick-Start

* Update docs/Quick-Start.md

Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>

* Update docs/Quick-Start.md

Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>

* Update docs/Quick-Start.md

Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>

* Update docs/Quick-Start.md

Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>

* update Quick-Start.md

added "Starting VM-Cluster via Docker" + Anchor fixes

* Apply suggestions from code review

Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-04-12 14:27:36 +03:00
Roman Khavronenko
e48a59eaa6
docs: add managed vm documentation section (#2437)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-04-12 13:04:11 +03:00
Aliaksandr Valialkin
70ad171070
lib/promscrape: follow-up after 7e79adfb55 2022-04-12 12:37:03 +03:00
Yurii Kravets
e5b8a5210e
url-examples.md (#2435)
* url-examples.md

added 1 more example

* Update url-examples.md
2022-04-12 11:30:53 +03:00
Aliaksandr Valialkin
81b7a31cb1
app/vmstorage: properly handle maxSeries limit passed from vmselect to vmstorage 2022-04-12 11:19:07 +03:00
Aliaksandr Valialkin
e3bf464f11
lib/protoparser/native: follow-up after fe01f4803d 2022-04-11 19:27:53 +03:00
Aliaksandr Valialkin
b3a3e9990f
vendor: update github.com/VictoriaMetrics/metricsql from v0.40.0 to v0.41.0
This allows using built-in function names as with template names
2022-04-11 18:32:18 +03:00
Aliaksandr Valialkin
5b265b3031
docs/Single-server-VictoriaMetrics.md: clarify that ingestion protocol means data ingestion protocol 2022-04-11 12:57:55 +03:00
Aliaksandr Valialkin
3c27bde77e
docs/CHANGELOG.md: document ed364a42e3 2022-04-11 12:12:07 +03:00
Aliaksandr Valialkin
e1b274d26c
docs/Single-server-VictoriaMetrics.md: mention that VictoriaMetrics supports data ingestion in Graphite protocol at Graphite API usage chapter 2022-04-10 16:21:19 +03:00
Aliaksandr Valialkin
cf9d261038
docs/Cluster-VictoriaMetrics.md: update docs after b843f0e229 2022-04-10 16:17:57 +03:00
Yurii Kravets
ace7dfe5d1
Update url-examples (#2410) 2022-04-08 13:07:06 +03:00
Aliaksandr Valialkin
4f5622b6fa
docs/Cluster-VictoriaMetrics.md: clarify high availability docs 2022-04-08 12:51:49 +03:00
Aliaksandr Valialkin
ce02d086d0
docs/CHANGELOG.md: document backwards-incompatible changes in cluster version of v1.76.0 2022-04-08 12:06:49 +03:00
Dmytro Kozlov
2aeb221691
docs/guides: Multi-regional setup with VictoriaMetrics (#2416)
* docs/guides: Multi-regional setup with VictoriaMetrics

* docs/guides: cleanup
2022-04-08 11:49:04 +03:00
Aliaksandr Valialkin
d7557b12ab
docs/CHANGELOG.md: document the bugfix in hitCount function 2022-04-08 11:48:36 +03:00
Aliaksandr Valialkin
949bf25a87
docs/CHANGELOG.md: typo fix 2022-04-07 17:20:20 +03:00
Aliaksandr Valialkin
e59019d088
docs/CHANGELOG.md: cut v1.76.0 2022-04-07 15:34:12 +03:00
Aliaksandr Valialkin
f082e64e0c
app/vmagent: reduce the probability of TLS handshake timeout when dialing the remote storage
The following actions are taken:

- Increase the TLS hashdshake timeout from 5 seconds to 10 seconds
- Increase dial timeout from 5 seconds to 30 seconds
- Specify DialContext instead of Dial in http.Transport. This allows properly handling
  the Context arg during dialing the remote storage

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1699
2022-04-06 12:35:14 +03:00
Aliaksandr Valialkin
087609a12b
docs/Release-Guide.md: add missing steps 2022-04-06 11:41:42 +03:00
Aliaksandr Valialkin
c24c5b8926
docs/CHANGELOG.md: document 0c0efc7781 2022-04-05 19:22:08 +03:00
Nikolay
7eb49d204f
vmctl verify-blocks command (#2390)
* lib/protoparser: changes ParseStream for native format
uses reader instead of http.Request
updates app/vmagent and app/vmagent method usage

* app/vmctl: add verify-block subcommand
it allows to check exported from VictoriaMetrics data block in native format
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2362

Update app/vmctl/README.md

Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
2022-04-05 17:46:36 +03:00
Aliaksandr Valialkin
8752cce157
app/vminsert: reduce the max packet size, which vminsert can send to vmstorage
This reduces the max memory usage for vminsert and vmstorage under heavy ingestion rate
by up to 50% on production workload
2022-04-05 15:39:58 +03:00
Roman Khavronenko
ce1629b70a
vmalert: add flag for disabling long-lived connections (keepalive) (#2395)
The new flag `datasource.disableKeepAlive` allows disabling keepalive
connections. This may be useful if there are multiple datasource
replicas (e.g. vmselects) behind the HTTP balancer to avoid uneven
load spread because of long-lived connections.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-04-04 13:08:06 +03:00
Aliaksandr Valialkin
816085a652
docs/CHANGELOG.md: document 70bb0d2708 2022-04-04 13:02:41 +03:00
Aliaksandr Valialkin
676c1f4fd7
docs/CHANGELOG.md: document 173073364e1bb1e0259ddc873dbd96ce62b07543 2022-04-04 12:56:11 +03:00
Artem Navoiev
23614d752a
Update release process. Actualize helm-charts release process. Add re… (#2397)
* Update release process. Actualize helm-charts release process. Add release guide for ansible playbooks

* add step to Ansible steps
2022-04-04 12:56:11 +03:00
Ross Dougherty
4df09074d3
fix object selectors link (#2391)
* fix object selectors link

* update kustomize url
2022-04-01 12:41:28 +03:00
Dima Lazerka
e756eae06d
Fix typo "vmanomapy" 2022-04-01 12:26:53 +03:00
Aliaksandr Valialkin
157555b022
docs/CHANGELOG.md: document a57e3807537914396ee3eb378648a464fa9e1b97 2022-04-01 12:25:54 +03:00
Aliaksandr Valialkin
151a00189e
docs/CHANGELOG.md: document 0989649ad0 2022-04-01 12:03:41 +03:00
Yurii Kravets
bd0068254d
url-examples (#2389) 2022-04-01 12:03:41 +03:00
Roman Khavronenko
d907e0d9f0
docs: fix typo in vmalert's API (#2380)
The API handler was changed in 1.75 but docs
still contain the old address.

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2366
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-04-01 12:03:41 +03:00
Roman Khavronenko
5b367ad594
docs: update increase_prometheus desc (#2381)
Remove note that this func is supported by PromQL.
2022-03-28 12:29:41 +03:00
Roman Khavronenko
44fa461e77
docs: escape chars for /label/values endpoint (#2379)
Without escaping the part wrapped with `<` `>` chars
won't be rendered properly.
2022-03-28 12:29:41 +03:00
Aliaksandr Valialkin
7c81ce296d
docs/CHANGELOG.md: cut v1.75.1 2022-03-28 12:28:27 +03:00
Yury Molodov
7cbf19812d
vmui: predefined panels (#2243)
* feat: add basic components for predefined dashboards

* fix: change display alert

* feat: add autosize and unit for axes

* feat: add component for CircularProgress

* feat: change layout for predefined dashboards

* feat: add override step for predefined panels

* feat: add override step for predefined panels

* feat: change yaxis limits for predefined panels

* fix: rename flag for hide legend

* feat: add formatted panel description

* feat: add README.md for dashboard setup

* feat: validate dashboard settings

* feat: add unit for y-ticks

* fix: correct display error for dashboards

* fix: disable auto refresh after route change

* update package-lock.json

* fix: add basename for BrowserRouter

* fix: add dynamic basename for routing

* update packages

* feat: add a pre-defined dashboard "per-job resource usage"

* feat: display unit in the hover-tooltip

* fix: change routing and home layout

* fix: change axis width calc

* updated packages

* app/vmselect: `make vmui-update`

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-03-26 13:05:21 +02:00
Aliaksandr Valialkin
b843f0e229
app/vmselect: add fine-grained limits for the number of returned/scanned time series for various APIs 2022-03-26 11:28:14 +02:00
Dima Lazerka
6e8e385375
VMAnomaly docs fixes (#2361)
* Added docs for vmanomaly

* Add example images

* Stylistic fixes

* Move images to root

* Update docs/vmanomaly.md

* Update docs/vmanomaly.md

Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>

* Squeeze vmanomaly after vmbackupmanager before Case Studies

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
2022-03-25 12:09:05 +02:00
Aliaksandr Valialkin
b7b06fb50e
Revert "docs/CHANGELOG.md: mention a bugfix in Graphite render API in v1.75.0"
This reverts commit 1e18c1c1ae.

The bugfix has been alread mentioned in the commit 9290605891
2022-03-24 19:21:15 +02:00
Aliaksandr Valialkin
1e18c1c1ae
docs/CHANGELOG.md: mention a bugfix in Graphite render API in v1.75.0 2022-03-24 19:20:42 +02:00
Dima Lazerka
50bc179dce
Add vmanomaly docs section (#2356)
* Added docs for vmanomaly

* Add example images

* Stylistic fixes

* Move images to root

* Update docs/vmanomaly.md

* Update docs/vmanomaly.md

Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
2022-03-24 11:57:07 +02:00
Yurii Kravets
b8d9c071e9
Update url-examples (#2358)
* Update url-examples

Add federate example

* Update docs/url-examples.md

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-03-24 11:57:06 +02:00
Aliaksandr Valialkin
c3d7522c19
docs: a follow-up after 76a477c609 2022-03-23 19:39:58 +02:00
Yurii Kravets
5f41e22cb1
Update Single-server-VictoriaMetrics.md (#2357)
* Update Single-server-VictoriaMetrics.md

Adding /federate link

* Apply suggestions from code review

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-03-23 19:39:57 +02:00
Yurii Kravets
a8b55103aa
Update url-examples.md (#2297)
* Update url-examples.md

+additional examples

* Update

* Update url-examples

Some changes requested by Roman.

* Update url-examples.md

* Update url-example

* Update url-examples

Additional info and marking for /labels part

* Update url-example

Added example with complex query which needs encoding:
 How to execute the query similar to - sum(increase(foo{status="bar"}[5m])) by (status)

* Update url-samples

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-03-22 15:12:36 +02:00
Aliaksandr Valialkin
8f678411cc
docs/Cluster-VictoriaMetrics.md: clarify mTLS protection docs 2022-03-22 14:01:06 +02:00
Arash Hatami
2e01691d5d
A good change for MD files (#2353)
* Lint YAML

* Remove extra comment

* Fix command problem

* Format MD files

* Format & fix problem of MD files for docs

* Another fix for MD files
2022-03-22 14:01:04 +02:00
Roman Khavronenko
bb594b34b8
docs: update release notes (#2349)
Warn about memory issue introduced in releases 1.73 - 1.74
2022-03-21 15:41:25 +02:00
Aliaksandr Valialkin
84b3234f3d
docs/CHANGELOG.md: document a1e17e91f8 2022-03-21 15:37:03 +02:00
hagen1778
7c7436b584
docs: add update note to v1.75.0 release note
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-03-21 15:37:01 +02:00
Aliaksandr Valialkin
7e67d61fdf
docs/CHANGELOG.md: cut v1.75.0 2022-03-18 19:54:13 +02:00
Aliaksandr Valialkin
85695faa94
app/vmselect/bufferedwriter: suppress trivial network errors, which can be generated by remote side
These errors include `broken pipe` and `reset by peer`.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2334
2022-03-18 19:27:33 +02:00
Aliaksandr Valialkin
9d91ad7124
app/vmagent/remotewrite: prevent from infinite recursion panic when pushing a time series with big number of samples to remote storage
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2335
2022-03-18 19:07:27 +02:00
Aliaksandr Valialkin
ab966f8a7a
docs: document 20bb5e703c 2022-03-18 18:42:09 +02:00
Aliaksandr Valialkin
e35c9124b7
lib/storage: reduce the interval for checking for free disk space from 30 seconds to 1 second
This should reduce the probability of out of disk space panics when -storage.minFreeDiskSpaceBytes is set to low values.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2305
2022-03-18 16:53:19 +02:00
Aliaksandr Valialkin
7c92aaeaa4
lib/blockcache: properly release memory occupied by deleted entries
Proviously the deleted entries could remain referenced via lastAccessHeap for long time.
This could lead to increased memory usage for the following caches starting from v1.73.0:

* indexdb/indexBlocks
* indexdb/dataBlocks
* storage/indexBlocks

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2242
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2007
2022-03-18 16:53:19 +02:00
Aliaksandr Valialkin
85a4b805e1
docs/CHANGELOG.md: document e5868b9c29
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/546
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2255
2022-03-18 13:08:56 +02:00
Aliaksandr Valialkin
3b7aefade2
docs/vmalert.md: sync after 11ae1ae924 2022-03-17 20:18:07 +02:00
Aliaksandr Valialkin
e1f70af14e
docs/CHANGELOG.md: document 11ae1ae924 2022-03-17 20:09:18 +02:00
Aliaksandr Valialkin
f0e96a84db
docs: document the addition of mTLS communication between cluster components 2022-03-17 20:00:14 +02:00
Aliaksandr Valialkin
9290605891
docs/CHANGELOG.md: document c1d07e7c52f0a2ab892921b0639cd42677aa33a8 2022-03-16 14:25:38 +02:00
Aliaksandr Valialkin
c175afbe02
docs/CHANGELOG.md: document changes from fb6eab03a2 2022-03-16 13:28:29 +02:00
Aliaksandr Valialkin
5b53373154
docs/CHANGELOG.md: document 565bd08c43
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1824
2022-03-16 13:28:29 +02:00
Aliaksandr Valialkin
bf131a4990
docs: make docs-sync 2022-03-16 13:28:29 +02:00
hagen1778
fd0f521bb9
docs: add update details for some releases
Some of the releases could negatively affect performance for a limited
period of time due to some changes in core. Update details are meant to
warn users about expected changes in peformance after the update.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-03-16 13:03:36 +02:00
Roman Khavronenko
ec48d022df
docs: fix broken links (#2303)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-03-16 13:01:46 +02:00
Dmytro Kozlov
7f0212f964
Issue-1824: added flags and different auth types support (#2287)
* vmalert/notifier: added flags and different auth types support

Co-authored-by: hagen1778 <roman@victoriametrics.com>
2022-03-16 12:52:01 +02:00
Ted Robertson
d4f2dd2a71
Clarify lack of support for InfluxQL (#2273) 2022-03-16 12:25:09 +02:00
Bastien Dronneau
da722e462e
docs(vmalert): typo in path (#2278) 2022-03-16 12:24:12 +02:00
Aliaksandr Valialkin
e7a0b4e095
docs/CHANGELOG.md: cut v1.74.0 2022-03-03 19:30:53 +02:00
Aliaksandr Valialkin
419f109f88
docs: make docs-sync after 702aa4948b 2022-03-03 18:37:07 +02:00
Dan Dascalescu
e15b27b792
docs: clarify Retention, improve English (#2266) 2022-03-03 18:37:07 +02:00
nemobis
cd522dee14
Fix typo, sentence flow in operator description (#2251) 2022-03-03 18:37:07 +02:00
Aliaksandr Valialkin
0b5ec1780d
docs/CHANGELOG.md: document performance improvements when registering new time series 2022-03-03 17:12:25 +02:00
Nikolay
d5ba1249f8
fixes incorrect step for calculation for MovingWindow functions (#283)
* fixes incorrect step for calculation for MovingWindow functions
https://victoriametrics.zendesk.com/agent/tickets/99

* wip

* wip

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-02-25 13:53:48 +02:00
Aliaksandr Valialkin
1bde8b4e22
docs/CHANGELOG.md: document cfc6c14dc48ae9dd35e65f1a6e5c7af8ccb9f029 2022-02-25 13:53:09 +02:00
Aliaksandr Valialkin
02a922b53f
lib/storage: properly handle series selector matching multiple metric names plus a negative filter
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2238

This is a follow-up for 00cbb099b6
2022-02-24 12:11:53 +02:00
Aliaksandr Valialkin
2431c9cf81
lib/promrelabel: add support for conditional relabeling via if filter
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1998
2022-02-24 02:35:13 +02:00
Nikolay
54221297d1
fixes jwt token parse with correct base64Url decoding (#281)
* fixes jwt token parse with correct base64Url decoding
it must be applied according to jwt RFC that requires token to be URL safe

added slow path for decoding tokens with std base64 decoding

adds error logging for vmgateway

* docs/CHANGELOG.md: document the bugfix

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-02-23 13:57:54 +02:00
Aliaksandr Valialkin
244c23ea2c
lib/workingsetcache: reduce the default cache rotation period from hour to 20 minutes
This should reduce memory usage under high time series churn rate
2022-02-23 13:42:27 +02:00
Aliaksandr Valialkin
cb100ce2af
docs/CHANGELOG.md: cut v1.73.1 2022-02-22 21:11:55 +02:00
Aliaksandr Valialkin
1ff42a0080
docs/CHANGELOG.md: link to the feature request for X-Influxdb-Version response header
Follow-up for 71ef3155c8

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2209
2022-02-22 20:34:03 +02:00
Yurii Kravets
538114cd7c
Update URL examples (#2207)
added div with copy function
2022-02-22 20:09:27 +02:00
Denys Holius
fc6f71fba8
Added minimal supported version of AlertManager (#2237)
* added minimal supported version of supported AlertManager

* docs: `make docs-sync`

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-02-22 20:09:26 +02:00
Aliaksandr Valialkin
89ead3daca
app/vmselect/netstorage: report vmstorage errors to vmselect clients even if partial responses are allowed
If a vmstorage is reachable and returns an application-level error to vmselect,
then such error must be returned to the caller even if partial responses are allowed,
since it usually means cluster mis-configuration.

Partial responses may be returned only if some vmstorage nodes are temporarily unavailable.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1941
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/678
2022-02-21 21:17:05 +02:00
Roman Khavronenko
5a4b16794d
Consul SD - update services on the watcher's start (#2202)
* lib/discovery/consul: update services on the watcher's start

Previously, watcher's start was only initing goroutines for discovery
but not waiting for the first iteration to end. It means first Consul
discovery wasn't returning discovered targets until the next iteration.

The change makes the watcher's start blocking until we get first discovery
iteration done and all registries updated.

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* vmalert: remove workarounds for consul SD

Now when consul SD lib properly updates services
on the first start, we don't need workarounds in vmalert.

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* lib/discovery/consul: update after review

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* wip

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-02-21 15:33:33 +02:00
Aliaksandr Valialkin
4a658cf955
app/vmagent: add -influxDBLabel in the same way as for app/vminsert
This is a follow-up for 3d19fa6932
2022-02-21 14:09:40 +02:00
Roman Khavronenko
bd7837d524
lib: allow to configure cache size by type (#2206)
* lib: allow to configure cache size by type

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1940
Signed-off-by: hagen1778 <roman@victoriametrics.com>

* Apply suggestions from code review

* wip

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-02-21 13:55:51 +02:00
Pablo Ley
4606d00d7d
Added missing inlineScrapeConfig field to the VMAgentSpec docs 2022-02-21 13:55:50 +02:00
Aliaksandr Valialkin
47b93a1337
docs: make docs-sync 2022-02-18 12:26:35 +02:00
Roman Khavronenko
f59685e5ff
docs: make profiling commands more copy-friendly (#2213)
* docs: make profiling commands more copy-friendly

The change adds `copy text` snippet to code examples
and replaces hostname placeholders with `0.0.0.0`.

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* Apply suggestions from code review

* Apply suggestions from code review

* Apply suggestions from code review

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-02-18 12:26:35 +02:00
Aliaksandr Valialkin
ce91456982
app/vminsert: add X-Influxdb-Version response header for InfluxDB API requests
This is needed for some clients, which expect this header.
See https://github.com/ntop/ntopng/issues/5449#issuecomment-1005347597
2022-02-17 12:49:24 +02:00
Aliaksandr Valialkin
8aaeffa7b4
docs: document 3d19fa6932 2022-02-16 23:30:52 +02:00
Aliaksandr Valialkin
ee066aa0d5
lib/storage: use binary search instead of full scan for skipping artificial tags when searching for tag names or tag values
This should improve performance for /api/v1/labels and /api/v1/label/<label_name>/values

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2200
2022-02-16 18:17:27 +02:00
Aliaksandr Valialkin
f227712d7d
docs/Single-server-VictoriaMetrics.md: mention that VictoriaMetrics doesnt check DD_API_KEY when accepting data via DataDog protocol 2022-02-16 14:40:01 +02:00
匠心零度
cbba78e091
Correct URL:How to delete time series (#2201)
Co-authored-by: lirenzuo <lirenzuo@shein.com>
2022-02-16 14:39:59 +02:00
Aliaksandr Valialkin
eaf29a5e8e
docs/url-examples.md: formatting fixes 2022-02-15 21:25:09 +02:00
Aliaksandr Valialkin
bb113efeb4
docs/CHANGELOG.md: document 2efa46a11c 2022-02-15 21:13:32 +02:00
Denys Holius
7e73fafc6f
Update revisions of dashboards (#2199)
* update revision of vmcluster for operator guide

* update revision of vmcluster for vmcluster guide
2022-02-15 21:13:32 +02:00
Yurii Kravets
759ea64b85
Create url-examples.md 2022-02-15 21:13:32 +02:00
Aliaksandr Valialkin
5aa3c896a8
docs/Single-server-VictoriaMetrics.md: clarify that the data is imported into VictoriaMetrics at how to import data section 2022-02-15 21:07:04 +02:00
Aliaksandr Valialkin
7e8596edb8
docs: update -help output for VictoriaMetrics components 2022-02-15 21:02:36 +02:00
Oct
c7c45bf842
docs: correct address in the docs (#2189)
* docs/FAQ.md:fix 404 url

* app/vmui/packages/vmui/README.md:rm 404 url
2022-02-15 12:59:50 +02:00
Aliaksandr Valialkin
e2be41f4dd
docs/CHANGELOG.md: document ad6bdd78d0 2022-02-15 12:48:23 +02:00
Aliaksandr Valialkin
38c73a00db
docs/CHANGELOG.md: cut v1.73.0 2022-02-14 17:54:53 +02:00
Aliaksandr Valialkin
5d8ea8c918
docs/CHANGELOG.md: document 3d890e89f1 2022-02-14 17:42:33 +02:00
Nikolay
48a9e068be
adds release build for macos darwin amd64 and arm64 (#2185)
* adds release build for macos darwin amd64 and arm64

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1896
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1851

* wip

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-02-14 17:42:33 +02:00
Aliaksandr Valialkin
fc0771a888
docs/CHANGELOG.md: document c90c1c4d54 2022-02-14 13:13:03 +02:00
Aliaksandr Valialkin
31b42e9c57
lib/promscrape: add expand all and collapse all buttons to /targets page
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2021
2022-02-12 18:42:01 +02:00
Aliaksandr Valialkin
989668beba
app/vmselect/promql: return at most one time series from absent_over_time() in the same way as Prometheus does
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2130
2022-02-12 15:46:43 +02:00
Aliaksandr Valialkin
216b13186d
docs/vmagent.md: add unsupported Prometheus config sections chapter 2022-02-12 01:08:34 +02:00
Aliaksandr Valialkin
3ab4aef140
docs/CHANGELOG.md: document ea153e5f90 2022-02-12 00:48:40 +02:00
Roman Khavronenko
d107f86fbc
lib/index: reduce read/write load after indexDB rotation (#2177)
* lib/index: reduce read/write load after indexDB rotation

IndexDB in VM is responsible for storing TSID - ID's used for identifying
time series. The index is stored on disk and used by both ingestion and read path.

IndexDB is stored separately to data parts and is global for all stored data.
It can't be deleted partially as VM deletes data parts. Instead, indexDB is
rotated once in `retention` interval.

The rotation procedure means that `current` indexDB becomes `previous`,
and new freshly created indexDB struct becomes `current`. So in any time,
VM holds indexDB for current and previous retention periods.
When time series is ingested or queried, VM checks if its TSID is present
in `current` indexDB. If it is missing, it checks the `previous` indexDB.
If TSID was found, it gets copied to the `current` indexDB. In this way
`current` indexDB stores only series which were active during the retention
period.

To improve indexDB lookups, VM uses a cache layer called `tsidCache`. Both
write and read path consult `tsidCache` and on miss the relad lookup happens.

When rotation happens, VM resets the `tsidCache`. This is needed for ingestion
path to trigger `current` indexDB re-population. Since index re-population
requires additional resources, every index rotation event may cause some extra
load on CPU and disk. While it may be unnoticeable for most of the cases,
for systems with very high number of unique series each rotation may lead
to performance degradation for some period of time.

This PR makes an attempt to smooth out resource usage after the rotation.
The changes are following:
1. `tsidCache` is no longer reset after the rotation;
2. Instead, each entry in `tsidCache` gains a notion of indexDB to which
they belong;
3. On ingestion path after the rotation we check if requested TSID was
found in `tsidCache`. Then we have 3 branches:
3.1 Fast path. It was found, and belongs to the `current` indexDB. Return TSID.
3.2 Slow path. It wasn't found, so we generate it from scratch,
add to `current` indexDB, add it to `tsidCache`.
3.3 Smooth path. It was found but does not belong to the `current` indexDB.
In this case, we add it to the `current` indexDB with some probability.
The probability is based on time passed since the last rotation with some threshold.
The more time has passed since rotation the higher is chance to re-populate `current` indexDB.
The default re-population interval in this PR is set to `1h`, during which entries from
`previous` index supposed to slowly re-populate `current` index.

The new metric `vm_timeseries_repopulated_total` was added to identify how many TSIDs
were moved from `previous` indexDB to the `current` indexDB. This metric supposed to
grow only during the first `1h` after the last rotation.

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1401

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* wip

* wip

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-02-12 00:34:44 +02:00
Roman Khavronenko
791cad8c2e
lib/promscrape: support prometheus-like duration in scrape configs (#2169)
* lib/promscrape: support prometheus-like duration in scrape configs

The change allows to specify duration values like `1d`, `1w`
for fields `scrape_interval`, `scrape_timeout`, etc.

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/817#issuecomment-1033384766
Signed-off-by: hagen1778 <roman@victoriametrics.com>

* lib/blockcache: make linter happy

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* lib/promscrape: support prometheus-like duration in scrape configs

* add support for extra fields `scrape_align_interval` and `scrape_offset`;
* support Prometheus duration parsing for `__scrape_interval__`
and `__scrape_duration__` labels;

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* wip

* wip

* docs/CHANGELOG.md: document the feature

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-02-11 16:17:51 +02:00
Aliaksandr Valialkin
727b29d4a3
lib/promscrape/discovery/kubernetes: add __meta_kubernetes_endpointslice_{label,annotation}* labels to be consistent with other role values for Kubernetes service discovery 2022-02-11 14:56:10 +02:00
Nikolay
265938a385
fixes service discovery for kubernetes (#2173)
* fixes service discovery for kubernetes
now it must take in account all pods that belong to the discovered endpoint and endpointslice
adds simple test for endpoints
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2134

* wip

* docs/CHANGELOG.md: document the change

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-02-11 13:35:34 +02:00
Aliaksandr Valialkin
6cb2954612
docs/CHANGELOG.md: document 4e722c459b
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2167
2022-02-10 12:21:05 +02:00
Aliaksandr Valialkin
718342a806
docs/Articles.md: add an article from Asserts.io, which mentions VictoriaMetrics components 2022-02-10 12:21:05 +02:00
Aliaksandr Valialkin
b81e6ba403
docs/CHANGELOG.md: add instructions on how to build VictoriaMetrics components from source code in order to test tip changes 2022-02-08 16:44:20 +02:00
Nikolay
feefbbab48
adds CGO build for arm64 (#2102)
* adds CGO build for arm64
it must improve performance for arm64 based deployments of vmstorage and
vmsingle for 15-20%

it depends on gozstd package update for correct musl gozstd vendoring

* typo fixes

* docs/CHANGELOG.md: document the change

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-02-08 16:26:55 +02:00
Roman Khavronenko
763cb68a67
docs: add Storage section to docs (#2155)
* docs: add Storage section to docs

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2151

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* Apply suggestions from code review

* docs: `make docs-sync`

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-02-08 16:00:18 +02:00
Aliaksandr Valialkin
eed66b6640
lib/promscrape: set -promscrape.config.strictParse to true by default
This allows detecting long-living silent errors in -promscrape.config
2022-02-08 15:42:33 +02:00
Aliaksandr Valialkin
6e05e3308e
docs/Single-server-VictoriaMetrics.md: add a link to exlanation on how downsampling works 2022-02-08 15:11:27 +02:00
Aliaksandr Valialkin
2149805886
docs/CHANGELOG.md: add links to issues, which could benefit from improved re-routing algorithm 2022-02-07 16:54:27 +02:00
Aliaksandr Valialkin
eb466e0c4e
docs: fix links to FAQ.md entries after they have been changed in ccf04239e6 2022-02-07 16:01:34 +02:00
Aliaksandr Valialkin
5da7f08be6
docs: cross-link downsampling docs from deduplication and vmalert docs 2022-02-07 14:53:50 +02:00
Aliaksandr Valialkin
c738739494
app/vminsert: add -dropSamplesOnOverload command-line flag
Drop incoming samples if the destination vmstorage node is unavailable
and/or accepts data at slower rate than other vmstorage nodes
2022-02-07 12:32:18 +02:00
Aliaksandr Valialkin
021ee53ba8
app/vminsert: improve re-routing logic in order to spread rows more evenly among the available storage nodes 2022-02-06 20:20:02 +02:00
Aliaksandr Valialkin
d24e5d9efd
lib/promscrape: show the total number of scrapes and the total number of scrape errors per target at /targets page
This information may be useful when debugging unreliable scrape targets
2022-02-03 20:23:27 +02:00
Aliaksandr Valialkin
678b3e71db
lib/promscrape: provide the ability to fetch target responses on behalf of vmagent or single-node VictoriaMetrics
This feature may be useful when debugging metrics for the given target located in isolated environment
2022-02-03 19:02:12 +02:00
Aliaksandr Valialkin
10367d7e4c
app/vmselect/promql: do not push down filters, which enumerate more than 10k unique values
Such filters may slow down time series search, so just skip them.

This is a follow-up for e7f1ceeb84

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1827
2022-02-02 23:42:25 +02:00
Aliaksandr Valialkin
97b7b94f91
docs/CHANGELOG.md: document 55e3bbd4cc
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1567
2022-02-02 23:42:25 +02:00
Aliaksandr Valialkin
dbead4813e
docs: updates after 5da71eb685
* Mention about the ability to configure vmalert notifiers via files in docs/CHANGELOG.md
* Mention about the ability to use Consul service discovery for vmalert notifiers in docs/CHANGELOG.md
* Run `make docs-sync` in order to sync app/vmalert/README.md to docs/vmalert.md
2022-02-02 23:42:25 +02:00
Pablo Ley
66271bfeee
Added missing links in EmbeddedHPA API docs 2022-02-02 23:42:25 +02:00
Aliaksandr Valialkin
566e12874d
lib/cgroup: expose process_cpu_cores_available metric
This metric shows the number of CPU cores available to the process.
This allows creating alerting rules on CPU saturation with the following query:

    rate(process_cpu_seconds_total[5m]) / process_cpu_cores_available > 0.9

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2107
2022-01-31 20:25:15 +02:00
Aliaksandr Valialkin
04d6596298
app/vmselect/promql: optimize queries, which join on _info metrics.
Automatically add common filters from one side of binary operation
to the other side before sending the query to storage subsystem.

See https://grafana.com/blog/2021/08/04/how-to-use-promql-joins-for-more-effective-queries-of-prometheus-metrics-at-scale/
and https://www.robustperception.io/exposing-the-software-version-to-prometheus
2022-01-31 20:25:15 +02:00
Aliaksandr Valialkin
ad5d646e4e
docs/Release-Guide.md: add instructions on how to create signed release tags 2022-01-31 20:25:15 +02:00
Aliaksandr Valialkin
35164d4dcf
docs/CHANGELOG.md: document 6a519896db 2022-01-31 12:42:27 +02:00
Aliaksandr Valialkin
069880fd3c
docs/vmauth.md: mention that backend services must be accessible only via vmauth 2022-01-28 12:07:12 +02:00
Denys Holius
9e7d6a585b
FAQ update (#2111)
* FAQ update: how downsampling and deduplication will work at the same time

* Update docs/FAQ.md

Co-authored-by: Aliaksandr Valialkin <valyala@gmail.com>
2022-01-27 20:36:15 +02:00
jaysony
3e22537248
correct VMAgentStatus description (#2109)
correct VMAgentStatus description from VMAlert to VMAgent.
2022-01-27 20:18:37 +02:00
Aliaksandr Valialkin
0ac2a51682
vendor: update github.com/VictoriaMetrics/metricsql from v0.37.0 to v0.38.0
This adds more optimization cases for https://utcc.utoronto.ca/~cks/space/blog/sysadmin/PrometheusLabelNonOptimization

For example:

* Multi-level transform functions. For example, abs(round(foo{a="b"})) + bar{x="y"}
  is now optimized to abs(round(foo{a="b",x="y"})) + bar{a="b",x="y"}
* Binary operations with `on()`, `without()`, `group_left()` and `group_right()` modifiers.
  For example, foo{a="b"} on (a) + bar is now optimized to foo{a="b"} on (a) + bar{a="b"}
* Multi-level binary operations. For example, foo{a="b"} + bar{x="y"} + baz{z="q"}
  is now optimized to foo{a="b",x="y",z="q"} + bar{a="b",x="y",z="q"} + baz{a="b",x="y",z="q"}
* Aggregate functions. For example, sum(foo{a="b"}) by (c) + bar{c="d"}
  is now optimized to sum(foo{a="b",c="d"}) by (c) + bar{c="d"}
2022-01-27 19:04:45 +02:00
Aliaksandr Valialkin
4911c0c045
docs/operator: fix links to Operator API docs 2022-01-25 17:38:41 +02:00
Aliaksandr Valialkin
49650fe6aa
lib/logger/throttler.go: show the original location of the error and warning message
Previously the location inside LogThrottler implementation was shown. This could complicate debugging.
2022-01-23 13:55:48 +02:00
f41gh7
2f56f484af
updates operator docs 2022-01-23 13:15:37 +02:00
Yury Molodov
196bef8348
vmui: fixed display type switching (#2088)
* fix: correct switch display type

* docs/CHANGELOG.md: document the bugfix

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-01-21 16:57:14 +02:00
Yurii Kravets
4b7650832e
Created Operator folder in docs (#2034)
* Created Operator folder in docs

Transferred Operator documentation

* Removed Contributing and Release

* Changed sort numbering

* Renamed folder

Operator -> operator

* 1

1

* Name change

Operator -> operator

* Removed colon symbol

* Useful links transformed to links style

* "updated at..." is no longer a header

* delete manager patch.yaml

* delete kustomization.yaml

* removed part with links

* community and contributions part removed

* Delete readme

* Docs navigation removed
2022-01-21 12:06:41 +02:00
Yury Molodov
034012c80f
vmui: fix time range selector (#2085)
* fix: add date validate for time range

* app/vmselect/vmui: `make vmui-update`

* docs/CHANGELOG.md: document the bugfix

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-01-21 12:03:41 +02:00