github-mirrors/VictoriaMetrics
1
0
Fork 0
mirror of https://github.com/VictoriaMetrics/VictoriaMetrics.git synced 2024-12-11 14:53:49 +00:00
Code Issues Projects Releases Packages Wiki Activity
7198 commits 131 branches 758 tags 363 MiB
Commit graph

7198 commits

This branch
This branch
All branches
Author SHA1 Message Date
Aliaksandr Valialkin
ddf88dddc0
docs/CHANGELOG.md: clarify the description of bugfixes at f7dda12b4d and b6ad581b45 
This is a follow-up for 8b01bc4a5c

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/4999
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5009
 2023-09-19 00:45:59 +02:00
Aliaksandr Valialkin
cc8f2bee0d
app/vlinsert: follow-up for d570763c91 
- Switch from summary to histogram for vl_http_request_duration_seconds metric.
  This allows calculating request duration quantiles across multiple hosts
  via histogram_quantile(0.99, sum(vl_http_request_duration_seconds_bucket) by (vmrange)).
- Take into account only successfully processed data ingestion requests
  when updating vl_http_request_duration_seconds histogram.
  Failed requests are ignored, since they may significantly skew measurements.
- Clarify the description of the change at docs/VictoriaLogs/CHANGELOG.md.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/4934
 2023-09-19 00:45:28 +02:00
Aliaksandr Valialkin
de2b3ff9b0
lib/promscrape/discovery/kubernetes: follow-up after 03fece44e0 
- Properly update vm_promscrape_discovery_kubernetes_url_watchers
  and vm_promscrape_discovery_kubernetes_group_watchers metrics after config changes

- Properly stop goroutine responsible for recreating scrapeWorks after the corresponding urlWatcher is stopped

- Log the event when urlWatcher is stopped in order to simplify debugging

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4850
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/4861
 2023-09-19 00:44:21 +02:00
Aliaksandr Valialkin
705b31c351
lib/promscrape/discovery/kubernetes: wait for 10 seconds before checking whether the urlWatcher must be stopped 
This should prevent from excess urlWatcher churn on config reload, since it leads to removal of all the apiWatchers
before creating new apiWatchers. So, every config reload would lead to stopping of all the previous urlWatchers
and starting new urlWatchers.

The new logic gives 10 seconds for config reload before stopping unused urlWatchers.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4850
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/4861
 2023-09-19 00:43:26 +02:00
Aliaksandr Valialkin
fe24523e19
lib/promscrape/discovery/kubernetes: follow-up after eeb862f3ff 
- Move the bugfix description to the correct place in docs/CHANGELOG.md
- Prevent from logging of 'context canceled' errors after the url watcher is stopped,
  since these errors are expected and may confuse users.
- Remove unused urlWatcher.refCount field.
- Remove unused urlWatcher.close() method.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4850
 2023-09-19 00:41:29 +02:00
Aliaksandr Valialkin
d0c103ad05
lib/backup: properly copy parts.json files inside indexdb directory additional to data directory 
This is a follow-up for 264ffe3fa1

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5005
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5006
 2023-09-19 00:38:31 +02:00
Aliaksandr Valialkin
c0b8143daf
lib/backup/common: consistently use canonical path with / directory separators at Part.Path 
Previously Part.Path could contain `\` directory separators on Windows OS,
which could result in incorrect filepaths generation when making backups at object storage.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4704

This is a follow-up for f2df8ad480
 2023-09-19 00:36:32 +02:00
Aliaksandr Valialkin
9e50ea6cce
app/vlinsert/insertutils: cosmetic changes after 8d3e574c31 2023-09-19 00:35:51 +02:00
dependabot[bot]
1870a5cac0
build(deps): bump codecov/codecov-action from 3 to 4 (#5011) 
Bumps [codecov/codecov-action](https://github.com/codecov/codecov-action) from 3 to 4.
- [Release notes](https://github.com/codecov/codecov-action/releases)
- [Changelog](https://github.com/codecov/codecov-action/blob/main/CHANGELOG.md)
- [Commits](https://github.com/codecov/codecov-action/compare/v3...v4)

---
updated-dependencies:
- dependency-name: codecov/codecov-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-09-19 00:35:10 +02:00
Nikolay
e5b62b0f62
docs: reflect recent changes at change logs (#5015) 2023-09-19 00:24:28 +02:00
crossoverJie
fb13887573
app/vlinsert: Add vl_http_request_duration_seconds metrics (#4934) 2023-09-19 00:10:24 +02:00
Zakhar Bessarab
55d25fb844
lib/promscrape/discovery/kubernetes: fix leaking api watcher (#4861) 
* lib/promscrape/discovery/kubernetes: fix leaking api watcher

goroutine which was polling k8s API had no execution control. This leaded to leaking goroutines during config reload.

See: https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4850
Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* lib/promscrape/discovery/kubernetes: use reference counting for urlWatcher cleanup

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* lib/promscrape/discovery/kubernetes: remove waitgroup sync for goroutines polling API server

This is unnecessary since context will is cancelled and new requests will not be sent. Also, using waitgroup will increase time required to perform reload which might result in missed scrapes.

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* lib/promscrape/discovery/kubernetes: clarify comment

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* Apply suggestions from code review

* lib/promscrape/discovery/kubernetes: address review feedback

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

---------

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
Co-authored-by: Nikolay <nik@victoriametrics.com>
 2023-09-18 17:13:55 +02:00
Konstantin
c1a8a2d54c
app/vmselect: return +Inf as null in graphite render api (#5009) 
Signed-off-by: Konstantin Kulikov <k.kulikov2@gmail.com>
 2023-09-18 16:41:39 +02:00
faceair
609c76eec9
lib/storage: remove ForceMergeAllParts internal loop (#4999) 
Signed-off-by: faceair <git@faceair.me>
 2023-09-18 16:35:37 +02:00
Zakhar Bessarab
aa583f0b9a
lib/backup: force copying of parts.json (#5006) 
* lib/backup: force copying of parts.json

Copying of parts.json is required because `part.key()` comparison can create same key value for files with different contents. This will result in inconsistent backup being created or restored.

See: https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5005
Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* lib/backup: ensure parts.json is only copied once

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

---------

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
Co-authored-by: Nikolay <nik@victoriametrics.com>
 2023-09-18 16:18:03 +02:00
Zakhar Bessarab
b0c8f7f718
app/vlinsert: add flag to limit amount of fields per line (#4976) 
Adding limit on ingestion allows to avoid issues like this one https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4762
Such issues are often caused by misconfigurtion on log persing/ingestion side and preventing such rows from being ingested allows to avoid performance implications created by storing such log rows.

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
 2023-09-18 12:00:02 +02:00
Zakhar Bessarab
eefdc60c88
docs: add changelog entry for downsampling.period and dedup.minScrapeInterval verification (#5000) 
* docs: add changelog entry for downsampling.period and dedup.minScrapeInterval verification

- added changelog entry
- documented requirements for dedup.minScrapeInterval and downsampling.period being multiples of each other

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* docs: `make docs-sync`

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

---------

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
 2023-09-18 11:38:13 +02:00
Artem Navoiev
d435939109
add annotation to VictoriaLogs dashboards - restarts and version change (#5008) 
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
 2023-09-18 11:35:16 +02:00
Dmytro Kozlov
5477b52991
vmagent: add validation of MetricsQL functions (#4991) 
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2023-09-15 13:16:22 +02:00
Aliaksandr Valialkin
24cec79763
lib/storage: handle fatal errors inside indexSearch.getTSIDByMetricID() instead of returning them to the caller 
This simplifies the code a bit at caller side
 2023-09-15 12:01:06 +02:00
Github Actions
bf30ded160
Automatic update operator docs from VictoriaMetrics/operator@49d245d (#5002) 2023-09-15 11:58:54 +02:00
Github Actions
622bd073b6
Automatic update operator docs from VictoriaMetrics/operator@a9c54f4 (#4998) 2023-09-15 11:58:54 +02:00
Aliaksandr Valialkin
f3eaeb8e5c
.github/workflow: remove automatic creation of pull request at github.com/VictoriaMetrics/ops repository on new tag 
This automation doesn't work as intended on LTS releases, bugfix releases and custom releases,
since it assumes every new tag is related only to new release.

Also the github.com/VictoriaMetrics/ops repository may contain manually set custom tags
for VictoriaMetrics components (for example, for testing the latest bugfixes or features),
which are overwritten by the generated pull request.

The way to go is to manually update tags at github.com/VictoriaMetrics/ops repository when needed
instead of trying to automate this process.
 2023-09-11 23:52:55 +02:00
Alexander Marshalov
c57ce9a55a
Updates in release guide (#4956) 
new helm release flow and operator release flow

Signed-off-by: Alexander Marshalov <_@marshalov.org>
 2023-09-11 23:51:27 +02:00
Github Actions
9baaa47ef1
Automatic update operator docs from VictoriaMetrics/operator@21f7203 (#4988) 2023-09-11 23:51:11 +02:00
Aliaksandr Valialkin
f9f3a871f7
docs/CHANGELOG.md: document v1.87.9 2023-09-10 21:41:56 +02:00
Aliaksandr Valialkin
1d9adc022d
deployment/docker: update VictoriaMetrics from v1.93.3 to v1.93.4 
See https://docs.victoriametrics.com/CHANGELOG.html#v1934
 2023-09-10 19:50:14 +02:00
Aliaksandr Valialkin
5f4ed9b45f
docs/CHANGELOG.md: document v1.93.4 2023-09-10 19:47:51 +02:00
Aliaksandr Valialkin
9c3a37597c
app/vmselect/netstorage: run make fmt after 58326dbf25 2023-09-10 15:18:15 +02:00
Artem Navoiev
973c74df55
Update VL daashboard. Add Resource Section, add ds and job filters, a… (#4981) 
* Update VL daashboard. Add Resource Section, add ds and job filters, add metric collection in docker compose from victorialogs, fix networkigs usage in docker compose

Signed-off-by: Artem Navoiev <tenmozes@gmail.com>

* add vl dashboard to docker compose

Signed-off-by: Artem Navoiev <tenmozes@gmail.com>

* add vl dashboard to docker compose

Signed-off-by: Artem Navoiev <tenmozes@gmail.com>

---------

Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
 2023-09-10 15:05:19 +02:00
Github Actions
d2bc7e2160
Automatic update operator docs from VictoriaMetrics/operator@83c07ed (#4970) 2023-09-08 23:33:42 +02:00
Aliaksandr Valialkin
a5a953fe1e
app/vmagent/remotewrite: fix data race when extra labels are added to samples before sending them to multiple remote storage systems 
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4972
 2023-09-08 23:26:40 +02:00
Aliaksandr Valialkin
23bb38ef80
app/vmauth: add ability to specify response status codes for retrying requests during load-balancing 
Response status codes for retrying can be specified via retry_status_codes list

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4893
 2023-09-08 23:26:36 +02:00
Zakhar Bessarab
b842a0cd25
docs: sync description for license flags (#4977) 
- update eula flag to add deprecation notice
- add new license flags description

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
 2023-09-08 23:26:25 +02:00
Roman Khavronenko
548acce6b3
vmalert: correctly add duplicated params to the query (#4955) 
Fix the bug when Group's `params` fields with multiple values were
overriding each other instead of adding up.
The bug was introduced in this commit eccecdf177
 starting from v1.91.1 https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.91.1

 https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4908

Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 6351d07da8)
 2023-09-08 09:33:45 +02:00
Aliaksandr Valialkin
3257fcfd8d
app/vmauth: retry requests at other backends on 5xx response status codes 
This should allow implementing high availability scheme described at https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4792#issuecomment-1674338561

See also https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4893
 2023-09-08 00:47:57 +02:00
Aliaksandr Valialkin
58326dbf25
app/vmselect: return 503 status code when partial responses are denied and some of vmstorage nodes are temporarily unavailable 
This should help detecting this case and automatic retrying the query at healthy cluster replica
in another availability zone.

This commit is needed as a preparation for automatic query retry at another backend at vmauth on 5xx errors
as described at https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4792#issuecomment-1674338561
 2023-09-07 16:07:06 +02:00
Aliaksandr Valialkin
4af79504f6
lib/logger: increase the maximum log arg size from 200 to 500 
The 200 chars limit has been appeared too small for typical log messages emitted by VictoriaMetrics components

This is a follow-up for 87fea7d8ac
 2023-09-07 15:58:57 +02:00
Aliaksandr Valialkin
dfe44c917c
lib/logger: limit the maximum arg length, which can be emitted to log lines 
This should prevent from emitting too long lines when too long args are passed to logger.* functions.
For example, too long MetricsQL queries or too long data samples.
 2023-09-07 15:28:53 +02:00
dependabot[bot]
5a16d0936b
build(deps): bump actions/checkout from 3 to 4 (#4950) 
Bumps [actions/checkout](https://github.com/actions/checkout) from 3 to 4.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/v3...v4)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-09-07 15:23:52 +02:00
Github Actions
c12b24f2df
Automatic update operator docs from VictoriaMetrics/operator@9c461d2 (#4968) 2023-09-07 13:04:49 +02:00
Aliaksandr Valialkin
1dd19df849
vendor: return back the latest version of golang.org/x/exp/slices, which works correctly with github.com/prometheus/prometheus/model/labels 2023-09-07 12:40:00 +02:00
Aliaksandr Valialkin
5cbb330555
vendor: run make vendor-update 2023-09-07 12:35:42 +02:00
Aliaksandr Valialkin
0bc0d2610c
go.mod: increase the minimum supported Go version from Go1.19 to Go1.20 2023-09-07 12:18:12 +02:00
Aliaksandr Valialkin
1dac069498
docs/CHANGELOG.md: return back accidentally deleted line at 45c0e4bb31 2023-09-07 12:04:14 +02:00
Aliaksandr Valialkin
677612c6cb
all: update Go builder from Go1.21.0 to Go1.21.1 
See https://github.com/golang/go/issues?q=milestone%3AGo1.21.1+label%3ACherryPickApproved
 2023-09-07 11:36:49 +02:00
Aliaksandr Valialkin
dddf092791
docs/CHANGELOG.md: clarify the scope of recent bugfixes 2023-09-07 11:25:18 +02:00
Aliaksandr Valialkin
8c656d0396
SECURITY.md: substitute v1.79.x LTS releases with v1.93.x LTS releases, because v1.79.x is deprecated 2023-09-07 11:19:40 +02:00
Aliaksandr Valialkin
fe739e771b
deployment/docker: properly build armv5 production builds for GOARCH=arm 
Pass GOARM=5 when building GOARCH=arm production builds, since the default value for this env var
has been changed to GOARM=6 since Go1.21.0.

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4965
and https://github.com/golang/go/issues/62475
 2023-09-07 11:19:40 +02:00
hagen1778
740725b7a3
docs: mention that quantiles can't be used in sharded mode 
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4942
Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 40c94b26dd)
 2023-09-07 10:59:19 +02:00