Commit graph

2499 commits

Author SHA1 Message Date
Aliaksandr Valialkin
7f3b5431a1
app/vmselect/graphite: follow-up after c7884f8686
- Consistently use -search.maxGraphiteTagValues for limiting tag values from auto-complete API
- Use -search.maxGraphiteSeries for limiting paths (aka series), which can be returned from Graphite series API
- Clarify the change in docs/CHANGELOG.md

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4339
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2841
2023-07-06 15:21:56 -07:00
Aliaksandr Valialkin
45e345806c
app/vmselect/netstorage: remove runtime.Gosched() call from unpackWorker()
This should improve scalability of unpackWorker() on systems with many CPU cores.
This is a follow-up for a2ecf4fa4a and 16f3b279a2

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3966
2023-07-06 10:05:58 -07:00
Roman Khavronenko
2f710ec77d
vmctl: interrupt explore procedure in influx mode if no numeric fields were found (#4576)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-07-04 13:45:42 +02:00
Sergey
9bea7610f0
security: update base Alpine image to 3.18.2 to avoid security risks (#4571)
libcrypto3 and libssl3 in Alpine 3.18.0 have versions `3.1.0-r4`
which contains CVE-2023-2650:
https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-2650
Use ALpine image 3.18.2 which contains fixed versions of libssl3
and libcrypto3: 3.1.1-r0
NB: In Openshift these containers are marked as vulnerabilities
because of these CVEs.
2023-07-04 09:23:53 +02:00
Dmytro Kozlov
9bde95bfff
app/vmalert: show on UI groups error after reload config (#4543)
show on UI groups error after reload config

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4076

Co-authored-by: hagen1778 <roman@victoriametrics.com>
2023-07-03 14:59:52 +02:00
Zakhar Bessarab
adc07b711e
app/vmagent/remotewrite: fix error message for auth config (#4545)
Error message will be present for any auth error, but message claims an error is about OAuth2 configuration which is confusing.

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2023-07-03 13:12:40 +02:00
Yury Molodov
3cdba1b1c6
vmui: fix app routing issues (#4408)
The change focuses on rectifying inconsistencies in the navigation behavior of the application
and eliminating issues encountered when manually altering the URL.

The key updates include:
- Refactoring of the routing mechanism to handle all possible routes and their states.
- Enhancement of the React Router usage to ensure a smoother navigation experience.
- Handling application state when the URL is manually changed.
2023-06-30 10:13:10 +02:00
Roman Khavronenko
72edc31ffb
vmauth: expose latency metrics per user (#4525)
expose `vmauth_user_request_duration_seconds`
and `vmauth_unauthorized_user_request_duration_seconds` summary metrics
for measuring requests latency per user.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-06-27 20:15:17 +02:00
Roman Khavronenko
c9ddfb118e
vmauth: rm ip filters from non-ent config example (#4526)
It is impossible to run OS vmauth with the provided config.
The example of using ip filters should be only a part of docs.
All other examples should work seamlessly with OS version.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-06-27 20:14:52 +02:00
Haleygo
a97887a2d9
vmalert: add vmalert_remotewrite_sent_duration_seconds_total metric (#4517)
add `vmalert_remotewrite_sent_duration_seconds_total` metric
2023-06-26 07:34:51 +02:00
Roman Khavronenko
37c9a631ca
vmalert: make linter happy (#4509)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-06-22 17:46:12 +02:00
Roman Khavronenko
5f9ad22884
vmalert: update retry policy for pushing data to -remoteWrite.url (#4504)
By default, vmalert will make multiple retry attempts with exponential delay.
The total time spent during retry attempts shouldn't exceed `-remoteWrite.retryMaxTime` (default is 30s).
When retry time is exceeded vmalert drops the data dedicated for `-remoteWrite.url`.
Before, vmalert dropped data after 5 retry attempts with 1s delay between attempts (not configurable).

See `-remoteWrite.retryMinInterval` and `-remoteWrite.retryMaxTime` cmd-line flags.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: Nikolay <nik@victoriametrics.com>
2023-06-22 15:14:23 +02:00
Roman Khavronenko
4aad7a43df
vmalert: properly interrupt remotewrite retries on shutdown (#4505)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-06-22 15:07:32 +02:00
Aliaksandr Valialkin
83aa78dfb4
app/vlstorage: export vl_active_merges and vl_merges_total metrics 2023-06-21 20:58:57 -07:00
Aliaksandr Valialkin
57541d5cea
Revert "app/vlselect/logsql: use buffered writer in order to save syscalls when sending big amounts of data to clients"
This reverts commit c19048dc13.

Reason for revert: it has been appeared that the net/http.ResponseWriter is already buffered,
so there in no need in double bufferring
2023-06-21 20:40:01 -07:00
Aliaksandr Valialkin
c19048dc13
app/vlselect/logsql: use buffered writer in order to save syscalls when sending big amounts of data to clients 2023-06-21 20:25:32 -07:00
Aliaksandr Valialkin
3ded68d4b8
app/vmui/Makefile: consistently use tabs instead of spaces in multi-line Makefile rules 2023-06-21 19:57:48 -07:00
Aliaksandr Valialkin
8be52ef217
app/vlselect: handle vmui at /select/vmui path instead of /vmui
This simplifies routing at auth proxies such as vmauth to vlselect component,
which serves VMUI - just route all the requests, which start with /select/, to vlselect.
2023-06-21 19:52:50 -07:00
Aliaksandr Valialkin
dde9ceed07
app/vlinsert/jsonline: code prettifying 2023-06-21 19:39:22 -07:00
Aliaksandr Valialkin
83a1249299
app/vlselect/logsql: properly handle the error from ParseLogMessage 2023-06-21 10:28:47 -07:00
Dmytro Kozlov
81c1124a0f
app/victoria-logs: remove header control (#4493) 2023-06-21 18:35:42 +02:00
Alexander Marshalov
944793c0f7
removed debug message from jsonlines handler of victorialogs (#4492)
Signed-off-by: Alexander Marshalov <_@marshalov.org>
2023-06-21 09:34:12 -07:00
dmitryk-dk
58c84ad90d app/victoria-logs: add vmui dependecies 2023-06-21 09:08:03 -07:00
Yury Molodov
4a7b17ed76
vmui: logs explorer (#4484)
* feat: add a logs page

* app/vixtoria-logs: add handlers for vmui

* feat: add group logs

* feat: add logs build

* app/vixtoria-logs: update make file

* app/vixtoria-logs: cleanup make

* app/vixtoria-logs: fix description

* fix: correct url for logs

* fix: save display view in query params

* fix: change logo for logs build

* app/vixtoria-logs: remove dashboards from vlselect

* app/vixtoria-logs: enable user

---------

Co-authored-by: dmitryk-dk <kozlovdmitriyy@gmail.com>
2023-06-21 16:57:09 +02:00
Alexander Marshalov
bf081d157e
jsonline support for data ingestion in vlinsert (#4487)
added json lines / json stream format for ingestion to vlinsert
2023-06-21 15:31:28 +02:00
Aliaksandr Valialkin
7346bb4f03
app/vlselect/logsql: sort query results by _time if their summary size doesnt exceed -select.maxSortBufferSize 2023-06-21 01:11:25 -07:00
Roman Khavronenko
94f516df43
docs/vmalert: specify version requirements for new features (#4480)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-06-21 09:44:00 +02:00
Aliaksandr Valialkin
332b295268
docs/VictoriaLogs: change the structure of the docs in order to be more maintainable
The change is based on https://github.com/VictoriaMetrics/VictoriaMetrics/pull/4477
2023-06-20 22:23:39 -07:00
Aliaksandr Valialkin
3edc548584
app/vlinsert/elasticsearch: allow empty lines in Elasticsearch bulk protocol
Empty lines may appear there during debugging and custom client implementation
2023-06-20 21:23:38 -07:00
Aliaksandr Valialkin
07bd118b1a
app/vlinsert/elasticsearch: optimize parsing command line
Just search for "create" or "index" substrings there instead of spending CPU time on its parsing
2023-06-20 21:18:12 -07:00
Aliaksandr Valialkin
e7bf36f0b6
app/vlstorage: log -storageDataPath and basic stats for the opened storage 2023-06-20 20:47:53 -07:00
Aliaksandr Valialkin
00c3dbd15d
app/victoria-logs: add ability to debug data ingestion by passing debug query arg to data ingestion API 2023-06-20 20:02:46 -07:00
Roman Khavronenko
79a5499cb2
vmalert: retry all errors except 4XX status codes (#4461)
vmalert: retry all errors except 4XX status codes

Retry all errors except 4XX status codes while pushing via remote-write
to the remote storage. Previously, errors like broken connection could
prevent vmalert from retrying the request.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-06-20 13:24:45 +02:00
Yury Molodov
66b42a6772
vmui: memory leak fix (#4455)
* fix: optimize the preparation of data for the graph

* fix: optimize tooltip rendering

* fix: optimize re-rendering of the chart

* vmui: memory leak fix
2023-06-20 11:29:24 +02:00
Aliaksandr Valialkin
a638f5e2bf
app/vmctl/utils: properly use timezone in TestGetTime() 2023-06-19 23:16:50 -07:00
Aliaksandr Valialkin
87b66db47d
app/victoria-logs: initial code release 2023-06-19 22:55:12 -07:00
Aliaksandr Valialkin
78eaa056c0
app/vmselect: move common http functionality from app/vmselect/searchutils to lib/httputils
While at it, move app/vmselect/bufferedwriter to lib/bufferedwriter, since it is going to be used in VictoriaLogs
2023-06-19 22:34:20 -07:00
Dmytro Kozlov
7a92263459
vmctl: increase retry backoff policy delay (#4447)
vmctl: update backoff policy on retries to reduce probability of overloading for `source` or `destination` databases
2023-06-14 09:47:44 +02:00
Roman Khavronenko
28b23e7a4c
docs/vmalert: mention same labelset error in docs (#4443)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-06-13 17:03:53 +02:00
Dmytro Kozlov
ddb3ae0f00
vmctl: finish retries if context canceled (#4442)
vmctl: interrupt backoff retries if import context is cancelled

Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
2023-06-13 13:54:24 +02:00
Roman Khavronenko
c4be49e21b
docs: mention stream aggregation as more efficient approach for aggregation (#4429)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-06-09 13:47:54 +02:00
Dmytro Kozlov
24f34347f1
docs: clarify -retentionPeriod flag usage (#4417)
app/vmstorage: clarify the min value for `-retentionPeriod` flag

Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
2023-06-09 09:46:25 +02:00
Roman Khavronenko
476c7bdd6f
all: update Go builder from Go1.20.4 to Go1.20.5 (#4427)
See https://github.com/golang/go/issues?q=milestone%3AGo1.20.5+label%3ACherryPickApproved

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-06-09 09:42:55 +02:00
Zakhar Bessarab
ce7141383d
app/vmagent/remotewrite: fix vmagent panic on shutdown (#4407)
app/vmagent/remotewrite: fix vmagent panic on shutdown

Currently, when vmagent is stopping it first flushes pending series in remote write context and proceeds to stop streaming aggregation. This leads to streaming aggregation being unable to write results into pending timeseries (since it is already nil) and panic.
This can lead to losing some aggregation results being lost almost silently.

The fix is reordering flow to first stop streaming aggregation and flush all pending time series after that.

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2023-06-07 15:45:43 +02:00
Dmytro Kozlov
fc5292d8ed
app/vmctl: add verbose output for docker installations or when TTY isn't available (#4333)
* app/vmctl: add verbose output for docker installations or when TTY isn't available

* app/vmctl: fix tests

* app/vmctl: make vmctl interactive if no tty

* app/vmctl: cleanup

* app/vmctl: add comment

---------

Co-authored-by: Nikolay <nik@victoriametrics.com>
2023-06-02 14:57:08 +02:00
Dmytro Kozlov
c7884f8686
app/{graphite,netstorage,prometheus}: fix graphite search tags api limits, remove redudant limit from SeriesHandler handler (#4352)
* app/{graphite,netstorage,prometheus}: fix graphite search tags api limits, remove unused limit from SeriesHandler handler,

* app/{graphite,netstorage,prometheus}: use search.maxTagValues for Graphite

* app/{graphite,netstorage,prometheus}: update CHANGELOG.md

* app/{graphite,netstorage,prometheus}: use own flags for Graphite API

* app/{graphite,netstorage,prometheus}: cleanup

* app/{graphite,netstorage,prometheus}: cleanup

* app/{graphite,netstorage,prometheus}: update docs

---------

Co-authored-by: Nikolay <nik@victoriametrics.com>
2023-06-02 14:34:04 +02:00
Roman Khavronenko
de94812088
vmalert: fix nil map assignment (#4392)
* vmalert: fix nil map assignment

The storage instance with nil map params was created for remote-read purposes.
And before change 7a9ae9de0d this map was ignored in ApplyParams.
Now, it started to be used and vmalert panics in runtime.

The fix properly inits map for at `NewVMStorage` and verifies it is not nil
on assignment in `ApplyParams`.

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* vmalert: add to changelog

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* vmalert: properly clone Storage params

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* vmalert: properly clone Storage params

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* vmalert: properly clone Storage params

Signed-off-by: hagen1778 <roman@victoriametrics.com>

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-06-02 11:38:55 +02:00
Roman Khavronenko
eccecdf177
app/vmalert: follow-up after 7a9ae9de0d (#4381)
7a9ae9de0d

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-06-01 11:38:48 +02:00
Dmytro Kozlov
9843ec0e1d
app/vmui: fix behavior when changing url in global settings (#4332)
* app/vmui: fix behavior when changing url in global settings

* app/vmctl: minor fix

* app/vmui: fix behavior when changing url in global settings
2023-06-01 12:19:03 +03:00
Roman Khavronenko
8185c2466c
docs: clarify deduplication docs (#4371)
The purpose of the change is too highlight what HA pair is
and how deduplication needs identical labels to be present
in raw samples.

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4367

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-06-01 10:28:21 +02:00