Commit graph

2491 commits

Author SHA1 Message Date
Roman Khavronenko
c9ddfb118e
vmauth: rm ip filters from non-ent config example (#4526)
It is impossible to run OS vmauth with the provided config.
The example of using ip filters should be only a part of docs.
All other examples should work seamlessly with OS version.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-06-27 20:14:52 +02:00
Haleygo
a97887a2d9
vmalert: add vmalert_remotewrite_sent_duration_seconds_total metric (#4517)
add `vmalert_remotewrite_sent_duration_seconds_total` metric
2023-06-26 07:34:51 +02:00
Roman Khavronenko
37c9a631ca
vmalert: make linter happy (#4509)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-06-22 17:46:12 +02:00
Roman Khavronenko
5f9ad22884
vmalert: update retry policy for pushing data to -remoteWrite.url (#4504)
By default, vmalert will make multiple retry attempts with exponential delay.
The total time spent during retry attempts shouldn't exceed `-remoteWrite.retryMaxTime` (default is 30s).
When retry time is exceeded vmalert drops the data dedicated for `-remoteWrite.url`.
Before, vmalert dropped data after 5 retry attempts with 1s delay between attempts (not configurable).

See `-remoteWrite.retryMinInterval` and `-remoteWrite.retryMaxTime` cmd-line flags.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: Nikolay <nik@victoriametrics.com>
2023-06-22 15:14:23 +02:00
Roman Khavronenko
4aad7a43df
vmalert: properly interrupt remotewrite retries on shutdown (#4505)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-06-22 15:07:32 +02:00
Aliaksandr Valialkin
83aa78dfb4
app/vlstorage: export vl_active_merges and vl_merges_total metrics 2023-06-21 20:58:57 -07:00
Aliaksandr Valialkin
57541d5cea
Revert "app/vlselect/logsql: use buffered writer in order to save syscalls when sending big amounts of data to clients"
This reverts commit c19048dc13.

Reason for revert: it has been appeared that the net/http.ResponseWriter is already buffered,
so there in no need in double bufferring
2023-06-21 20:40:01 -07:00
Aliaksandr Valialkin
c19048dc13
app/vlselect/logsql: use buffered writer in order to save syscalls when sending big amounts of data to clients 2023-06-21 20:25:32 -07:00
Aliaksandr Valialkin
3ded68d4b8
app/vmui/Makefile: consistently use tabs instead of spaces in multi-line Makefile rules 2023-06-21 19:57:48 -07:00
Aliaksandr Valialkin
8be52ef217
app/vlselect: handle vmui at /select/vmui path instead of /vmui
This simplifies routing at auth proxies such as vmauth to vlselect component,
which serves VMUI - just route all the requests, which start with /select/, to vlselect.
2023-06-21 19:52:50 -07:00
Aliaksandr Valialkin
dde9ceed07
app/vlinsert/jsonline: code prettifying 2023-06-21 19:39:22 -07:00
Aliaksandr Valialkin
83a1249299
app/vlselect/logsql: properly handle the error from ParseLogMessage 2023-06-21 10:28:47 -07:00
Dmytro Kozlov
81c1124a0f
app/victoria-logs: remove header control (#4493) 2023-06-21 18:35:42 +02:00
Alexander Marshalov
944793c0f7
removed debug message from jsonlines handler of victorialogs (#4492)
Signed-off-by: Alexander Marshalov <_@marshalov.org>
2023-06-21 09:34:12 -07:00
dmitryk-dk
58c84ad90d app/victoria-logs: add vmui dependecies 2023-06-21 09:08:03 -07:00
Yury Molodov
4a7b17ed76
vmui: logs explorer (#4484)
* feat: add a logs page

* app/vixtoria-logs: add handlers for vmui

* feat: add group logs

* feat: add logs build

* app/vixtoria-logs: update make file

* app/vixtoria-logs: cleanup make

* app/vixtoria-logs: fix description

* fix: correct url for logs

* fix: save display view in query params

* fix: change logo for logs build

* app/vixtoria-logs: remove dashboards from vlselect

* app/vixtoria-logs: enable user

---------

Co-authored-by: dmitryk-dk <kozlovdmitriyy@gmail.com>
2023-06-21 16:57:09 +02:00
Alexander Marshalov
bf081d157e
jsonline support for data ingestion in vlinsert (#4487)
added json lines / json stream format for ingestion to vlinsert
2023-06-21 15:31:28 +02:00
Aliaksandr Valialkin
7346bb4f03
app/vlselect/logsql: sort query results by _time if their summary size doesnt exceed -select.maxSortBufferSize 2023-06-21 01:11:25 -07:00
Roman Khavronenko
94f516df43
docs/vmalert: specify version requirements for new features (#4480)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-06-21 09:44:00 +02:00
Aliaksandr Valialkin
332b295268
docs/VictoriaLogs: change the structure of the docs in order to be more maintainable
The change is based on https://github.com/VictoriaMetrics/VictoriaMetrics/pull/4477
2023-06-20 22:23:39 -07:00
Aliaksandr Valialkin
3edc548584
app/vlinsert/elasticsearch: allow empty lines in Elasticsearch bulk protocol
Empty lines may appear there during debugging and custom client implementation
2023-06-20 21:23:38 -07:00
Aliaksandr Valialkin
07bd118b1a
app/vlinsert/elasticsearch: optimize parsing command line
Just search for "create" or "index" substrings there instead of spending CPU time on its parsing
2023-06-20 21:18:12 -07:00
Aliaksandr Valialkin
e7bf36f0b6
app/vlstorage: log -storageDataPath and basic stats for the opened storage 2023-06-20 20:47:53 -07:00
Aliaksandr Valialkin
00c3dbd15d
app/victoria-logs: add ability to debug data ingestion by passing debug query arg to data ingestion API 2023-06-20 20:02:46 -07:00
Roman Khavronenko
79a5499cb2
vmalert: retry all errors except 4XX status codes (#4461)
vmalert: retry all errors except 4XX status codes

Retry all errors except 4XX status codes while pushing via remote-write
to the remote storage. Previously, errors like broken connection could
prevent vmalert from retrying the request.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-06-20 13:24:45 +02:00
Yury Molodov
66b42a6772
vmui: memory leak fix (#4455)
* fix: optimize the preparation of data for the graph

* fix: optimize tooltip rendering

* fix: optimize re-rendering of the chart

* vmui: memory leak fix
2023-06-20 11:29:24 +02:00
Aliaksandr Valialkin
a638f5e2bf
app/vmctl/utils: properly use timezone in TestGetTime() 2023-06-19 23:16:50 -07:00
Aliaksandr Valialkin
87b66db47d
app/victoria-logs: initial code release 2023-06-19 22:55:12 -07:00
Aliaksandr Valialkin
78eaa056c0
app/vmselect: move common http functionality from app/vmselect/searchutils to lib/httputils
While at it, move app/vmselect/bufferedwriter to lib/bufferedwriter, since it is going to be used in VictoriaLogs
2023-06-19 22:34:20 -07:00
Dmytro Kozlov
7a92263459
vmctl: increase retry backoff policy delay (#4447)
vmctl: update backoff policy on retries to reduce probability of overloading for `source` or `destination` databases
2023-06-14 09:47:44 +02:00
Roman Khavronenko
28b23e7a4c
docs/vmalert: mention same labelset error in docs (#4443)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-06-13 17:03:53 +02:00
Dmytro Kozlov
ddb3ae0f00
vmctl: finish retries if context canceled (#4442)
vmctl: interrupt backoff retries if import context is cancelled

Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
2023-06-13 13:54:24 +02:00
Roman Khavronenko
c4be49e21b
docs: mention stream aggregation as more efficient approach for aggregation (#4429)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-06-09 13:47:54 +02:00
Dmytro Kozlov
24f34347f1
docs: clarify -retentionPeriod flag usage (#4417)
app/vmstorage: clarify the min value for `-retentionPeriod` flag

Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
2023-06-09 09:46:25 +02:00
Roman Khavronenko
476c7bdd6f
all: update Go builder from Go1.20.4 to Go1.20.5 (#4427)
See https://github.com/golang/go/issues?q=milestone%3AGo1.20.5+label%3ACherryPickApproved

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-06-09 09:42:55 +02:00
Zakhar Bessarab
ce7141383d
app/vmagent/remotewrite: fix vmagent panic on shutdown (#4407)
app/vmagent/remotewrite: fix vmagent panic on shutdown

Currently, when vmagent is stopping it first flushes pending series in remote write context and proceeds to stop streaming aggregation. This leads to streaming aggregation being unable to write results into pending timeseries (since it is already nil) and panic.
This can lead to losing some aggregation results being lost almost silently.

The fix is reordering flow to first stop streaming aggregation and flush all pending time series after that.

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2023-06-07 15:45:43 +02:00
Dmytro Kozlov
fc5292d8ed
app/vmctl: add verbose output for docker installations or when TTY isn't available (#4333)
* app/vmctl: add verbose output for docker installations or when TTY isn't available

* app/vmctl: fix tests

* app/vmctl: make vmctl interactive if no tty

* app/vmctl: cleanup

* app/vmctl: add comment

---------

Co-authored-by: Nikolay <nik@victoriametrics.com>
2023-06-02 14:57:08 +02:00
Dmytro Kozlov
c7884f8686
app/{graphite,netstorage,prometheus}: fix graphite search tags api limits, remove redudant limit from SeriesHandler handler (#4352)
* app/{graphite,netstorage,prometheus}: fix graphite search tags api limits, remove unused limit from SeriesHandler handler,

* app/{graphite,netstorage,prometheus}: use search.maxTagValues for Graphite

* app/{graphite,netstorage,prometheus}: update CHANGELOG.md

* app/{graphite,netstorage,prometheus}: use own flags for Graphite API

* app/{graphite,netstorage,prometheus}: cleanup

* app/{graphite,netstorage,prometheus}: cleanup

* app/{graphite,netstorage,prometheus}: update docs

---------

Co-authored-by: Nikolay <nik@victoriametrics.com>
2023-06-02 14:34:04 +02:00
Roman Khavronenko
de94812088
vmalert: fix nil map assignment (#4392)
* vmalert: fix nil map assignment

The storage instance with nil map params was created for remote-read purposes.
And before change 7a9ae9de0d this map was ignored in ApplyParams.
Now, it started to be used and vmalert panics in runtime.

The fix properly inits map for at `NewVMStorage` and verifies it is not nil
on assignment in `ApplyParams`.

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* vmalert: add to changelog

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* vmalert: properly clone Storage params

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* vmalert: properly clone Storage params

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* vmalert: properly clone Storage params

Signed-off-by: hagen1778 <roman@victoriametrics.com>

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-06-02 11:38:55 +02:00
Roman Khavronenko
eccecdf177
app/vmalert: follow-up after 7a9ae9de0d (#4381)
7a9ae9de0d

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-06-01 11:38:48 +02:00
Dmytro Kozlov
9843ec0e1d
app/vmui: fix behavior when changing url in global settings (#4332)
* app/vmui: fix behavior when changing url in global settings

* app/vmctl: minor fix

* app/vmui: fix behavior when changing url in global settings
2023-06-01 12:19:03 +03:00
Roman Khavronenko
8185c2466c
docs: clarify deduplication docs (#4371)
The purpose of the change is too highlight what HA pair is
and how deduplication needs identical labels to be present
in raw samples.

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4367

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-06-01 10:28:21 +02:00
gsakun
20dc3db71e
app/vmalert: fix datasource.roundDigits Parameter (#4341)
app/vmalert: fix querybuild clone and extraParams merge logic

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4340
2023-06-01 09:44:11 +02:00
Roman Khavronenko
1b24f4729b
vmalert: mention default value for external.url flag (#4365)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-05-30 12:33:17 +02:00
Artem Navoiev
83c1944184
docs: fix markdown for title
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2023-05-29 22:22:25 +02:00
Artem Navoiev
6967c3cb95 fix image path
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2023-05-29 07:11:15 -07:00
Artem Navoiev
8a4c89ea22 docs: change images from markdown tag to html for migration
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2023-05-29 07:11:15 -07:00
Roman Khavronenko
51cea6cad4
vmalert: properly form assets address if httpPrefix set (#4351)
Properly form path to static assets in WEB UI
 if `http.pathPrefix` set.

 https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4349

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-05-29 07:38:13 +02:00
Nikolay
228ea03bda
app/vmselect/graphite: fixes tests for arm (#4348)
at arm based CPUs only 9 digits after comma matches for tests.
Especially at holtWinters functions. Since it only takes effect at tests
it makes no sense for changing float prescision at actual functions
2023-05-26 09:34:15 +02:00
Dmytro Kozlov
4060f3f261
app/vmctl: fix tests (#4345) 2023-05-25 17:04:04 +02:00