Commit graph

2596 commits

Author SHA1 Message Date
Aliaksandr Valialkin
772852ff4f
app/vlselect/logsql: properly handle the error from ParseLogMessage 2023-07-06 21:33:22 -07:00
Dmytro Kozlov
caf4743e45
app/victoria-logs: remove header control (#4493) 2023-07-06 21:33:00 -07:00
Alexander Marshalov
db910dd336
removed debug message from jsonlines handler of victorialogs (#4492)
Signed-off-by: Alexander Marshalov <_@marshalov.org>
2023-07-06 21:32:38 -07:00
dmitryk-dk
058fbbdb16
app/victoria-logs: add vmui dependecies 2023-07-06 21:32:19 -07:00
Yury Molodov
a04a206cd2
vmui: logs explorer (#4484)
* feat: add a logs page

* app/vixtoria-logs: add handlers for vmui

* feat: add group logs

* feat: add logs build

* app/vixtoria-logs: update make file

* app/vixtoria-logs: cleanup make

* app/vixtoria-logs: fix description

* fix: correct url for logs

* fix: save display view in query params

* fix: change logo for logs build

* app/vixtoria-logs: remove dashboards from vlselect

* app/vixtoria-logs: enable user

---------

Co-authored-by: dmitryk-dk <kozlovdmitriyy@gmail.com>
2023-07-06 21:31:33 -07:00
Alexander Marshalov
d9d759bc90
jsonline support for data ingestion in vlinsert (#4487)
added json lines / json stream format for ingestion to vlinsert
2023-07-06 21:30:35 -07:00
Aliaksandr Valialkin
efee71986f
app/vlselect/logsql: sort query results by _time if their summary size doesnt exceed -select.maxSortBufferSize 2023-07-06 21:25:00 -07:00
Roman Khavronenko
4e99bf8c9e
docs/vmalert: specify version requirements for new features (#4480)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-07-06 21:24:38 -07:00
Aliaksandr Valialkin
fd6c2dd02e
docs/VictoriaLogs: change the structure of the docs in order to be more maintainable
The change is based on https://github.com/VictoriaMetrics/VictoriaMetrics/pull/4477
2023-07-06 21:22:59 -07:00
Aliaksandr Valialkin
e21b3bceab
app/vlinsert/elasticsearch: allow empty lines in Elasticsearch bulk protocol
Empty lines may appear there during debugging and custom client implementation
2023-07-06 21:22:22 -07:00
Aliaksandr Valialkin
7f146543ef
app/vlinsert/elasticsearch: optimize parsing command line
Just search for "create" or "index" substrings there instead of spending CPU time on its parsing
2023-07-06 21:21:57 -07:00
Aliaksandr Valialkin
c8cffc9b1b
app/vlstorage: log -storageDataPath and basic stats for the opened storage 2023-07-06 21:21:00 -07:00
Aliaksandr Valialkin
1c39af56ab
app/victoria-logs: add ability to debug data ingestion by passing debug query arg to data ingestion API 2023-07-06 21:19:58 -07:00
Roman Khavronenko
d4ee505f6f
vmalert: retry all errors except 4XX status codes (#4461)
vmalert: retry all errors except 4XX status codes

Retry all errors except 4XX status codes while pushing via remote-write
to the remote storage. Previously, errors like broken connection could
prevent vmalert from retrying the request.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-07-06 17:34:32 -07:00
Yury Molodov
0ad966a898
vmui: memory leak fix (#4455)
* fix: optimize the preparation of data for the graph

* fix: optimize tooltip rendering

* fix: optimize re-rendering of the chart

* vmui: memory leak fix
2023-07-06 17:33:54 -07:00
Aliaksandr Valialkin
2db55cd081
app/vmctl/utils: properly use timezone in TestGetTime() 2023-07-06 17:30:51 -07:00
Aliaksandr Valialkin
374890294e
app/victoria-logs: initial code release 2023-07-06 17:30:05 -07:00
Aliaksandr Valialkin
427ce69426
app/vmselect: move common http functionality from app/vmselect/searchutils to lib/httputils
While at it, move app/vmselect/bufferedwriter to lib/bufferedwriter, since it is going to be used in VictoriaLogs
2023-07-06 17:22:23 -07:00
Dmytro Kozlov
b32a270da7
vmctl: increase retry backoff policy delay (#4447)
vmctl: update backoff policy on retries to reduce probability of overloading for `source` or `destination` databases
2023-07-06 17:00:06 -07:00
Roman Khavronenko
b76c0d182c
docs/vmalert: mention same labelset error in docs (#4443)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-07-06 16:56:42 -07:00
Dmytro Kozlov
2e81c5f740
vmctl: finish retries if context canceled (#4442)
vmctl: interrupt backoff retries if import context is cancelled

Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
2023-07-06 16:56:00 -07:00
Roman Khavronenko
91612b38cd
docs: mention stream aggregation as more efficient approach for aggregation (#4429)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-07-06 16:46:17 -07:00
Aliaksandr Valialkin
dff199a745
app/vmselect/graphite: follow-up after c7884f8686
- Consistently use -search.maxGraphiteTagValues for limiting tag values from auto-complete API
- Use -search.maxGraphiteSeries for limiting paths (aka series), which can be returned from Graphite series API
- Clarify the change in docs/CHANGELOG.md

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4339
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2841
2023-07-06 15:19:07 -07:00
Aliaksandr Valialkin
eb47ad4b69
app/vmselect/netstorage: remove runtime.Gosched() call from unpackWorker()
This should improve scalability of unpackWorker() on systems with many CPU cores.
This is a follow-up for a2ecf4fa4a and 16f3b279a2

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3966
2023-07-06 10:07:42 -07:00
Aliaksandr Valialkin
ec75d9097d
app/vmselect/netstorage: follow-up after 11ac551d52
- Clarify the scope of the fix at docs/CHANGELOG.md
- Handle the case when -search.maxSamplesPerSeries limit is exceeded
  in the same way as the -search.maxSamplesPerQuery limit.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/4472
2023-07-05 21:13:34 -07:00
Aliaksandr Valialkin
643e99a157
app/vmselect/netstorage: improve code readability a bit after 6c84b61893
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4364
2023-07-05 20:48:38 -07:00
Roman Khavronenko
11ac551d52
app/vmselect/netstorage: properly process -search.maxSamplesPerQuery limit (#4472)
Properly return the error to user when `-search.maxSamplesPerQuery` limit is exceeded.
Before, user could have received a partial response instead.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-06-23 13:17:34 +02:00
Dmytro Kozlov
88ac6116bd
docs: clarify -retentionPeriod flag usage (#4417)
app/vmstorage: clarify the min value for `-retentionPeriod` flag

Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>

(cherry picked from commit 24f34347f1)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-06-09 10:44:22 +02:00
Roman Khavronenko
4624fda00d
all: update Go builder from Go1.20.4 to Go1.20.5 (#4427)
See https://github.com/golang/go/issues?q=milestone%3AGo1.20.5+label%3ACherryPickApproved

Signed-off-by: hagen1778 <roman@victoriametrics.com>

(cherry picked from commit 476c7bdd6f)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-06-09 10:42:15 +02:00
Zakhar Bessarab
7925e9698f
app/vmagent/remotewrite: fix vmagent panic on shutdown (#4407)
app/vmagent/remotewrite: fix vmagent panic on shutdown

Currently, when vmagent is stopping it first flushes pending series in remote write context and proceeds to stop streaming aggregation. This leads to streaming aggregation being unable to write results into pending timeseries (since it is already nil) and panic.
This can lead to losing some aggregation results being lost almost silently.

The fix is reordering flow to first stop streaming aggregation and flush all pending time series after that.

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
(cherry picked from commit ce7141383d)
2023-06-09 10:40:52 +02:00
Dmytro Kozlov
dd89fb2e12
app/vmctl: add verbose output for docker installations or when TTY isn't available (#4333)
* app/vmctl: add verbose output for docker installations or when TTY isn't available

* app/vmctl: fix tests

* app/vmctl: make vmctl interactive if no tty

* app/vmctl: cleanup

* app/vmctl: add comment

---------

Co-authored-by: Nikolay <nik@victoriametrics.com>

(cherry picked from commit fc5292d8ed)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-06-09 10:39:38 +02:00
Dmytro Kozlov
c5debee3f4
app/{graphite,netstorage,prometheus}: fix graphite search tags api limits, remove redudant limit from SeriesHandler handler (#4352)
* app/{graphite,netstorage,prometheus}: fix graphite search tags api limits, remove unused limit from SeriesHandler handler,

* app/{graphite,netstorage,prometheus}: use search.maxTagValues for Graphite

* app/{graphite,netstorage,prometheus}: update CHANGELOG.md

* app/{graphite,netstorage,prometheus}: use own flags for Graphite API

* app/{graphite,netstorage,prometheus}: cleanup

* app/{graphite,netstorage,prometheus}: cleanup

* app/{graphite,netstorage,prometheus}: update docs

---------

Co-authored-by: Nikolay <nik@victoriametrics.com>

(cherry picked from commit c7884f8686)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-06-09 10:39:12 +02:00
Roman Khavronenko
a1b6a9317d
vmalert: fix nil map assignment (#4392)
* vmalert: fix nil map assignment

The storage instance with nil map params was created for remote-read purposes.
And before change 7a9ae9de0d this map was ignored in ApplyParams.
Now, it started to be used and vmalert panics in runtime.

The fix properly inits map for at `NewVMStorage` and verifies it is not nil
on assignment in `ApplyParams`.

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* vmalert: add to changelog

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* vmalert: properly clone Storage params

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* vmalert: properly clone Storage params

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* vmalert: properly clone Storage params

Signed-off-by: hagen1778 <roman@victoriametrics.com>

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit de94812088)
2023-06-02 13:29:51 +02:00
Roman Khavronenko
fde5a59726
app/vmalert: follow-up after 7a9ae9de0d (#4381)
7a9ae9de0d

Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit eccecdf177)
2023-06-02 13:29:47 +02:00
Dmytro Kozlov
ce98e68fe1
app/vmui: fix behavior when changing url in global settings (#4332)
* app/vmui: fix behavior when changing url in global settings

* app/vmctl: minor fix

* app/vmui: fix behavior when changing url in global settings

(cherry picked from commit 9843ec0e1d)
2023-06-02 13:29:46 +02:00
Roman Khavronenko
7da0c37d2b
docs: clarify deduplication docs (#4371)
The purpose of the change is too highlight what HA pair is
and how deduplication needs identical labels to be present
in raw samples.

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4367

Signed-off-by: hagen1778 <roman@victoriametrics.com>

(cherry picked from commit 8185c2466c)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-06-02 13:29:45 +02:00
gsakun
72b72f06a0
app/vmalert: fix datasource.roundDigits Parameter (#4341)
app/vmalert: fix querybuild clone and extraParams merge logic

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4340

(cherry picked from commit 20dc3db71e)
2023-06-02 13:29:16 +02:00
Roman Khavronenko
288e8fec6b
vmalert: mention default value for external.url flag (#4365)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 1b24f4729b)
2023-06-02 13:28:08 +02:00
Artem Navoiev
27960fd6ec
docs: fix markdown for title
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
(cherry picked from commit 83c1944184)
2023-06-02 13:28:04 +02:00
Artem Navoiev
bbd1dd9d03
fix image path
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
(cherry picked from commit 6967c3cb95)
2023-06-02 13:28:04 +02:00
Artem Navoiev
756f0e1545
docs: change images from markdown tag to html for migration
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
(cherry picked from commit 8a4c89ea22)
2023-06-02 13:28:04 +02:00
Roman Khavronenko
74c4c1d909
vmalert: properly form assets address if httpPrefix set (#4351)
Properly form path to static assets in WEB UI
 if `http.pathPrefix` set.

 https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4349

Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 51cea6cad4)
2023-06-02 13:28:03 +02:00
Nikolay
e3ce736ce2
app/vmselect/graphite: fixes tests for arm (#4348)
at arm based CPUs only 9 digits after comma matches for tests.
Especially at holtWinters functions. Since it only takes effect at tests
it makes no sense for changing float prescision at actual functions

(cherry picked from commit 228ea03bda)
2023-06-02 13:19:34 +02:00
Roman Khavronenko
033e1ac3d2
vmalert: do not return nil rules for /api/v1/rules (#4344)
The fix addresses a case when vmalert is configured with a group
which has `name`, but doesn't have `rules` configured. In this
case it still returns a `nil` instead of `[]` slice.

Fixing this via current commit.

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4221

Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 66ed6fe62f)
2023-06-02 13:19:33 +02:00
Roman Khavronenko
5950a94e63
vmalert: fix the typo in popup (#4331)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 76f7e66d8e)
2023-06-02 13:19:32 +02:00
Roman Khavronenko
576e59d82c
cluster: standardize default HTTP responses (#4368)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-06-01 10:26:52 +02:00
Haleygo
6c84b61893
vmselect:fix init sn take too much time (#4366)
* vmselect: descrease start time for vmselect

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4364
2023-05-30 13:04:31 +02:00
Aliaksandr Valialkin
934a7f485c
app/vmselect: log locations of sendPrometheusError() calls
Previously the location inside the sendPrometheusError() was logged.
This could make hard investigating error locations via `vm_log_messages_total` metric.
2023-05-18 20:39:50 -07:00
Aliaksandr Valialkin
1ff67bb036
app/vmselect/vmui: run make vmui-update after 39c1b0f8d1 2023-05-18 12:15:22 -07:00
Dmytro Kozlov
6da5cfa3a8
app/vmctl: set default value for --vm-native-step-interval flag (#4327)
* app/vmctl: set default value for `--vm-native-step-interval` flag

* app/vmctl: update CHANGELOG.md

* app/vmctl: update CHANGELOG.md, fix docs

* app/vmctl: fix typo

* app/vmctl: fix typo
2023-05-18 11:03:26 -07:00