Commit graph

2632 commits

Author SHA1 Message Date
Dmytro Kozlov
f31ac064f9
app/vmctl: fix panic --remote-read-filter-time-start flag not defined (#4605)
* app/vmctl: fix panic `--remote-read-filter-time-start` flag not defined

* app/vmctl: update CHANGELOG.md

---------

Co-authored-by: Nikolay <nik@victoriametrics.com>
2023-07-13 12:13:21 -07:00
Roman Khavronenko
fdccb56620
vmalert: check for negative offset for missed rounds (#4628)
It could happen for low evaluation intervals and irregular
delays during execution that evaluation time would get
a negative offset. This could result into cumulative
discrepancy between the actual time and evaluation time for rules.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-07-13 12:05:52 -07:00
Aliaksandr Valialkin
b07a1c85b9
all: update Go builder from 1.20.5 to 1.20.6
See https://github.com/golang/go/issues?q=milestone%3AGo1.20.6+label%3ACherryPickApproved
2023-07-12 01:00:24 -07:00
Dmytro Kozlov
5c4ca4aea8
app/vmctl: remove undefined flag from the documentation. See: https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4552. (#4606) 2023-07-10 15:01:54 -07:00
Aliaksandr Valialkin
f65153018b
app/{vmselect,vlselect}: run make vmui-update vmui-logs-update 2023-07-09 12:44:04 -07:00
Zakhar Bessarab
ddd918b93c
docs: make httpAuth.* flags description less ambiguous (#4588)
* docs: make `httpAuth.*` flags description less ambiguous

Currently, it may confuse users whether `httpAuth.*` flags are used by HTTP client or server configuration(see https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4586 for example).

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* docs: fix a typo

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

---------

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2023-07-09 12:36:14 -07:00
Haleygo
ef8e3eb9b3
vmselect: fix result in Prometheus query when time is small (#4578)
vmselect: fix result in Prometheus query when time is small

Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
2023-07-09 12:33:29 -07:00
Aliaksandr Valialkin
e1a2404db5
app/vmselect/netstorage: follow-up after 173ccf4333
- Clarify docs about -replicationFactor command-line flag at vmselect
- Clarify description for -replicationFactor and -search.skipSlowReplicas command-line flags
- Fix the logic for returning responses if -search.skipSlowReplicas command-line flag
  is enabled. The logic was broken in the 173ccf4333,
  so it could return responses only if some of vmstorage nodes return error,
  while it should return when query results are successfully collected from more than
  (len(storageNodes) - replicationFactor) vmstorage nodes.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1207
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/711
2023-07-09 11:58:22 -07:00
Haleygo
3c2308fd52
vmalert:fix query request using rfc3339 format (#4577)
vmalert: consistently use time.RFC3339 format for time in queries

Co-authored-by: hagen1778 <roman@victoriametrics.com>
2023-07-09 11:03:10 -07:00
Haleygo
14e242d0b9
vmselect: fix result collect count (#4599) 2023-07-08 08:21:27 +02:00
Roman Khavronenko
173ccf4333
vmselect: introduce search.skipSlowReplicas cmd-line flag (#4538)
* vmselect: introduce `search.skipSlowReplicas` cmd-line flag

vmselect has two logical conditions during request processing when
`-replicationFactor` cmd-line flag is set:
1. If at least `len(storageNodes) - replicationFactor` responded, it could skip
waiting for the rest of nodes to respond. This could lead to problems described
here https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1207.
2. Mark response as partial if less than `len(storageNodes) - replicationFactor` responded
without an error.

The P1 showed itself error-prone and became the main reason why
`-replicationFactor` wasn't recommended to use at vmselect level.
However, this optimization could be still very useful in situations
when there are slow and fast replicas in cluster.

But P2 remains viable and important conditionless.
Hiding P1 behind the feature-flag `search.skipSlowReplicas`
should make `-replicationFactor` flag usable again. And let users
choose whether they want P1 to be respected.

Related issues
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1207
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/711

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* docs: update changelog

Signed-off-by: hagen1778 <roman@victoriametrics.com>

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-07-07 11:50:26 +02:00
Roman Khavronenko
109e55f865
vmalert: allow disabling of step param attached to instant queries (#4574)
vmalert: allow disabling of `step` param attached to instant queries

This might be useful for using vmalert with datasources that to not support this param,
unlike VictoriaMetrics.

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4573

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-07-06 23:13:56 -07:00
Aliaksandr Valialkin
0107d78639
docs/vmgateway.md: update -help output 2023-07-06 23:07:47 -07:00
Aliaksandr Valialkin
ee4280d132
docs/vmbackupmanager.md: update -help output 2023-07-06 22:57:31 -07:00
Aliaksandr Valialkin
921d8b36b5
docs/vmrestore.md: update -help output 2023-07-06 22:55:26 -07:00
Aliaksandr Valialkin
e1993dadc2
docs/vmbackup.md: update -help output 2023-07-06 22:54:15 -07:00
Aliaksandr Valialkin
e35abdd2e4
docs/vmauth.md: update -help output 2023-07-06 22:52:48 -07:00
Aliaksandr Valialkin
316abe550d
docs/vmalert.md: update -help output 2023-07-06 22:50:47 -07:00
Aliaksandr Valialkin
b9790515e4
docs/vmagent.md: update -help output 2023-07-06 22:48:23 -07:00
Roman Khavronenko
bd5abb74fd
vmctl: interrupt explore procedure in influx mode if no numeric fields were found (#4576)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-07-06 22:21:18 -07:00
Sergey
747c39d714
security: update base Alpine image to 3.18.2 to avoid security risks (#4571)
libcrypto3 and libssl3 in Alpine 3.18.0 have versions `3.1.0-r4`
which contains CVE-2023-2650:
https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-2650
Use ALpine image 3.18.2 which contains fixed versions of libssl3
and libcrypto3: 3.1.1-r0
NB: In Openshift these containers are marked as vulnerabilities
because of these CVEs.
2023-07-06 22:12:20 -07:00
Dmytro Kozlov
dd412a3757
app/vmalert: show on UI groups error after reload config (#4543)
show on UI groups error after reload config

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4076

Co-authored-by: hagen1778 <roman@victoriametrics.com>
2023-07-06 22:11:36 -07:00
Zakhar Bessarab
e42f856b56
app/vmagent/remotewrite: fix error message for auth config (#4545)
Error message will be present for any auth error, but message claims an error is about OAuth2 configuration which is confusing.

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2023-07-06 22:10:13 -07:00
Yury Molodov
8c190ec8fb
vmui: fix app routing issues (#4408)
The change focuses on rectifying inconsistencies in the navigation behavior of the application
and eliminating issues encountered when manually altering the URL.

The key updates include:
- Refactoring of the routing mechanism to handle all possible routes and their states.
- Enhancement of the React Router usage to ensure a smoother navigation experience.
- Handling application state when the URL is manually changed.
2023-07-06 21:58:09 -07:00
Roman Khavronenko
cf433c066a
vmauth: expose latency metrics per user (#4525)
expose `vmauth_user_request_duration_seconds`
and `vmauth_unauthorized_user_request_duration_seconds` summary metrics
for measuring requests latency per user.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-07-06 21:55:37 -07:00
Roman Khavronenko
8a15397b5c
vmauth: rm ip filters from non-ent config example (#4526)
It is impossible to run OS vmauth with the provided config.
The example of using ip filters should be only a part of docs.
All other examples should work seamlessly with OS version.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-07-06 21:52:51 -07:00
Haleygo
9e49a9e924
vmalert: add vmalert_remotewrite_sent_duration_seconds_total metric (#4517)
add `vmalert_remotewrite_sent_duration_seconds_total` metric
2023-07-06 21:51:31 -07:00
Roman Khavronenko
a677509b38
vmalert: make linter happy (#4509)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-07-06 21:46:22 -07:00
Roman Khavronenko
d5e7ea5ef3
vmalert: update retry policy for pushing data to -remoteWrite.url (#4504)
By default, vmalert will make multiple retry attempts with exponential delay.
The total time spent during retry attempts shouldn't exceed `-remoteWrite.retryMaxTime` (default is 30s).
When retry time is exceeded vmalert drops the data dedicated for `-remoteWrite.url`.
Before, vmalert dropped data after 5 retry attempts with 1s delay between attempts (not configurable).

See `-remoteWrite.retryMinInterval` and `-remoteWrite.retryMaxTime` cmd-line flags.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: Nikolay <nik@victoriametrics.com>
2023-07-06 21:44:18 -07:00
Roman Khavronenko
311a81c7b0
vmalert: properly interrupt remotewrite retries on shutdown (#4505)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-07-06 21:43:04 -07:00
Aliaksandr Valialkin
a9eb2409ea
app/vlstorage: export vl_active_merges and vl_merges_total metrics 2023-07-06 21:38:09 -07:00
Aliaksandr Valialkin
352429486a
Revert "app/vlselect/logsql: use buffered writer in order to save syscalls when sending big amounts of data to clients"
This reverts commit c19048dc13.

Reason for revert: it has been appeared that the net/http.ResponseWriter is already buffered,
so there in no need in double bufferring
2023-07-06 21:37:38 -07:00
Aliaksandr Valialkin
19870d42c5
app/vlselect/logsql: use buffered writer in order to save syscalls when sending big amounts of data to clients 2023-07-06 21:37:38 -07:00
Aliaksandr Valialkin
33625610c6
app/vmui/Makefile: consistently use tabs instead of spaces in multi-line Makefile rules 2023-07-06 21:37:38 -07:00
Aliaksandr Valialkin
4b10432435
app/vlselect: handle vmui at /select/vmui path instead of /vmui
This simplifies routing at auth proxies such as vmauth to vlselect component,
which serves VMUI - just route all the requests, which start with /select/, to vlselect.
2023-07-06 21:36:28 -07:00
Aliaksandr Valialkin
08634ae612
app/vlinsert/jsonline: code prettifying 2023-07-06 21:35:55 -07:00
Aliaksandr Valialkin
772852ff4f
app/vlselect/logsql: properly handle the error from ParseLogMessage 2023-07-06 21:33:22 -07:00
Dmytro Kozlov
caf4743e45
app/victoria-logs: remove header control (#4493) 2023-07-06 21:33:00 -07:00
Alexander Marshalov
db910dd336
removed debug message from jsonlines handler of victorialogs (#4492)
Signed-off-by: Alexander Marshalov <_@marshalov.org>
2023-07-06 21:32:38 -07:00
dmitryk-dk
058fbbdb16
app/victoria-logs: add vmui dependecies 2023-07-06 21:32:19 -07:00
Yury Molodov
a04a206cd2
vmui: logs explorer (#4484)
* feat: add a logs page

* app/vixtoria-logs: add handlers for vmui

* feat: add group logs

* feat: add logs build

* app/vixtoria-logs: update make file

* app/vixtoria-logs: cleanup make

* app/vixtoria-logs: fix description

* fix: correct url for logs

* fix: save display view in query params

* fix: change logo for logs build

* app/vixtoria-logs: remove dashboards from vlselect

* app/vixtoria-logs: enable user

---------

Co-authored-by: dmitryk-dk <kozlovdmitriyy@gmail.com>
2023-07-06 21:31:33 -07:00
Alexander Marshalov
d9d759bc90
jsonline support for data ingestion in vlinsert (#4487)
added json lines / json stream format for ingestion to vlinsert
2023-07-06 21:30:35 -07:00
Aliaksandr Valialkin
efee71986f
app/vlselect/logsql: sort query results by _time if their summary size doesnt exceed -select.maxSortBufferSize 2023-07-06 21:25:00 -07:00
Roman Khavronenko
4e99bf8c9e
docs/vmalert: specify version requirements for new features (#4480)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-07-06 21:24:38 -07:00
Aliaksandr Valialkin
fd6c2dd02e
docs/VictoriaLogs: change the structure of the docs in order to be more maintainable
The change is based on https://github.com/VictoriaMetrics/VictoriaMetrics/pull/4477
2023-07-06 21:22:59 -07:00
Aliaksandr Valialkin
e21b3bceab
app/vlinsert/elasticsearch: allow empty lines in Elasticsearch bulk protocol
Empty lines may appear there during debugging and custom client implementation
2023-07-06 21:22:22 -07:00
Aliaksandr Valialkin
7f146543ef
app/vlinsert/elasticsearch: optimize parsing command line
Just search for "create" or "index" substrings there instead of spending CPU time on its parsing
2023-07-06 21:21:57 -07:00
Aliaksandr Valialkin
c8cffc9b1b
app/vlstorage: log -storageDataPath and basic stats for the opened storage 2023-07-06 21:21:00 -07:00
Aliaksandr Valialkin
1c39af56ab
app/victoria-logs: add ability to debug data ingestion by passing debug query arg to data ingestion API 2023-07-06 21:19:58 -07:00
Roman Khavronenko
d4ee505f6f
vmalert: retry all errors except 4XX status codes (#4461)
vmalert: retry all errors except 4XX status codes

Retry all errors except 4XX status codes while pushing via remote-write
to the remote storage. Previously, errors like broken connection could
prevent vmalert from retrying the request.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-07-06 17:34:32 -07:00