Commit graph

2489 commits

Author SHA1 Message Date
Aliaksandr Valialkin
b83024330f
wip 2024-05-25 18:26:07 +02:00
Aliaksandr Valialkin
bbb9887ccf
wip 2024-05-25 17:29:24 +02:00
Aliaksandr Valialkin
22c2671205
wip 2024-05-25 16:09:59 +02:00
Aliaksandr Valialkin
46bc1c3435
wip 2024-05-25 15:51:47 +02:00
Aliaksandr Valialkin
33610de341
wip 2024-05-25 14:37:26 +02:00
Aliaksandr Valialkin
e93a71d741
wip 2024-05-25 11:59:47 +02:00
Aliaksandr Valialkin
c2050495c4
wip 2024-05-25 10:51:44 +02:00
Aliaksandr Valialkin
07d244dab0
Merge branch 'public-single-node' into victorialogs-wip 2024-05-25 00:28:20 +02:00
Aliaksandr Valialkin
7654bd60f8
wip 2024-05-25 00:22:14 +02:00
Aliaksandr Valialkin
f8e034f061
wip 2024-05-24 22:17:21 +02:00
Aliaksandr Valialkin
aa23832921
wip 2024-05-24 19:45:43 +02:00
Aliaksandr Valialkin
b59ed0bc79
wip 2024-05-24 19:20:46 +02:00
Aliaksandr Valialkin
a15a56def0
wip 2024-05-24 19:04:24 +02:00
Aliaksandr Valialkin
37ff352ed5
wip 2024-05-24 18:54:58 +02:00
Aliaksandr Valialkin
a3032067bd
wip 2024-05-24 18:31:49 +02:00
Nikolay
69d244e6fb
lib/mergeset: adds tracking for indexdb records drop (#6297)
It allows to create alert for possible item drops at indexdb. It may
happen, if ingested metric size exceeds max indexdb item size.

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
2024-05-24 14:55:20 +02:00
Aliaksandr Valialkin
4b458370c1
lib/logstorage: work-in-progress 2024-05-24 03:06:55 +02:00
Aliaksandr Valialkin
869b2fabc4
wip 2024-05-24 03:03:12 +02:00
Aliaksandr Valialkin
03b9d7977d
wip 2024-05-23 22:54:21 +02:00
Aliaksandr Valialkin
892afe25d1
wip 2024-05-23 22:21:00 +02:00
Aliaksandr Valialkin
63ffa557ba
wip 2024-05-23 22:06:33 +02:00
Aliaksandr Valialkin
59d52cec67
wip 2024-05-23 21:47:21 +02:00
Aliaksandr Valialkin
b0afef1e2b
wip 2024-05-23 21:24:08 +02:00
Aliaksandr Valialkin
32e96050f9
wip 2024-05-23 17:41:35 +02:00
Aliaksandr Valialkin
91b006f0a7
wip 2024-05-23 17:32:42 +02:00
Aliaksandr Valialkin
ceae8a7e08
wip 2024-05-23 14:11:16 +02:00
Aliaksandr Valialkin
a686c7dd74
wip 2024-05-23 13:58:30 +02:00
Aliaksandr Valialkin
e3cbf97bdd
wip 2024-05-23 12:24:09 +02:00
Aliaksandr Valialkin
a4337149a2
wip 2024-05-23 12:01:15 +02:00
Nikolay
a5d1013042
lib/storage: change default value for maxLabelValueLen to 1024 (#6313)
* It must reduce memory usage for misbehaving clients. Since
VictoriaMetrics stores sparse index inmemory.
* Reduce disk space usage for indexdb.
* Prevent possible indexDB items drops.
* It may trigger slow insert and new timeseries registration due to
default value for flag change

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6176

---------

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2024-05-22 21:53:53 +02:00
Alexander Marshalov
7da541360e
[vmlogs] fixed time parsing with millisecond precision time (#6293) (#6295)
fix for #6293

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2024-05-22 21:46:50 +02:00
Aliaksandr Valialkin
22107421eb
lib/logstorage: work-in-progress 2024-05-22 21:01:20 +02:00
Roman Khavronenko
ac836bcf6c
lib/backup: add -s3TLSInsecureSkipVerify command-line flag (#6318)
* The new flag can be used for for skipping TLS certificates
verification when connecting to S3 endpoint. Affects vmbackup,
vmrestore, vmbackupmanager.

* replace deprecated `EndpointResolver` with `BaseEndpoint`

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1056

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-05-22 13:58:39 +02:00
Hui Wang
d7b5062917
app/vmalert: support DNS SRV record in -remoteWrite.url (#6299)
part of https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6053,
supports [DNS SRV](https://en.wikipedia.org/wiki/SRV_record) address in
`-remoteWrite.url` command-line option.
2024-05-22 10:52:51 +02:00
Roman Khavronenko
7ce052b32d
lib/streamaggr: skip empty aggregators (#6307)
Prevent excessive resource usage when stream aggregation config file
contains no matchers by prevent pushing data into Aggregators object.
Before this change a lot of extra work was invoked without reason.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-05-20 14:03:28 +02:00
Aliaksandr Valialkin
bc4a0b8f37
lib/logstorage: fix golangci-lint warnings 2024-05-20 11:04:12 +02:00
Aliaksandr Valialkin
ad505a7a9a
lib/logstorage: work-in-progress 2024-05-20 04:08:30 +02:00
Andrii Chubatiuk
f153f54d11
app/vmagent: add global aggregator (#6268)
Add global stream aggregation for VMAgent

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5467
2024-05-17 14:00:47 +02:00
Nikolay
b2765c45d0
follow-up for c6c5a5a186 (#6265)
* adds datadog extensions for statsd:
  - multiple packed values (v1.1)
  - additional types distribution, histogram

* adds type check and append metric type to the labels with special tag
name `__statsd_metric_type__`. It simplifies streaming aggregation
config.

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
2024-05-16 09:25:42 +02:00
Aliaksandr Valialkin
0aa19a2837
lib/logstorage: work-in-progress 2024-05-15 04:55:44 +02:00
Aliaksandr Valialkin
b617dc9c0b
lib/streamaggr: properly return output key from getOutputKey
The bug has been introduced in cc2647d212
2024-05-14 17:47:21 +02:00
Aliaksandr Valialkin
da3af090c6
lib/logstorage: work-in-progress 2024-05-14 03:05:03 +02:00
Aliaksandr Valialkin
cb35e62e04
lib/logstorage: work-in-progress
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6258
2024-05-14 01:49:23 +02:00
Aliaksandr Valialkin
cc2647d212
lib/encoding: optimize UnmarshalVarUint64, UnmarshalVarInt64 and UnmarshalBytes a bit
Change the return values for these functions - now they return the unmarshaled result plus
the size of the unmarshaled result in bytes, so the caller could re-slice the src for further unmarshaling.

This improves performance of these functions in hot loops of VictoriaLogs a bit.
2024-05-14 01:23:54 +02:00
Aliaksandr Valialkin
707f3a69db
lib/stringsutil: add LessNatural() function for natural sorting
Natural sorting is needed for sort_by_label_natural() and sort_by_label_natural_desc()
functions in MetricsQL - see https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6192
and https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6256

Natural sorting will be also used by `| sort ...` pipe in VictoriaLogs -
see https://docs.victoriametrics.com/victorialogs/logsql/#sort-pipe
2024-05-13 16:56:47 +02:00
Hui Wang
4c80b17027
storage: correctly apply -inmemoryDataFlushInterval when it's set t… (#6221)
…o minimum supported value 1s
pendingRowsFlushInterval was bumped to 2s in
73f0a805e2
2024-05-13 16:44:30 +02:00
Andrii Chubatiuk
ce25d68b45
lib/streamaggr: added rate_sum and rate_avg to benchmarks, lint fix (#6264)
fixed lint for rate outputs
2024-05-13 16:40:37 +02:00
Andrii Chubatiuk
9c3d44c8c9
lib/streamaggr: added rate and rate_avg output (#6243)
Added `rate` and `rate_avg` output
Resource usage is the same as for increase output, tested on a benchmark

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
2024-05-13 15:39:49 +02:00
hagen1778
17283fab6c
lib/logstorage: make linter happy
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-05-13 15:35:11 +02:00
Aliaksandr Valialkin
9dbd0f9085
lib/logstorage: initial implementation of pipes in LogsQL
See https://docs.victoriametrics.com/victorialogs/logsql/#pipes
2024-05-12 16:33:31 +02:00