Commit graph

2491 commits

Author SHA1 Message Date
Aliaksandr Valialkin
a013cb0ca3
wip 2024-05-28 17:39:42 +02:00
Aliaksandr Valialkin
9052c6b734
wip 2024-05-28 17:19:06 +02:00
Aliaksandr Valialkin
e391d7b69c
wip 2024-05-28 16:21:05 +02:00
Aliaksandr Valialkin
ba40c1a2d5
wip 2024-05-28 15:58:36 +02:00
Aliaksandr Valialkin
22e14a3dbe
wip 2024-05-28 15:53:21 +02:00
Aliaksandr Valialkin
bd70c94a27
wip 2024-05-28 15:50:39 +02:00
Aliaksandr Valialkin
e8e49405ef
wip 2024-05-28 14:51:54 +02:00
Aliaksandr Valialkin
e25b2e7f05
wip 2024-05-28 14:21:29 +02:00
Aliaksandr Valialkin
dfcb8dfd65
wip 2024-05-28 02:06:40 +02:00
Aliaksandr Valialkin
dc245f82f3
wip 2024-05-27 23:24:07 +02:00
Aliaksandr Valialkin
9903654e0f
wip 2024-05-27 21:29:24 +02:00
Aliaksandr Valialkin
ca4b5686c1
wip 2024-05-27 20:33:19 +02:00
Aliaksandr Valialkin
741fb6b038
wip 2024-05-27 18:50:22 +02:00
Aliaksandr Valialkin
66e294edfb
Merge branch 'public-single-node' into victorialogs-wip 2024-05-27 16:50:04 +02:00
Aliaksandr Valialkin
fc6a923c5e
wip 2024-05-27 16:48:34 +02:00
Aliaksandr Valialkin
c01bc0282a
wip 2024-05-27 16:18:53 +02:00
Aliaksandr Valialkin
401e79e0d8
wip 2024-05-27 14:22:18 +02:00
Aliaksandr Valialkin
9af6c63774
wip 2024-05-27 00:58:41 +02:00
Aliaksandr Valialkin
99138e15c0
lib/logstorage: fix golangci-lint warnings 2024-05-26 02:01:32 +02:00
Aliaksandr Valialkin
1e203f35f7
lib/logstorage: work-in-progress 2024-05-26 01:55:21 +02:00
Aliaksandr Valialkin
ad0cc2d55e
wip 2024-05-26 01:54:39 +02:00
Aliaksandr Valialkin
7994b5b22a
wip 2024-05-26 00:21:57 +02:00
Aliaksandr Valialkin
5be91507fa
wip 2024-05-26 00:17:09 +02:00
Aliaksandr Valialkin
7ac529c235
lib/logstorage: work-in-progress 2024-05-25 22:59:13 +02:00
Aliaksandr Valialkin
1c8d97b95e
wip 2024-05-25 22:58:35 +02:00
Aliaksandr Valialkin
8248afc9bd
wip 2024-05-25 22:20:47 +02:00
Aliaksandr Valialkin
0b629ce5a5
lib/logstorage: re-use per-shard fields across processed blocks in pipePackJSON and pipeUnroll 2024-05-25 22:13:32 +02:00
Aliaksandr Valialkin
dc55146752
lib/logstorage: work-in-progress 2024-05-25 21:36:16 +02:00
Aliaksandr Valialkin
e2590f0485
lib/logstorage: work-in-progress 2024-05-25 00:30:58 +02:00
Nikolay
69d244e6fb
lib/mergeset: adds tracking for indexdb records drop (#6297)
It allows to create alert for possible item drops at indexdb. It may
happen, if ingested metric size exceeds max indexdb item size.

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
2024-05-24 14:55:20 +02:00
Aliaksandr Valialkin
4b458370c1
lib/logstorage: work-in-progress 2024-05-24 03:06:55 +02:00
Nikolay
a5d1013042
lib/storage: change default value for maxLabelValueLen to 1024 (#6313)
* It must reduce memory usage for misbehaving clients. Since
VictoriaMetrics stores sparse index inmemory.
* Reduce disk space usage for indexdb.
* Prevent possible indexDB items drops.
* It may trigger slow insert and new timeseries registration due to
default value for flag change

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6176

---------

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2024-05-22 21:53:53 +02:00
Alexander Marshalov
7da541360e
[vmlogs] fixed time parsing with millisecond precision time (#6293) (#6295)
fix for #6293

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2024-05-22 21:46:50 +02:00
Aliaksandr Valialkin
22107421eb
lib/logstorage: work-in-progress 2024-05-22 21:01:20 +02:00
Roman Khavronenko
ac836bcf6c
lib/backup: add -s3TLSInsecureSkipVerify command-line flag (#6318)
* The new flag can be used for for skipping TLS certificates
verification when connecting to S3 endpoint. Affects vmbackup,
vmrestore, vmbackupmanager.

* replace deprecated `EndpointResolver` with `BaseEndpoint`

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1056

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-05-22 13:58:39 +02:00
Hui Wang
d7b5062917
app/vmalert: support DNS SRV record in -remoteWrite.url (#6299)
part of https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6053,
supports [DNS SRV](https://en.wikipedia.org/wiki/SRV_record) address in
`-remoteWrite.url` command-line option.
2024-05-22 10:52:51 +02:00
Roman Khavronenko
7ce052b32d
lib/streamaggr: skip empty aggregators (#6307)
Prevent excessive resource usage when stream aggregation config file
contains no matchers by prevent pushing data into Aggregators object.
Before this change a lot of extra work was invoked without reason.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-05-20 14:03:28 +02:00
Aliaksandr Valialkin
bc4a0b8f37
lib/logstorage: fix golangci-lint warnings 2024-05-20 11:04:12 +02:00
Aliaksandr Valialkin
ad505a7a9a
lib/logstorage: work-in-progress 2024-05-20 04:08:30 +02:00
Andrii Chubatiuk
f153f54d11
app/vmagent: add global aggregator (#6268)
Add global stream aggregation for VMAgent

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5467
2024-05-17 14:00:47 +02:00
Nikolay
b2765c45d0
follow-up for c6c5a5a186 (#6265)
* adds datadog extensions for statsd:
  - multiple packed values (v1.1)
  - additional types distribution, histogram

* adds type check and append metric type to the labels with special tag
name `__statsd_metric_type__`. It simplifies streaming aggregation
config.

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
2024-05-16 09:25:42 +02:00
Aliaksandr Valialkin
0aa19a2837
lib/logstorage: work-in-progress 2024-05-15 04:55:44 +02:00
Aliaksandr Valialkin
b617dc9c0b
lib/streamaggr: properly return output key from getOutputKey
The bug has been introduced in cc2647d212
2024-05-14 17:47:21 +02:00
Aliaksandr Valialkin
da3af090c6
lib/logstorage: work-in-progress 2024-05-14 03:05:03 +02:00
Aliaksandr Valialkin
cb35e62e04
lib/logstorage: work-in-progress
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6258
2024-05-14 01:49:23 +02:00
Aliaksandr Valialkin
cc2647d212
lib/encoding: optimize UnmarshalVarUint64, UnmarshalVarInt64 and UnmarshalBytes a bit
Change the return values for these functions - now they return the unmarshaled result plus
the size of the unmarshaled result in bytes, so the caller could re-slice the src for further unmarshaling.

This improves performance of these functions in hot loops of VictoriaLogs a bit.
2024-05-14 01:23:54 +02:00
Aliaksandr Valialkin
707f3a69db
lib/stringsutil: add LessNatural() function for natural sorting
Natural sorting is needed for sort_by_label_natural() and sort_by_label_natural_desc()
functions in MetricsQL - see https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6192
and https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6256

Natural sorting will be also used by `| sort ...` pipe in VictoriaLogs -
see https://docs.victoriametrics.com/victorialogs/logsql/#sort-pipe
2024-05-13 16:56:47 +02:00
Hui Wang
4c80b17027
storage: correctly apply -inmemoryDataFlushInterval when it's set t… (#6221)
…o minimum supported value 1s
pendingRowsFlushInterval was bumped to 2s in
73f0a805e2
2024-05-13 16:44:30 +02:00
Andrii Chubatiuk
ce25d68b45
lib/streamaggr: added rate_sum and rate_avg to benchmarks, lint fix (#6264)
fixed lint for rate outputs
2024-05-13 16:40:37 +02:00
Andrii Chubatiuk
9c3d44c8c9
lib/streamaggr: added rate and rate_avg output (#6243)
Added `rate` and `rate_avg` output
Resource usage is the same as for increase output, tested on a benchmark

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
2024-05-13 15:39:49 +02:00