Commit graph

2482 commits

Author SHA1 Message Date
Aliaksandr Valialkin
c38d3dab4d
wip 2024-05-17 11:32:27 +02:00
Aliaksandr Valialkin
bddc715c0c
wip 2024-05-17 11:23:59 +02:00
Aliaksandr Valialkin
160034bda3
wip 2024-05-17 11:04:50 +02:00
Aliaksandr Valialkin
53a378faab
wip 2024-05-17 04:11:10 +02:00
Aliaksandr Valialkin
7e4769abad
wip 2024-05-16 00:49:11 +02:00
Aliaksandr Valialkin
e4d89f3584
wip 2024-05-16 00:41:13 +02:00
Aliaksandr Valialkin
65dd152387
wip 2024-05-16 00:35:49 +02:00
Aliaksandr Valialkin
2368a3afce
wip 2024-05-16 00:21:01 +02:00
Aliaksandr Valialkin
17026756a5
wip 2024-05-16 00:11:49 +02:00
Aliaksandr Valialkin
0d71dc22ca
wip 2024-05-15 23:11:48 +02:00
Aliaksandr Valialkin
d50273d6c0
wip 2024-05-15 22:31:21 +02:00
Aliaksandr Valialkin
3010034c7a
wip 2024-05-15 22:22:24 +02:00
Aliaksandr Valialkin
164705cf20
wip 2024-05-15 22:19:21 +02:00
Aliaksandr Valialkin
7ffcdabcbe
wip 2024-05-15 16:53:40 +02:00
Aliaksandr Valialkin
6026112913
wip 2024-05-15 16:50:26 +02:00
Aliaksandr Valialkin
15c66abbe0
wip 2024-05-15 16:40:32 +02:00
Aliaksandr Valialkin
bc72ac0519
wip 2024-05-15 16:21:00 +02:00
Aliaksandr Valialkin
454f781cd1
wip 2024-05-15 16:04:10 +02:00
Aliaksandr Valialkin
639b3091b5
wip 2024-05-15 15:46:42 +02:00
Aliaksandr Valialkin
6f798c628f
wip 2024-05-15 13:23:51 +02:00
Aliaksandr Valialkin
d3e464a68b
wip 2024-05-15 13:07:15 +02:00
Aliaksandr Valialkin
21f09ab823
wip 2024-05-15 12:15:16 +02:00
Aliaksandr Valialkin
20918a2810
wip 2024-05-15 04:42:03 +02:00
Aliaksandr Valialkin
a080c9e4e5
wip 2024-05-15 04:03:05 +02:00
Aliaksandr Valialkin
12fe2b265c
wip 2024-05-15 03:55:46 +02:00
Aliaksandr Valialkin
ff2b6fbe35
wip 2024-05-15 03:23:33 +02:00
Aliaksandr Valialkin
fcce0fc6e1
wip 2024-05-15 03:02:05 +02:00
Aliaksandr Valialkin
edd493431e
wip 2024-05-15 02:52:23 +02:00
Aliaksandr Valialkin
33a01c659b
wip 2024-05-15 02:45:43 +02:00
Aliaksandr Valialkin
42c49c37ff
wip 2024-05-14 22:34:45 +02:00
Aliaksandr Valialkin
87183112f3
wip 2024-05-14 22:31:21 +02:00
Aliaksandr Valialkin
f26d593c7b
wip 2024-05-14 22:11:51 +02:00
Aliaksandr Valialkin
447a7f0bdf
wip 2024-05-14 19:35:08 +02:00
Aliaksandr Valialkin
b617dc9c0b
lib/streamaggr: properly return output key from getOutputKey
The bug has been introduced in cc2647d212
2024-05-14 17:47:21 +02:00
Aliaksandr Valialkin
da3af090c6
lib/logstorage: work-in-progress 2024-05-14 03:05:03 +02:00
Aliaksandr Valialkin
cb35e62e04
lib/logstorage: work-in-progress
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6258
2024-05-14 01:49:23 +02:00
Aliaksandr Valialkin
cc2647d212
lib/encoding: optimize UnmarshalVarUint64, UnmarshalVarInt64 and UnmarshalBytes a bit
Change the return values for these functions - now they return the unmarshaled result plus
the size of the unmarshaled result in bytes, so the caller could re-slice the src for further unmarshaling.

This improves performance of these functions in hot loops of VictoriaLogs a bit.
2024-05-14 01:23:54 +02:00
Aliaksandr Valialkin
707f3a69db
lib/stringsutil: add LessNatural() function for natural sorting
Natural sorting is needed for sort_by_label_natural() and sort_by_label_natural_desc()
functions in MetricsQL - see https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6192
and https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6256

Natural sorting will be also used by `| sort ...` pipe in VictoriaLogs -
see https://docs.victoriametrics.com/victorialogs/logsql/#sort-pipe
2024-05-13 16:56:47 +02:00
Hui Wang
4c80b17027
storage: correctly apply -inmemoryDataFlushInterval when it's set t… (#6221)
…o minimum supported value 1s
pendingRowsFlushInterval was bumped to 2s in
73f0a805e2
2024-05-13 16:44:30 +02:00
Andrii Chubatiuk
ce25d68b45
lib/streamaggr: added rate_sum and rate_avg to benchmarks, lint fix (#6264)
fixed lint for rate outputs
2024-05-13 16:40:37 +02:00
Andrii Chubatiuk
9c3d44c8c9
lib/streamaggr: added rate and rate_avg output (#6243)
Added `rate` and `rate_avg` output
Resource usage is the same as for increase output, tested on a benchmark

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
2024-05-13 15:39:49 +02:00
hagen1778
17283fab6c
lib/logstorage: make linter happy
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-05-13 15:35:11 +02:00
Aliaksandr Valialkin
9dbd0f9085
lib/logstorage: initial implementation of pipes in LogsQL
See https://docs.victoriametrics.com/victorialogs/logsql/#pipes
2024-05-12 16:33:31 +02:00
Aliaksandr Valialkin
e66465cb03
lib/encoding: optimizing UnmarshalVarUint64 and UnmarshalVarInt64 a bit 2024-05-12 16:32:11 +02:00
Aliaksandr Valialkin
590160ddbb
lib/slicesutil: add helper functions for setting slice length and extending its capacity
The added helper functions - SetLength() and ExtendCapacity() - replace error-prone code with simple function calls.
2024-05-12 11:32:17 +02:00
Aliaksandr Valialkin
f20d452196
lib/storage: remove outdated misleading comments 2024-05-12 10:24:04 +02:00
Roman Khavronenko
87fd400dfc
Feature allow configuring disableOnDiskQueue and dropSamplesOnOverload per url (#6248)
* FEATURE: [vmagent](https://docs.victoriametrics.com/vmagent.html):
allow configuring `-remoteWrite.disableOnDiskQueue` and
`-remoteWrite.dropSamplesOnOverload` cmd-line flags per each
`-remoteWrite.url`. See this [pull
request](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6065).
Thanks to @rbizos for implementaion!
* FEATURE: [vmagent](https://docs.victoriametrics.com/vmagent.html): add
labels `path` and `url` to metrics
`vmagent_remotewrite_push_failures_total` and
`vmagent_remotewrite_samples_dropped_total`. Now number of failed pushes
and dropped samples can be tracked per `-remoteWrite.url`.

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: Raphael Bizos <r.bizos@criteo.com>
2024-05-10 12:09:21 +02:00
Roman Khavronenko
8a03e987cb
lib/streamaggr: set correct suffix <output>_prometheus (#6228)
Set correct suffix `<output>_prometheus` for aggregation outputs
`increase_prometheus` and `total_prometheus`
Before, outputs `total` and `total_prometheus` or `increase` and
`increase_prometheus` had the same suffix.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-05-08 13:11:30 +02:00
Andrii Chubatiuk
a9283e06a3
streamaggr: made labels compressor shared (#6173)
Though labels compressor is quite resource intensive, each aggregator
and deduplicator instance has it's own compressor. Made it shared across
all aggregators to consume less resources while using multiple
aggregators.

Co-authored-by: Roman Khavronenko <hagen1778@gmail.com>
2024-05-08 13:10:53 +02:00
Zhu Jiekun
17e3d019d2
feature: [vmagent] Add service discovery support for Vultr (#6068)
### Describe Your Changes
related issue:
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6041

#### Added
- Added service discovery support for Vultr.

#### Docs
- `CHANGELOG.md`, `sd_configs.md`, `vmagent.md` are updated.

#### Note
- Useful links: 
- Vultr API:
https://www.vultr.com/api/#tag/instances/operation/list-instances
    - Vultr client SDK: https://github.com/vultr/govultr
- Prometheus SD:
https://github.com/prometheus/prometheus/tree/main/discovery/vultr

---
### Checklist

The following checks are mandatory:

- [X] I have read the [Contributing
Guidelines](https://github.com/VictoriaMetrics/VictoriaMetrics/blob/master/CONTRIBUTING.md)
- [x] All commits are signed and include `Signed-off-by` line. Use `git
commit -s` to include `Signed-off-by` your commits. See this
[doc](https://git-scm.com/book/en/v2/Git-Tools-Signing-Your-Work) about
how to sign your commits.
- [x] Tests are passing locally. Use `make test` to run all tests
locally.
- [x] Linting is passing locally. Use `make check-all` to run all
linters locally.

Further checks are optional for External Contributions:

- [X] Include a link to the GitHub issue in the commit message, if issue
exists.
- [x] Mention the change in the
[Changelog](https://github.com/VictoriaMetrics/VictoriaMetrics/blob/master/docs/CHANGELOG.md).
Explain what has changed and why. If there is a related issue or
documentation change - link them as well.

  Tips for writing a good changelog message::

* Write a human-readable changelog message that describes the problem
and solution.
* Include a link to the issue or pull request in your changelog message.
* Use specific language identifying the fix, such as an error message,
metric name, or flag name.
* Provide a link to the relevant documentation for any new features you
add or modify.

- [ ] After your pull request is merged, please add a message to the
issue with instructions for how to test the fix or try the feature you
added. Here is an
[example](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4048#issuecomment-1546453726)
- [x] Do not close the original issue before the change is released.
Please note, in some cases Github can automatically close the issue once
PR is merged. Re-open the issue in such case.
- [x] If the change somehow affects public interfaces (a new flag was
added or updated, or some behavior has changed) - add the corresponding
change to documentation.

Signed-off-by: Jiekun <jiekun.dev@gmail.com>
2024-05-08 10:01:48 +02:00