Commit graph

34 commits

Author SHA1 Message Date
Aliaksandr Valialkin
e309b5a83b app/vmagent/remotewrite: increase the maximum possible number of inmemory blocks for systems with high amounts of RAM
This should reduce the probability of using much slower file-based persistent queue
when vmagent processes metrics at high rate (millions of metrics per second).

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1235
2021-04-23 22:05:00 +03:00
Aliaksandr Valialkin
392ba94d1d app/vmagent/remotewrite: reduce memory usage when -remoteWrite.queues is set to a big value
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1167
2021-03-31 16:17:23 +03:00
Aliaksandr Valialkin
b873b965af app/vmagent/remotewrite: reduce memory usage when samples with big number of labels are sent to remote storage 2021-03-31 00:45:42 +03:00
Aliaksandr Valialkin
2041008d64 app/vmagent: mention -remoteWrite.maxDiskUsagePerURL in the descriptio of -remoteWrite.tmpDataPath flag 2021-03-23 16:38:41 +02:00
Aliaksandr Valialkin
197ecca426 lib/promrelabel: add more optimizations for relabeling for common cases 2021-02-22 16:36:54 +02:00
Aliaksandr Valialkin
72eef964d9 app/vmagent: properly perform graceful shutdown, which was broken in the commit 1d1ba889fe
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1065
2021-02-19 00:34:17 +02:00
Nikolay
40973eda1c adds pushback for fastqueue, (#1075)
during shutdown currently sending block was lost,
now its pushed back to fast queue and will be flushed on disk,
it may lead to data duplication.
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1065
2021-02-17 21:43:46 +02:00
Aliaksandr Valialkin
fdf9de98f8 app/vmagent: add -remoteWrite.roundDigits command-line option for limiting the number of digits after the point for stored values
This commit also adds --vm-round-digits command-line option to vmctl tool.
2021-02-01 14:42:15 +02:00
Aliaksandr Valialkin
1a237c6903 all: properly handle CPU limits set on the host system/container
This can reduce memory usage on systems with enabled CPU limits.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/946
2020-12-08 21:07:03 +02:00
Aliaksandr Valialkin
5dca7bbe85 app/vmagent/remotewrite: do not show -remoteWrite.url in logs if -remoteWrite.showURL isn't set
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/773
2020-09-29 19:49:19 +03:00
Aliaksandr Valialkin
8fc9b77496 app/vmagent: reduce memory usage when importing data via /api/v1/import
Previously vmagent could use big amounts of RAM when each ingested JSON line
contained many samples.
2020-09-26 04:10:13 +03:00
Aliaksandr Valialkin
1c07d7bee7 app/vmagent: increase default value for -remoteWrite.queues from 1 to 4, since it has been appeared that many users hit this limit 2020-09-18 14:22:02 +03:00
Aliaksandr Valialkin
406f4fe445 app/vmagent: substitute -remoteWrite.url with secret-url value in logs, since it may contain sensitive info such as passwords or auth tokens
Pass `-remoteWrite.showURL` command-line flag in order to see real `-remoteWrite.url` values in logs and at `/metrics` page.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/773
2020-09-16 22:36:18 +03:00
Aliaksandr Valialkin
6382e8081a app/vmagent: allow setting multiple identical -remoteWrite.url values
This may be useful when each url is authenticated via different `-remoteWrite.basicAuth.username`.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/755
2020-09-11 15:17:29 +03:00
Aliaksandr Valialkin
3b1ecac04b app/vmagent: apply sane limits to -remoteWrite.queues
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/707
2020-08-30 21:25:51 +03:00
Roman Khavronenko
4b89da9463 lib/decimal: rename significant decimal digits to significant figures (#698)
The previous notion was inconsistent with what `decimal.Round` does.
According to [wiki](https://en.wikipedia.org/wiki/Significant_figures) rounding
applied to all significant figures, not just decimal ones.
2020-08-16 17:22:40 +03:00
Aliaksandr Valialkin
6aab2f4989 all: allow using KB, MB, GB, KiB, MiB and GiB suffixes in command-line flag values related to byte sizes or byte rates 2020-08-16 17:08:28 +03:00
Aliaksandr Valialkin
a3f48e395e app/vmagent: add -remoteWrite.decimalPlaces command-line flag, which may be used for reducing disk space usage on the remote storage 2020-07-21 21:55:42 +03:00
Aliaksandr Valialkin
c2b4b9138d app/vmagent/remotewrite: return proper value from tssRelabelPool.New
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/599
2020-07-14 14:28:14 +03:00
Aliaksandr Valialkin
4353ff7ef1 app/vmagent: fix data race when multiple -remoteWrite.urlRelabelConfig options are set
Previously multiple goroutines could access remoteWriteCtx.tss concurrently, which could lead to data race
and improper relabeling. Now each goroutine has its own copy of tss during relabeling.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/599
2020-07-10 15:17:23 +03:00
Aliaksandr Valialkin
805a90f642 app/vmagent/remotewrite: typo fix in -remoteWrite.showURL help message 2020-07-10 14:07:14 +03:00
Aliaksandr Valialkin
37aa4fe282 app/vmagent: reload -remoteWrite.relabelConfig and -remoteWrite.urlRelabelConfig on SIGHUP and on /-/reload
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/518
2020-05-30 14:37:02 +03:00
Aliaksandr Valialkin
b59e089ac7 app/vmagent: add -dryRun option for checking all the configs mentioned in command-line flags without running vmagent
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/362
2020-05-21 15:23:18 +03:00
Aliaksandr Valialkin
1d71253653 app/vmagent/remotewrite: allow ingesting time series with multiple samples at once
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/481
2020-05-15 17:37:27 +03:00
Aliaksandr Valialkin
3b0f66a227 app/vmagent: fix a bug with improper relabeling when multiple -remoteWrite.urlRelableConfig args are set
This bug could result in incorrect relabeling and metrics' drop.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/467
2020-05-12 22:03:45 +03:00
Aliaksandr Valialkin
20538a2a5d app/vmagent: allow setting independent auth configs per each configured -remoteWrite.url 2020-05-06 16:52:32 +03:00
Aliaksandr Valialkin
47e986c26f app/vmagent: properly add labels set via -remoteWrite.label to metrics before sending them to -remoteWrite.url 2020-03-06 19:28:14 +02:00
Aliaksandr Valialkin
f01d1bf4a8 app/vmagent: add -remoteWrite.maxDiskUsagePerURL for limiting the maximum disk usage for each -remoteWrite.url buffer
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/352
2020-03-03 19:49:20 +02:00
Aliaksandr Valialkin
808c17e250 app/vmagent/remotewrite: do not reset empty relabelCtx 2020-03-03 15:01:21 +02:00
Aliaksandr Valialkin
af19ca2483 app/vmagent: add -remoteWrite.urlRelabelConfig for applying individual relabeling for each -remoteWrite.url
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/320
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/308
2020-03-03 13:13:06 +02:00
Aliaksandr Valialkin
0eed71c7f4 app/vmagent/remotewrite: yet another typo fix 2020-02-28 20:07:00 +02:00
Aliaksandr Valialkin
6cdc97a53f app/vmagent/remotewrite: typo fix 2020-02-28 19:05:11 +02:00
Aliaksandr Valialkin
cc39c9d74b app/vmagent/remotewrite: limit memory usage when big scrape blocks are pushed to remote storage 2020-02-28 18:58:13 +02:00
Aliaksandr Valialkin
7ee7614e90 app/vmagent: initial implementation for vmagent 2020-02-23 17:31:54 +02:00