VictoriaMetrics/lib/bytesutil
Roman Khavronenko f28f496a9d
lib/bytesutil: smooth buffer growth rate (#6761)
Before, buffer growth was always x2 of its size, which could lead to
excessive memory usage when processing big amount of data.
For example, scraping a target with hundreds of MBs in response could
result into hih memory spikes in vmagent because buffer has to double
its size to fit the response. See
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6759

The change smoothes out the growth rate, trading higher allocation rate
for lower mem usage at certain conditions.

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-08-07 16:49:43 +02:00
..
bytebuffer.go lib/bytesutil: smooth buffer growth rate (#6761) 2024-08-07 16:49:43 +02:00
bytebuffer_test.go all: rename ReadAt* to MustReadAt* in order to dont clash with io.ReaderAt 2020-01-30 15:08:58 +02:00
bytesutil.go lib/bytesutil: use unsafe.String instead of unsafe conversion of slice header to string header 2024-02-29 17:27:51 +02:00
bytesutil_test.go lib/bytesutil: split Resize* funcs to MayOverallocate and NoOverallocate for more fine-grained control over memory allocations 2022-02-01 00:18:42 +02:00
bytesutil_timing_test.go lib/bytesutil: make BenchmarkToUnsafeString and BenchmarkToUnsafeBytes more reliable 2024-02-29 17:11:03 +02:00
fast_string_matcher.go all: consistently use 'any' instead of 'interface{}' 2024-07-10 00:20:37 +02:00
fast_string_matcher_test.go lib: consistently use atomic.* types instead of atomic.* functions 2024-02-24 02:07:53 +02:00
fast_string_matcher_timing_test.go lib: consistently use atomic.* types instead of atomic.* functions 2024-02-24 02:07:53 +02:00
fast_string_transformer.go all: consistently use 'any' instead of 'interface{}' 2024-07-10 00:20:37 +02:00
fast_string_transformer_test.go lib/bytesutil: add FastStringTransformer and use it in the rest of the code where needed 2022-09-28 10:41:00 +03:00
fast_string_transformer_timing_test.go lib: consistently use atomic.* types instead of atomic.* functions 2024-02-24 02:07:53 +02:00
internstring.go lib/bytesutil: optimize internStringMap cleanup 2024-06-13 15:06:51 +02:00
internstring_test.go Makefile: update golangci-lint from v1.51.2 to v1.54.2 2023-09-01 10:16:42 +02:00
internstring_timing_test.go lib/bytesutil: move InternString() from lib/promscrape/discoverytutils to lib/bytesutil 2022-09-30 07:44:35 +03:00
itoa.go lib/bytesutil: add InternBytes() function as a shortcut to InternString(ToUnsafeString(..)) 2023-01-03 22:16:22 -08:00
itoa_test.go lib/promscrape: optimize service discovery speed 2022-11-29 21:26:00 -08:00