VictoriaMetrics/lib/bytesutil
Roman Khavronenko c41a9b8d17
lib/bytesutil: smooth buffer growth rate (#6761)
Before, buffer growth was always x2 of its size, which could lead to
excessive memory usage when processing big amount of data.
For example, scraping a target with hundreds of MBs in response could
result into hih memory spikes in vmagent because buffer has to double
its size to fit the response. See
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6759

The change smoothes out the growth rate, trading higher allocation rate
for lower mem usage at certain conditions.

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit f28f496a9d)
2024-08-07 16:59:23 +02:00
..
bytebuffer.go lib/bytesutil: smooth buffer growth rate (#6761) 2024-08-07 16:59:23 +02:00
bytebuffer_test.go all: rename ReadAt* to MustReadAt* in order to dont clash with io.ReaderAt 2020-01-30 15:16:16 +02:00
bytesutil.go lib/bytesutil: use unsafe.String instead of unsafe conversion of slice header to string header 2024-02-29 17:28:04 +02:00
bytesutil_test.go lib/bytesutil: split Resize* funcs to MayOverallocate and NoOverallocate for more fine-grained control over memory allocations 2022-02-01 11:20:20 +02:00
bytesutil_timing_test.go lib/bytesutil: make BenchmarkToUnsafeString and BenchmarkToUnsafeBytes more reliable 2024-02-29 17:12:30 +02:00
fast_string_matcher.go all: consistently use 'any' instead of 'interface{}' 2024-07-10 00:23:26 +02:00
fast_string_matcher_test.go lib: consistently use atomic.* types instead of atomic.* functions 2024-02-24 02:10:04 +02:00
fast_string_matcher_timing_test.go lib: consistently use atomic.* types instead of atomic.* functions 2024-02-24 02:10:04 +02:00
fast_string_transformer.go all: consistently use 'any' instead of 'interface{}' 2024-07-10 00:23:26 +02:00
fast_string_transformer_test.go lib/bytesutil: add FastStringTransformer and use it in the rest of the code where needed 2022-09-28 10:39:42 +03:00
fast_string_transformer_timing_test.go lib: consistently use atomic.* types instead of atomic.* functions 2024-02-24 02:10:04 +02:00
internstring.go lib/bytesutil: optimize internStringMap cleanup 2024-06-13 15:09:42 +02:00
internstring_test.go Makefile: update golangci-lint from v1.51.2 to v1.54.2 2023-09-01 10:25:49 +02:00
internstring_timing_test.go lib/bytesutil: move InternString() from lib/promscrape/discoverytutils to lib/bytesutil 2022-09-30 07:34:59 +03:00
itoa.go lib/bytesutil: add InternBytes() function as a shortcut to InternString(ToUnsafeString(..)) 2023-01-03 22:15:49 -08:00
itoa_test.go lib/promscrape: optimize service discovery speed 2022-11-29 21:26:23 -08:00