VictoriaMetrics/app/vmselect/netstorage
YuDong Tang cab3ef8294
app/vmselect:add command-line flag -search.inmemoryBufSizeBytes (#6869)
add command-line flag `-search.inmemoryBufSizeBytes` for configuring size of in-memory buffers used by vmselect during processing of vmstorage responses. A new summary metric `vm_tmp_blocks_inmemory_file_size_bytes` is exposed to show the size of the buffer during requests processing. 

The new setting can be used by experienced users to adjust memory usage by vmselect when processing
many small read requests. Instead of allocating 4MB buffers each time, vmselect can be instructed to lower
the buffer size via `-search.inmemoryBufSizeBytes`. To make the decision whether this flag needs to be adjusted
users can consult with `vm_tmp_blocks_inmemory_file_size_bytes` which shows the actual size of buffers used
during query processing.

----------

The detailed information of this PR can be found in
https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6851

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Co-authored-by: hagen1778 <roman@victoriametrics.com>
2024-08-26 14:37:45 +02:00
..
netstorage.go app/{vminsert,vmselect}: pass proper args to metrics.UnregisterSet() after a8356f3a26 2024-07-15 20:27:40 +02:00
netstorage_test.go app/vmselect/netstorage: eliminate memory allocation for sortBlocksHeap arg when calling mergeSortBlocks() 2023-01-09 21:29:01 -08:00
netstorage_timing_test.go app/vmselect/netstorage: eliminate memory allocation for sortBlocksHeap arg when calling mergeSortBlocks() 2023-01-09 21:29:01 -08:00
tmp_blocks_file.go app/vmselect:add command-line flag -search.inmemoryBufSizeBytes (#6869) 2024-08-26 14:37:45 +02:00
tmp_blocks_file_test.go app/{vminsert,vmselect}: limit the access to storageNodes to getStorageNodesBucket and setStorageNodesBucket functions 2022-10-28 11:41:55 +03:00