Aliaksandr Valialkin
7b59466667
lib/memory: attempt #2 to determine memory limit inside LXC container
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/84
2019-06-28 18:08:15 +03:00
Aliaksandr Valialkin
593bd35aaa
lib/memory: an attempt to read proper memory limit inside LXC container
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/84
2019-06-28 15:34:30 +03:00
Aliaksandr Valialkin
c1be1e4342
lib/storage: optimize time series search by regexp filter
...
This should improve search speed on label filters like `{foo=~"bar.+baz"}`
2019-06-27 16:17:43 +03:00
Aliaksandr Valialkin
683bf2a11f
lib/storage: make sure non-nil args are passed to openIndexDB
2019-06-25 20:10:04 +03:00
Aliaksandr Valialkin
eb2283a029
lib/storage: reduce too big maxMetrics in getTagFilterWithMinMetricIDsCountAdaptive
...
This should improve performance on inverted index search for big amount of unique time series
when big -search.maxUniqueTimeseries is set.
2019-06-25 19:55:27 +03:00
Aliaksandr Valialkin
e8377011ab
lib/storage: free up memory from caches owned by indexDB when it is deleted
2019-06-25 14:42:44 +03:00
Aliaksandr Valialkin
33ea2120c3
lib/storage: use unversioned keys for tag cache in extDB
...
Data in ExtDB cannot be changed, so it is OK to use unversioned keys for tag cache.
This should improve performance for index lookups over big amount of time series.
2019-06-25 13:08:58 +03:00
Aliaksandr Valialkin
cf63669303
lib/storage: skip searching in extDB if it doesn't contain items for the given time range
...
This should improve inverted index search performance for big amount
of unique time series when the search is performed only on recent data.
2019-06-25 13:00:37 +03:00
Aliaksandr Valialkin
af2ceaaa0b
lib/storage: mention source parts on merge error
...
This should improve determining broken source part.
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/76
2019-06-24 14:08:43 +03:00
Aliaksandr Valialkin
e88a03323a
all: initial stubs for Windows support; see https://github.com/VictoriaMetrics/VictoriaMetrics/issues/70
2019-06-20 20:07:10 +03:00
Aliaksandr Valialkin
80db24386e
lib/storage: typo fixes found by golangci-lint; updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/69
2019-06-20 14:37:55 +03:00
Aliaksandr Valialkin
296c14317f
lib/netutil: remove unused TCPListener.name; updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/69
2019-06-20 14:36:15 +03:00
Aliaksandr Valialkin
a78b3dba7f
app/vmstorage: add vm_cache_entries{type="storage/hour_metric_ids"} metric for tracking active time series count
2019-06-19 18:36:47 +03:00
Aliaksandr Valialkin
710d6c33ea
lib/prompb: remove superflouos bytes copying in ReadSnappy
2019-06-18 20:37:51 +03:00
Aliaksandr Valialkin
6d6c9eb1f8
lib/flagutil: remove unused package
2019-06-18 10:43:55 +03:00
Aliaksandr Valialkin
d2c801029b
lib/storage: persist metric ids for the current and the previous hour on graceful shutdown
...
This should improve performance after restart when the db contains a lot of time series
with high time series churn (i.e. metrics from Kubernetes with many pods and frequent deployments)
2019-06-14 07:55:14 +03:00
Aliaksandr Valialkin
26f8d7ea1b
lib/fs: sync parent dir in MustRemoveAll only if it exists
...
The parent directory may be non-existing when the deleted directory
didn't exist before the MustRemoveAll call
2019-06-12 02:14:44 +03:00
Aliaksandr Valialkin
419197ba08
lib/fs: consolidate *RemoveAll* funcs into a single MustRemoveAll func
...
The func syncs parent dir in order to persist directory removal
in the event of power loss
2019-06-12 01:53:46 +03:00
Aliaksandr Valialkin
c1276edab5
lib/fs: panic with fatal error when directories cannot be removed
...
Unremoved directories may lead to inconsistent data directory,
so VictoriaMetrics will fail to start next time.
So panic on the first error when trying to remove directory in order
to simplify recover process.
2019-06-12 01:20:54 +03:00
Aliaksandr Valialkin
2322c9a45a
lib/fs: attempt #2 to work around NFS issue with directory removal
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/61
2019-06-12 01:07:05 +03:00
Aliaksandr Valialkin
935bfd7a18
lib/fs: consistency renaming SyncPath -> MustSyncPath, since it doesnt return error
2019-06-11 23:13:49 +03:00
Aliaksandr Valialkin
3dd36b8088
lib/fs: make sure the created directory remains visible in the fs in the event of power loss
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/63
2019-06-11 23:08:09 +03:00
Aliaksandr Valialkin
afb964670a
lib/fs: use filepath.Dir instead of filepath.Split, since the filename is unused
2019-06-11 22:54:26 +03:00
Aliaksandr Valialkin
20fc0e0e54
lib/{storage,mergeset}: sync filenames inside part when finalizing the part
...
Fixes https://github.com/VictoriaMetrics/VictoriaMetrics/issues/63
2019-06-11 21:51:13 +03:00
Aliaksandr Valialkin
ac7b186f13
all: try hard removing directory with contents
...
Fixes https://github.com/VictoriaMetrics/VictoriaMetrics/issues/61
2019-06-11 01:57:59 +03:00
Aliaksandr Valialkin
cbe692f0e2
app/vmselect: add /api/v1/labels/count handler for quick detection of labels with the maximum number of distinct values
2019-06-10 19:55:38 +03:00
Aliaksandr Valialkin
7b6623558f
lib/storage: skip adaptive searching for tag filter matching the minimum number of metrics if the identical previous search didn't found such filter
...
This should improve speed for searching metrics among high number of time series
with high churn rate like in big Kubernetes clusters with frequent deployments.
2019-06-10 14:07:39 +03:00
Aliaksandr Valialkin
a1351bbaee
lib/storage: factor out getTagFilterWithMinMetricIDsCountAdaptive from updateMetricIDsForTagFilters
2019-06-10 13:26:44 +03:00
Aliaksandr Valialkin
b4d707d9bb
lib/storage: give clearer names to more functions
2019-06-10 13:01:23 +03:00
Aliaksandr Valialkin
bee7298f81
lib/storage: give more clear names to functions
2019-06-10 12:50:44 +03:00
Aliaksandr Valialkin
dbd217b8f0
lib/storage: test GetSeriesCount
2019-06-10 12:43:34 +03:00
Aliaksandr Valialkin
4d936b1524
lib/storage: make getSeriesCount func indexSearch method
2019-06-10 12:29:11 +03:00
Aliaksandr Valialkin
d37924900b
lib/storage: optimize time series lookup for recent hours when the db contains many millions of time series with high churn rate (aka frequent deployments in Kubernetes)
2019-06-09 19:13:56 +03:00
Aliaksandr Valialkin
a3e26de45e
lib/procutil: typo fix in comment to WaitForSigterm
2019-06-04 17:31:47 +03:00
Aliaksandr Valialkin
28f6c36ab4
lib/storage: tune updating a map with today`s metric ids
...
- Increase update iterval from 1s to 10s. This should reduce CPU usage
for large amounts of metric ids with constant churn.
- Reduce pendingTodayMetricIDsLock lock duration during the update.
2019-06-02 21:58:16 +03:00
Aliaksandr Valialkin
4794f894a4
lib/storage: speed up checking metricID existence in the list for the current date
2019-06-02 18:34:08 +03:00
Aliaksandr Valialkin
e307a4d92c
lib/timerpool: use timer pool in concurrency limiters
...
This should reduce the number of memory allocations in highly loaded system
2019-05-28 17:20:10 +03:00
Aliaksandr Valialkin
54fb8b21f9
all: fix misspellings
2019-05-25 21:51:11 +03:00
Aliaksandr Valialkin
d6523ffe90
Makefile: add -s flag to go fmt in make fmt command
2019-05-25 21:43:35 +03:00
Aliaksandr Valialkin
6b0ae332f8
lib/encoding: add vm_zstd_block_{compress|decompress}_calls_total for determining the number CompressZSTD / DecompressZSTD calls
2019-05-24 13:01:02 +03:00
Aliaksandr Valialkin
6fb9dd09f5
lib/encoding: add vm_zstd_block_{original|compressed}_bytes_total metrics for rough estimation of block compression ratio
2019-05-24 12:34:32 +03:00
Aliaksandr Valialkin
19b6643e5c
lib/encoding: substitute CompressZSTD with CompressZSTDLevel
2019-05-24 12:32:55 +03:00
Aliaksandr Valialkin
08b889ef09
lib/httpserver: add -http.disableResponseCompression flag, which may help saving CPU resources at the cost of higher network bandwidth usage
2019-05-24 12:18:40 +03:00
Aliaksandr Valialkin
674888fdc9
lib/decimal: add a comment explaining weird code in maxUpExponent. Fixes #29
2019-05-23 17:18:35 +03:00
Aliaksandr Valialkin
1836c415e6
all: open-sourcing single-node version
2019-05-23 00:18:06 +03:00
