Aliaksandr Valialkin
f9e1d32168
lib/storage: persist metric ids for the current and the previous hour on graceful shutdown
...
This should improve performance after restart when the db contains a lot of time series
with high time series churn (i.e. metrics from Kubernetes with many pods and frequent deployments)
2019-06-14 07:55:09 +03:00
Aliaksandr Valialkin
b0b93e3d50
lib/fs: sync parent dir in MustRemoveAll only if it exists
...
The parent directory may be non-existing when the deleted directory
didn't exist before the MustRemoveAll call
2019-06-12 02:16:15 +03:00
Aliaksandr Valialkin
18d6f293f7
lib/fs: consolidate *RemoveAll* funcs into a single MustRemoveAll func
...
The func syncs parent dir in order to persist directory removal
in the event of power loss
2019-06-12 01:55:18 +03:00
Aliaksandr Valialkin
28d9904efc
lib/fs: panic with fatal error when directories cannot be removed
...
Unremoved directories may lead to inconsistent data directory,
so VictoriaMetrics will fail to start next time.
So panic on the first error when trying to remove directory in order
to simplify recover process.
2019-06-12 01:20:10 +03:00
Aliaksandr Valialkin
d897bc3f08
lib/fs: attempt #2 to work around NFS issue with directory removal
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/61
2019-06-12 01:07:29 +03:00
Aliaksandr Valialkin
51e2e255a6
lib/fs: consistency renaming SyncPath -> MustSyncPath, since it doesnt return error
2019-06-11 23:13:45 +03:00
Aliaksandr Valialkin
3fa4c28f6b
lib/fs: make sure the created directory remains visible in the fs in the event of power loss
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/63
2019-06-11 23:08:17 +03:00
Aliaksandr Valialkin
0b7f751f60
lib/fs: use filepath.Dir instead of filepath.Split, since the filename is unused
2019-06-11 22:54:23 +03:00
Aliaksandr Valialkin
b491045a4b
lib/{storage,mergeset}: sync filenames inside part when finalizing the part
...
Fixes https://github.com/VictoriaMetrics/VictoriaMetrics/issues/63
2019-06-11 21:51:19 +03:00
Aliaksandr Valialkin
3437c30180
all: try hard removing directory with contents
...
Fixes https://github.com/VictoriaMetrics/VictoriaMetrics/issues/61
2019-06-11 01:58:08 +03:00
Aliaksandr Valialkin
75a0acf72d
app/vmselect: add /api/v1/labels/count handler for quick detection of labels with the maximum number of distinct values
2019-06-10 19:54:55 +03:00
Aliaksandr Valialkin
0ccedbdfd2
lib/storage: mention the accountID and projectID in error message when filtering out other (accountID, projectID) entries
2019-06-10 14:43:53 +03:00
Aliaksandr Valialkin
d54f5fec0b
lib/storage: skip adaptive searching for tag filter matching the minimum number of metrics if the identical previous search didn't found such filter
...
This should improve speed for searching metrics among high number of time series
with high churn rate like in big Kubernetes clusters with frequent deployments.
2019-06-10 14:07:47 +03:00
Aliaksandr Valialkin
27e50e86f4
lib/storage: factor out getTagFilterWithMinMetricIDsCountAdaptive from updateMetricIDsForTagFilters
2019-06-10 13:26:00 +03:00
Aliaksandr Valialkin
b69d3dbd0c
lib/storage: filter out metricIDs from another (AccountID, ProjectID) in getMetricIDsForRecentHours
2019-06-10 13:05:16 +03:00
Aliaksandr Valialkin
3059ae7be0
lib/storage: give clearer names to more functions
2019-06-10 12:59:33 +03:00
Aliaksandr Valialkin
d3a024d2d6
lib/storage: give more clear names to functions
2019-06-10 12:50:22 +03:00
Aliaksandr Valialkin
00e0760608
lib/storage: test GetSeriesCount
2019-06-10 12:40:33 +03:00
Aliaksandr Valialkin
e4cba5a7ed
lib/storage: make getSeriesCount func indexSearch method
2019-06-10 12:29:24 +03:00
Aliaksandr Valialkin
d882afa905
lib/storage: optimize time series lookup for recent hours when the db contains many millions of time series with high churn rate (aka frequent deployments in Kubernetes)
2019-06-09 19:14:04 +03:00
Aliaksandr Valialkin
8cf0a0e59c
app/vminsert: split vm_rows_inserted_total into per-(accountID, projectID) metrics
...
Fixes https://github.com/VictoriaMetrics/VictoriaMetrics/issues/59
2019-06-07 22:11:20 +03:00
Aliaksandr Valialkin
1d87abc8eb
lib/procutil: typo fix in comment to WaitForSigterm
2019-06-03 16:54:37 +03:00
Aliaksandr Valialkin
a2986cde70
lib/storage: tune updating a map with today`s metric ids
...
- Increase update iterval from 1s to 10s. This should reduce CPU usage
for large amounts of metric ids with constant churn.
- Reduce pendingTodayMetricIDsLock lock duration during the update.
2019-06-02 22:00:13 +03:00
Aliaksandr Valialkin
e27fd5148a
lib/storage: speed up checking metricID existence in the list for the current date
2019-06-02 18:34:20 +03:00
Aliaksandr Valialkin
a6d02ff275
lib/timerpool: use timer pool in concurrency limiters
...
This should reduce the number of memory allocations in highly loaded system
2019-05-28 17:30:10 +03:00
Aliaksandr Valialkin
bdf696ef18
all: fix misspellings
2019-05-25 21:51:24 +03:00
Aliaksandr Valialkin
121a920a18
Makefile: add -s flag to go fmt in make fmt command
2019-05-25 21:44:36 +03:00
Aliaksandr Valialkin
9b64dfee4b
lib/encoding: add vm_zstd_block_{compress|decompress}_calls_total for determining the number CompressZSTD / DecompressZSTD calls
2019-05-24 13:01:15 +03:00
Aliaksandr Valialkin
b23352dc9e
lib/encoding: add vm_zstd_block_{original|compressed}_bytes_total metrics for rough estimation of block compression ratio
2019-05-24 12:34:51 +03:00
Aliaksandr Valialkin
f67f40d63a
lib/encoding: substitute CompressZSTD with CompressZSTDLevel
2019-05-24 12:32:49 +03:00
Aliaksandr Valialkin
a26e774eca
lib/httpserver: add -http.disableResponseCompression flag, which may help saving CPU resources at the cost of higher network bandwidth usage
2019-05-24 12:20:36 +03:00
Aliaksandr Valialkin
820cdae88d
lib/decimal: add a comment explaining weird code in maxUpExponent. Fixes #29
2019-05-23 17:18:50 +03:00
Aliaksandr Valialkin
24578b4bb1
all: open-sourcing cluster version
2019-05-23 00:25:38 +03:00
Aliaksandr Valialkin
1836c415e6
all: open-sourcing single-node version
2019-05-23 00:18:06 +03:00
