github-mirrors/VictoriaMetrics
1
0
Fork 0
mirror of https://github.com/VictoriaMetrics/VictoriaMetrics.git synced 2024-11-21 14:44:00 +00:00
Code Issues Projects Releases Packages Wiki Activity
342 commits 150 branches 748 tags 347 MiB
Commit graph

84 commits

Author SHA1 Message Date
Aliaksandr Valialkin
ac004665b5 all: return 503 http error if service is temporarily unavailable 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/156
 2019-08-23 09:55:07 +03:00
Aliaksandr Valialkin
8c03a8c4b4 app/vminsert: allow setting the maximum number of labels per time series via -maxLabelsPerTimeseries 2019-08-23 08:45:26 +03:00
Aliaksandr Valialkin
380cae23a0 lib/storage: add benchmarks for regexp filter match / mismatch 
These benchmarks allow estimate the performance of regexp filters in promql
 2019-08-22 16:36:42 +03:00
Aliaksandr Valialkin
4f738c8a15 lib/storage: try slower path for searching the tag filter with the minimum number of matching time series before giving up with increase -search.maxUniqueTimeseries error 2019-08-19 16:04:21 +03:00
Aliaksandr Valialkin
c23b66a1ad lib/storage: pre-allocate memory for blockHeader slice in unmarshalBlockHeaders 
This reduces memory usage and memory fragmentation when working with big number of time series
 2019-08-19 12:46:33 +03:00
Aliaksandr Valialkin
483de1cc06 lib/workingsetcache: automatically detect when it is better to double cache capacity 2019-08-15 22:57:55 +03:00
Aliaksandr Valialkin
5b41122292 lib/storage: properly cache tagFilters -> TSIDs entries from historical index 2019-08-14 02:29:58 +03:00
Aliaksandr Valialkin
964c296f96 lib/storage: compress contents of cache for tagFilters -> TSIDs 
This should increase cache capacity
 2019-08-14 02:29:52 +03:00
Aliaksandr Valialkin
09fc6e22e5 all: use workingsetcache instead of fastcache 
This should reduce the amount of RAM required for processing time series
with non-zero churn rate.

The previous cache behavior can be restored with `-cache.oldBehavior` command-line flag.
 2019-08-13 21:39:34 +03:00
Aliaksandr Valialkin
99c37c2c96 lib/fs: add test for IsTemporaryFileName 2019-08-13 21:33:45 +03:00
Aliaksandr Valialkin
ec1b185991 lib/storage: remove broken BenchmarkIndexDBSearchTSIDs 2019-08-13 20:22:08 +03:00
Aliaksandr Valialkin
0967683ae9 lib: move common code for creating flock.lock file into fs.CreateFlockFile 2019-08-13 01:45:46 +03:00
Aliaksandr Valialkin
5d8d110010 lib/fs: atomically create file with the given contents on WriteFileAtomically 
This should prevent from `transaction` and `metadata.json` files corruption
on unclean shutdown such as OOM, `kill -9`, power loss, etc.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/148
 2019-08-12 15:02:55 +03:00
Aliaksandr Valialkin
0b488f1e37 lib/storage: do not change timestamps to constant rate if values are constant or have constant delta 
This breaks the original timestamps, which results in issues like
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/120 and
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/141 .
 2019-08-06 15:40:07 +03:00
Aliaksandr Valialkin
b8bb74ffc6 app/vmstorage: add vm_concurrent_addrows_* metrics for tracking concurrency for Storage.AddRows calls 
Track also the number of dropped rows due to the exceeded timeout
on concurrency limit for Storage.AddRows. This number is tracked in `vm_concurrent_addrows_dropped_rows_total`
 2019-08-06 15:08:33 +03:00
Aliaksandr Valialkin
8822079b77 lib/storage: properly reset partSearch.fetchData in partSearch.reset 2019-08-05 09:56:06 +03:00
Aliaksandr Valialkin
47e4b50112 app/vmselect: optimize /api/v1/series by skipping storage data 
Fetch and process only time series metainfo.
 2019-08-04 23:01:28 +03:00
Aliaksandr Valialkin
10caad4728 lib/decimal: modernize tests a bit 2019-07-31 21:10:03 +03:00
Aliaksandr Valialkin
c14fd6c43f lib/storage: typo fixes after a77e88db7d 2019-07-30 15:38:52 +03:00
Aliaksandr Valialkin
a77e88db7d lib/storage: fix matching against tag filter with empty name 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/137
 2019-07-30 15:15:09 +03:00
Aliaksandr Valialkin
f586e1f83c lib/storage: add metrics for calculating skipped rows outside the retention 
The metrics are:

    - vm_too_big_timestamp_rows_total
    - vm_too_small_timestamp_rows_total
 2019-07-26 14:11:01 +03:00
Aliaksandr Valialkin
b339890dca lib/encoding/zstd: go fmt 2019-07-25 01:37:16 +03:00
Aliaksandr Valialkin
6c4ca89d75 lib/encoding/zstd: disable CRC checks in pure Go build 
This should give slightly better compression and decompressions performance.
Additionally this shaves off 4 bytes per each compressed block.
 2019-07-24 19:17:16 +03:00
Aliaksandr Valialkin
d07f616609 lib/encoding: small fixes in tests after the PR #114 2019-07-23 19:37:51 +03:00
Roman Khavronenko
5bf4e5ffb5 all: add Pure Go build (pull request #114) 
Updates #94
 2019-07-23 19:26:39 +03:00
Aliaksandr Valialkin
0f2ecde0e6 lib/encoding: improve gauge series detection 
- Series with negative values are always gauges
- Counters may only have increasing values with possible counter resets

This should improve compression ratio for gauge series which
were previously mistakenly detected as counters.
 2019-07-20 14:05:09 +03:00
Aliaksandr Valialkin
daba0cdb05 lib/netutil: do not count timeouts as network errors 2019-07-15 23:05:35 +03:00
Aliaksandr Valialkin
e75d5f47c4 lib/storage: remove unused function isTooBigTimeRangeForDateMetricIDs 2019-07-12 02:28:23 +03:00
Aliaksandr Valialkin
fc90ebf43c lib/storage: do not reduce maxMetrics on time ranges exceeding maxDaysForDateMetricIDs 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/95
 2019-07-12 02:20:34 +03:00
Aliaksandr Valialkin
2bd1a01d1a lib/storage: do not pollute inverted index with data for samples outside the retention period 2019-07-11 17:04:56 +03:00
Aliaksandr Valialkin
d031e04023 lib/storage: use fast path for orSuffix when searching for metricIDs against plain tag value 2019-07-11 14:48:37 +03:00
Aliaksandr Valialkin
43ea4ce428 lib/storage: remember and skip individual tag filters matching too many metrics 
This saves CPU time by skipping useless matching for individual tag filters.
 2019-07-11 14:48:30 +03:00
Aliaksandr Valialkin
1fe6d784d8 all: consistency renaming: bytesSize -> sizeBytes 2019-07-10 00:47:36 +03:00
Aliaksandr Valialkin
56c154f45b all: add vm_data_size_bytes metrics for easy monitoring of on-disk data size and on-disk inverted index size 2019-07-04 19:42:30 +03:00
Aliaksandr Valialkin
c7034fc51b lib/memory: attempt #3 to determine memory limit for LXC container 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/84
 2019-07-01 14:01:13 +03:00
Aliaksandr Valialkin
a41955863a lib/mergeset: make fmt 2019-06-29 14:25:26 +03:00
Aliaksandr Valialkin
2ecb117082 lib/storage: skip non-matching metricIDs in sortedFilter 
This should improve performance for big sorteFilter lists.
 2019-06-29 13:48:32 +03:00
Aliaksandr Valialkin
0c88afa386 lib/mergeset: speed up binarySearchKey by skipping the first item during binary search 2019-06-29 13:45:49 +03:00
Aliaksandr Valialkin
828078eb45 lib/memory: remove TestReadLXCMemoryLimit, since it doesnt work in Travis 2019-06-28 18:22:46 +03:00
Aliaksandr Valialkin
7b59466667 lib/memory: attempt #2 to determine memory limit inside LXC container 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/84
 2019-06-28 18:08:15 +03:00
Aliaksandr Valialkin
593bd35aaa lib/memory: an attempt to read proper memory limit inside LXC container 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/84
 2019-06-28 15:34:30 +03:00
Aliaksandr Valialkin
c1be1e4342 lib/storage: optimize time series search by regexp filter 
This should improve search speed on label filters like `{foo=~"bar.+baz"}`
 2019-06-27 16:17:43 +03:00
Aliaksandr Valialkin
683bf2a11f lib/storage: make sure non-nil args are passed to openIndexDB 2019-06-25 20:10:04 +03:00
Aliaksandr Valialkin
eb2283a029 lib/storage: reduce too big maxMetrics in getTagFilterWithMinMetricIDsCountAdaptive 
This should improve performance on inverted index search for big amount of unique time series
when big -search.maxUniqueTimeseries is set.
 2019-06-25 19:55:27 +03:00
Aliaksandr Valialkin
e8377011ab lib/storage: free up memory from caches owned by indexDB when it is deleted 2019-06-25 14:42:44 +03:00
Aliaksandr Valialkin
33ea2120c3 lib/storage: use unversioned keys for tag cache in extDB 
Data in ExtDB cannot be changed, so it is OK to use unversioned keys for tag cache.
This should improve performance for index lookups over big amount of time series.
 2019-06-25 13:08:58 +03:00
Aliaksandr Valialkin
cf63669303 lib/storage: skip searching in extDB if it doesn't contain items for the given time range 
This should improve inverted index search performance for big amount
of unique time series when the search is performed only on recent data.
 2019-06-25 13:00:37 +03:00
Aliaksandr Valialkin
af2ceaaa0b lib/storage: mention source parts on merge error 
This should improve determining broken source part.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/76
 2019-06-24 14:08:43 +03:00
Aliaksandr Valialkin
e88a03323a all: initial stubs for Windows support; see https://github.com/VictoriaMetrics/VictoriaMetrics/issues/70 2019-06-20 20:07:10 +03:00
Aliaksandr Valialkin
80db24386e lib/storage: typo fixes found by golangci-lint; updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/69 2019-06-20 14:37:55 +03:00