github-mirrors/VictoriaMetrics
1
0
Fork 0
mirror of https://github.com/VictoriaMetrics/VictoriaMetrics.git synced 2024-12-11 14:53:49 +00:00
Code Issues Projects Releases Packages Wiki Activity
3771 commits 131 branches 758 tags 363 MiB
Commit graph

501 commits

Author SHA1 Message Date
Aliaksandr Valialkin
c93cee8de8 lib/{mergeset,storage}: reduce the maximum lifetime for cached indexdb and data blocks from 2 minutes to a minute 
This should reduce memory usage on a system with high number of active time series and a high churn rate.
One minute is enough for caching the blocks needed for repeated queries (e.g. alerting rules, recording rules and dashboard refreshes).
 2021-06-29 19:57:07 +03:00
Aliaksandr Valialkin
5506cff76e lib/storage: put indexDBName into the key for dateTagFilter cache and for uselessTagFilters cache 
This should prevent from stats overwriting when the previous indexdb is queried.
 2021-06-29 12:40:05 +03:00
Aliaksandr Valialkin
452720c5dc Merge branch 'public-single-node' into pmm-6401-read-prometheus-data-files 2021-06-25 13:24:42 +03:00
Aliaksandr Valialkin
c22114c6f0 lib/storage: tune tag filters search logic 
Tune the logic according to the logs provided at https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1338#issuecomment-864293624

The previous logic had a race when multiple concurrent queries execute the same tag filter without prior stats.
This could result in incorrectly stored stats for such tag filter, which then could result in non-optimal sorting of tag filters
for further queries.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1338
 2021-06-23 13:29:39 +03:00
Aliaksandr Valialkin
bbca1740c1 Merge branch 'public-single-node' into pmm-6401-read-prometheus-data-files 2021-06-18 10:55:54 +03:00
Aliaksandr Valialkin
dcbc22552f lib/storage: fix infinite loop introduced in aa9b56a046 2021-06-17 14:28:10 +03:00
Aliaksandr Valialkin
aa9b56a046 lib/{mergeset,storage}: reduce the number of fsync calls on data ingestion path on systems with many cpu cores 
VictoriaMetrics maintains a buffer per CPU core for the ingested data. These buffers are flushed to disk every second.
These buffers are flushed to disk in parallel starting from the commit 56b6b893ce .
This resulted in increased write disk IO usage on systems with many cpu cores
as described at https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1338#issuecomment-863046999 .

This commit merges the per-CPU buffers into bigger in-memory buffers before flushing them to disk.
This should reduce the rate of fsync syscalls and, consequently, the write disk IO on systems with many CPU cores.

This should help https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1338
See also https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1244
 2021-06-17 13:52:08 +03:00
Aliaksandr Valialkin
84fb59b0ba lib/storage: move deletedMetricIDs set from indexDB to Storage 
This makes consitent the list of deleted metricIDs when it is used from both the current indexDB and the previous indexDB (aka extDB).
This should fix the issue, which could lead to storing new samples under deleted metricIDs after indexDB rotation.
See more details at https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1347#issuecomment-861232136 .

Thanks to @tangqipengleoo for the initial analysis and the pull request - https://github.com/VictoriaMetrics/VictoriaMetrics/pull/1383 .

This commit resolves the issue in more generic way compared to https://github.com/VictoriaMetrics/VictoriaMetrics/pull/1383 .

The downside of the commit is the deletedMetricIDs set isn't cleaned from the metricIDs outside the retention. It needs app restart.
This should be OK in most cases.
 2021-06-15 15:04:30 +03:00
Aliaksandr Valialkin
b348114dab Merge branch 'public-single-node' into pmm-6401-read-prometheus-data-files 2021-06-11 13:00:09 +03:00
Aliaksandr Valialkin
c4f3fbfa5d lib/storage: reset cache on disk during series deletion and during indexdb rotation 
This should prevent from inconsistent behavior (aka partially missing data for some time series) after unclean shutdown.

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1347
 2021-06-11 12:42:28 +03:00
Aliaksandr Valialkin
69b1482bdb lib/storage: consistency renaming: getMaxRawRowsPerPartition -> getMaxRawRowsPerShard 2021-06-11 10:57:23 +03:00
Aliaksandr Valialkin
044ab46824 lib/storage: reduce the amounts of memory which can be occupied by rawRow items during data ingestion on a system with many CPU cores 2021-06-11 10:57:23 +03:00
Aliaksandr Valialkin
fae6e4fc85 Merge branch 'public-single-node' into pmm-6401-read-prometheus-data-files 2021-06-09 19:10:11 +03:00
Aliaksandr Valialkin
96b691a0ab lib/storage: properly account the number of loops spent when matching for or suffixes 
This may help https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1338
 2021-06-08 13:06:12 +03:00
Aliaksandr Valialkin
2d8bd41f8a lib/storage: reduce memory allocations when syncing dateMetricIDCache 2021-06-03 16:20:42 +03:00
Aliaksandr Valialkin
e49bf9bc73 Merge branch 'public-single-node' into pmm-6401-read-prometheus-data-files 2021-05-24 16:03:14 +03:00
Aliaksandr Valialkin
39ef1e7a51 lib/storage: do not stop data ingestion on the first error in Storage.AddRows 
Continue data ingestion for the rest of blocks.
 2021-05-24 15:32:47 +03:00
Aliaksandr Valialkin
4b01c9fb2e lib/storage: limit the number of rows per each block in Storage.AddRows() 
This should reduce memory usage when ingesting big blocks or rows.
 2021-05-24 15:24:07 +03:00
Aliaksandr Valialkin
a4ff4b8e65 lib/storage: allow filling all the rows up to their capacity in rawRowsShard.addRows 
This should reduce memory usage a bit on data ingestion path
 2021-05-24 15:22:59 +03:00
Aliaksandr Valialkin
f54133b200 lib/storage: do not populate MetricID->MetricName cache during data ingestion 
This cache isn't needed during data ingestion, so there is no need in spending RAM on it.

This reduces RAM usage on data ingestion path by 30%
 2021-05-24 03:02:46 +03:00
Aliaksandr Valialkin
ec79abc382 lib/{mergeset,storage}: reduce the number of IFNO log messages like merged ... items across ... blocks in ... seconds 
Log these messages if the merge takes more than 30 seconds instead of 10 seconds.
 2021-05-23 14:03:21 +03:00
Aliaksandr Valialkin
ad73f226ff app/vmstorage: add ability to limit series cardinality via -storage.maxHourlySeries and -storage.maxDailySeries command-line flags 2021-05-20 14:15:19 +03:00
Aliaksandr Valialkin
009e136d88 lib/storage: remove possible data race when logging dropped labels 2021-05-20 02:47:22 +03:00
Aliaksandr Valialkin
d7be2753c0 lib/storage: substitute GetTSDBStatusForDate with GetTSDBStatusWithFiltersForDate with nil tfss 2021-05-13 09:02:33 +03:00
Aliaksandr Valialkin
a22a17dc66 lib/storage: merge getTSDBStatusForDate with getTSDBStatusWithFiltersForDate 
These functions are non-trivial, while their code has minimal differences.
It is better from maintainability PoV to merge these functions into a single function.
 2021-05-12 17:56:53 +03:00
Aliaksandr Valialkin
832651c6c2 app/vmselect: follow up after 8a0678678b 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1168
 2021-05-12 17:18:30 +03:00
Nikolay
8a0678678b
Adds tsdb match filters (#1282) 
* init work on filters

* init propose for status filters

* fixes tsdb status
adds test

* fix bug

* removes checks from test
 2021-05-12 15:18:45 +03:00
Aliaksandr Valialkin
33f7bacb01 lib/storage: properly apply time range when matching an empty filter 
It must match all the time series on the given time range.
Previously it was matched to all the time series without the restriction on the given time range.
 2021-05-11 01:12:05 +03:00
Aliaksandr Valialkin
75c2c813fc lib/storage: remove dead code after the commit 3ccf7ea20c 2021-05-08 20:14:11 +03:00
Aliaksandr Valialkin
12d733dd5d app/vminsert: add support for data ingestion via other vminsert nodes 2021-05-08 19:52:57 +03:00
Aliaksandr Valialkin
76027093ca lib/storage: use WARNING instead of INFO level for logging dropped labels 2021-05-03 13:56:43 +03:00
Nikolay
477369b62f
adds stalePartsRemover (#1261) 
for new created partitions
 2021-05-03 11:34:00 +03:00
Aliaksandr Valialkin
a142390014 Merge branch 'public-single-node' into pmm-6401-read-prometheus-data-files 2021-05-01 10:05:48 +03:00
Aliaksandr Valialkin
58d1e6eeea lib/storage: log dropped labels if the number of labels in a metric exceeds -maxLabelsPerTimeseries command-line flag value 
This should improve debuggability for this case.
 2021-05-01 09:27:55 +03:00
Aliaksandr Valialkin
bceb8082f6 Merge branch 'public-single-node' into pmm-6401-read-prometheus-data-files 2021-04-30 10:11:24 +03:00
Aliaksandr Valialkin
87179c6839 lib/{storage,mergeset}: fix unaligned 64-bit atomic operation panic for 32-bit architectures 
The panic has been introduced in 56b6b893ce
 2021-04-27 16:41:32 +03:00
Aliaksandr Valialkin
56b6b893ce lib/mergeset: split rows ingestion among multiple shards 
This improves rows ingestion on systems with many CPU cores by reducing lock contention.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1244

Thanks to @waldoweng for the original idea and draft implementation at https://github.com/VictoriaMetrics/VictoriaMetrics/pull/1243
 2021-04-27 15:36:34 +03:00
Aliaksandr Valialkin
f89c1f7f49 lib/storage: typo fix in info message when deleting the part outside the configured retention 
Previously the message was displaying incorrect retention time
 2021-04-27 13:32:46 +03:00
Aliaksandr Valialkin
276969500e Merge branch 'public-single-node' into pmm-6401-read-prometheus-data-files 2021-04-24 01:42:54 +03:00
Aliaksandr Valialkin
bbebdf9ba1 lib/{storage,mergeset}: remove empty directories on startup. Such directories can be left after unclean shutdown on NFS storage 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1142
 2021-04-22 13:02:44 +03:00
Aliaksandr Valialkin
030e3a63f2 Merge branch 'public-single-node' into pmm-6401-read-prometheus-data-files 2021-04-21 11:00:07 +03:00
Aliaksandr Valialkin
251747e253 lib/storage: code clarification: remove caching the found metricName in searchMetricName 2021-04-13 10:22:21 +03:00
Aliaksandr Valialkin
660c3c7251 Merge branch 'public-single-node' into pmm-6401-read-prometheus-data-files 2021-04-08 00:54:19 +03:00
Aliaksandr Valialkin
df32d2836c lib/storage: properly handle big time ranges passed to /api/v1/labels and /api/v1/label/<labelName>/values 
It should be faster querying all the labels and/or all the values instead of querying per-day labels/values on time ranges exceeding maxDaysForPerDaySearch
 2021-04-07 13:33:46 +03:00
Aliaksandr Valialkin
dc9eafcd02 app/{vminsert,vmagent}: add -sortLabels command-line option for sorting time series labels before ingesting them in the storage 
This option can be useful when samples for the same time series are ingested with distinct order of labels.
For example, metric{k1="v1",k2="v2"} and metric{k2="v2",k1="v1"}.
 2021-03-31 23:27:58 +03:00
Aliaksandr Valialkin
e1f699bb6c lib/storage: reduce memory usage when ingesting samples for the same time series with distinct order of labels 2021-03-31 21:24:46 +03:00
Aliaksandr Valialkin
11ded82e60 Merge branch 'public-single-node' into pmm-6401-read-prometheus-data-files 2021-03-29 19:15:52 +03:00
Aliaksandr Valialkin
2601cc0fb0 lib/storage: do not update b.nextIdx if no samples are removed because of retention 2021-03-29 12:00:21 +03:00
Aliaksandr Valialkin
aa81039b42 app/vmselect: log the metric which trigger rollup result cache reset 
This should help finding the source of stale metrics
 2021-03-25 21:31:39 +02:00
Aliaksandr Valialkin
343f444e87 Merge branch 'public-single-node' into pmm-6401-read-prometheus-data-files 2021-03-25 19:15:08 +02:00
Aliaksandr Valialkin
6e855d4b82 lib/storage: tune loopsCountPerMetricNameMatch according to production workload 2021-03-25 13:27:47 +02:00
Aliaksandr Valialkin
3a3d2165f9 lib/storage: do not reload metricName for the same metricID in Search.NextMetricBlock 
This should speed up Search.NextMetricBlock a bit
 2021-03-23 17:56:49 +02:00
Aliaksandr Valialkin
3cfb3a3683 lib/storage: respect the deadline passed to Storage.SearchMetricNames 2021-03-22 23:03:17 +02:00
Aliaksandr Valialkin
8e2afdf568 lib/storage: improve Search.NextMetricBlock performance by using MetricID->MetricName cache 2021-03-22 22:49:18 +02:00
Aliaksandr Valialkin
910092ca4d lib/storage: tune loopsCountPerMetricNameMatch 2021-03-22 12:53:17 +02:00
Aliaksandr Valialkin
726f6ad804 lib/storage: small code simplification after 6cee5338b2 2021-03-18 15:21:13 +02:00
Aliaksandr Valialkin
6cee5338b2 lib/storage: prevent from infinite loop if {__graphite__="..."} filter matches a metric name with *, [ or { chars 
The idea has been borrowed from https://github.com/VictoriaMetrics/VictoriaMetrics/pull/1137
 2021-03-18 14:53:47 +02:00
Aliaksandr Valialkin
45dabfac1b lib/storage: faster move heavy filters to the end of list 2021-03-17 15:12:13 +02:00
Aliaksandr Valialkin
16884c20c0 Merge branch 'public-single-node' into pmm-6401-read-prometheus-data-files 2021-03-17 02:05:46 +02:00
Aliaksandr Valialkin
ccfb0ae2d3 lib/storage: limit loops count in order to reduce max CPU usage during filter search 2021-03-17 00:49:26 +02:00
Aliaksandr Valialkin
576a80b3d9 lib/storage: do not modify filterLoopsCount stats with loopsCount stats 
Such a modification can result in incorrect filter sorting later
 2021-03-17 00:49:26 +02:00
Aliaksandr Valialkin
fd86a7dc1d lib/storage: time series search optimization according to production workload profiling 
Do not pass filter metric ids to getMetricIDsForTagFilter, since it has been appeared that this slows down
the function by multiple times when it finds big number of metricIDs (tens of millions).
 2021-03-16 20:01:43 +02:00
Aliaksandr Valialkin
e36fbfae5b lib/storage: further tuning for time series search 2021-03-16 18:46:22 +02:00
Aliaksandr Valialkin
7d44cdd8ce Merge branch 'public-single-node' into pmm-6401-read-prometheus-data-files 2021-03-15 22:44:24 +02:00
Aliaksandr Valialkin
dd7e82c34f app/vmstorage: add -logNewSeries command-line flag for determining the source of series churn rate 2021-03-15 22:38:50 +02:00
Aliaksandr Valialkin
9c77e34ef9 lib/storage: further tuning for time series selector code 2021-03-15 20:31:34 +02:00
Aliaksandr Valialkin
3ccf7ea20c lib/storage: tune per-day index search 2021-03-15 13:31:55 +02:00
Aliaksandr Valialkin
f669531506 lib/storage: further tune filters sorting logic 2021-03-12 00:53:04 +02:00
Aliaksandr Valialkin
8582fba4b1 Merge branch 'public-single-node' into pmm-6401-read-prometheus-data-files 2021-03-09 11:46:39 +02:00
John Belmonte
364fdf4a56
spelling fix: adjacent (#1115) 2021-03-09 09:18:19 +02:00
Aliaksandr Valialkin
345980f78f lib/storage: go fmt 2021-03-08 12:03:31 +02:00
Aliaksandr Valialkin
18fe0ff14b lib/storage: tune loopsCount estimations in getMetricIDsForTagFilterSlow 
The adjusted estmations give up to 2x lower median response times on 200qps /api/v1/query_range workload
 2021-03-07 21:12:35 +02:00
Aliaksandr Valialkin
6197440bb9 Merge branch 'public-single-node' into pmm-6401-read-prometheus-data-files 2021-03-02 21:46:03 +02:00
Aliaksandr Valialkin
4de4da1e2a lib/storage: typo fix: umarshal -> unmarshal 2021-03-02 20:47:59 +02:00
Aliaksandr Valialkin
0ad887fd4d Merge branch 'public-single-node' into pmm-6401-read-prometheus-data-files 2021-02-26 23:00:40 +02:00
Aliaksandr Valialkin
2c44178645 lib/storage: consistency renaming: durationsPerDateTagFilterCache -> loopsPerDateTagFilterCache 2021-02-23 15:47:19 +02:00
faceair
15d61c4879
lib/storage: correct tagfilter match cost (#1079) 2021-02-22 21:46:56 +02:00
Aliaksandr Valialkin
636c55b526 lib/mergeset: reduce memory usage for inmemoryBlock by using more compact items representation 
This also should reduce CPU time spent by GC, since inmemoryBlock.items don't have pointers now,
so GC doesn't need visiting them.
 2021-02-21 22:06:47 +02:00
Aliaksandr Valialkin
388cdb1980 lib/storage: do not re-calculate stats for heavy tag filters 
This should reduce the number of slow queries when stats for heavy tag filters was recalculated.
 2021-02-21 21:39:01 +02:00
Aliaksandr Valialkin
48656dcc38 lib/{mergeset,storage}: allow merging smaller number of small parts 
While this may increase CPU and disk IO usage needed for background merge,
this also recudes CPU usage during queries in production. This is because
such queries tend to read recently added data and it is better to have lower number
of parts for such data in order to reduce CPU usage.

This partially reverts ebf8da3730
 2021-02-21 21:28:36 +02:00
Aliaksandr Valialkin
cb311bb156 lib/{mergeset,storage}: do not use pools for indexBlock and inmemoryBlock during their caching, since this results in higher memory usage in production without any performance gains 2021-02-21 21:18:59 +02:00
Aliaksandr Valialkin
d5dde7f6b1 Merge branch 'public-single-node' into pmm-6401-read-prometheus-data-files 2021-02-18 19:14:41 +02:00
Aliaksandr Valialkin
f9084611bd lib/storage: use composite index for a query with a name filter and negative filters 2021-02-18 18:57:23 +02:00
Aliaksandr Valialkin
a537c4f602 lib/storage: properly handle queries containing a filter on metric name plus any number of negative filters and zero non-negative filters 
Example: `node_cpu_seconds_total{mode!="idle"}`
 2021-02-18 18:46:36 +02:00
Aliaksandr Valialkin
3588687f84 Merge branch 'public-single-node' into pmm-6401-read-prometheus-data-files 2021-02-18 14:53:19 +02:00
Aliaksandr Valialkin
e540c02014 lib/storage: prevent from running identical heavy tag filters in concurrent queries when measuring the number of loops for such tag filter. 
This should reduce CPU usage spikes when measuring the number of loops needed for heavy tag filters
 2021-02-18 13:58:18 +02:00
Aliaksandr Valialkin
711f8a5b8d lib/storage: sort tag filters by the number of loops they need for the execution 
This metric should work better than the filter execution duration, since it cannot be distorted
by concurrently running queries.
 2021-02-18 12:47:38 +02:00
Aliaksandr Valialkin
ce99b48a9a Revert "lib/mergeset: tune lifetime for entries inside block caches" 
This reverts commit 458c89324d.

Production testing revealed zero improvements for memory usage with reduced lifetime for entries in block caches.
 2021-02-17 20:42:21 +02:00
Aliaksandr Valialkin
939d5ffc2b lib/storage: move composite filters to the top during sorting 2021-02-17 20:26:51 +02:00
Aliaksandr Valialkin
faad6f84a4 lib/storage: return back filter arg to getMetricIDsForTagFilter function 
The filter arg has been removed in the commit c7ee2fabb8
because it was preventing from caching the number of matching time series per each tf.

Now the cache contains duration for tf execution, so the filter shouldn't break such caching.
 2021-02-17 19:33:22 +02:00
Aliaksandr Valialkin
d4849561ef app/vmstorage: export vm_composite_filter_success_conversions_total and vm_composite_filter_missing_conversions_total metrics 2021-02-17 19:13:38 +02:00
Aliaksandr Valialkin
33806264ec lib/storage: revert ecf132933e, since negative filters require the same amount of work as non-negative filters 2021-02-17 18:55:04 +02:00
Aliaksandr Valialkin
63fc140624 lib/storage: tag filters sorting... 2021-02-17 17:55:29 +02:00
Aliaksandr Valialkin
74424b55ee lib/storage: further tune tag filters sorting 2021-02-17 17:28:15 +02:00
Aliaksandr Valialkin
442fcfec5a lib/storage: tune the logic for sorting tag filters according the their exeuction times 2021-02-17 15:00:08 +02:00
Aliaksandr Valialkin
4a07820048 lib/storage: make sure that nobody uses partitions when closing the table 2021-02-17 14:59:04 +02:00
Aliaksandr Valialkin
687eb4ab00 Merge branch 'public-single-node' into pmm-6401-read-prometheus-data-files 2021-02-16 22:29:45 +02:00
Aliaksandr Valialkin
d61f7b7279 lib/storage: more tuning for tag filters sorting according the time they take 2021-02-16 21:22:23 +02:00
Aliaksandr Valialkin
458c89324d lib/mergeset: tune lifetime for entries inside block caches 
This should reduce memory usage in general case without significant CPU usage increase
 2021-02-16 18:11:51 +02:00
Aliaksandr Valialkin
ca191696fe lib/storage: tune sorting for tag filters 2021-02-16 13:04:49 +02:00