Aliaksandr Valialkin
8df8c414de
Merge branch 'public-single-node' into pmm-6401-read-prometheus-data-files
2021-07-15 14:05:30 +03:00
Aliaksandr Valialkin
61cc13c16f
docs/CHANGELOG.md: cut v1.63.0
2021-07-15 14:02:13 +03:00
Aliaksandr Valialkin
b060d8bf53
vendor: make vendor-update
2021-07-15 12:55:40 +03:00
Aliaksandr Valialkin
d472b03e34
lib/storage: make sure the second call to DeduplicateSamples and deduplicateSamplesDuringMerge doesnt change samples
2021-07-15 12:17:45 +03:00
Aliaksandr Valialkin
682662b2ae
lib/storage: remove cache directory if it contains reset_cache_on_startup file
...
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1447
2021-07-13 17:58:51 +03:00
Aliaksandr Valialkin
2b0e3efa5c
.github/workflows/wiki.yml: properly copy subdirectories
2021-07-13 17:35:02 +03:00
Aliaksandr Valialkin
8d99e94a52
docs: clarify why spare CPU and RAM resources are needed in capacity planning
2021-07-13 15:48:25 +03:00
Aliaksandr Valialkin
8d0ec47be9
docs/Cluster-VictoriaMetrics.md: clarify the docs about the needed values for -dedup.minScrapeInterval
at vmselect
during replication when the data is pushed from HA pair
2021-07-13 15:43:02 +03:00
Aliaksandr Valialkin
2df66dad7b
lib/httpserver: add is_set
label to flag
metrics
...
This label allows determining the set flags with the query `flag{is_set="true"}`
2021-07-13 15:10:13 +03:00
Aliaksandr Valialkin
5bc240bffe
vendor: make vendor-update
2021-07-13 14:30:19 +03:00
Aliaksandr Valialkin
244d0fe5d7
deployment/docker: update Go builder from v1.16.5 to v1.16.6
...
Ths Go release has the following bugfixes: https://github.com/golang/go/issues?q=milestone%3AGo1.16.6+label%3ACherryPickApproved
2021-07-13 14:25:41 +03:00
Aliaksandr Valialkin
a925d5a3e1
app/vmselect/promql: duration handling improvements in MetricsQL queries
...
- Support durations anywhere in MetricsQL queries. E.g. sum_over_time(m[1h])/1h is equivalent to sum_over_time(m[1h])/3600
- Support durations without suffix. E.g. rate(m[300]) is equivalent to rate(m[5m])
2021-07-12 17:16:41 +03:00
Aliaksandr Valialkin
f9de546139
lib/storage: reset perKeyMisses stats less frequently
...
This should reduce CPU usage for queries executed with intervals higher than 30 seconds
2021-07-12 14:33:42 +03:00
Aliaksandr Valialkin
4f80b2f230
lib/storage: properly limit the size of storage/date_metricID
cache
2021-07-12 14:25:44 +03:00
Aliaksandr Valialkin
f3a5465ece
docs/CHANGELOG.md: document the change from bfba4c28a4
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1444
2021-07-12 12:44:06 +03:00
Aliaksandr Valialkin
bfba4c28a4
app/vmalert: accept Prometheus-like durations in interval
config option inside group
section
2021-07-12 12:35:17 +03:00
Aliaksandr Valialkin
a684408e27
docs: update http://slack.victoriametrics.com to https://slack.victoriametrics.com
2021-07-12 10:57:16 +03:00
Aliaksandr Valialkin
8ca2799478
lib/storage: properly determine when the deduplication is needed in needsDedup
...
Previously needsDedup() could return true if the de-duplication wasn't needed for the following case:
d < interval
/ \
| v | v |
interval interval
Now it properly returns false for this case
2021-07-12 10:53:30 +03:00
Aliaksandr Valialkin
f539772ca6
docs: sync with the cluster branch
2021-07-10 12:45:38 +03:00
Aliaksandr Valialkin
8c764e88f0
app/vmui: move source code from https://github.com/VictoriaMetrics/vmui to app/vmui
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1413
2021-07-09 17:15:23 +03:00
Aliaksandr Valialkin
2be340a4c9
docs: clarify what does "workload" mean in capacity planning docs
2021-07-09 12:49:53 +03:00
Aliaksandr Valialkin
c5f0b454f0
app/vmselect: follow-up after aa11ef6d3b
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1413
2021-07-07 17:43:35 +03:00
tony
e9e35a7d6a
add vmui for vmselect component ( #1431 )
...
Co-authored-by: Aliaksandr Valialkin <valyala@gmail.com>
2021-07-07 17:33:02 +03:00
Aliaksandr Valialkin
92b8380fdf
docs/Cluster-VictoriaMetrics.md: improve capacity planning recommendations
2021-07-07 16:22:51 +03:00
Aliaksandr Valialkin
1152de4321
vendor: make vendor-update
2021-07-07 16:05:04 +03:00
Aliaksandr Valialkin
7e77980608
vendor: update github.com/VictoriaMetrics/metrics from v1.17.2 to v1.17.3
2021-07-07 16:00:25 +03:00
Aliaksandr Valialkin
6e0553c92e
lib/mergeset: cache indexBlock items only on the second request
...
This should reduce the indexdb/indexBlocks cache size, since it won't contain one-time-wonders items.
2021-07-07 15:23:06 +03:00
Aliaksandr Valialkin
ed944313b0
app/{vminsert,vmselect}: export vminsert_request_duration_seconds and vmselect_request_duration_seconds histograms
2021-07-07 13:25:21 +03:00
Aliaksandr Valialkin
766edbc421
lib/httpserver: print full requestURI in httpserver.Errorf
...
This should simplify debugging.
2021-07-07 13:09:40 +03:00
Aliaksandr Valialkin
c55a64ba08
docs: clarify capacity planning
docs
2021-07-07 12:46:55 +03:00
Aliaksandr Valialkin
e843bd7bd7
lib/storage: do not cache inmemoryBlock entries requested only once (aka one-time-wonder items)
...
This should reduce the cache size and memory usage for the indexdb/dataBlocks cache
2021-07-07 10:58:51 +03:00
Aliaksandr Valialkin
8b262d4ba7
lib/storage: periodically reset prefetchedMetricIDs cache in order to limit its size under high churn rate
2021-07-07 10:58:51 +03:00
Roman Khavronenko
2f54559c89
alerts: sync alert expression for DiskRunsOutOfSpaceIn3Days
with dashboard ( #1436 )
2021-07-07 10:31:09 +03:00
Aliaksandr Valialkin
a7694092b8
Revert "lib/uint64set: allow reusing bucket16 structs inside uint64set.Set via uint64set.Release method"
...
This reverts commit 7c6d3981bf
.
Reason for revert: high contention at bucket16Pool on systems with big number of CPU cores.
This slows down query processing significantly.
2021-07-06 18:21:35 +03:00
Aliaksandr Valialkin
8aa9bba9bd
lib/{mergeset,storage}: switch from sync.Pool to chan-based pool for inmemoryPart objects
...
This should reduce memory usage on systems with big number of CPU cores,
since every inmemoryPart object occupies at least 64KB of memory and sync.Pool maintains
a separate pool inmemoryPart objects per each CPU core.
Though the new scheme for the pool worsens per-cpu cache locality, this should be amortized
by big sizes of inmemoryPart objects.
2021-07-06 16:28:41 +03:00
Aliaksandr Valialkin
7c6d3981bf
lib/uint64set: allow reusing bucket16 structs inside uint64set.Set via uint64set.Release method
...
This reduces the load on memory allocator in Go runtime in production workload.
2021-07-06 15:35:03 +03:00
Aliaksandr Valialkin
78c9174682
lib/mergeset: increase pool capacity for inmemoryBlock according to collected profiles from production workload
...
CPU and memory profiles show that the pool capacity for inmemoryBlock objects is too small.
This results in the increased load on memory allocation code in Go runtime.
Increase the pool capacity in order to reduce the load on Go runtime.
2021-07-06 13:41:34 +03:00
Aliaksandr Valialkin
f71e4d1853
lib/mergeset: limit the frequency for flushCallback calls to once per 10 seconds
...
This should improve hit ratio for tagFiltersCache when big number of new time series are constantly registered
(aka high churn rate). This, in turn, should reduce CPU usage for queries over such time series.
2021-07-06 12:17:17 +03:00
Aliaksandr Valialkin
f3acf065c9
lib/storage: consistency renaming: tagCache -> tagFiltersCache
...
This improves code readability
2021-07-06 11:03:51 +03:00
Aliaksandr Valialkin
0020b9f904
lib/workingsetcache: properly update stats for requests and cache misses
...
Previously the stats for cache misses could be improperly counted, because it had inflated cache misses
if the entry was missing in the curr cache, but was existing in the prev cache.
The same applies to cache requests - they were inflated if the entry was missing in the curr cache.
2021-07-06 10:53:32 +03:00
Roman Khavronenko
75f12bfe78
add option to add Copy button for code snippets ( #1433 )
...
To add a Copy button wrap code snippet with the following element:
```
<div class="with-copy" markdown="1">
<your-code-snippet>
</div>
```
See the changes to `Kubernetes monitoring with VictoriaMetrics Single` for details.
2021-07-06 08:23:39 +03:00
Aliaksandr Valialkin
4cf47163c1
lib/workingsetcache: fix cache capacity calculations after 4f0003f182
2021-07-05 17:11:57 +03:00
Aliaksandr Valialkin
4f0003f182
lib/workingsetcache: typo fixes after d0c830039d
2021-07-05 15:35:37 +03:00
Aliaksandr Valialkin
d0c830039d
lib/storage: tune cache sizes according to production workload
2021-07-05 15:16:11 +03:00
Aliaksandr Valialkin
8f973e34fb
lib/workingsetcache: properly switch to whole
mode
...
Previously the switch from `split` to `whole` mode had been performed too early,
e.g. when the current cache size became bigger than 1/4 of the allowed cache size.
Now it is performed when the current cache size becomes bigger than 1/2 of the allowed cache size.
This change can reduce memory usage for data ingestion path when big number of active time series are ingested.
2021-07-05 15:16:11 +03:00
Aliaksandr Valialkin
43103be011
lib/{storage,mergeset}: increase cache timeout for data and index blocks from a minute to two minutes
...
One minute cache timeout result in slower queries in some production workloads where the interval
between query execution is in the range 1 minute - 2 minutes.
2021-07-05 15:16:11 +03:00
Aliaksandr Valialkin
54b9e1d3cb
lib/cgroup: set GOGC to 50 by default if it isn't set
...
This should reduce memory usage for typical VictoriaMetrics workloads by up to 50%
2021-07-05 15:16:11 +03:00
Roman Khavronenko
bd6b8f7e31
move github-pages docs to the main repo ( #1432 )
...
* move github-pages docs to the main repo
* rm github actions for copying docs to VictoriaMetrics/VictoriaMetrics.github.io
2021-07-05 14:34:10 +03:00
Aliaksandr Valialkin
888d62e40c
docs/CHANGELOG.md: document the bugfix for vm_merge_need_free_disk_space
metric at 9a83e9018d
2021-07-05 12:01:08 +03:00
Aliaksandr Valialkin
6bef227388
docs/Articles.md: add an url to https://medium.com/ibm-garage/monitoring-of-multiple-openshift-clusters-with-victoriametrics-d4f0979e2544
2021-07-05 11:51:37 +03:00