Aliaksandr Valialkin
1caee74235
lib/blockcache: eliminate possible race when Cache.Put is called for the same entry from multiple goroutines
...
The race could result in incorrect cache size tracking, which, in turn, could result in too frequent cache cleaning
2022-02-08 01:18:27 +02:00
Aliaksandr Valialkin
10476738a8
lib/blockcache: increase the lifetime for rarely accessed blocks from 2 minutes to 5 minutes
...
This should improve data ingestion speed if time series samples are ingested with interval bigger than 2 minutes.
The actual interval could exceed 2 minutes if the original interval between samples doesn't exceed 2 minutes
in the case of slow inserts. Slow inserts may appear in the following cases:
* Big number of new time series are pushed to VictoriaMetrics, so they couldn't be registered in 2 minutes.
* MetricName->tsid cache reset on indexdb rotation or due to unclean shutdown.
In this case VictoriaMetrics needs to load MetricName->tsid entries for all the incoming series from IndexDB.
IndexDB uses the block cache for increasing lookup performance. If the cache has no the needed block,
then IndexDB reads and unpacks the block from disk. This requires an extra disk read IO and CPU.
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1401
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2007
This also should increase performance for periodically executed queries with intervals from 2 minutes to 5 minutes.
See the previous similar commit - 43103be011
2022-02-08 01:18:27 +02:00
Aliaksandr Valialkin
b7cefff7b0
lib/workingsetcache: use the original cache size limits when rotating caches
...
Previously limits for new caches were taken from cache stats.
These limits could mismatch the original limits. This could result in failed cache load
if the stored cache has been created with the limits obtained from cache stats.
2022-02-08 01:18:27 +02:00
Aliaksandr Valialkin
178178bdd4
vendor: update github.com/VictoriaMetrics/fastcache from v1.8.0 to v1.9.0
2022-02-08 01:18:27 +02:00
Aliaksandr Valialkin
87071640a7
lib/blockcache: return proper number of entries from the cache
...
This has been broken in 0d7374ad2f
2022-02-08 01:18:27 +02:00
Aliaksandr Valialkin
75c5c33694
vendor: update github.com/valyala/gozstd from v1.15.1 to v1.16.0
2022-02-07 18:31:13 +02:00
Aliaksandr Valialkin
2149805886
docs/CHANGELOG.md: add links to issues, which could benefit from improved re-routing algorithm
2022-02-07 16:54:27 +02:00
Aliaksandr Valialkin
eb466e0c4e
docs: fix links to FAQ.md entries after they have been changed in ccf04239e6
2022-02-07 16:01:34 +02:00
Aliaksandr Valialkin
755e26e67b
app/vminsert/netstorage: throttle warning logs, which can be too verbose when vminsert cannot send data to vmstorage nodes
2022-02-07 15:39:06 +02:00
Aliaksandr Valialkin
b4b40774ec
vendor: make vendor-update
2022-02-07 15:07:43 +02:00
Aliaksandr Valialkin
9fb5ce5fb6
app/vminsert/netstorage: log vmstorage addr, which cannot accept new samples due to overload and/or unavailability
2022-02-07 15:03:05 +02:00
Aliaksandr Valialkin
5da7f08be6
docs: cross-link downsampling docs from deduplication and vmalert docs
2022-02-07 14:53:50 +02:00
Aliaksandr Valialkin
34d14c4940
all: substitute zeroTime with time.Time{}, since this generates more optimal binary code
2022-02-07 14:36:41 +02:00
Aliaksandr Valialkin
5aee6eb406
app/vminsert/netstorage: tune re-routing algorithm further
2022-02-07 14:35:39 +02:00
Aliaksandr Valialkin
1a5546006d
app/vminsert/netstorage: typo fix after 021ee53ba8
2022-02-07 13:20:43 +02:00
Aliaksandr Valialkin
e2d12a25e0
lib/netutil: increase dial timeout from 1 second to 5 seconds
...
There are real-world cases when TCP connection needs more than 1 second to be established.
2022-02-07 12:33:40 +02:00
Aliaksandr Valialkin
c738739494
app/vminsert: add -dropSamplesOnOverload command-line flag
...
Drop incoming samples if the destination vmstorage node is unavailable
and/or accepts data at slower rate than other vmstorage nodes
2022-02-07 12:32:18 +02:00
Aliaksandr Valialkin
021ee53ba8
app/vminsert: improve re-routing logic in order to spread rows more evenly among the available storage nodes
2022-02-06 20:20:02 +02:00
Aliaksandr Valialkin
d24e5d9efd
lib/promscrape: show the total number of scrapes and the total number of scrape errors per target at /targets page
...
This information may be useful when debugging unreliable scrape targets
2022-02-03 20:23:27 +02:00
Aliaksandr Valialkin
678b3e71db
lib/promscrape: provide the ability to fetch target responses on behalf of vmagent or single-node VictoriaMetrics
...
This feature may be useful when debugging metrics for the given target located in isolated environment
2022-02-03 19:02:12 +02:00
Aliaksandr Valialkin
1173964d8d
app/vmselect/promql: clarify comments on why the right side of if and and operators are executed at first
2022-02-03 00:26:42 +02:00
Aliaksandr Valialkin
10367d7e4c
app/vmselect/promql: do not push down filters, which enumerate more than 10k unique values
...
Such filters may slow down time series search, so just skip them.
This is a follow-up for e7f1ceeb84https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1827
2022-02-02 23:42:25 +02:00
Aliaksandr Valialkin
97b7b94f91
docs/CHANGELOG.md: document 55e3bbd4cc
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1567
2022-02-02 23:42:25 +02:00
Aliaksandr Valialkin
dbead4813e
docs: updates after 5da71eb685
...
* Mention about the ability to configure vmalert notifiers via files in docs/CHANGELOG.md
* Mention about the ability to use Consul service discovery for vmalert notifiers in docs/CHANGELOG.md
* Run `make docs-sync` in order to sync app/vmalert/README.md to docs/vmalert.md
2022-02-02 23:42:25 +02:00
Pablo Ley
66271bfeee
Added missing links in EmbeddedHPA API docs
2022-02-02 23:42:25 +02:00
hagen1778
a67e843257
vmalert: add support of -notifier.basicAuth.passwordFile flag for notifiers
...
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1567
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-02-02 23:42:25 +02:00
hagen1778
605de7bdd6
vmalert: remove trailing slash for static notifier addresses
...
This would make addresses `http://localhost:9093 ` and `http://localhost:9093/ `
both to result into `http://localhost:9093/api/v2/alerts `.
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-02-02 23:42:25 +02:00
Roman Khavronenko
2a3a62dc41
vmalert: support configuration file for notifiers ( #2127 )
...
vmalert: support configuration file for notifiers
* vmalert notifiers now can be configured via file
see https://docs.victoriametrics.com/vmalert.html#notifier-configuration-file
* add support of Consul service discovery for notifiers config
see https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1947
* add UI section for currently loaded/discovered notifiers
* deprecate `-rule.configCheckInterval` in favour of `-configCheckInterval`
* add ability to suppress logs for duplicated targets for notifiers discovery
* change behaviour of `vmalert_alerts_send_errors_total` - it now accounts
for failed alerts, not HTTP calls.
2022-02-02 23:42:25 +02:00
Aliaksandr Valialkin
5f266370c5
all: follow-up after 4bdd10ab90
...
Properly use new bytesutil.Resize* functions
2022-02-01 17:49:28 +02:00
Aliaksandr Valialkin
13c2692ca1
app/vmselect/promql: properly handle foo or bar queries
...
Such queries may miss `bar` results after the commit e7f1ceeb84
2022-02-01 17:43:10 +02:00
Aliaksandr Valialkin
2cc5f2940f
vendor: update github.com/VictoriaMetrics/metricsql from v0.39.0 to v0.40.0
2022-02-01 17:43:09 +02:00
Aliaksandr Valialkin
d8d59ff760
lib/mergeset: pre-allocate data and items for inmemoryBlock in order to reduce memory allocations under high churn rate
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2007
2022-02-01 11:20:20 +02:00
Aliaksandr Valialkin
02b2bfcff3
lib/bytesutil: split Resize* funcs to MayOverallocate and NoOverallocate for more fine-grained control over memory allocations
...
Follow-up for f4989edd96https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2007
2022-02-01 11:20:20 +02:00
Aliaksandr Valialkin
084664d780
lib/encoding: substitute 64-bits.LeadingZeros64() with bits.Len64()
2022-02-01 11:20:20 +02:00
Aliaksandr Valialkin
0fbfa8c245
lib/storage: avoid allocations of tsidPrev on every blockStreamReader.NextBlock() call
...
This is a follow-up for 00b7c97d2ahttps://github.com/VictoriaMetrics/VictoriaMetrics/issues/2082
2022-01-31 22:47:16 +02:00
Aliaksandr Valialkin
784cdcb080
app/vmselect/promql: check for binary operation in case-insensitive manner when deciding which side of the operation to perform the first
...
PromQL and MetricsQL operators are case-insensitive
2022-01-31 22:07:35 +02:00
Aliaksandr Valialkin
a02dde6cc7
lib/cgroup: fall back to runtime.NumCPU() when determining process_cpu_cores_available metric if it is impossible to determine cpu quota via cgroups
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2107
2022-01-31 20:31:12 +02:00
Aliaksandr Valialkin
566e12874d
lib/cgroup: expose process_cpu_cores_available metric
...
This metric shows the number of CPU cores available to the process.
This allows creating alerting rules on CPU saturation with the following query:
rate(process_cpu_seconds_total[5m]) / process_cpu_cores_available > 0.9
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2107
2022-01-31 20:25:15 +02:00
Aliaksandr Valialkin
04d6596298
app/vmselect/promql: optimize queries, which join on _info metrics.
...
Automatically add common filters from one side of binary operation
to the other side before sending the query to storage subsystem.
See https://grafana.com/blog/2021/08/04/how-to-use-promql-joins-for-more-effective-queries-of-prometheus-metrics-at-scale/
and https://www.robustperception.io/exposing-the-software-version-to-prometheus
2022-01-31 20:25:15 +02:00
Aliaksandr Valialkin
ad5d646e4e
docs/Release-Guide.md: add instructions on how to create signed release tags
2022-01-31 20:25:15 +02:00
Aliaksandr Valialkin
f3250307b1
vendor: make vendor-update
2022-01-31 12:46:44 +02:00
Aliaksandr Valialkin
35164d4dcf
docs/CHANGELOG.md: document 6a519896db
2022-01-31 12:42:27 +02:00
Aliaksandr Valialkin
d95d945046
app/vmselect/vmui: make vmui-update
2022-01-31 12:42:21 +02:00
Yury Molodov
f6a76e7bfd
fix: change logic set free fields for legend ( #2117 )
2022-01-31 12:42:21 +02:00
dependabot[bot]
76421f5535
build(deps): bump @mui/material in /app/vmui/packages/vmui ( #2129 )
...
Bumps [@mui/material](https://github.com/mui-org/material-ui/tree/HEAD/packages/mui-material ) from 5.3.0 to 5.3.1.
- [Release notes](https://github.com/mui-org/material-ui/releases )
- [Changelog](https://github.com/mui-org/material-ui/blob/master/CHANGELOG.md )
- [Commits](https://github.com/mui-org/material-ui/commits/v5.3.1/packages/mui-material )
---
updated-dependencies:
- dependency-name: "@mui/material"
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-01-31 12:42:21 +02:00
dependabot[bot]
80068504db
build(deps): bump preact in /app/vmui/packages/vmui ( #2128 )
...
Bumps [preact](https://github.com/preactjs/preact ) from 10.6.4 to 10.6.5.
- [Release notes](https://github.com/preactjs/preact/releases )
- [Commits](https://github.com/preactjs/preact/compare/10.6.4...10.6.5 )
---
updated-dependencies:
- dependency-name: preact
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-01-31 12:42:21 +02:00
dependabot[bot]
25db81459c
build(deps-dev): bump @typescript-eslint/parser ( #2123 )
...
Bumps [@typescript-eslint/parser](https://github.com/typescript-eslint/typescript-eslint/tree/HEAD/packages/parser ) from 5.10.0 to 5.10.1.
- [Release notes](https://github.com/typescript-eslint/typescript-eslint/releases )
- [Changelog](https://github.com/typescript-eslint/typescript-eslint/blob/main/packages/parser/CHANGELOG.md )
- [Commits](https://github.com/typescript-eslint/typescript-eslint/commits/v5.10.1/packages/parser )
---
updated-dependencies:
- dependency-name: "@typescript-eslint/parser"
dependency-type: direct:development
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-01-31 12:42:21 +02:00
dependabot[bot]
ab526bf18c
build(deps): bump @mui/icons-material in /app/vmui/packages/vmui ( #2122 )
...
Bumps [@mui/icons-material](https://github.com/mui-org/material-ui/tree/HEAD/packages/mui-icons-material ) from 5.3.0 to 5.3.1.
- [Release notes](https://github.com/mui-org/material-ui/releases )
- [Changelog](https://github.com/mui-org/material-ui/blob/master/CHANGELOG.md )
- [Commits](https://github.com/mui-org/material-ui/commits/v5.3.1/packages/mui-icons-material )
---
updated-dependencies:
- dependency-name: "@mui/icons-material"
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-01-31 12:42:21 +02:00
dependabot[bot]
7d3b0b7c44
build(deps-dev): bump @typescript-eslint/eslint-plugin ( #2124 )
...
Bumps [@typescript-eslint/eslint-plugin](https://github.com/typescript-eslint/typescript-eslint/tree/HEAD/packages/eslint-plugin ) from 5.10.0 to 5.10.1.
- [Release notes](https://github.com/typescript-eslint/typescript-eslint/releases )
- [Changelog](https://github.com/typescript-eslint/typescript-eslint/blob/main/packages/eslint-plugin/CHANGELOG.md )
- [Commits](https://github.com/typescript-eslint/typescript-eslint/commits/v5.10.1/packages/eslint-plugin )
---
updated-dependencies:
- dependency-name: "@typescript-eslint/eslint-plugin"
dependency-type: direct:development
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-01-31 12:42:21 +02:00
dependabot[bot]
8d2f113a8a
build(deps): bump @mui/lab in /app/vmui/packages/vmui ( #2125 )
...
Bumps [@mui/lab](https://github.com/mui-org/material-ui/tree/HEAD/packages/mui-lab ) from 5.0.0-alpha.65 to 5.0.0-alpha.66.
- [Release notes](https://github.com/mui-org/material-ui/releases )
- [Changelog](https://github.com/mui-org/material-ui/blob/master/CHANGELOG.md )
- [Commits](https://github.com/mui-org/material-ui/commits/HEAD/packages/mui-lab )
---
updated-dependencies:
- dependency-name: "@mui/lab"
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-01-31 12:42:21 +02:00
