Commit graph

4588 commits

Author SHA1 Message Date
Aliaksandr Valialkin
b5b3c585b3
lib/promscrape: show the total number of scrapes and the total number of scrape errors per target at /targets page
This information may be useful when debugging unreliable scrape targets
2022-02-03 20:22:41 +02:00
Aliaksandr Valialkin
2968779f16
lib/promscrape: provide the ability to fetch target responses on behalf of vmagent or single-node VictoriaMetrics
This feature may be useful when debugging metrics for the given target located in isolated environment
2022-02-03 19:00:55 +02:00
Aliaksandr Valialkin
96b7de6736
app/vmselect/promql: clarify comments on why the right side of if and and operators are executed at first 2022-02-03 00:26:14 +02:00
Aliaksandr Valialkin
4b850c2a59
app/vmselect/promql: do not push down filters, which enumerate more than 10k unique values
Such filters may slow down time series search, so just skip them.

This is a follow-up for e7f1ceeb84

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1827
2022-02-02 23:40:02 +02:00
Aliaksandr Valialkin
4ef32df4fa
docs/CHANGELOG.md: document 55e3bbd4cc
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1567
2022-02-02 23:32:35 +02:00
Aliaksandr Valialkin
6530bcedec
docs: updates after 5da71eb685
* Mention about the ability to configure vmalert notifiers via files in docs/CHANGELOG.md
* Mention about the ability to use Consul service discovery for vmalert notifiers in docs/CHANGELOG.md
* Run `make docs-sync` in order to sync app/vmalert/README.md to docs/vmalert.md
2022-02-02 23:26:18 +02:00
Pablo Ley
a6587ded51 Added missing links in EmbeddedHPA API docs 2022-02-02 18:59:42 +03:00
hagen1778
55e3bbd4cc vmalert: add support of -notifier.basicAuth.passwordFile flag for notifiers
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1567

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-02-02 18:58:54 +03:00
hagen1778
f57982eddc vmalert: remove trailing slash for static notifier addresses
This would make addresses `http://localhost:9093` and `http://localhost:9093/`
both to result into `http://localhost:9093/api/v2/alerts`.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-02-02 18:58:17 +03:00
Roman Khavronenko
5da71eb685
vmalert: support configuration file for notifiers (#2127)
vmalert: support configuration file for notifiers

* vmalert notifiers now can be configured via file
see https://docs.victoriametrics.com/vmalert.html#notifier-configuration-file
* add support of Consul service discovery for notifiers config
see https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1947
* add UI section for currently loaded/discovered notifiers
* deprecate `-rule.configCheckInterval` in favour of `-configCheckInterval`
* add ability to suppress logs for duplicated targets for notifiers discovery
* change behaviour of `vmalert_alerts_send_errors_total` - it now accounts
for failed alerts, not HTTP calls.
2022-02-02 14:11:41 +02:00
Aliaksandr Valialkin
2016a2c899
app/vmselect/promql: properly handle foo or bar queries
Such queries may miss `bar` results after the commit e7f1ceeb84
because common label filters from `foo` could be mistakenly applied to `bar`.
2022-02-01 17:40:51 +02:00
Aliaksandr Valialkin
d4b09896fa
vendor: update github.com/VictoriaMetrics/metricsql from v0.39.0 to v0.40.0 2022-02-01 17:31:38 +02:00
Aliaksandr Valialkin
9c62b25ad6
lib/mergeset: pre-allocate data and items for inmemoryBlock in order to reduce memory allocations under high churn rate
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2007
2022-02-01 00:57:14 +02:00
Aliaksandr Valialkin
4bdd10ab90
lib/bytesutil: split Resize* funcs to MayOverallocate and NoOverallocate for more fine-grained control over memory allocations
Follow-up for f4989edd96

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2007
2022-02-01 00:18:42 +02:00
Aliaksandr Valialkin
e13ce2ee98
lib/encoding: substitute 64-bits.LeadingZeros64() with bits.Len64() 2022-01-31 23:36:48 +02:00
Aliaksandr Valialkin
a8509c112a
lib/storage: avoid allocations of tsidPrev on every blockStreamReader.NextBlock() call
This is a follow-up for 00b7c97d2a

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2082
2022-01-31 22:46:53 +02:00
Aliaksandr Valialkin
a8d22e1223
app/vmselect/promql: check for binary operation in case-insensitive manner when deciding which side of the operation to perform the first
PromQL and MetricsQL operators are case-insensitive
2022-01-31 22:06:15 +02:00
Aliaksandr Valialkin
f50cf60534
lib/cgroup: fall back to runtime.NumCPU() when determining process_cpu_cores_available metric if it is impossible to determine cpu quota via cgroups
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2107
2022-01-31 20:30:14 +02:00
Aliaksandr Valialkin
ead66155ef
lib/cgroup: expose process_cpu_cores_available metric
This metric shows the number of CPU cores available to the process.
This allows creating alerting rules on CPU saturation with the following query:

    rate(process_cpu_seconds_total[5m]) / process_cpu_cores_available > 0.9

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2107
2022-01-31 20:24:41 +02:00
Aliaksandr Valialkin
e7f1ceeb84
app/vmselect/promql: optimize queries, which join on _info metrics.
Automatically add common filters from one side of binary operation
to the other side before sending the query to storage subsystem.

See https://grafana.com/blog/2021/08/04/how-to-use-promql-joins-for-more-effective-queries-of-prometheus-metrics-at-scale/
and https://www.robustperception.io/exposing-the-software-version-to-prometheus
2022-01-31 19:32:36 +02:00
Aliaksandr Valialkin
15475a9d1f
docs/Release-Guide.md: add instructions on how to create signed release tags 2022-01-31 19:18:48 +02:00
Aliaksandr Valialkin
d2ac954fe1
vendor: make vendor-update 2022-01-31 12:46:17 +02:00
Aliaksandr Valialkin
3d8a4bf023
docs/CHANGELOG.md: document 6a519896db 2022-01-31 12:41:57 +02:00
Aliaksandr Valialkin
7edf8be3bc
app/vmselect/vmui: make vmui-update 2022-01-31 12:36:59 +02:00
Yury Molodov
6a519896db
fix: change logic set free fields for legend (#2117) 2022-01-31 12:33:40 +02:00
dependabot[bot]
02a1a39796
build(deps): bump @mui/material in /app/vmui/packages/vmui (#2129)
Bumps [@mui/material](https://github.com/mui-org/material-ui/tree/HEAD/packages/mui-material) from 5.3.0 to 5.3.1.
- [Release notes](https://github.com/mui-org/material-ui/releases)
- [Changelog](https://github.com/mui-org/material-ui/blob/master/CHANGELOG.md)
- [Commits](https://github.com/mui-org/material-ui/commits/v5.3.1/packages/mui-material)

---
updated-dependencies:
- dependency-name: "@mui/material"
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-01-31 13:08:17 +03:00
dependabot[bot]
4477a2e513
build(deps): bump preact in /app/vmui/packages/vmui (#2128)
Bumps [preact](https://github.com/preactjs/preact) from 10.6.4 to 10.6.5.
- [Release notes](https://github.com/preactjs/preact/releases)
- [Commits](https://github.com/preactjs/preact/compare/10.6.4...10.6.5)

---
updated-dependencies:
- dependency-name: preact
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-01-31 13:02:43 +03:00
dependabot[bot]
53852e35d8
build(deps-dev): bump @typescript-eslint/parser (#2123)
Bumps [@typescript-eslint/parser](https://github.com/typescript-eslint/typescript-eslint/tree/HEAD/packages/parser) from 5.10.0 to 5.10.1.
- [Release notes](https://github.com/typescript-eslint/typescript-eslint/releases)
- [Changelog](https://github.com/typescript-eslint/typescript-eslint/blob/main/packages/parser/CHANGELOG.md)
- [Commits](https://github.com/typescript-eslint/typescript-eslint/commits/v5.10.1/packages/parser)

---
updated-dependencies:
- dependency-name: "@typescript-eslint/parser"
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-01-31 12:51:52 +03:00
dependabot[bot]
b8a47c6589
build(deps): bump @mui/icons-material in /app/vmui/packages/vmui (#2122)
Bumps [@mui/icons-material](https://github.com/mui-org/material-ui/tree/HEAD/packages/mui-icons-material) from 5.3.0 to 5.3.1.
- [Release notes](https://github.com/mui-org/material-ui/releases)
- [Changelog](https://github.com/mui-org/material-ui/blob/master/CHANGELOG.md)
- [Commits](https://github.com/mui-org/material-ui/commits/v5.3.1/packages/mui-icons-material)

---
updated-dependencies:
- dependency-name: "@mui/icons-material"
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-01-31 12:51:43 +03:00
dependabot[bot]
9226b9917a
build(deps-dev): bump @typescript-eslint/eslint-plugin (#2124)
Bumps [@typescript-eslint/eslint-plugin](https://github.com/typescript-eslint/typescript-eslint/tree/HEAD/packages/eslint-plugin) from 5.10.0 to 5.10.1.
- [Release notes](https://github.com/typescript-eslint/typescript-eslint/releases)
- [Changelog](https://github.com/typescript-eslint/typescript-eslint/blob/main/packages/eslint-plugin/CHANGELOG.md)
- [Commits](https://github.com/typescript-eslint/typescript-eslint/commits/v5.10.1/packages/eslint-plugin)

---
updated-dependencies:
- dependency-name: "@typescript-eslint/eslint-plugin"
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-01-31 12:25:46 +03:00
dependabot[bot]
5ae9892f5f
build(deps): bump @mui/lab in /app/vmui/packages/vmui (#2125)
Bumps [@mui/lab](https://github.com/mui-org/material-ui/tree/HEAD/packages/mui-lab) from 5.0.0-alpha.65 to 5.0.0-alpha.66.
- [Release notes](https://github.com/mui-org/material-ui/releases)
- [Changelog](https://github.com/mui-org/material-ui/blob/master/CHANGELOG.md)
- [Commits](https://github.com/mui-org/material-ui/commits/HEAD/packages/mui-lab)

---
updated-dependencies:
- dependency-name: "@mui/lab"
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-01-31 12:25:32 +03:00
dependabot[bot]
86a7a72400
build(deps): bump @types/node in /app/vmui/packages/vmui (#2126)
Bumps [@types/node](https://github.com/DefinitelyTyped/DefinitelyTyped/tree/HEAD/types/node) from 17.0.10 to 17.0.13.
- [Release notes](https://github.com/DefinitelyTyped/DefinitelyTyped/releases)
- [Commits](https://github.com/DefinitelyTyped/DefinitelyTyped/commits/HEAD/types/node)

---
updated-dependencies:
- dependency-name: "@types/node"
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-01-31 12:25:20 +03:00
Aliaksandr Valialkin
96aa3761fc
lib/storage/table.go: add missing tb.ptwsLock.Unlock() before the return
This is a follow-up for a1083d0531

See https://github.com/VictoriaMetrics/VictoriaMetrics/pull/2103
2022-01-28 14:15:42 +02:00
匠心零度
1999bbfe82
optimized code (#2103)
* optimized code ,because only the first error,so no need var errors []error

* optimized code ,because only the first error,so no need var errors []error

Co-authored-by: lirenzuo <lirenzuo@shein.com>
2022-01-28 14:15:41 +02:00
Aliaksandr Valialkin
97947c5fcf
docs/vmauth.md: mention that backend services must be accessible only via vmauth 2022-01-28 12:06:45 +02:00
Aliaksandr Valialkin
f6899cc289
Revert ".github/workflows: disable updating VictoriaMetrics wiki with docs"
This reverts commit 2ef3fabcb8.

The reason: there are many links to https://github.com/VictoriaMetrics/VictoriaMetrics/wiki/* all over the Internet.

This commit breaks such links :(
2022-01-28 11:43:06 +02:00
Denys Holius
527bee4b1e
FAQ update (#2111)
* FAQ update: how downsampling and deduplication will work at the same time

* Update docs/FAQ.md

Co-authored-by: Aliaksandr Valialkin <valyala@gmail.com>
2022-01-27 20:35:16 +02:00
jaysony
2e59b17108
correct VMAgentStatus description (#2109)
correct VMAgentStatus description from VMAlert to VMAgent.
2022-01-27 20:18:05 +02:00
Aliaksandr Valialkin
e02e0508da
vendor: update github.com/VictoriaMetrics/metricsql from v0.37.0 to v0.38.0
This adds more optimization cases for https://utcc.utoronto.ca/~cks/space/blog/sysadmin/PrometheusLabelNonOptimization

For example:

* Multi-level transform functions. For example, abs(round(foo{a="b"})) + bar{x="y"}
  is now optimized to abs(round(foo{a="b",x="y"})) + bar{a="b",x="y"}
* Binary operations with `on()`, `without()`, `group_left()` and `group_right()` modifiers.
  For example, foo{a="b"} on (a) + bar is now optimized to foo{a="b"} on (a) + bar{a="b"}
* Multi-level binary operations. For example, foo{a="b"} + bar{x="y"} + baz{z="q"}
  is now optimized to foo{a="b",x="y",z="q"} + bar{a="b",x="y",z="q"} + baz{a="b",x="y",z="q"}
* Aggregate functions. For example, sum(foo{a="b"}) by (c) + bar{c="d"}
  is now optimized to sum(foo{a="b",c="d"}) by (c) + bar{c="d"}
2022-01-27 19:03:54 +02:00
Aliaksandr Valialkin
ac92d471a6
vendor: make vendor-update 2022-01-27 13:16:33 +02:00
Denys Holius
f0eb1f3749 Update version of golangci-lint
Update version of golangci-lint from v1.43.0 to v1.44.0 in Makefile.
See https://github.com/golangci/golangci-lint/releases/tag/v1.44.0 .
2022-01-25 19:57:43 +03:00
Aliaksandr Valialkin
74a2297dcc
app/vmselect/vmui: make vmui-update after e3995572bb 2022-01-25 17:51:46 +02:00
Yury Molodov
e3995572bb
feat: add sortable columns (#2101) 2022-01-25 17:48:28 +02:00
Aliaksandr Valialkin
2ef3fabcb8
.github/workflows: disable updating VictoriaMetrics wiki with docs
The https://github.com/VictoriaMetrics/VictoriaMetrics/wiki has been broken and unmaintained
after VictoriaMetrics documentation has been moved to https://docs.victoriametrics.com .

Let's remove the broken GitHub action, which tries keeping in sync VictoriaMetrics wiki at https://github.com/VictoriaMetrics/VictoriaMetrics/wiki .
2022-01-25 17:43:09 +02:00
Aliaksandr Valialkin
a41c34705e
docs/operator: fix links to Operator API docs 2022-01-25 17:38:16 +02:00
Aliaksandr Valialkin
f4989edd96
lib/bytesutil: split Resize() into ResizeNoCopy() and ResizeWithCopy() functions
Previously bytesutil.Resize() was copying the original byte slice contents to a newly allocated slice.
This wasted CPU cycles and memory bandwidth in some places, where the original slice contents wasn't needed
after slize resizing. Switch such places to bytesutil.ResizeNoCopy().

Rename the original bytesutil.Resize() function to bytesutil.ResizeWithCopy() for the sake of improved readability.

Additionally, allocate new slice with `make()` instead of `append()`. This guarantees that the capacity of the allocated slice
exactly matches the requested size. The `append()` could return a slice with bigger capacity as an optimization for further `append()` calls.
This could result in excess memory usage when the returned byte slice was cached (for instance, in lib/blockcache).

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2007
2022-01-25 15:24:44 +02:00
Aliaksandr Valialkin
91f2af2d7a
lib/mergeset: allocate the needed amounts of memory when unmarshaling inmemoryBlock
This should reduce the memory required for indexdb/dataBlocks cache.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2007
2022-01-24 18:50:40 +02:00
Aliaksandr Valialkin
285bb2bbec
app/vmselect/vmui: make vmui-update 2022-01-24 12:19:49 +02:00
Aliaksandr Valialkin
4c13bae1cf
lib/logger: removed broken test after 746ee191e8 2022-01-24 12:14:32 +02:00
dependabot[bot]
132425eb46
build(deps-dev): bump @typescript-eslint/parser (#2097)
Bumps [@typescript-eslint/parser](https://github.com/typescript-eslint/typescript-eslint/tree/HEAD/packages/parser) from 5.9.1 to 5.10.0.
- [Release notes](https://github.com/typescript-eslint/typescript-eslint/releases)
- [Changelog](https://github.com/typescript-eslint/typescript-eslint/blob/main/packages/parser/CHANGELOG.md)
- [Commits](https://github.com/typescript-eslint/typescript-eslint/commits/v5.10.0/packages/parser)

---
updated-dependencies:
- dependency-name: "@typescript-eslint/parser"
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-01-24 13:09:46 +03:00