Aliaksandr Valialkin
7ecb72648d
docs/CHANGELOG.md: document 0c0efc7781
2022-04-05 19:21:49 +03:00
Aliaksandr Valialkin
4ecb86c179
app/vminsert: reduce the max packet size, which vminsert can send to vmstorage
...
This reduces the max memory usage for vminsert and vmstorage under heavy ingestion rate
by up to 50% on production workload
2022-04-05 15:43:07 +03:00
Aliaksandr Valialkin
d011446f6f
docs/CHANGELOG.md: document 70bb0d2708
2022-04-04 13:02:27 +03:00
Aliaksandr Valialkin
43df19a742
docs/CHANGELOG.md: document 173073364e1bb1e0259ddc873dbd96ce62b07543
2022-04-04 12:55:43 +03:00
Aliaksandr Valialkin
f977ca8eaf
docs/CHANGELOG.md: document a57e3807537914396ee3eb378648a464fa9e1b97
2022-04-01 12:24:49 +03:00
Aliaksandr Valialkin
1c38ff6f48
docs/CHANGELOG.md: document 0989649ad0
2022-04-01 12:01:34 +03:00
Aliaksandr Valialkin
e210384f7e
docs/CHANGELOG.md: cut v1.75.1
2022-03-28 12:28:48 +03:00
Yury Molodov
c8d29ed78e
vmui: predefined panels ( #2243 )
...
* feat: add basic components for predefined dashboards
* fix: change display alert
* feat: add autosize and unit for axes
* feat: add component for CircularProgress
* feat: change layout for predefined dashboards
* feat: add override step for predefined panels
* feat: add override step for predefined panels
* feat: change yaxis limits for predefined panels
* fix: rename flag for hide legend
* feat: add formatted panel description
* feat: add README.md for dashboard setup
* feat: validate dashboard settings
* feat: add unit for y-ticks
* fix: correct display error for dashboards
* fix: disable auto refresh after route change
* update package-lock.json
* fix: add basename for BrowserRouter
* fix: add dynamic basename for routing
* update packages
* feat: add a pre-defined dashboard "per-job resource usage"
* feat: display unit in the hover-tooltip
* fix: change routing and home layout
* fix: change axis width calc
* updated packages
* app/vmselect: `make vmui-update`
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-03-26 13:03:11 +02:00
Aliaksandr Valialkin
6e364e19ef
app/vmselect: add fine-grained limits for the number of returned/scanned time series for various APIs
2022-03-26 11:29:49 +02:00
Dima Lazerka
1fa0f3ec89
VMAnomaly docs fixes ( #2361 )
...
* Added docs for vmanomaly
* Add example images
* Stylistic fixes
* Move images to root
* Update docs/vmanomaly.md
* Update docs/vmanomaly.md
Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
* Squeeze vmanomaly after vmbackupmanager before Case Studies
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
2022-03-25 12:08:17 +02:00
Arash Hatami
a8de1ab000
A good change for MD files ( #2353 )
...
* Lint YAML
* Remove extra comment
* Fix command problem
* Format MD files
* Format & fix problem of MD files for docs
* Another fix for MD files
2022-03-22 13:40:55 +02:00
Roman Khavronenko
f367ff086c
docs: update release notes ( #2349 )
...
Warn about memory issue introduced in releases 1.73 - 1.74
2022-03-21 15:40:50 +02:00
Aliaksandr Valialkin
5ab6c350ec
docs/CHANGELOG.md: document a1e17e91f8
2022-03-21 15:34:49 +02:00
hagen1778
82659ab5b6
docs: add update note to v1.75.0 release note
...
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-03-19 17:55:38 +01:00
Aliaksandr Valialkin
87d356348b
docs/CHANGELOG.md: cut v1.75.0
2022-03-18 19:53:45 +02:00
Aliaksandr Valialkin
e78f3ac8ac
app/vmselect/bufferedwriter: suppress trivial network errors, which can be generated by remote side
...
These errors include `broken pipe` and `reset by peer`.
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2334
2022-03-18 19:28:02 +02:00
Aliaksandr Valialkin
ec03dec72d
app/vmagent/remotewrite: prevent from infinite recursion panic when pushing a time series with big number of samples to remote storage
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2335
2022-03-18 19:06:22 +02:00
Aliaksandr Valialkin
620b605786
docs: document 20bb5e703c
2022-03-18 18:41:19 +02:00
Aliaksandr Valialkin
2ae3a9a8a3
lib/storage: reduce the interval for checking for free disk space from 30 seconds to 1 second
...
This should reduce the probability of out of disk space panics when -storage.minFreeDiskSpaceBytes is set to low values.
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2305
2022-03-18 16:52:27 +02:00
Aliaksandr Valialkin
88605a7ea2
lib/blockcache: properly release memory occupied by deleted entries
...
Proviously the deleted entries could remain referenced via lastAccessHeap for long time.
This could lead to increased memory usage for the following caches starting from v1.73.0:
* indexdb/indexBlocks
* indexdb/dataBlocks
* storage/indexBlocks
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2242
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2007
2022-03-18 16:52:27 +02:00
Aliaksandr Valialkin
db781a9342
docs/CHANGELOG.md: document e5868b9c29
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/546
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2255
2022-03-18 13:07:32 +02:00
Aliaksandr Valialkin
e6acd16daf
docs/CHANGELOG.md: document 11ae1ae924
2022-03-17 20:08:05 +02:00
Aliaksandr Valialkin
11869a8307
docs: document the addition of mTLS communication between cluster components
2022-03-17 20:00:56 +02:00
Aliaksandr Valialkin
8ae9825bb4
docs/CHANGELOG.md: document c1d07e7c52f0a2ab892921b0639cd42677aa33a8
2022-03-16 14:25:44 +02:00
Aliaksandr Valialkin
54ec080bbc
docs/CHANGELOG.md: document changes from fb6eab03a2
2022-03-16 13:21:57 +02:00
Aliaksandr Valialkin
370024c7ed
docs/CHANGELOG.md: document 565bd08c43
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1824
2022-03-16 12:50:49 +02:00
hagen1778
894416b4ca
docs: add update details for some releases
...
Some of the releases could negatively affect performance for a limited
period of time due to some changes in core. Update details are meant to
warn users about expected changes in peformance after the update.
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-03-15 07:06:48 +03:00
Roman Khavronenko
0fa7effc4b
docs: fix broken links ( #2303 )
...
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-03-13 15:56:01 +02:00
Aliaksandr Valialkin
74bb9ea734
docs/CHANGELOG.md: cut v1.74.0
2022-03-03 19:30:41 +02:00
Aliaksandr Valialkin
ce8d28f8f4
docs/CHANGELOG.md: document performance improvements when registering new time series
2022-03-03 17:11:30 +02:00
Aliaksandr Valialkin
e757ebc58b
app/vmselect/netstorage: report vmstorage errors to vmselect clients even if partial responses are allowed
...
If a vmstorage is reachable and returns an application-level error to vmselect,
then such error must be returned to the caller even if partial responses are allowed,
since it usually means cluster mis-configuration.
Partial responses may be returned only if some vmstorage nodes are temporarily unavailable.
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1941
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/678
2022-02-25 13:56:42 +02:00
Nikolay
9fe2e4e2c2
fixes incorrect step for calculation for MovingWindow functions ( #283 )
...
* fixes incorrect step for calculation for MovingWindow functions
https://victoriametrics.zendesk.com/agent/tickets/99
* wip
* wip
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-02-25 13:54:03 +02:00
Aliaksandr Valialkin
0d79c8cbef
docs/CHANGELOG.md: document cfc6c14dc48ae9dd35e65f1a6e5c7af8ccb9f029
2022-02-25 13:52:48 +02:00
Aliaksandr Valialkin
a16f1ae565
lib/storage: properly handle series selector matching multiple metric names plus a negative filter
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2238
This is a follow-up for 00cbb099b6
2022-02-24 12:15:54 +02:00
Aliaksandr Valialkin
3f49bdaeff
lib/promrelabel: add support for conditional relabeling via if
filter
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1998
2022-02-24 02:27:26 +02:00
Nikolay
fbac1a9dad
fixes jwt token parse with correct base64Url decoding ( #281 )
...
* fixes jwt token parse with correct base64Url decoding
it must be applied according to jwt RFC that requires token to be URL safe
added slow path for decoding tokens with std base64 decoding
adds error logging for vmgateway
* docs/CHANGELOG.md: document the bugfix
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-02-23 13:57:36 +02:00
Aliaksandr Valialkin
62b46007c5
lib/workingsetcache: reduce the default cache rotation period from hour to 20 minutes
...
This should reduce memory usage under high time series churn rate
2022-02-23 13:41:45 +02:00
Aliaksandr Valialkin
acbea6c1ee
docs/CHANGELOG.md: cut v1.73.1
2022-02-22 21:11:42 +02:00
Aliaksandr Valialkin
0a38542a45
docs/CHANGELOG.md: link to the feature request for X-Influxdb-Version response header
...
Follow-up for 71ef3155c8
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2209
2022-02-22 20:35:53 +02:00
Roman Khavronenko
69d1893f4c
Consul SD - update services on the watcher's start ( #2202 )
...
* lib/discovery/consul: update services on the watcher's start
Previously, watcher's start was only initing goroutines for discovery
but not waiting for the first iteration to end. It means first Consul
discovery wasn't returning discovered targets until the next iteration.
The change makes the watcher's start blocking until we get first discovery
iteration done and all registries updated.
Signed-off-by: hagen1778 <roman@victoriametrics.com>
* vmalert: remove workarounds for consul SD
Now when consul SD lib properly updates services
on the first start, we don't need workarounds in vmalert.
Signed-off-by: hagen1778 <roman@victoriametrics.com>
* lib/discovery/consul: update after review
Signed-off-by: hagen1778 <roman@victoriametrics.com>
* wip
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-02-21 15:32:45 +02:00
Roman Khavronenko
b6ed9afd6d
lib: allow to configure cache size by type ( #2206 )
...
* lib: allow to configure cache size by type
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1940
Signed-off-by: hagen1778 <roman@victoriametrics.com>
* Apply suggestions from code review
* wip
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-02-21 13:50:34 +02:00
Aliaksandr Valialkin
71ef3155c8
app/vminsert: add X-Influxdb-Version
response header for InfluxDB API requests
...
This is needed for some clients, which expect this header.
See https://github.com/ntop/ntopng/issues/5449#issuecomment-1005347597
2022-02-17 12:47:43 +02:00
Aliaksandr Valialkin
3c3805865b
docs: document 3d19fa6932
2022-02-16 23:30:17 +02:00
Aliaksandr Valialkin
b71be42d90
lib/storage: use binary search instead of full scan for skipping artificial tags when searching for tag names or tag values
...
This should improve performance for /api/v1/labels and /api/v1/label/<label_name>/values
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2200
2022-02-16 18:15:41 +02:00
Aliaksandr Valialkin
424121c126
docs/CHANGELOG.md: document 2efa46a11c
2022-02-15 21:11:56 +02:00
Aliaksandr Valialkin
0f9e107e36
docs/CHANGELOG.md: document ad6bdd78d0
2022-02-15 12:47:55 +02:00
Aliaksandr Valialkin
ac502785b6
docs/CHANGELOG.md: cut v1.73.0
2022-02-14 17:54:32 +02:00
Aliaksandr Valialkin
1215f51043
docs/CHANGELOG.md: document 3d890e89f1
2022-02-14 17:39:12 +02:00
Nikolay
75e84144c7
adds release build for macos darwin amd64 and arm64 ( #2185 )
...
* adds release build for macos darwin amd64 and arm64
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1896
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1851
* wip
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-02-14 17:28:56 +02:00
Aliaksandr Valialkin
578a37aa14
docs/CHANGELOG.md: document c90c1c4d54
2022-02-14 13:09:12 +02:00
Aliaksandr Valialkin
f10c38b827
lib/promscrape: add expand all
and collapse all
buttons to /targets
page
2022-02-12 18:41:29 +02:00
Aliaksandr Valialkin
b1f94f7f0e
app/vmselect/promql: return at most one time series from absent_over_time()
in the same way as Prometheus does
...
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2130
2022-02-12 15:45:09 +02:00
Aliaksandr Valialkin
d8ffbf55a2
docs/CHANGELOG.md: document ea153e5f90
2022-02-12 00:48:06 +02:00
Roman Khavronenko
cf1a8bce6b
lib/index: reduce read/write load after indexDB rotation ( #2177 )
...
* lib/index: reduce read/write load after indexDB rotation
IndexDB in VM is responsible for storing TSID - ID's used for identifying
time series. The index is stored on disk and used by both ingestion and read path.
IndexDB is stored separately to data parts and is global for all stored data.
It can't be deleted partially as VM deletes data parts. Instead, indexDB is
rotated once in `retention` interval.
The rotation procedure means that `current` indexDB becomes `previous`,
and new freshly created indexDB struct becomes `current`. So in any time,
VM holds indexDB for current and previous retention periods.
When time series is ingested or queried, VM checks if its TSID is present
in `current` indexDB. If it is missing, it checks the `previous` indexDB.
If TSID was found, it gets copied to the `current` indexDB. In this way
`current` indexDB stores only series which were active during the retention
period.
To improve indexDB lookups, VM uses a cache layer called `tsidCache`. Both
write and read path consult `tsidCache` and on miss the relad lookup happens.
When rotation happens, VM resets the `tsidCache`. This is needed for ingestion
path to trigger `current` indexDB re-population. Since index re-population
requires additional resources, every index rotation event may cause some extra
load on CPU and disk. While it may be unnoticeable for most of the cases,
for systems with very high number of unique series each rotation may lead
to performance degradation for some period of time.
This PR makes an attempt to smooth out resource usage after the rotation.
The changes are following:
1. `tsidCache` is no longer reset after the rotation;
2. Instead, each entry in `tsidCache` gains a notion of indexDB to which
they belong;
3. On ingestion path after the rotation we check if requested TSID was
found in `tsidCache`. Then we have 3 branches:
3.1 Fast path. It was found, and belongs to the `current` indexDB. Return TSID.
3.2 Slow path. It wasn't found, so we generate it from scratch,
add to `current` indexDB, add it to `tsidCache`.
3.3 Smooth path. It was found but does not belong to the `current` indexDB.
In this case, we add it to the `current` indexDB with some probability.
The probability is based on time passed since the last rotation with some threshold.
The more time has passed since rotation the higher is chance to re-populate `current` indexDB.
The default re-population interval in this PR is set to `1h`, during which entries from
`previous` index supposed to slowly re-populate `current` index.
The new metric `vm_timeseries_repopulated_total` was added to identify how many TSIDs
were moved from `previous` indexDB to the `current` indexDB. This metric supposed to
grow only during the first `1h` after the last rotation.
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1401
Signed-off-by: hagen1778 <roman@victoriametrics.com>
* wip
* wip
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-02-12 00:30:08 +02:00
Roman Khavronenko
e3adcbec6e
lib/promscrape: support prometheus-like duration in scrape configs ( #2169 )
...
* lib/promscrape: support prometheus-like duration in scrape configs
The change allows to specify duration values like `1d`, `1w`
for fields `scrape_interval`, `scrape_timeout`, etc.
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/817#issuecomment-1033384766
Signed-off-by: hagen1778 <roman@victoriametrics.com>
* lib/blockcache: make linter happy
Signed-off-by: hagen1778 <roman@victoriametrics.com>
* lib/promscrape: support prometheus-like duration in scrape configs
* add support for extra fields `scrape_align_interval` and `scrape_offset`;
* support Prometheus duration parsing for `__scrape_interval__`
and `__scrape_duration__` labels;
Signed-off-by: hagen1778 <roman@victoriametrics.com>
* wip
* wip
* docs/CHANGELOG.md: document the feature
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-02-11 16:17:00 +02:00
Aliaksandr Valialkin
3cb72ccc2a
lib/promscrape/discovery/kubernetes: add __meta_kubernetes_endpointslice_{label,annotation}*
labels to be consistent with other role
values for Kubernetes service discovery
2022-02-11 14:54:47 +02:00
Nikolay
4e7f7f3302
fixes service discovery for kubernetes ( #2173 )
...
* fixes service discovery for kubernetes
now it must take in account all pods that belong to the discovered endpoint and endpointslice
adds simple test for endpoints
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2134
* wip
* docs/CHANGELOG.md: document the change
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-02-11 13:34:22 +02:00
Aliaksandr Valialkin
8f2d03fdc7
docs/CHANGELOG.md: document 4e722c459b
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2167
2022-02-10 12:20:12 +02:00
Aliaksandr Valialkin
0028b2c6d1
docs/CHANGELOG.md: add instructions on how to build VictoriaMetrics components from source code in order to test tip changes
2022-02-08 16:44:08 +02:00
Nikolay
a8acad7453
adds CGO build for arm64 ( #2102 )
...
* adds CGO build for arm64
it must improve performance for arm64 based deployments of vmstorage and
vmsingle for 15-20%
it depends on gozstd package update for correct musl gozstd vendoring
* typo fixes
* docs/CHANGELOG.md: document the change
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-02-08 16:25:59 +02:00
Aliaksandr Valialkin
9bb60ab00f
lib/promscrape: set -promscrape.config.strictParse
to true by default
...
This allows detecting long-living silent errors in -promscrape.config
2022-02-08 15:41:43 +02:00
Aliaksandr Valialkin
8b36044c93
docs/CHANGELOG.md: add links to issues, which could benefit from improved re-routing algorithm
2022-02-07 16:55:28 +02:00
Aliaksandr Valialkin
865f09ecbb
docs: sync with cluster branch
2022-02-07 14:54:16 +02:00
Aliaksandr Valialkin
b5b3c585b3
lib/promscrape: show the total number of scrapes and the total number of scrape errors per target at /targets page
...
This information may be useful when debugging unreliable scrape targets
2022-02-03 20:22:41 +02:00
Aliaksandr Valialkin
2968779f16
lib/promscrape: provide the ability to fetch target responses on behalf of vmagent or single-node VictoriaMetrics
...
This feature may be useful when debugging metrics for the given target located in isolated environment
2022-02-03 19:00:55 +02:00
Aliaksandr Valialkin
4b850c2a59
app/vmselect/promql: do not push down filters, which enumerate more than 10k unique values
...
Such filters may slow down time series search, so just skip them.
This is a follow-up for e7f1ceeb84
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1827
2022-02-02 23:40:02 +02:00
Aliaksandr Valialkin
4ef32df4fa
docs/CHANGELOG.md: document 55e3bbd4cc
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1567
2022-02-02 23:32:35 +02:00
Aliaksandr Valialkin
6530bcedec
docs: updates after 5da71eb685
...
* Mention about the ability to configure vmalert notifiers via files in docs/CHANGELOG.md
* Mention about the ability to use Consul service discovery for vmalert notifiers in docs/CHANGELOG.md
* Run `make docs-sync` in order to sync app/vmalert/README.md to docs/vmalert.md
2022-02-02 23:26:18 +02:00
Aliaksandr Valialkin
ead66155ef
lib/cgroup: expose process_cpu_cores_available
metric
...
This metric shows the number of CPU cores available to the process.
This allows creating alerting rules on CPU saturation with the following query:
rate(process_cpu_seconds_total[5m]) / process_cpu_cores_available > 0.9
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2107
2022-01-31 20:24:41 +02:00
Aliaksandr Valialkin
e7f1ceeb84
app/vmselect/promql: optimize queries, which join on _info
metrics.
...
Automatically add common filters from one side of binary operation
to the other side before sending the query to storage subsystem.
See https://grafana.com/blog/2021/08/04/how-to-use-promql-joins-for-more-effective-queries-of-prometheus-metrics-at-scale/
and https://www.robustperception.io/exposing-the-software-version-to-prometheus
2022-01-31 19:32:36 +02:00
Aliaksandr Valialkin
3d8a4bf023
docs/CHANGELOG.md: document 6a519896db
2022-01-31 12:41:57 +02:00
Aliaksandr Valialkin
e02e0508da
vendor: update github.com/VictoriaMetrics/metricsql from v0.37.0 to v0.38.0
...
This adds more optimization cases for https://utcc.utoronto.ca/~cks/space/blog/sysadmin/PrometheusLabelNonOptimization
For example:
* Multi-level transform functions. For example, abs(round(foo{a="b"})) + bar{x="y"}
is now optimized to abs(round(foo{a="b",x="y"})) + bar{a="b",x="y"}
* Binary operations with `on()`, `without()`, `group_left()` and `group_right()` modifiers.
For example, foo{a="b"} on (a) + bar is now optimized to foo{a="b"} on (a) + bar{a="b"}
* Multi-level binary operations. For example, foo{a="b"} + bar{x="y"} + baz{z="q"}
is now optimized to foo{a="b",x="y",z="q"} + bar{a="b",x="y",z="q"} + baz{a="b",x="y",z="q"}
* Aggregate functions. For example, sum(foo{a="b"}) by (c) + bar{c="d"}
is now optimized to sum(foo{a="b",c="d"}) by (c) + bar{c="d"}
2022-01-27 19:03:54 +02:00
Aliaksandr Valialkin
746ee191e8
lib/logger/throttler.go: show the original location of the error and warning message
...
Previously the location inside LogThrottler implementation was shown. This could complicate debugging.
2022-01-23 13:55:00 +02:00
Yury Molodov
ad5059f2d3
vmui: fixed display type switching ( #2088 )
...
* fix: correct switch display type
* docs/CHANGELOG.md: document the bugfix
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-01-21 16:56:22 +02:00
Yury Molodov
adbb821eac
vmui: fix time range selector ( #2085 )
...
* fix: add date validate for time range
* app/vmselect/vmui: `make vmui-update`
* docs/CHANGELOG.md: document the bugfix
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-01-21 12:02:38 +02:00
Aliaksandr Valialkin
145337792d
lib/{mergeset,storage}: properly limit cache sizes for indexdb
...
Previously these caches could exceed limits set via `-memory.allowedPercent` and/or `-memory.allowedBytes`,
since limits were set independently per each data part. If the number of data parts was big, then limits could be exceeded,
which could result to out of memory errors.
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2007
2022-01-20 18:37:17 +02:00
Aliaksandr Valialkin
84f6b3014c
docs/CHANGELOG.md: document the bugfix for highestMax() function is Graphite render API
2022-01-20 12:16:18 +02:00
Aliaksandr Valialkin
109363de49
docs/CHANGELOG.md: add missing parens in example for @
modifier
2022-01-19 13:04:51 +02:00
Aliaksandr Valialkin
98edeac7b7
docs/CHANGELOG.md: fix incorrect link to the issue
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1911
2022-01-19 00:06:44 +02:00
Aliaksandr Valialkin
8f5902dfcf
docs/CHANGELOG.md: cut v1.72.0
2022-01-18 22:43:18 +02:00
Yury Molodov
8bdc45ba00
fix: remove buffer period ( #2078 )
...
* fix: remove buffer period
* app/vmselect/vmui: `make vmui-update`
* docs/CHANGELOG.md: document the implemented feature
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2064
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-01-18 21:42:56 +02:00
Yury Molodov
70737ea4ac
vmui: correct url encoding ( #2067 )
...
* fix: correct encode multi-line queries
* fix: change autocomplete for correct arrows work
* app/vmselect/vmui: `make vmui-update`
* docs/CHANGELOG.md: document the bugfix for https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2039
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-01-18 21:31:46 +02:00
Aliaksandr Valialkin
dcadec65b6
docs/CHANGELOG.md: document 8e3f9c1fbb
2022-01-18 21:24:49 +02:00
Aliaksandr Valialkin
e933e3150d
docs/CHANGELOG.md: document fcd33fc409
2022-01-18 12:46:33 +02:00
Aliaksandr Valialkin
c2a3911bb5
docs/CHANGELOG.md: document that vmgateway now supports extra_filters option
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1863
2022-01-18 12:39:25 +02:00
Aliaksandr Valialkin
1bdc71d917
app/vmselect/promql: implement keep_metric_names
modifier for transform and rollup functions
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/949
2022-01-14 04:14:59 +02:00
Aliaksandr Valialkin
f41846d002
app/vmselect/promql: add stale_samples_over_time()
function
2022-01-14 01:48:04 +02:00
Aliaksandr Valialkin
96707223db
docs/CHANGELOG.md: yet another attempt to fix formatting for yaml snippet
2022-01-14 01:14:55 +02:00
Aliaksandr Valialkin
d7d83d6d93
docs/CHANGELOG.md: fix formatting for scrape_configs
example
2022-01-14 01:02:32 +02:00
Aliaksandr Valialkin
1d05444b33
lib/promscrape: expose promscrape_stale_samples_created_total metric for monitoring the number of created stale samples
2022-01-14 01:00:46 +02:00
Aliaksandr Valialkin
831b93a755
app/vmalert: add parseDuration
function in the same way as Prometheus does
...
See https://github.com/prometheus/prometheus/pull/8817
2022-01-13 23:30:41 +02:00
Aliaksandr Valialkin
80f03177c4
lib/promscrape/discovery/kubernetes: add __meta_kubernetes_node_provider_id
label for discovered Kubernetes nodes in the same way as Prometheus does
...
See https://github.com/prometheus/prometheus/pull/9603
2022-01-13 23:16:02 +02:00
Aliaksandr Valialkin
80f966b80c
app/vmalert: add stripPort
template function in the same way as Prometheus does
...
See https://github.com/prometheus/prometheus/pull/10002
2022-01-13 22:53:42 +02:00
Aliaksandr Valialkin
355a63733d
lib/promscrape/discovery/kubernetes: add the ability to limit service discovery to the current namespace
...
See https://github.com/prometheus/prometheus/issues/9782 and https://github.com/prometheus/prometheus/pull/9881
2022-01-13 22:44:35 +02:00
Aliaksandr Valialkin
c883c15878
app/vmselect/promql: add support for @
modifier
...
Add support for `@` modifier in MetricsQL according to https://prometheus.io/docs/prometheus/latest/querying/basics/#modifier
Extend the support with the following features:
* Allow using `@` modifier everywhere in the query. For example, `sum(foo) @ end()`
* Allow using arbitrary expression as `@` modifier. For example, `foo @ (end() - 1h)`
returns `foo` value at `end - 1 hour` timestamp on the selected time range `[start ... end]`
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1348
2022-01-13 22:12:06 +02:00
Aliaksandr Valialkin
17eb86a689
lib/promscrape/discovery/dockerswarm: follow up after 68a117a25a
...
- Document the bugfix at docs/CHANGELOG.md
- Set __address__ field after copying commonLabels to the resulting map of discovered labels.
This makes sure that the correct __address__ label is used.
2022-01-11 09:20:10 +02:00
Aliaksandr Valialkin
7a2c46d951
docs/CHANGELOG.md: document 77bfa8181d
2022-01-11 08:53:49 +02:00
Aliaksandr Valialkin
bdba50432b
docs/CHANGELOG.md: add release dates for every release
2022-01-07 12:27:32 +02:00
Aliaksandr Valialkin
e4e36383e2
lib/promscrape: do not send staleness markers on graceful shutdown
...
This follows Prometheus behavior.
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2013#issuecomment-1006994079
2022-01-07 01:17:57 +02:00
Denys Holius
ae89b4e818
Old links replaced for newest ( #2033 )
...
* replaced old links to the website
* fixed deletion main README.md file
* fix: added docs files after docs-sync
2022-01-05 16:30:13 +02:00
Aliaksandr Valialkin
dbe592597f
docs/CHANGELOG.md: clarify the issue description, which is fixed by 38bf5fc136
2022-01-05 16:19:06 +02:00
Aliaksandr Valialkin
178dd87e26
lib/storage: follow-up for 38bf5fc136
2022-01-05 16:00:11 +02:00
Aliaksandr Valialkin
e1d7cbfc77
docs/CHANGELOG.md: document 60266078ca
2022-01-04 11:44:57 +02:00
Aliaksandr Valialkin
5ce94e1dd3
docs/CHANGELOG.md: document ac47733044
2022-01-03 21:15:44 +02:00
Aliaksandr Valialkin
b5ce35dfc8
docs/CHANGELOG.md: document 543bd0ea0c
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1999
2021-12-23 12:08:06 +02:00
Aliaksandr Valialkin
cbaa2af280
lib/promscrape: scrape replicated targets at different offsets in vmagent replicated clustering mode
...
This guarantees that the deduplication consistently leaves samples from the same vmagent replica.
See https://docs.victoriametrics.com/vmagent.html#scraping-big-number-of-targets
2021-12-23 00:20:39 +02:00
Nikolay
8ff7da7202
adds restore.lock ( #1988 )
...
* adds restore.lock
it must prevent from running storage after incomplete restore process
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1958
* return back flock file deletion
* Apply suggestions from code review
* wip
* docs/CHANGELOG.md: document https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1958
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2021-12-22 13:10:15 +02:00
Aliaksandr Valialkin
3cfb90b227
docs/CHANGELOG.md: document 34fdc8881b
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1911
2021-12-21 16:40:39 +02:00
Aliaksandr Valialkin
4701c108ff
docs/CHANGELOG.md: cut v1.71.0
2021-12-20 19:10:24 +02:00
Aliaksandr Valialkin
974d9c0eee
app/vmselect/promql: follow-up after 177e345d8a
...
* Document changes_prometheus(), increase_prometheus() and delta_prometheus() functions.
* Simplify their implementation
* Mention these functions in docs/CHANGELOG.md
2021-12-20 13:19:44 +02:00
Aliaksandr Valialkin
16636a458f
docs/CHANGELOG.md: document 6814cc6809
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1943
2021-12-17 20:16:22 +02:00
Aliaksandr Valialkin
193331d522
app/vmselect: de-duplicate data exported via /api/v1/export/csv
by default
...
Previously the exported data wasn't de-duplicated.
Now it is possible to export the raw data without deduplication
by passing reduce_mem_usage=1 query arg to /api/v1/export/csv
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1837
2021-12-17 10:57:39 +02:00
Aliaksandr Valialkin
a3adf24527
lib/promscrape: allow up to 5 redirects when scraping a target by default
...
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1945
2021-12-16 00:14:14 +02:00
Aliaksandr Valialkin
5efe377a26
app/vmselect/promql: add timestamp_with_name(m[d])
function
...
This function works the same as `timestamp()`, but doesn't remove source time series names.
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/949#issuecomment-995222388
2021-12-15 23:37:07 +02:00
Aliaksandr Valialkin
8e1a87491a
docs: document the added dowsnampling support in VictoriaMetrics enterprise
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/36
2021-12-15 16:25:51 +02:00
Aliaksandr Valialkin
4ff647137a
lib/storage: deduplicate samples more thoroughly
...
Previously some duplicate samples may be left on disk for time series with high churn rate.
This may result in higher disk space usage.
2021-12-15 15:59:58 +02:00
Aliaksandr Valialkin
e1a715b0f5
lib/storage: convert alternate regexps into Graphite wildcards inside __graphite__
pseudo-label
...
For example, `{__graphite__=~"foo.(bar|baz)"}` is automatically converted to `{__graphite__=~"foo.{bar,baz}"}` before execution.
This allows using multi-value Grafana template variables such as `{__graphite__=~"foo.($app)"}`.
2021-12-14 19:51:49 +02:00
Aliaksandr Valialkin
d456af7499
docs/CHANGELOG.md: link to the issue about unaligned 64-bit atomic opertion
panic on 32-bit architectures
2021-12-14 15:00:10 +02:00
Aliaksandr Valialkin
80996d916b
docs/CHANGELOG.md: an attempt to properly show $labels.alertname at https://docs.victoriametrics.com/CHANGELOG.html
2021-12-14 14:56:57 +02:00
Yury Molodov
49e6a921df
vmui: custom step ( #1942 )
...
* feat: add a label for the Query field
* fix: change zoom position
* fix: add description and error code to alerts
* fix: correct logic query history
* fix: correct update query history
* feat: add custom step
* update package-lock.json
* docs: document that VMUI now supports overriding of `step` query arg, which is passed to `/api/v1/query_range`
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2021-12-14 14:51:45 +02:00
Yury Molodov
b5b701d590
vmui: minor fixes ( #1936 )
...
* feat: add a label for the Query field
* fix: change zoom position
* fix: add description and error code to alerts
* fix: correct logic query history
* fix: correct update query history
* app/vmselect/vmui: `make vmui-update`
* docs/CHANGELOG.md: document bugfixes
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2021-12-13 13:42:37 +02:00
Aliaksandr Valialkin
c82cc9cd11
docs/CHANGELOG.md: document 7c3b6365f0
2021-12-12 19:09:20 +02:00
Aliaksandr Valialkin
d1f8915ed1
app/vmselect/promql: preserve the order of time series passed to limit_offset()
function
...
Previously time series passed to `limit_offset()` were shuffled according to hash for their labels.
This was unexpected behaviour for most users.
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1920 and https://github.com/VictoriaMetrics/VictoriaMetrics/issues/951
2021-12-12 18:04:58 +02:00
Aliaksandr Valialkin
3d4349343d
docs/CHANGELOG.md: document 2851709745
...
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1921
2021-12-10 12:13:34 +02:00
Aliaksandr Valialkin
52eb9c99e2
docs: document the ability to investigate correlation between two queries at /vmui
...
This is a follow-up for c1fd93e8a0
2021-12-08 17:24:01 +02:00
Aliaksandr Valialkin
ff15a752c1
app/vmselect: accept optional extra_filters[]
query args for all the supported Prometheus querying APIs
...
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1863
2021-12-06 17:07:09 +02:00
Aliaksandr Valialkin
45d082bbe2
app/vminsert: add -maxLabelValueLen
command-line flag
...
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1908
2021-12-06 11:40:34 +02:00
Aliaksandr Valialkin
da402fbdfa
lib/workingsetcache: fix unaligned 64-bit atomic operation
panic on 32-bit architectures
...
The panic has been introduced in 7275ebf91a
2021-12-03 01:21:51 +02:00
Aliaksandr Valialkin
06642d97f5
app: allow specifying http and https urls in the following command-line flags
...
* -promscrape.config
* -relabelConfig
* -remoteWrite.relabelConfig
* -remoteWrite.urlRelabelConfig
2021-12-03 00:10:02 +02:00
Aliaksandr Valialkin
62b4efb3e7
app/vmauth: follow-up for 13368bed18
...
* Document the ability to specify http or https urls in `-auth.config` at docs/CHANGELOG.md
* Move the ReadFileOrHTTP to lib/fs, so it can be re-used in other places where a file
should be read from the given path. For example, in `-promscrape.config` at `vmagent`.
2021-12-02 23:32:05 +02:00
Aliaksandr Valialkin
7f2f26b25f
docs/CHANGELOG.md: cut v1.70.0
2021-12-02 15:01:25 +02:00
Aliaksandr Valialkin
ed9ef7733b
docs/CHANGELOG.md: document 0afd14a14a
2021-12-02 14:47:20 +02:00
Aliaksandr Valialkin
394a345ae0
lib/httpserver: expose /-/healthy
and /-/ready
endpoints as Prometheus does
...
This improves integration with third-party solutions, which rely on these endpoints.
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1833
2021-12-02 14:36:58 +02:00
Aliaksandr Valialkin
03f5ad3060
lib/protoparser/graphite: allow multiple separators between metric name, value and timestamp
2021-12-02 13:43:49 +02:00
Aliaksandr Valialkin
49a18b8660
lib/protoparser/graphite: properly parse Graphite line with whitespace after the timestamp
...
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1865
2021-12-02 13:33:26 +02:00
Aliaksandr Valialkin
c0cbf0de2a
app/{vmbackup,vmrestore}: export internal metrics at /metrics
http handler
2021-12-02 11:55:58 +02:00
Aliaksandr Valialkin
7275ebf91a
app/vmstorage: export vm_cache_size_max_bytes metrics for determining capacity of various caches
...
The vm_cache_size_max_bytes metric can be used for determining caches which reach their capacity via the following query:
vm_cache_size_bytes / vm_cache_size_max_bytes > 0.9
2021-12-02 10:30:43 +02:00
Aliaksandr Valialkin
2f63dec2e3
lib/fs: add vm_filestream_read_duration_seconds_total
and vm_filestream_write_duration_seconds_total
metrics
...
These metrics help determining persistent disk saturation with `rate(vm_filestream_read_duration_seconds_total) > 0.9`
2021-12-02 10:30:42 +02:00
Aliaksandr Valialkin
ed54b27b85
docs/CHANGELOG.md: document 06eff5a72c
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/487
2021-12-01 12:28:06 +02:00
Aliaksandr Valialkin
10bd8b1d86
docs/CHANGELOG.md: document 852a895b70
2021-11-30 01:22:37 +02:00
Aliaksandr Valialkin
ca6fc0265e
docs/CHANGELOG.md: document f05cddd2fc
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1830
2021-11-30 01:15:41 +02:00
Aliaksandr Valialkin
841647643a
docs/CHANGELOG.md: document 9dd650f67f
2021-11-30 01:09:10 +02:00
Aliaksandr Valialkin
7e79fc6e3c
docs/CHANGELOG.md: document 624ad73705
2021-11-30 01:05:05 +02:00
Aliaksandr Valialkin
98d244b288
docs/CHANGELOG.md: document c6d5927281
2021-11-30 01:00:36 +02:00
Aliaksandr Valialkin
d44c585ca4
lib/protoparser: do not log connection reset by peer
error when reading the data via InfluxDB, Graphite and OpenTSDB protocols over plain TCP connections
...
This error is expected, so there is no need in spamming the log with this error.
2021-11-29 21:47:56 +02:00
Aliaksandr Valialkin
03509025bc
docs/CHANGELOG.md: document 695cb617b2
2021-11-29 12:13:11 +02:00
Aliaksandr Valialkin
9bee043ff2
app/vmselect/promql: consistently return zero from deriv(const)
2021-11-17 18:02:05 +02:00
Aliaksandr Valialkin
b688960db0
lib/persistentqueue: add vm_persistentqueue_read_duration_seconds_total and vm_persistentqueue_write_duration_seconds_total metrics for determining disk usage saturation at vmagent
2021-11-17 16:41:35 +02:00
Aliaksandr Valialkin
b900560b83
app/vmselect/promql: add now()
function, which returns the current timestamp as a floating-point value in seconds
2021-11-17 16:35:30 +02:00
Aliaksandr Valialkin
52e19a0577
docs: document -s3ForcePathStyle command-line option
...
This is a follow-up for b72eed1f5e
2021-11-17 01:09:32 +02:00
Aliaksandr Valialkin
f3ac945d74
app/vmauth: add ability to override the username
label value for vmauth_user_requests_total
metric by specifying name
option in -auth.config
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1805
2021-11-17 00:47:34 +02:00
Aliaksandr Valialkin
5a180c6659
docs/CHANGELOG.md: document the addition of vm_tenant_used_tenant_bytes
metric, which shows the per-tenant disk usage
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1605
2021-11-16 23:36:48 +02:00
Aliaksandr Valialkin
8b6c89423d
docs/CHANGELOG.md: document bugfixes in enteprise versions of vmagent and vmalert
2021-11-12 13:24:07 +02:00
Aliaksandr Valialkin
e5d4c7f4a7
app/vmauth: initialize reverse proxy only after flag.Parse() is called
...
This should properly take into accoun the `-maxIdleConnsPerBackend` command-line flag value.
Previously it was hardcoded to 100.
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1300
2021-11-09 19:22:34 +02:00
Aliaksandr Valialkin
69e655ba7f
docs/CHANGELOG.md: cut v1.69.0
2021-11-08 15:47:36 +02:00
Aliaksandr Valialkin
3419ac1d36
app/vmselect/promql: add duration_over_time(m[d], max_interval)
function
...
This function calculates the actual lifetime of the time series on the given lookbehdind window `d`
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1780
2021-11-08 13:14:09 +02:00
Aliaksandr Valialkin
5edf695bc9
docs/CHANGELOG.md: document b9cdbcb5046315db96e1e7ca9923d09d0f30dc25
2021-11-08 12:11:30 +02:00
Aliaksandr Valialkin
237885e0d2
docs/vmalert.md: document the addition of -defaultTenant.prometheus and -defaultTenant.graphite command-line options to enterprise version of vmalert
2021-11-05 20:04:09 +02:00
Aliaksandr Valialkin
cbfc7b7c92
app/{vminsert,vmagent}: hide passwords and auth tokens by default at /config
page
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1764
2021-11-05 14:41:16 +02:00
Aliaksandr Valialkin
a69264e885
app/vmagent: add -remoteWrite.maxRowsPerBlock
command-line option, which may be used for improving data ingestion performance under high load
2021-11-04 15:39:14 +02:00
Aliaksandr Valialkin
e0d2ba5608
app/vmagent/remotewrite: send data to remote storage systems in parallel
...
This should improve data ingestion speed when many `-remoteWrite.url` command-line flags are configured
2021-11-04 15:04:16 +02:00
Aliaksandr Valialkin
aa534c2582
lib/promscrape: add -promscrape.maxResponseHeadersSize
command-line flag for tuning the maximum http response headers size from Prometheus scrape targets
2021-11-03 22:26:56 +02:00
Aliaksandr Valialkin
27044b84d2
app/vmselect/promql: add limit_offset(limit, offset, q)
function, which can be used for paging over big number of time series
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1778
2021-11-03 16:02:27 +02:00
Aliaksandr Valialkin
43a58bd618
app/vmselect/promql: add label_graphite_group()
function for extracting groups from Graphite metric names
2021-11-03 13:19:08 +02:00
Aliaksandr Valialkin
da2e0e29a4
docs/CHANGELOG.md: document e3a91b186a
2021-11-02 18:39:14 +02:00
Aliaksandr Valialkin
d1eb87c831
app/{vmagent,vminsert}: add ability to restrict access to /config page with authKey query arg
...
The authKey can be configured via `-configAuthKey` command-line flag.
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1764
2021-11-01 16:44:54 +02:00
Aliaksandr Valialkin
bb87949d5c
lib/protoparser/influx: automatically detect timestamp precision depending on the number of decimal digits in the timestamp
2021-10-28 12:47:22 +03:00
Aliaksandr Valialkin
74b8af9891
lib/promscrape: add collapse
and expand
buttons per each group of targets from the same scrape job
2021-10-27 20:03:24 +03:00
Aliaksandr Valialkin
1952ab99aa
all: fix build issues and tests for Apple M1
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1653
2021-10-27 15:06:34 +03:00
Aliaksandr Valialkin
9ec0175e83
docs/CHANGELOG.md: mention the issue about missing proxy_url
config option at /config
page
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1755
2021-10-26 22:06:35 +03:00
Aliaksandr Valialkin
4821adfd95
lib/promscrape: properly show proxy_url
option value at /config
page
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1755
2021-10-26 21:23:54 +03:00
Aliaksandr Valialkin
a7e3cbd6ad
docs/CHANGELOG.md: document 3dbdf1632e
2021-10-25 12:16:37 +03:00
Aliaksandr Valialkin
8d0fafc377
docs/CHANGELOG.md: typo fix
2021-10-22 21:11:54 +03:00
Aliaksandr Valialkin
7f7cac20c1
docs/CHANGELOG.md: cut v1.68.0
2021-10-22 19:37:48 +03:00
Aliaksandr Valialkin
013d626889
app/vmauth: add ability to specify http headers to send in requests to backends
...
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1736
2021-10-22 19:10:29 +03:00
Aliaksandr Valialkin
7e88713ca3
docs/CHANGELOG.md: document 43a7984cd8
2021-10-22 14:00:20 +03:00
Aliaksandr Valialkin
8568003bb1
docs/CHANGELOG.md: document a3684fe3de
2021-10-22 12:28:01 +03:00
Yury Molodov
2b266cb87e
vmui: query history ( #1732 )
...
* feat: add query history
* fix: change detect keyUp for nav query history
* feat: set default query history
* app/vmselect/vmui: `make vmui-update`
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2021-10-22 12:21:22 +03:00
Aliaksandr Valialkin
8ad95f0db7
lib/httpserver: expose command-line flags at /flags
page
...
This should simplify debugging.
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1695
2021-10-20 00:45:09 +03:00
Aliaksandr Valialkin
0d48b89afe
docs/CHANGELOG.md: document 146a5b504c
2021-10-19 11:25:02 +03:00
Aliaksandr Valialkin
c64a134146
docs/CHANGELOG.md: document cbcc622786
2021-10-19 08:56:23 +03:00
Yury Molodov
a3e09a57c2
vmui: features ( #1711 )
...
* feat: initial uPlot graph
* feat: add zoom/pan for graph
* fix: add zoom by ctrl/mac
* fix: remove unused code
* feat: add toggle cache for fetch
* feat: add fix y-axis limits
* fix: stop point events while panning
* fix: change getting cursor position when scaling
* feat: add cursor tooltip to graph
* fix: uninstall chart.js
* fix: change link for create an issue
* fix: set default cache value to true
* app/vmalert: follow-up after 0e2486df56
* docs/CHANGELOG.md: document 5416e18007
* app/vmui: `make vmui-update`
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2021-10-18 15:16:57 +03:00
Aliaksandr Valialkin
c37f285466
lib/promscrape: set honor_timestamps: true
by default if this option isnt set explicitly in scrape configs
...
This aligns the behavior to Prometheus - see https://prometheus.io/docs/prometheus/latest/configuration/configuration/#scrape_config
2021-10-16 20:49:08 +03:00
Aliaksandr Valialkin
c055bc478c
lib/promscrape: expose promscrape_series_limit_max_series
and promscrape_series_limit_current_series
metrics per each scrape target with the enabled unique series limiter
2021-10-16 18:47:13 +03:00
Aliaksandr Valialkin
9866dd95c1
lib/promscrape: store the full response in stream parsing mode in scrapeWork.lastScrape byte slice
...
This allows sending staleness marks and properly calculate scrape_series_added metric in stream parsing mode
at the cost of the increased memory usage, since now the potentially big response is kept
in the lastScrape byte slice per each scrapeWork.
In practice the memory usage increase shouldn't be big, since the response size
is usually much smaller than the parsed metrics from this response after the relabeling,
which usually adds a big pile of target-specific labels per each metric.
2021-10-15 15:39:23 +03:00
Aliaksandr Valialkin
78425561ce
docs/CHANGELOG.md: document the change at 7fcbd3fa4b
2021-10-14 14:37:44 +03:00
Aliaksandr Valialkin
bbd34fa15e
lib/promscrape: add -promscrape.minResponseSizeForStreamParse
command-line option for automatic switching to stream parsing mode when scraping targets with big responses
...
This should reduce memory usage when vmagent scrapes targets with non-uniform response sizes.
This is common case in Kubernetes monitoring.
2021-10-14 12:29:35 +03:00
Aliaksandr Valialkin
1a7287c408
lib/promscrape: return error if sample_limit
or series_limit
options are set when stream parsing mode is enabled
2021-10-14 12:11:23 +03:00
Aliaksandr Valialkin
1c17fe70e0
docs/CHANGELOG.md: document e3c8304deb
2021-10-13 16:00:50 +03:00
Aliaksandr Valialkin
e3c8304deb
lib/promscrape: add ability to show the original labels for discovered targets at /targets page
...
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1698
2021-10-13 15:59:58 +03:00
Aliaksandr Valialkin
9882cda8b9
lib/promscrape: shard targets among cluster nodes after relabeling is applied
...
This guarantees that targets with the same set of labels go to the same vmagent node.
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1687#issuecomment-940629495
2021-10-12 17:06:00 +03:00
Aliaksandr Valialkin
5a58c041c2
app/vmagent: expose -promscrape.config contents at /config page as Prometheus does
...
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1695
2021-10-12 16:25:37 +03:00
Aliaksandr Valialkin
a5001b9c20
app/vmselect/promql: add atan2
binary operator, which is going to be added in Prometheus 2.31
...
See https://github.com/prometheus/prometheus/pull/9248
2021-10-11 21:15:53 +03:00
Aliaksandr Valialkin
81c6720392
app/vmselect/promql: add missing trigonometric functions, which are going to be added in Prometheus 2.31
...
See https://github.com/prometheus/prometheus/issues/9233
2021-10-11 21:01:33 +03:00
Aliaksandr Valialkin
92b92d4d2c
app/vmselect/promql: consistently return the same set of time series from limitk()
function
...
This is the expected behaviour by most users.
2021-10-08 19:53:52 +03:00
Aliaksandr Valialkin
001750c239
lib/storage: fix unaligned access on 32-bit architectures.
...
The bug has been introduced at a171916ef5
2021-10-08 19:43:03 +03:00
Aliaksandr Valialkin
0a3a774202
docs/CHANGELOG.md: cut v1.67.0
2021-10-08 16:00:33 +03:00
Aliaksandr Valialkin
0ff8fcac6a
app/vmui: follow-up after 7bfb44113e
...
* Run `vmui-update`
* Document the changes in README.md and CHANGELOG.md
2021-10-08 15:09:29 +03:00
Aliaksandr Valialkin
cf5cbd1c70
app/{vminsert,vmstorage}: follow-up after a171916ef5
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/269
2021-10-08 14:35:49 +03:00
Aliaksandr Valialkin
a19e7c7ce8
app/vminsert: fix uneven distribution of time series among storage nodes
...
Use distinct seed for distribution hash calculations on the second level of vminsert nodes.
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1672
2021-10-07 12:23:39 +03:00
Aliaksandr Valialkin
9515e58e28
docs/vmagent.md: document how to write data to Kafka
2021-09-30 17:45:53 +03:00
Aliaksandr Valialkin
0e3de5a0cc
app/vmselect/promql: add topk_last
and bottomk_last
functions
2021-09-30 13:22:52 +03:00
Aliaksandr Valialkin
2b623ae302
docs/CHANGELOG.md: link to Kafka integration docs
2021-09-29 12:31:23 +03:00
Aliaksandr Valialkin
ead59bdebf
docs/CHANGELOG.md: document the bugfix from de810031bf
2021-09-29 00:41:35 +03:00
Aliaksandr Valialkin
50ec259750
docs/CHANGELOG.md: document 3d17112a7e
2021-09-29 00:33:08 +03:00
Aliaksandr Valialkin
91b3c601bc
app/{vminsert,vmagent}: add ability to ingest data via DataDog "submit metrics" API
...
See https://docs.datadoghq.com/api/latest/metrics/#submit-metrics
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/206
2021-09-29 00:13:08 +03:00
Yury Molodov
a64155d91e
vmui: use Chart.js as default engine for graph ( #1634 )
...
* feat: add Plotly as default engine for graph
* fix: remove unused components
* feat: use Chart.js as default engine graph
* fix: correct styles for loader
* feat: add zoom/pan for chart
* feat: add height for chart
* fix: remove unused code
* fix: remove empty units from duration
* fix: change debounce for pan to 500ms
* fix: add utility for plugins register globally
* fix: optimize render graph
* feat: add buffer data for zoom
* fix: add limits for zoom in/out
* fix: change update data while zooming
* app/vmselect: `make vmui-update`
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2021-09-27 22:26:14 +03:00
Aliaksandr Valialkin
2efe0acfc9
app/vmselect/promql: add rollup_scrape_interval(m[d])
function
...
It calculates the min, max and avg scrape intervals for m over the given lookbehind window d
2021-09-27 19:21:24 +03:00
Aliaksandr Valialkin
c4c77aa2dd
app/vmselect/promql: follow-up after 526dd93b32
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1625
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1612
2021-09-27 18:55:39 +03:00
Aliaksandr Valialkin
e564411a62
app/vmselect/promql: align the behavior of or
, and
and unless
operators with on (labels)
modifier to Prometheus
...
Previously VictoriaMetrics could return unexpected result of the right-hand side operand
had multiple time series with the given set of labels mentioned in `on(labels)`.
See https://github.com/VictoriaMetrics/VictoriaMetrics/pull/1643
2021-09-24 00:46:25 +03:00
Aliaksandr Valialkin
9f19649672
docs/CHANGELOG.md: cut v1.66.2
2021-09-23 22:53:36 +03:00
Aliaksandr Valialkin
718eca33ab
lib/storage: properly handle {__name__=~"prefix(suffix1|suffix2)",other_label="..."}
queries
...
They were broken in the commit 00cbb099b6
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1644
2021-09-23 21:48:51 +03:00
Aliaksandr Valialkin
f5896b7420
docs/CHANGELOG.md: document 0e35fc9538
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1641
2021-09-23 20:46:23 +03:00
Aliaksandr Valialkin
a0313c046b
lib/promscrape: add vm_promscrape_max_scrape_size_exceeded_errors_total
metric for counting of the failed scrapes due to the exceeded response size
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1639
2021-09-23 14:47:54 +03:00
Aliaksandr Valialkin
9b3d1a1996
docs/CHANGELOG.md: cut v1.66.1
2021-09-22 01:47:05 +03:00
Aliaksandr Valialkin
a13c3de36f
docs/CHANGELOG.md: document 9ca1cbced1
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1635
2021-09-21 23:17:08 +03:00
Aliaksandr Valialkin
207c5760ce
lib/promrelabel: fix parsing regex: true
in relabeling rules
2021-09-21 23:00:53 +03:00
Aliaksandr Valialkin
a22aa0608b
app/vmselect: fix accessing /graphite/*
endpoints
2021-09-21 13:56:35 +03:00
Aliaksandr Valialkin
76811c2f60
docs/CHANGELOG.md: cut v1.66.0
2021-09-20 15:20:25 +03:00
Aliaksandr Valialkin
9a3d0c43b5
app/vmselect/promql: add quantiles_over_time("phiLabel", phi1, ..., phiN, m[d])
function for calculating multiple quantiles at once
2021-09-17 23:35:10 +03:00
Aliaksandr Valialkin
e1e5a20b36
docs/CHANGELOG.md: document 0e09fdb8b0
2021-09-17 18:47:06 +03:00
Aliaksandr Valialkin
2951dd0a57
app/vmselect/promql: add histogram_quantiles("phiLabel", phi1, ..., phiN, buckets) function
...
This function calculates multiple quantiles over the given buckets at once
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1573
2021-09-17 13:32:39 +03:00
Aliaksandr Valialkin
8c504d6efa
docs/CHANGELOG.md: document the change in enterprise apps, which allows passing -version
without -eula
flag
...
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1621
2021-09-17 12:37:45 +03:00
Aliaksandr Valialkin
e60dfc96ff
app/vmselect/promql: add mad(q)
and outliers_mad(tolerance, q)
functions to MetricsQL
2021-09-16 13:33:53 +03:00
Aliaksandr Valialkin
44b01fff13
app/{vminsert,vmselect}: automatically add missing port in -storageNode
lists passed to vminsert
and vmselect
...
This should simplify manual setup of the cluster according to https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#cluster-setup
2021-09-15 18:08:30 +03:00
Aliaksandr Valialkin
06ed694ad9
docs/CHANGELOG.md: document 777ff75874
2021-09-15 17:45:08 +03:00
Aliaksandr Valialkin
68ea3d18f7
vendor: update github.com/valyala/histogram from v1.1.2 to v1.2.0
...
This fixes the non-repeatable quantile_over_time() results when the number of input samples exceeds 1000.
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1612
2021-09-15 09:34:07 +03:00
Aliaksandr Valialkin
3e1683756b
docs/vmalert.md: follow-up after 68721f6e7d
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1608
2021-09-14 14:47:47 +03:00
Aliaksandr Valialkin
8f685d81c6
lib/storage: follow up after 00cbb099b6
2021-09-14 14:16:25 +03:00
Aliaksandr Valialkin
adedc83b3b
app/vmauth: do not log invalid auth tokens by default for security reasons
...
The logging can be enabled by passing `-logInvalidAuthTokens` command-line flag to vmauth
2021-09-14 12:20:03 +03:00
Aliaksandr Valialkin
8a6a36429a
app/vminsert/netstorage: disable rerouting by default
...
Production clusters work more stable with the disabled rerouting during rolling restarts and/or
during spikes in time series churn rate. So it would be better disabling the rerouting by default.
The re-routing can be enabled by passing `-disableRerouting=false` command-line flag to `vminsert` nodes.
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/791
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1054
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1165
2021-09-13 18:51:56 +03:00
Aliaksandr Valialkin
c4f11a49f8
docs/CHANGELOG.md: document 5494bc02a6
2021-09-13 17:11:23 +03:00
Aliaksandr Valialkin
7f0a8d4bdb
docs: consistency renaming: Influx -> InfluxDB
2021-09-13 17:05:16 +03:00
Aliaksandr Valialkin
d1a16e0891
app/vmselect/promql: use Prometheus-compatible label value formatting for count_values
function
2021-09-13 13:48:06 +03:00
Aliaksandr Valialkin
fb6ed0ce19
lib/promscrape/discovery/docker: support host networking mode
...
See https://github.com/prometheus/prometheus/issues/9116
2021-09-13 13:30:16 +03:00
Aliaksandr Valialkin
6295861acd
lib/promscrape/discovery/kubernetes: properly use https scheme for wildcard TLS certificates in ingress target discovery
2021-09-13 13:03:42 +03:00
Aliaksandr Valialkin
728c4c3841
lib/promscrape: generate scrape_timeout_seconds
metric per each scrape target in the same way as Prometheus 2.30 does
...
See https://github.com/prometheus/prometheus/pull/9247
2021-09-12 15:20:44 +03:00
Aliaksandr Valialkin
48e3e6c8df
lib/promscrape: add ability to configure scrape_timeout and scrape_interval via relabeling
...
See https://github.com/prometheus/prometheus/pull/8911
2021-09-12 13:33:41 +03:00
Aliaksandr Valialkin
f3e89754a9
lib/promscrape: reduce CPU usage for common case when calculating scrape_series_added
metric
...
Also reduce CPU usage when applying `series_limit` to scrape targets with constant set of metrics.
The main idea is to perform the calculations on scrape_series_added and series_limit
only if the set of metrics exposed by the target has been changed.
Scrape targets rarely change the set of exposed metrics,
so this optimization should reduce CPU usage in general case.
2021-09-12 12:53:14 +03:00
Aliaksandr Valialkin
9286107e82
lib/promscrape: send stale markers for disappeared metrics like Prometheus does
2021-09-11 10:51:04 +03:00
Aliaksandr Valialkin
f4dead529f
lib/storage: properly search series by multiple tag filters matching empty labels such as foo{bar=~"baz|",x=~"y|"}
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1601
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/395
2021-09-09 21:09:21 +03:00
Aliaksandr Valialkin
ea943911bc
app/vmselect/promql: keep metric name in rollup_candlestick
results, since they don't change the original series meaning
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1600
2021-09-09 19:21:18 +03:00
Aliaksandr Valialkin
f27980dcb3
docs/CHANGELOG.md: typo fix
2021-09-09 18:57:30 +03:00
Aliaksandr Valialkin
4aeb8db83f
lib/promscrape: add ability to set series_limit
and stream_parse
options via relabeling
...
This allows managing these options on a per-target basis.
Typical use case: to manage these options for pods via Kubernetes annotations.
2021-09-09 18:49:39 +03:00
Aliaksandr Valialkin
d6bd956930
lib/promrelabel: add keep_metrics
and drop_metrics
actions to relabeling rules
...
These actions simlify metrics filtering. For example,
- action: keep_metrics
regex: 'foo|bar|baz'
would leave only metrics with `foo`, `bar` and `baz` names, while the rest of metrics will be deleted.
The commit also makes possible to split long regexps into multiple lines. For example, the following config is equivalent to the config above:
- action: keep_metrics
regex:
- foo
- bar
- baz
2021-09-09 16:18:21 +03:00
Aliaksandr Valialkin
e6c9869d86
app/vmalert: follow-up after 21f022e5f0
2021-09-07 22:43:37 +03:00
Aliaksandr Valialkin
6fbaf8f978
docs/CHANGELOG.md: document 42e07cfaea
2021-09-07 22:34:39 +03:00
Aliaksandr Valialkin
f529058d3a
docs/CHANGELOG.md: cut v1.65.0
2021-09-01 17:12:32 +03:00
Aliaksandr Valialkin
f77dde837a
lib/promscrape: add the ability to limit the number of unique series per each scrape target
...
The number of series per target can be limited with the following options:
* Global limit with `-promscrape.maxSeriesPerTarget` command-line option.
* Per-target limit with `max_series: N` option in `scrape_config` section.
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1561
2021-09-01 16:03:59 +03:00
Aliaksandr Valialkin
9e41b05401
docs/CHANGELOG.md: document eff940aa76
2021-09-01 12:00:02 +03:00
Aliaksandr Valialkin
fce87bfe8d
docs/CHANGELOG.md: document 7c70dcbe3b
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1471
2021-09-01 11:56:23 +03:00
Aliaksandr Valialkin
8b228a5873
docs/CHANGELOG.md: add a link to Prometheus staleness tracking
2021-08-31 11:48:32 +03:00
Aliaksandr Valialkin
5c63d69454
lib/promscrape/discovery/kubernetes: return back support role: endpointslices
, since it is used by VictoriaMetrics operator
...
This is a follow up commit after 31b42b30b6
2021-08-29 12:37:03 +03:00
Aliaksandr Valialkin
db330232ac
lib/protoparser/opentsdb: follow-up after 8ee75ca45a
2021-08-29 11:49:21 +03:00
Aliaksandr Valialkin
31b42b30b6
lib/promscrape/discovery/kubernetes: rename role: endpointslices
to role: endpointslice
to be consistent with Prometheus
...
See 2ec6c7dbb8/discovery/kubernetes/kubernetes.go (L99)
2021-08-29 11:23:08 +03:00
Aliaksandr Valialkin
2e001db4de
lib/promscrape/discovery/kubernetes: use v1 API instead of v1beta1 API for role: ingress
and role: endpointslices
...
This should fix service discovery for these roles in Kubernetes v1.22 and newer versions.
See https://kubernetes.io/docs/reference/using-api/deprecation-guide/#ingress-v122
The corresponding change in Prometheus - https://github.com/prometheus/prometheus/pull/9205
2021-08-29 11:16:59 +03:00
Aliaksandr Valialkin
5ea689d61b
app/vmselect/promql: add quantile("phiLabel", phi1, ..., phiN, q)
aggregate function to MetricsQL
...
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1573
2021-08-27 18:37:20 +03:00
Aliaksandr Valialkin
bec18e4fe9
app/vmselect: add -search.disableAutoCacheReset
command-line option for disabling automatic cache reset when a sample with old timestamp outside -search.cacheTimestampOffset is inserted
...
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1570
2021-08-27 17:15:31 +03:00
Aliaksandr Valialkin
c8c153fb91
docs/CHANGELOG.md: document the bugfix for possible timeout error in vmbackupmanager when making snapshots
...
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1571
2021-08-27 13:04:06 +03:00
Aliaksandr Valialkin
10f960fa0c
lib/promscrape: add ability to load scrape configs from multiple files
...
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1559
2021-08-26 08:51:16 +03:00
Aliaksandr Valialkin
e24203cde8
docs/CHANGELOG.md: document 48f33d098b
2021-08-25 13:31:35 +03:00
Aliaksandr Valialkin
c27ee35c5c
lib/promscrape: expose promscrape_discovery_http_errors_total metric for tracking errors per each http_sd config
2021-08-25 13:05:49 +03:00
Aliaksandr Valialkin
ffc0ab1774
lib/{mergeset,storage}: improve the detection of the needed free space for background merge
...
This should prevent from possible out of disk space crashes during big merges.
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1560
2021-08-25 09:35:44 +03:00
Aliaksandr Valialkin
0078486ea7
app/vmselect/promql: rename sign()
function to sgn()
in order to be consistent with Prometheus
...
See https://github.com/prometheus/prometheus/pull/8457 for details.
2021-08-23 11:45:51 +03:00
Aliaksandr Valialkin
69c291353b
deployment/docker: update Go builder from Go1.16.0 to Go1.17.0
...
This improves data ingestion and query performance by up to 5% according to benchmarks.
See https://go.dev/blog/go1.17
2021-08-21 22:20:49 +03:00
Aliaksandr Valialkin
d5622b32e2
lib/promscrape: reduce memory and CPU usage when Prometheus staleness tracking is enabled for metrics from deleted / disappeared scrape targets
...
Store the scraped response body instead of storing the parsed and relabeld metrics.
This should reduce memory usage, since the response body takes less memory than the parsed and relabeled metrics.
This is especially true for Kubernetes service discovery, which adds many long labels for all the scraped metrics.
This should also reduce CPU usage, since the marshaling of the parsed
and relabeld metrics has been substituted by response body copying.
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1526
2021-08-21 21:17:26 +03:00
Aliaksandr Valialkin
6ffbb46aef
docs/CHANGELOG.md: document 9ee3d0378f
2021-08-21 20:20:08 +03:00
Aliaksandr Valialkin
c14edc860b
docs/CHANGELOG.md: document b5d6a0e499
2021-08-19 14:03:20 +03:00
Aliaksandr Valialkin
80ddade4ed
docs/CHANGELOG.md: clarify the change, which adds -search.noStaleMarkers
command-line flag
2021-08-19 13:56:04 +03:00
Aliaksandr Valialkin
a724229b5d
app/vmselect/promql: do not override [d]
at last_over_time(m[d])
if [d]
is smaller than scrape_interval
...
Since most users do not expect the overriding of explicitly set `[d]`.
2021-08-19 10:31:48 +03:00
Aliaksandr Valialkin
ce0c270e75
docs/CHANGELOG.md: cut v1.64.1
...
This is mostly bugfix release, which includes fixes for staleness handling and a security update for Alpine base image
2021-08-18 22:06:05 +03:00
Aliaksandr Valialkin
c09446a9aa
lib/promscrape: send stale markers for the previously scraped metrics on failed scrapes like Prometheus does
2021-08-18 21:59:03 +03:00
Aliaksandr Valialkin
04c3e9916d
docs/CHANGELOG.md: document 06bf21c21b
2021-08-18 14:01:04 +03:00
Aliaksandr Valialkin
cdc372bb98
app/vmselect: add -search.noStaleMarkers
command-line flag for disabling stale markers handling in queries
...
This option allows reducing CPU usage a bit when VictoriaMetrics is used
for collecting and processing non-Prometheus data. For example, InfluxDB line protocol, Graphite, OpenTSDB, CSV, etc.
2021-08-18 13:59:02 +03:00
Aliaksandr Valialkin
226143f31b
lib/promscrape: add ability to disable sending Prometheus staleness markers with -promscrape.disableStaleMarkers command-line flag
...
This option can be useful when vmagent consumes too much additional memory
for staleness markers functionality and when staleness markers aren't needed.
2021-08-18 13:43:21 +03:00
Aliaksandr Valialkin
db1e62495b
app/vmselect/promql: add bitmap_and(), bitmap_or() and bitmap_xor() functions to MetricsQL
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1541
2021-08-17 13:21:21 +03:00
Aliaksandr Valialkin
03c959f1df
lib/promscrape: stop scrapers for the removed targets before starting scrapers for the added targets
...
This should prevent from possible time series overlap when old target is substituted by new target (for example, during Kubernetes deployments).
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1526
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1530
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/748
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1509
2021-08-17 00:55:51 +03:00
Aliaksandr Valialkin
f37b963619
app/vmalert: follow-up for 2400f85761
2021-08-16 15:20:22 +03:00
Aliaksandr Valialkin
4547d4f692
docs/CHANGELOG.md: update urls to Prometheus 2.29 release
...
Previously these urls were pointing to rc0 release
2021-08-16 14:53:38 +03:00
Aliaksandr Valialkin
ae9f923449
docs/CHANGELOG.md: typo fix: satureated -> saturated
2021-08-16 14:53:38 +03:00
Aliaksandr Valialkin
c518858145
docs/CHANGELOG.md: cut v1.64.0
2021-08-15 23:52:03 +03:00
Aliaksandr Valialkin
a0e18f06eb
lib/promscrape: restore red highlighting for DOWN targets at /targets page
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1461
2021-08-15 16:03:57 +03:00
Aliaksandr Valialkin
40a859a760
docs/CHANGELOG.md: mention the bugfix when more than 27 time series are selected at /vmui
2021-08-15 15:10:41 +03:00
Aliaksandr Valialkin
386ee5b82c
docs/CHANGELOG.md: mention that VMUI automatically fills Server URL field
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1506
2021-08-15 14:45:09 +03:00
Aliaksandr Valialkin
aefba16d5e
app/vmagent/remotewrite: expose vmagent_remotewrite_send_duration_seconds_total metric
...
This metric can be used for determining high saturation of every connection to remote storage with
an alerting query `rate(vmagent_remotewrite_send_duration_seconds_total) > 0.9s`.
This query triggers when a connection is satureated by more than 90%
2021-08-15 13:34:12 +03:00
Aliaksandr Valialkin
4401464c22
all: add support for Prometheus staleness markers
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1526
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/748
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1509
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1530
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/845
2021-08-13 12:10:17 +03:00
Aliaksandr Valialkin
755f65f4bc
app/vminsert: add vm_rpc_send_duration_seconds_total metric per each vminsert->vmstorage
link
...
This metric is useful for determining high link saturation with the following alerting rule:
rate(vm_rpc_send_duration_seconds_total) > 0.9s
2021-08-11 11:44:40 +03:00
Aliaksandr Valialkin
5ce531027f
docs/CHANGELOG.md: document new metrics added to vmalert at 7416fdaa8b
2021-08-05 10:13:08 +03:00
Aliaksandr Valialkin
d826352688
app/vmagent: follow-up after fe445f753b
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1491
2021-08-05 09:52:32 +03:00
Aliaksandr Valialkin
50663ba41f
lib/promscrape/discovery/gce: add __meta_gce_interface_ipv4_<name> labels as in Prometheus 2.29
...
See https://github.com/prometheus/prometheus/pull/8978
2021-08-03 16:11:49 +03:00
Aliaksandr Valialkin
3cad8b4564
lib/promscrape/discovery/ec2: add __meta_ec2_availability_zone_id
label as Prometheus 2.29 does
2021-08-03 16:11:49 +03:00
Aliaksandr Valialkin
e92fde7945
app/vmselect/promql: add present_over_time(m[d])
function, which will be available starting from Prometheus 2.29.0
...
See https://github.com/prometheus/prometheus/releases/tag/v2.29.0-rc.0 and https://github.com/prometheus/prometheus/pull/9097
2021-08-03 16:11:49 +03:00
Aliaksandr Valialkin
8a3c13fd53
docs/CHANGELOG.md: typo fix
2021-07-30 12:35:57 +03:00
Aliaksandr Valialkin
a3b4fc0474
docs/CHANGELOG.md: document d05cac6c98
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1486
2021-07-30 12:19:53 +03:00
Aliaksandr Valialkin
a1911e1330
app/vmselect/netstorage: unpack time series data in mostly local big chunks
...
This should improve performance on multi-CPU systems for queries selecting time series with big number of raw samples
2021-07-30 12:03:17 +03:00
Aliaksandr Valialkin
192dfbfd90
app/vmselect: follow-up for ed95bc9531
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1493
2021-07-29 09:53:28 +03:00
Aliaksandr Valialkin
8ee8660ac4
app/vmselect: follow-up for 626073bca8
...
* Rename -search.maxMetricsPointSearch to -search.maxSamplesPerQuery, so it is more consistent with the existing -search.maxSamplesPerSeries
* Move the -search.maxSamplesPerQuery from vmstorage to vmselect, so it could effectively limit the number of raw samples obtained from all the vmstorage nodes
* Document the -search.maxSamplesPerQuery in docs/CHANGELOG.md
2021-07-28 18:00:23 +03:00
Aliaksandr Valialkin
8481f4f004
docs/CHANGELOG.md: document 9d45b46f4c
2021-07-27 12:38:31 +03:00
Aliaksandr Valialkin
95aff47330
app/vmselect: prevent from possible deadlock when f callback blocks inside RunParallel
2021-07-26 15:47:30 +03:00
Aliaksandr Valialkin
05672ffc32
app/vmselect/promql: properly handle (a op b) default N
if (a op b)
returns NaN series
...
The result should be a series with `N` values and `a op b` labels. Previously such series has been removed from the result.
2021-07-16 01:44:58 +03:00
Aliaksandr Valialkin
f4e81aef7e
app/vmselect/netstorage: add -search.maxSamplesPerSeries
command-line option for limiting the number of samples a query can process per each series
...
This should prevent from out of memory crashes like in https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1067
2021-07-15 16:03:28 +03:00
Aliaksandr Valialkin
171d44acd8
docs/CHANGELOG.md: typo fix: suffxies->suffixes
2021-07-15 15:02:20 +03:00
Aliaksandr Valialkin
61cc13c16f
docs/CHANGELOG.md: cut v1.63.0
2021-07-15 14:02:13 +03:00
Aliaksandr Valialkin
682662b2ae
lib/storage: remove cache directory if it contains reset_cache_on_startup file
...
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1447
2021-07-13 17:58:51 +03:00
Aliaksandr Valialkin
2df66dad7b
lib/httpserver: add is_set
label to flag
metrics
...
This label allows determining the set flags with the query `flag{is_set="true"}`
2021-07-13 15:10:13 +03:00
Aliaksandr Valialkin
a925d5a3e1
app/vmselect/promql: duration handling improvements in MetricsQL queries
...
- Support durations anywhere in MetricsQL queries. E.g. sum_over_time(m[1h])/1h is equivalent to sum_over_time(m[1h])/3600
- Support durations without suffix. E.g. rate(m[300]) is equivalent to rate(m[5m])
2021-07-12 17:16:41 +03:00
Aliaksandr Valialkin
f3a5465ece
docs/CHANGELOG.md: document the change from bfba4c28a4
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1444
2021-07-12 12:44:06 +03:00
Aliaksandr Valialkin
f539772ca6
docs: sync with the cluster branch
2021-07-10 12:45:38 +03:00
Aliaksandr Valialkin
8c764e88f0
app/vmui: move source code from https://github.com/VictoriaMetrics/vmui to app/vmui
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1413
2021-07-09 17:15:23 +03:00
Aliaksandr Valialkin
c5f0b454f0
app/vmselect: follow-up after aa11ef6d3b
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1413
2021-07-07 17:43:35 +03:00
Aliaksandr Valialkin
6e0553c92e
lib/mergeset: cache indexBlock items only on the second request
...
This should reduce the indexdb/indexBlocks cache size, since it won't contain one-time-wonders items.
2021-07-07 15:23:06 +03:00
Aliaksandr Valialkin
ed944313b0
app/{vminsert,vmselect}: export vminsert_request_duration_seconds and vmselect_request_duration_seconds histograms
2021-07-07 13:25:21 +03:00
Aliaksandr Valialkin
766edbc421
lib/httpserver: print full requestURI in httpserver.Errorf
...
This should simplify debugging.
2021-07-07 13:09:40 +03:00
Aliaksandr Valialkin
8f973e34fb
lib/workingsetcache: properly switch to whole
mode
...
Previously the switch from `split` to `whole` mode had been performed too early,
e.g. when the current cache size became bigger than 1/4 of the allowed cache size.
Now it is performed when the current cache size becomes bigger than 1/2 of the allowed cache size.
This change can reduce memory usage for data ingestion path when big number of active time series are ingested.
2021-07-05 15:16:11 +03:00
Aliaksandr Valialkin
888d62e40c
docs/CHANGELOG.md: document the bugfix for vm_merge_need_free_disk_space
metric at 9a83e9018d
2021-07-05 12:01:08 +03:00
Aliaksandr Valialkin
158c50c0ee
lib/promauth: reload TLS certificates from disk on every mTLS connection as Prometheus does
...
This allows updating client certificates without the need to restart vmagent and/or single-node VictoriaMetrics.
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1420
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/470
2021-07-01 15:27:47 +03:00
Aliaksandr Valialkin
02ed4b8b46
docs/CHANGELOG.md: document ae485c2bfd
2021-07-01 11:51:19 +03:00
Aliaksandr Valialkin
c25b839078
lib/workingsetcache: reset the cache mode when the cache is reset
...
This should reduce memory usage if the working set is reduced after the cache reset.
2021-07-01 11:50:11 +03:00
Aliaksandr Valialkin
1b0501a09e
lib/promscrape: typo fix in /targets
output
...
The typo has been introduced in fb72a2133f
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1408
2021-06-28 21:26:37 +03:00
Aliaksandr Valialkin
008033a374
docs/CHANGELOG.md: cut v1.62.0
2021-06-25 13:29:38 +03:00
Aliaksandr Valialkin
cb5453953f
lib/promscrape: split docker and dockerswarm service discovery code bases, since they have very little in common
...
This is a follow up after c85a5b7fcb
2021-06-25 13:20:20 +03:00
Aliaksandr Valialkin
ca42410afd
docs/CHANGELOG.md: document the bugfix in increase_pure() function from the commit fb4f758715
2021-06-24 12:05:39 +03:00
Aliaksandr Valialkin
5d64ed73c5
lib/protoparser/clusternative: do not pool unmarshalWork structs, since they can occupy big amounts of memory (more than 100MB per each struct)
...
This should reduce memory usage for vmstorage under high ingestion rate when the vmstorage runs on a system with big number of CPU cores
2021-06-23 15:46:50 +03:00
Aliaksandr Valialkin
cdfae0117a
app/vmselect/promql: return the last timestamp for the max / min value from tmax_over_time()
and tmin_over_time()
function as most users expect
2021-06-23 14:19:00 +03:00
Aliaksandr Valialkin
70e2852376
docs/CHANGELOG.md: document the bugfix for incorrect stats collection for concurrently executed tag filter
...
Follow up for c22114c6f0
2021-06-23 14:05:28 +03:00
Aliaksandr Valialkin
94f3e40ab3
app/vminsert/netstorage: sort the -storageNode
list passed to vminsert
nodes
...
This should reduce resource usage (CPU, RAM, disk IO) at vmstorage nodes
if the addresses of vmstorage nodes are passed in random order to vminsert nodes.
2021-06-23 14:01:41 +03:00
Aliaksandr Valialkin
ac54f34f9e
lib/promscrape/discovery/http: follow up after e307bbb29a
2021-06-22 13:40:33 +03:00
Aliaksandr Valialkin
755040a171
lib/promscrape/discovery: support generic auth configs in Consul service discovery in the same way as Prometheus 2.28 does
2021-06-22 13:34:02 +03:00
Aliaksandr Valialkin
59e7755df9
docs/CHANGELOG.md: document the support for Consul namepsace
...
See 58a2989fe7
2021-06-22 13:34:02 +03:00
Aliaksandr Valialkin
a3262daac0
docs/CHANGELOG.md: typo fixes
2021-06-18 19:14:53 +03:00
Aliaksandr Valialkin
83a4db813e
app/vmselect: log slow requests to all the /api/v1/*
handlers if their execution time exceeds -search.logSlowQueryDuration
2021-06-18 19:04:42 +03:00
Aliaksandr Valialkin
570f36b344
app/vmctl: limit JSON line size by 10K samples ( #1394 )
...
This should reduce the maximum memory usage at VictoriaMetrics when importing time series with big number of samples.
2021-06-18 15:26:47 +03:00
Aliaksandr Valialkin
cbd9159a22
docs/CHANGELOG.md: document the reduced disk write IO usage
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1338
2021-06-18 14:02:59 +03:00
Aliaksandr Valialkin
fb72a2133f
lib/promscrape: show jobs with empty scrape targets on /targets page
2021-06-18 10:53:52 +03:00
Aliaksandr Valialkin
6d17a4e12d
docs/CHANGELOG.md: document the changed -remoteWrite.queues
value
...
This is a follow-up for 0a796f7c3a
See https://github.com/VictoriaMetrics/VictoriaMetrics/pull/1385
2021-06-16 12:35:46 +03:00
Aliaksandr Valialkin
84fb59b0ba
lib/storage: move deletedMetricIDs set from indexDB to Storage
...
This makes consitent the list of deleted metricIDs when it is used from both the current indexDB and the previous indexDB (aka extDB).
This should fix the issue, which could lead to storing new samples under deleted metricIDs after indexDB rotation.
See more details at https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1347#issuecomment-861232136 .
Thanks to @tangqipengleoo for the initial analysis and the pull request - https://github.com/VictoriaMetrics/VictoriaMetrics/pull/1383 .
This commit resolves the issue in more generic way compared to https://github.com/VictoriaMetrics/VictoriaMetrics/pull/1383 .
The downside of the commit is the deletedMetricIDs set isn't cleaned from the metricIDs outside the retention. It needs app restart.
This should be OK in most cases.
2021-06-15 15:04:30 +03:00
Aliaksandr Valialkin
36d55bff66
lib/promscrape: show the number of samples collected during the last scrape at /targets and /api/v1/targets pages
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1377
2021-06-14 14:04:00 +03:00
Aliaksandr Valialkin
05bc9667c1
docs/CHANGELOG.md: document the addition of DigitalOcean service discovery
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1367
2021-06-14 13:18:19 +03:00
Aliaksandr Valialkin
2ef2e3d6f8
docs/CHANGELOG.md: cut v1.61.1
2021-06-11 13:01:31 +03:00
Aliaksandr Valialkin
c4f3fbfa5d
lib/storage: reset cache on disk during series deletion and during indexdb rotation
...
This should prevent from inconsistent behavior (aka partially missing data for some time series) after unclean shutdown.
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1347
2021-06-11 12:42:28 +03:00
Aliaksandr Valialkin
d979e14da2
docs/CHANGELOG.md: document the bugfix from 7adfe878e1
2021-06-11 11:28:10 +03:00
Aliaksandr Valialkin
b83a51366e
docs/CHANGELOG.md: cut v1.61.0
2021-06-09 19:04:44 +03:00
Aliaksandr Valialkin
329b6cd146
docs/CHANGELOG.md: document the enterprise bugfix for the target
property in Graphite Render API
2021-06-09 13:50:52 +03:00
Aliaksandr Valialkin
db1c548cb4
docs/CHANGELOG.md: document improvements in re-routing handling in vminsert
...
See the following commits:
* 1c09e71f5b
* 0d067eb112
* 2c6b917749
2021-06-09 13:40:37 +03:00
Aliaksandr Valialkin
ab15bf8c90
docs: document rules replay feature for vmalert
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/836
This is a follow-up for 2a259ef5e7
2021-06-09 12:27:34 +03:00
Aliaksandr Valialkin
c16edf8287
docs/CHANGELOG.md: document the bugfix, which prevents panics for aborted http requests in vmauth
...
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1353
This is a follow-up for 6b29b955c0
2021-06-09 12:11:53 +03:00
Aliaksandr Valialkin
78f83dc5ad
app/{vmagent,vminsert}: follow-up after 2fe045e2a4
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1343
2021-06-04 20:27:58 +03:00
Aliaksandr Valialkin
5ac25d2585
docs/CHANGELOG.md: document the bugfix from 6f19bb23a1
2021-06-04 11:53:00 +03:00
Aliaksandr Valialkin
f963f04d3d
app/vminsert: add -disableRerouting
command-line flag for disabling re-routing if some vmstorage nodes have lower performance than the others
2021-06-04 04:42:01 +03:00
Aliaksandr Valialkin
d2d746c4fc
docs/CHANGELOG.md: document that it is possible to build VictoriaMetrics components for Solaris
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1322
This is a follow-up for ddc8022702
2021-05-31 09:33:29 +03:00
Aliaksandr Valialkin
b22e380a34
app/vmauth: allow balancing the load among multiple backend nodes by specifying multiple urls in url_prefix
config
2021-05-29 01:03:37 +03:00
Aliaksandr Valialkin
60341722d5
docs: document f0c21b6300
2021-05-27 15:03:30 +03:00
Aliaksandr Valialkin
7f531e3a60
docs/CHANGELOG.md: document changes from 2233d6ed8a
and d210958fd0
2021-05-26 12:23:22 +03:00
Aliaksandr Valialkin
08234aa7a0
docs/CHANGELOG.md: cut v1.60.0
2021-05-24 15:55:08 +03:00
Aliaksandr Valialkin
f54133b200
lib/storage: do not populate MetricID->MetricName cache during data ingestion
...
This cache isn't needed during data ingestion, so there is no need in spending RAM on it.
This reduces RAM usage on data ingestion path by 30%
2021-05-24 03:02:46 +03:00
Aliaksandr Valialkin
24858820b5
docs/CHANGELOG.md: small typo fix
2021-05-23 14:15:01 +03:00
Aliaksandr Valialkin
04eb37a590
docs/CHANGELOG.md: document the addition of extra_filter_labels
at 84cc0513e1
2021-05-23 14:10:33 +03:00
Aliaksandr Valialkin
78dddfb98f
lib/promauth: follow-up after 5b8176c68e
2021-05-22 18:01:11 +03:00
Aliaksandr Valialkin
8e2985b53d
lib/fs: wait for a while before giving up on NFS file removal if the removal queue is full
...
This should reduce the probability of the panic on a highly loaded VictoriaMetrics
accepting millions of samples per second.
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1313
2021-05-21 17:21:00 +03:00
Aliaksandr Valialkin
14ec2b9f26
docs/CHANGELOG.md: mention the bugfix from d626c5c2a9
...
Updates https://github.com/VictoriaMetrics/operator/issues/243
2021-05-21 16:37:14 +03:00
Aliaksandr Valialkin
c54bb73867
all: do not skip SIGHUP signal during service initialization
...
This can lead to stale or incomplete configs like in the https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1240
2021-05-21 16:34:06 +03:00
Aliaksandr Valialkin
0d3e78b9ee
docs/CHANGELOG.md: move tip to proper place
2021-05-20 17:56:03 +03:00
Aliaksandr Valialkin
ad73f226ff
app/vmstorage: add ability to limit series cardinality via -storage.maxHourlySeries
and -storage.maxDailySeries
command-line flags
2021-05-20 14:15:19 +03:00
Aliaksandr Valialkin
7e526effaa
app/vmagent: add ability to limit series cardinality on a per-hour and per-day basis
2021-05-20 13:13:40 +03:00
Aliaksandr Valialkin
3cd8606abd
docs/CHANGELOG.md: document the bugfix in vmctl import for InfluxDB lines with identical names for field and tag
...
See dcf8803bbd
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1299
2021-05-20 12:06:16 +03:00
Aliaksandr Valialkin
98e425ee09
docs/CHANGELOG.md: refer to the issue related to timezone_offset() function
2021-05-20 12:03:39 +03:00
Aliaksandr Valialkin
0842bb9294
app/vmselect/promql: add timezone_offset(tz)
function
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1306
2021-05-20 11:53:09 +03:00
Aliaksandr Valialkin
b30925738b
docs/vmalert.md: document multitenant support
...
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/740
2021-05-18 16:26:14 +03:00
Aliaksandr Valialkin
66aba00549
app/vmauth: reload -auth.config
on the request to /-/reload
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1194
2021-05-18 02:23:55 +03:00
Aliaksandr Valialkin
ede2ba5a45
docs/CHANGELOG.md: document b38edec7ee
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1293
2021-05-17 01:57:34 +03:00
Aliaksandr Valialkin
733706e6c6
lib/promscrape: reload auth tokens from files every second
...
Previously auth tokens were loaded at startup and couldn't be updated without vmagent restart.
Now there is no need in vmagent restart.
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1297
2021-05-14 20:00:08 +03:00
Aliaksandr Valialkin
10a47af631
app/{vmalert,vmauth}: explicitly set MaxIdleConnsPerHost in net/http.Client.Transport
...
By default MaxIdleConnsPerHost is set to 2. This limits the possibility to re-use http keep-alive connections.
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1300
2021-05-14 18:12:24 +03:00
Aliaksandr Valialkin
fc3519fa26
lib/promscrape: limit scrape_timeout
by scrape_interval
like Prometheus does
2021-05-13 16:09:45 +03:00
Aliaksandr Valialkin
1f75ae6006
docs/CHANGELOG.md: document the bugfix from b4f5be8bd8
2021-05-13 11:18:39 +03:00
Aliaksandr Valialkin
e6fda03e8f
vendor: update github.com/VictoriaMetrics/fasthttp from v1.0.14 to v1.0.15
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1289
2021-05-13 10:44:14 +03:00
Aliaksandr Valialkin
c0ec541559
lib/cgroup: document the ability to detect cgroup v2 memory and cpu limits. This is follow-up for b50024812e
2021-05-13 09:26:20 +03:00
Aliaksandr Valialkin
832651c6c2
app/vmselect: follow up after 8a0678678b
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1168
2021-05-12 17:18:30 +03:00
Aliaksandr Valialkin
f2d5c4e2d0
lib/httpserver: return X-Server-Hostname http header in all the responses for better debuggability
2021-05-11 22:03:48 +03:00
Aliaksandr Valialkin
229d9d6dd7
docs/CHANGELOG.md: document -datasource.roundDigits
added at 5c448126dc
2021-05-10 11:18:26 +03:00
Aliaksandr Valialkin
3b0966c00c
docs/CHANGELOG.md: document vmalert fix for state restoration on startup
2021-05-10 11:10:04 +03:00
Aliaksandr Valialkin
cbd0569ce2
docs/CHANGELOG.md: mention the comment, which gives an example of multi-level vminsert setup
2021-05-08 22:57:04 +03:00
Aliaksandr Valialkin
7aea5f58c4
lib/ingestserver: properly close incoming connections during graceful shutdown
2021-05-08 19:52:58 +03:00
Aliaksandr Valialkin
12d733dd5d
app/vminsert: add support for data ingestion via other vminsert nodes
2021-05-08 19:52:57 +03:00
Aliaksandr Valialkin
d9e3872b1c
docs/CHANGELOG.md: document 904bbffc7f
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1240
2021-05-05 20:33:05 +03:00
Aliaksandr Valialkin
4bea1afc6d
docs/CHANGELOG.md: document 9cdd4696fe
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1252
2021-05-05 20:28:58 +03:00
Aliaksandr Valialkin
ce9e163e94
lib/httpserver: stop the process on panics in request handlers
...
Panics may leave the process in inconsistent state. That's why it is better to stop the process after the panic
instead of recovering from the panic. Unfortunately, the standard net/http.Server recovers panics in request handlers.
See https://github.com/golang/go/issues/16542 . That's lib/httpserver must stop the process on itself after the panic.
2021-05-03 11:59:40 +03:00
Aliaksandr Valialkin
988c3b386f
docs/CHANGELOG.md: document the bugfix for proper removal of stale parts ( 477369b62f
)
2021-05-03 11:38:15 +03:00
Aliaksandr Valialkin
fb097ff774
docs/CHANGELOG.md: cut v1.59.0
2021-05-01 09:39:48 +03:00
Aliaksandr Valialkin
58d1e6eeea
lib/storage: log dropped labels if the number of labels in a metric exceeds -maxLabelsPerTimeseries
command-line flag value
...
This should improve debuggability for this case.
2021-05-01 09:27:55 +03:00
Aliaksandr Valialkin
4394dc6cbb
docs/CHANGELOG.md: document the change from f3a048288e
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1232
2021-04-30 09:54:41 +03:00
Aliaksandr Valialkin
6fa5981e68
app/vmagent: list user-visible endpoints at
http://vmagent:8429/
...
While at it, use common WriteAPIHelp function for the listing in vmagent, vmalert and victoria-metrics
2021-04-30 09:36:43 +03:00
Aliaksandr Valialkin
56b6b893ce
lib/mergeset: split rows ingestion among multiple shards
...
This improves rows ingestion on systems with many CPU cores by reducing lock contention.
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1244
Thanks to @waldoweng for the original idea and draft implementation at https://github.com/VictoriaMetrics/VictoriaMetrics/pull/1243
2021-04-27 15:36:34 +03:00
Aliaksandr Valialkin
60947fb2d5
lib/persistentqueue: eliminate possible data race when obtaining vm_persistentqueue_bytes_pending metric value
2021-04-27 00:25:52 +03:00
Aliaksandr Valialkin
20b77abc17
docs: update docs order
2021-04-24 01:27:13 +03:00
Aliaksandr Valialkin
908e35affd
lib/promscrape: apply scrape_timeout
on receiving the first response byte for stream_parse: true
scrape targets
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1017#issuecomment-767235047
2021-04-23 21:53:35 +03:00
Aliaksandr Valialkin
ae37cfd528
lib/promscrape/discovery/kubernetes: refresh endpoints and endpointslices targets on service object update like Prometheus does
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1240
2021-04-23 20:11:40 +03:00
Aliaksandr Valialkin
bbebdf9ba1
lib/{storage,mergeset}: remove empty directories on startup. Such directories can be left after unclean shutdown on NFS storage
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1142
2021-04-22 13:02:44 +03:00
Aliaksandr Valialkin
6bc52fe41a
all: rename https://victoriametrics.github.io to https://docs.victoriametrics.com
2021-04-20 20:16:17 +03:00
Aliaksandr Valialkin
187e3ec909
app/vmauth: follow-up for 6a81a89b3d
2021-04-20 10:58:29 +03:00
f41gh7
e5db518c0f
updates per-tenant stats docs
...
changes docs order
changes per-tenant-stats pic
2021-04-14 12:14:51 +03:00
Roman Khavronenko
7644efae01
Docs update ( #1199 )
...
* docs: drop table of contents for `vmctl`
We already have it autogenerated on .github.io, so no need to keep it.
* docs: mention OpenTSDB migration feature for vmctl
* docs: sync docs for `vmalert`
2021-04-10 15:46:08 +03:00
Aliaksandr Valialkin
f1a22b097a
docs/CHANGELOG.md: cut v1.58.0
2021-04-08 00:48:16 +03:00
Aliaksandr Valialkin
1177dca3da
app/vmselect: do not sort series returned from topk*
and bottomk*
functions, since these series are already sorted in user-expected order
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1189
2021-04-07 14:16:08 +03:00
Roman Khavronenko
c6a8ebb11f
docs: update docs ordering and formatting ( #1192 )
...
The major change is adding `sort` directive to docs. For those docs which are copied
from internal packages `sort` is added via makefile command. For the rest it is added
manually since they're updated manually as well.
The rest of changes is connected with markdown formatting. For example, changing headers
in some files (`##` => `#`) makes navigation on .github.io to look better. This especially
useful for `changelog` docs.
Table of contents for `vmctl` is dropped, since we already have it autogenerated on .github.io.
No link changes expected. The corresponding PR to `cluster` branch will be made in follow-up PR.
2021-04-07 13:39:16 +03:00
Aliaksandr Valialkin
df32d2836c
lib/storage: properly handle big time ranges passed to /api/v1/labels
and /api/v1/label/<labelName>/values
...
It should be faster querying all the labels and/or all the values instead of querying per-day labels/values on time ranges exceeding maxDaysForPerDaySearch
2021-04-07 13:33:46 +03:00
Aliaksandr Valialkin
78d35d4f46
Makefile: prepare arm64
and amd64
release archives for cluster version on make release
command
2021-04-05 23:03:19 +03:00
Aliaksandr Valialkin
95dbebf512
lib/persistentqueue: delete corrupted persistent queue instead of throwing a fatal error
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1030
2021-04-05 19:26:11 +03:00
Aliaksandr Valialkin
f010d773d6
lib/promscrape/discovery/kubernetes: synchronously load Kubernetes objects on first access
...
Remove async registration of apiWatchers, since it breaks discovering `role: endpoints` and `role: endpointslices` targets,
which depend on pod and service objects.
There is no need in reloading `endpoints` and `endpointslices` targets if the referenced `pod` or `service` objects change,
since in this case the corresponding `endpoints` and `endpointslices` objects should also change because they contain
ResourceVersion of the referenced `pod` or `service` objects, which is modified on object update.
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1182
2021-04-05 14:20:12 +03:00
Aliaksandr Valialkin
a5c5b54c22
lib/proxy: add support for socks5 over tls
proxy
2021-04-05 13:00:05 +03:00
Aliaksandr Valialkin
6742839fd6
lib/promscrape: pass X-Prometheus-Scrape-Timeout-Seconds
header to scrape targets as Prometheus does
2021-04-05 12:15:24 +03:00
Aliaksandr Valialkin
9ff3ecb991
docs/CHANGELOG.md: explain why -sortLabels
is set to false by default
2021-04-04 01:59:25 +03:00
Aliaksandr Valialkin
fc2240fb22
docs/CHANGELOG.md: document the ability to use socks5 proxy
...
Follow-up for a4c6a3b3e1
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1177
2021-04-04 01:42:00 +03:00
Aliaksandr Valialkin
5153410ced
lib/promscrape: support for simple HTTP proxies without CONNECT
method support such as https://github.com/prometheus-community/PushProx
...
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1179
2021-04-04 00:40:40 +03:00
Aliaksandr Valialkin
7a0b964e8d
app/vmselect/promql: do not delete dst_label
if src_label
is empty in label_copy(q, src_label, dst_label)
and label_move(q, src_label, dst_label)
2021-04-03 22:05:06 +03:00
Aliaksandr Valialkin
6f3080f9fb
docs/CHANGELOG.md: document the change from 3055ab0115
(add ability to pass "label=value" to the third argument to topk_*
and bottomk_*
functions
2021-04-03 21:42:08 +03:00
Aliaksandr Valialkin
b8d9d6c326
docs/CHANGELOG.md: yet another typo fix
2021-04-03 00:25:39 +03:00
Aliaksandr Valialkin
22949911e9
docs/CHANGELOG.md: typo fix
2021-04-03 00:23:50 +03:00
Aliaksandr Valialkin
b1d0028e79
app/vmauth: add support for authorization via Authorization: Bearer <token>
2021-04-02 22:14:53 +03:00
Aliaksandr Valialkin
df148f48b7
lib/promscrape: add support for authorization
config in -promscrape.config
as Prometheus 2.26 does
...
See https://github.com/prometheus/prometheus/pull/8512
2021-04-02 21:17:45 +03:00
Aliaksandr Valialkin
7f9c68cdcb
lib/promscrape: add follow_redirect
option to scrape_configs
section like Prometheus does
...
See https://github.com/prometheus/prometheus/pull/8546
2021-04-02 19:56:40 +03:00
Aliaksandr Valialkin
12e4785fe8
lib/promscrape/discovery/kubernetes: properly discover targets in multiple namespaces
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1170
2021-04-02 14:28:30 +03:00
Aliaksandr Valialkin
3967bd705a
docs/CHANGELOG.md: mention about AWS IAM roles for tasks support for EC2 service discovery
...
Follow-up for fdb8995642
2021-04-02 13:10:03 +03:00
Aliaksandr Valialkin
dc9eafcd02
app/{vminsert,vmagent}: add -sortLabels
command-line option for sorting time series labels before ingesting them in the storage
...
This option can be useful when samples for the same time series are ingested with distinct order of labels.
For example, metric{k1="v1",k2="v2"} and metric{k2="v2",k1="v1"}.
2021-03-31 23:27:58 +03:00
Aliaksandr Valialkin
e1f699bb6c
lib/storage: reduce memory usage when ingesting samples for the same time series with distinct order of labels
2021-03-31 21:24:46 +03:00
Aliaksandr Valialkin
48275d8c12
app/vmagent/remotewrite: reduce memory usage when -remoteWrite.queues
is set to a big value
...
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1167
2021-03-31 16:16:33 +03:00
Aliaksandr Valialkin
2d3082bb55
docs/CHANGELOG.md: cut v1.57.1
2021-03-30 15:40:06 +03:00
Aliaksandr Valialkin
0fe8f11090
docs/CHANGELOG.md: mention about returned back type
label for vm_tenant_inserted_rows_total
metric
...
See 9b4e608199
2021-03-30 15:16:57 +03:00
Aliaksandr Valialkin
d58d5562f1
app/vmselect: remove -search.storageTimeout
command-line flag, since it has the same meaning as -search.maxQueryDuration
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/711
2021-03-30 15:04:54 +03:00
Aliaksandr Valialkin
7962cf1af8
app/vmselect: prevent from possible incomplete query results after timed out query
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/711
2021-03-30 13:35:45 +03:00
Aliaksandr Valialkin
75991277fa
Makefile: build vmutils for arm on make release-vmutils
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1147
2021-03-29 23:15:29 +03:00
Aliaksandr Valialkin
947b37ba8e
docs/CHANGELOG.md: cut v1.57.0
2021-03-29 19:14:49 +03:00
Aliaksandr Valialkin
e6fd1f7875
docs/CHANGELOG.md: typo fixes
2021-03-29 15:46:35 +03:00
Aliaksandr Valialkin
5c7c0b2bda
docs/CHANGELOG.md: mention about logging of metrics with too old timestamps in a single-node VictoriaMetrics
...
This is a follow up for aa81039b42
2021-03-29 15:42:46 +03:00
Aliaksandr Valialkin
5f4a9f782f
docs/CHANGELOG.md: mention Graphite Render API fixes
2021-03-29 14:27:42 +03:00
Aliaksandr Valialkin
602a3d99ae
docs/CHANGELOG.md: mention optimized query performance on systems with many CPU cores
2021-03-29 13:55:10 +03:00
Aliaksandr Valialkin
78188decf9
docs: document that vmagent drops data blocks when remote storage replies with 400 and 409 http status codes
...
This is a follow up for 1b7dc1e5a5
.
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1149
2021-03-26 14:44:06 +02:00
Aliaksandr Valialkin
9761ffd161
lib/promscrape/discovery/kubernetes: properly handle too old resource version
error message from Kubernetes watch API
2021-03-26 12:28:10 +02:00
Aliaksandr Valialkin
0b2a66db30
app/vmselect/promql: do not merge time series during requests to /api/v1/query
...
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1141
2021-03-25 13:56:07 +02:00
Aliaksandr Valialkin
d4aadba9fa
app/vmagent: add -promscrape.consul.waitTime
command-line flag for configuring Consul service discovery wait time
...
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1144
2021-03-23 19:33:25 +02:00
Aliaksandr Valialkin
edc0a94a3c
docs/CHANGELOG.md: mention the feature from 44a6cc5eca
2021-03-23 19:00:18 +02:00
Aliaksandr Valialkin
331a6a2015
app/vmselect/graphite: accept and enforce extra_label
in all the Graphite APIs
2021-03-23 15:29:16 +02:00
Aliaksandr Valialkin
b0c956a178
app/vmselect/graphite: follow-up after 529d7be26b
2021-03-18 16:30:20 +02:00
Aliaksandr Valialkin
6cee5338b2
lib/storage: prevent from infinite loop if {__graphite__="..."}
filter matches a metric name with *
, [
or {
chars
...
The idea has been borrowed from https://github.com/VictoriaMetrics/VictoriaMetrics/pull/1137
2021-03-18 14:53:47 +02:00
Aliaksandr Valialkin
17d4a6e900
vendor: update github.com/VictoriaMetrics/metrics from v1.16.0 to v1.17.0
2021-03-17 23:23:04 +02:00
Aliaksandr Valialkin
904dababcc
vendor: update github.com/VictoriaMetrics/metrics from v1.15.3 to v1.16.0
...
This adds the following new metrics for each VictoriaMetrics app:
* process_resident_memory_anonymous_bytes - the RSS share for memory allocated by the process itself.
This share cannot be freed by the OS, so it must be taken into account by OOM killer.
* process_resident_memory_pagecache_bytes - the RSS share for page cache memory (aka memory-mapped files).
This share can be freed by the OS at any time, so it must be ignored by OOM killer.
2021-03-17 17:59:40 +02:00
Aliaksandr Valialkin
20ac89c4e0
docs/CHANGELOG.md: cut v1.56.0
2021-03-17 02:04:55 +02:00
Aliaksandr Valialkin
bd0c6a095e
docs/CHANGELOG.md: do not mention reduction in query duration
...
There was a signficant refactoring in the code responsible for time series search,
so it can result in both speed ups and slow downs depending on used queries.
2021-03-17 01:56:58 +02:00
Aliaksandr Valialkin
7bc728bf53
app/vmselect: add vm_index_search_duration_seconds
histogram for monitoring the performance of index search
2021-03-17 01:17:41 +02:00
Aliaksandr Valialkin
6378205415
lib/netutil: enable IPv6 UDP listening if -enableTCP6
command-line flag is passed to VictoriaMetrics
...
This is a follow-up for 18cfc4be7b
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1131
2021-03-17 00:16:17 +02:00
Aliaksandr Valialkin
aba955fa16
Makefile: prepare vmutils-windows-*.zip archive on make release-vmutils
command
...
The archive contains the following executables for Windows:
* vmagent
* vmalert
* vmauth
* vmctl
Other components - vmbackup, vmrestore, victoria-metrics - aren't supported for Windows yet
2021-03-16 20:52:41 +02:00
Aliaksandr Valialkin
f0a4157f89
app/vmselect/promql: do not crash if histogram_over_time()
function name contains uppercase letters such as Histogram_over_time()
2021-03-16 12:24:21 +02:00
Aliaksandr Valialkin
dd7e82c34f
app/vmstorage: add -logNewSeries
command-line flag for determining the source of series churn rate
2021-03-15 22:38:50 +02:00
Aliaksandr Valialkin
7c37e9aea9
app/{vminsert,vmagent}: a follow-up for b1aa8c3d8f
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1124
2021-03-15 21:37:55 +02:00
Aliaksandr Valialkin
fb935e6e2c
docs/CHANGELOG.md: typo fix: FATURE -> FEATURE
2021-03-15 14:58:19 +02:00
Aliaksandr Valialkin
2dae0a2c47
app/vmselect: add round_digits
query arg to /api/v1/query
and /api/v1/query_range
handlers for limiting the number of decimal digits after the point
2021-03-15 12:36:33 +02:00
Aliaksandr Valialkin
7c002023d7
app/vmselect/prometheus: do not include datapoints with timestamps matching t-d
when returning results from /api/v1/query?query=m[d]&time=t
as Prometheus does
2021-03-12 12:16:50 +02:00
Aliaksandr Valialkin
a6a71ef861
lib/promscrape: add ability to configure proxy options via proxy_tls_config
, proxy_basic_auth
, proxy_bearer_token
and proxy_bearer_token_file
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1116
2021-03-12 03:36:19 +02:00
Aliaksandr Valialkin
bc5c4add89
lib/httpserver: export vm_available_memory_bytes
and vm_available_cpu_cores
metrics
...
These metrics are useful for tracking the available memory and CPU cores for VictoriaMetrics apps.
2021-03-10 12:02:42 +02:00
Aliaksandr Valialkin
2728b25783
docs/CHANGELOG.md: mention about the bugfix from 787242d7b0
2021-03-09 20:56:06 +02:00
Aliaksandr Valialkin
36fd007247
lib/proxy: set missing ServerName in TLS config for proxy_url
.
...
While at it, allow setting Proxy-Authorization for `proxy_url` via `basic_auth` and `bearer_token` configs.
2021-03-09 18:58:18 +02:00
Aliaksandr Valialkin
3fd8653b40
lib/promscrape: apply sample_limit
after metric relabeling is applied as Prometheus does
...
See the description for `sample_limit` option from Prometheus docs:
Per-scrape limit on number of scraped samples that will be accepted.
If more than this number of samples are present after metric relabeling
the entire scrape will be treated as failed. 0 means no limit.
https://prometheus.io/docs/prometheus/latest/configuration/configuration/#scrape_config
2021-03-09 15:47:18 +02:00
Aliaksandr Valialkin
e44532b760
docs/CHANGELOG.md: mention about improved query performance at 18fe0ff14b
2021-03-09 13:12:25 +02:00
Aliaksandr Valialkin
fe8b12fbad
app/vmselect/promql: follow up for 433fff0006
2021-03-09 12:48:44 +02:00
Aliaksandr Valialkin
f6878eac36
vendor: update github.com/VictoriaMetrics/fasthttp from 1.0.12 to 1.0.13
...
This should fix a bug in vmagent with high CPU usage during failed scrapes with small `scrape_timeout`.
2021-03-09 11:45:02 +02:00
John Belmonte
364fdf4a56
spelling fix: adjacent ( #1115 )
2021-03-09 09:18:19 +02:00
Aliaksandr Valialkin
14a399dd06
lib/promscrape: add scrape_offset
option to scrape_config
...
This option can be used for specifying the particular offset per each scrape interval for target scraping
2021-03-08 12:03:33 +02:00
Aliaksandr Valialkin
1187ee5e16
lib/decimal: prevent exponent overflow when processing values close to zero
...
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1114
2021-03-05 18:52:47 +02:00
Aliaksandr Valialkin
47ac2051bb
app/vmauth: allow using regexps in url_map
paths
...
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1112
2021-03-05 18:21:36 +02:00
Aliaksandr Valialkin
17b87725ed
lib/promscrape/discovery/kubernetes: reduce memory usage when Kubernetes service discovery is configured on a big number of scrape jobs
...
Previously vmagent was creating a separate Kubernetes object cache per each scrape job.
This could result in increased memory usage when monitoring a Kubernetes cluster with big number of objects (pods / nodes / services, etc.)
as seen at https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1113
Now it uses a shared map of scrape objects across multiple scrape jobs.
2021-03-05 17:29:55 +02:00
Aliaksandr Valialkin
a14053ffa0
app/vmselect/promql: add histogram_avg()
, histogram_stddev()
and histogram_stdvar()
functions to MetricsQL
2021-03-04 14:12:07 +02:00
Aliaksandr Valialkin
423cd981fb
lib/promscrape: add -promscrape.cluster.replicationFactor
command-line flag for replicating scrape targets among vmagent
instances in the cluster
2021-03-04 10:20:15 +02:00
Aliaksandr Valialkin
3a68b94487
docs/CHANGELOG.md: cut v1.55.1 release
2021-03-03 11:49:10 +02:00
Aliaksandr Valialkin
4c3ef78c05
docs/CHANGELOG.md: mention recent bugfixes from commits 7906316741
and e154f4a644
2021-03-03 10:50:31 +02:00
Aliaksandr Valialkin
1252ca44d8
docs/CHANGELOG.md: cut v1.55.0
2021-03-02 21:37:58 +02:00
Aliaksandr Valialkin
22b1941cfc
lib/promscrape/discovery/ec2: follow-up after f6114345de
2021-03-02 13:46:26 +02:00
Aliaksandr Valialkin
937f382938
lib/protoparser/prometheus: properly unescape label values in Prometheus exposition format
...
Unescape only `\n`, `\"` and `\\` sequences as Prometheus does. Other escape sequences shouldn't be unescaped.
2021-03-02 13:21:43 +02:00
Aliaksandr Valialkin
019d8e88d8
lib/protoparser/graphite: fix parsing of a Graphite line with empty tags such as foo; 1 2
...
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1100
2021-03-01 17:16:35 +02:00
Aliaksandr Valialkin
2c43e846a9
docs/CHANGELOG.md: mention the out of range panic bugfix d6a41b6ea2
2021-03-01 16:54:24 +02:00
Aliaksandr Valialkin
732e729ef9
docs/CHANGELOG.md: mention the issue related to using Kubernetes watch API for service discovery
2021-03-01 01:42:11 +02:00
Aliaksandr Valialkin
cb943f35c7
app/vmagent: remove data race when applying rate limits to -remoteWrite.url
with multiple queues
2021-03-01 00:29:07 +02:00
Aliaksandr Valialkin
1da1d502a8
docs/CHANGELOG.md: mention about https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1074
2021-02-28 19:31:44 +02:00
Aliaksandr Valialkin
9fc7726d84
lib/httpserver: make sure the gzipResponseWriter.Write() is called on Flush() and Close() calls
...
This should fix the `http: superfluous response.WriteHeader call` issue
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1078
2021-02-28 19:22:50 +02:00
Aliaksandr Valialkin
e5ca8ac0db
lib/promscrape: add ability to spread scrape targets among multiple vmagent instances
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1084
2021-02-28 18:41:08 +02:00
Aliaksandr Valialkin
03903c1176
docs/CHANGELOG.md: mention 317b0cbed2
2021-02-28 14:02:49 +02:00
Aliaksandr Valialkin
7cc3d96a41
lib/fs: follow-up after f3a03c4164
2021-02-27 01:01:47 +02:00
Aliaksandr Valialkin
975dac9086
docs/CHANGELOG.md: mentioned a bugfix with extra_label
handling during caching query results
...
Related to 186c078fac
2021-02-27 00:20:29 +02:00
Aliaksandr Valialkin
8683ea85e6
lib/fs: properly handle stale NFS file handle
error during file deletion
...
This error can appear when -storageDataPath points to NFS volume and the given file has been already removed.
2021-02-26 23:25:14 +02:00
Aliaksandr Valialkin
d86e9b49c4
app/vmselect/promql: increase accuracy for buckets_limit()
function for small limits by skipping the first and the last buckets during merge
...
The first and the last buckets are usually `[0 ... leMin]` and `(leMax ... +Inf)`. If they are merged with adjancent buckets,
then the resulting accuracy can suffer.
2021-02-26 22:56:36 +02:00
Aliaksandr Valialkin
c8f2f9b2e8
lib/promscrape: cleanup after 9b2246c29b
...
Main points:
* Revert changes outside lib/promscrape/discovery/kuberntes . These changes can be applied later in a separate commit
* Minimize changes in lib/promscrape/discovery/kubernetes compared to a93e644001
* Corner case fixes.
2021-02-26 16:54:05 +02:00
Aliaksandr Valialkin
98854e5f2b
app/vmselect: add sign(q)
and clamp(q, min, max)
functions, which will be added in the upcoming Prometheus release
...
See https://twitter.com/roidelapluie/status/1363428376162295811
The `last_over_time(m[d])` function already exists in MetricsQL.
2021-02-24 17:24:56 +02:00
Aliaksandr Valialkin
f050e3f492
docs/CHANGELOG.md: mention about a bugfix from 4805b80977
2021-02-24 11:49:10 +02:00
Aliaksandr Valialkin
f4135b0d14
app/vmselect/promql: properly calculate histogram_quantile() over zero buckets and only a single non-zero
le="+Inf"` bucket like Prometheus does
2021-02-24 00:42:22 +02:00
Aliaksandr Valialkin
fa03e0d210
app/vmselect/promql: add increase_pure()
function to MetricsQL
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/962
2021-02-22 19:14:15 +02:00
Aliaksandr Valialkin
dd1e53b119
lib/promrelabel: optimize relabeling performance for common cases
2021-02-22 00:51:13 +02:00
Aliaksandr Valialkin
ff5bbc4b88
lib/promscrape: export vm_promscrape_target_relabel_duration_seconds metric
2021-02-21 23:21:42 +02:00
Aliaksandr Valialkin
901e12024d
vendor: update github.com/VictoriaMetrics/metrics from v1.14.0 to v1.15.0
...
The v1.15.0 exports the following additional metrics:
process_io_read_bytes_total - the number of bytes read via io syscalls such as read and pread
process_io_written_bytes_total - the number of bytes written via io syscalls such as write and pwrite
process_io_read_syscalls_total - the number of read syscalls such as read and pread
process_io_write_syscalls_total - the number of write syscalls such as write and pwrite
process_io_storage_read_bytes_total - the number of bytes read from storage layer
process_io_storage_written_bytes_total - the number of bytes written to storage layer
These metrics can be used for monitoring process io
2021-02-21 22:54:00 +02:00
Aliaksandr Valialkin
c2678754e4
app/vmagent: properly perform graceful shutdown, which was broken in the commit 1d1ba889fe
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1065
2021-02-19 00:31:34 +02:00
Aliaksandr Valialkin
f26162ec99
lib/promscrape: add scrape_align_interval config option into scrape config
...
This option allows aligning scrapes to a particular intervals.
2021-02-18 23:53:44 +02:00
Aliaksandr Valialkin
9c70c1f21f
app/vmselect/promql: reduce the probability of duplicate time series
errors when querying Kubernetes metrics
2021-02-18 22:07:29 +02:00
Aliaksandr Valialkin
5e341ccb59
docs/CHANGELOG.md: cut v1.54.1
2021-02-18 19:09:59 +02:00
Aliaksandr Valialkin
a537c4f602
lib/storage: properly handle queries containing a filter on metric name plus any number of negative filters and zero non-negative filters
...
Example: `node_cpu_seconds_total{mode!="idle"}`
2021-02-18 18:46:36 +02:00
Aliaksandr Valialkin
a7697cc88b
docs/CHANGELOG.md: cut v1.54.0
2021-02-18 14:52:38 +02:00
Aliaksandr Valialkin
edcdc39eb3
app/vmagent/remotewrite: cleanup after 1d1ba889fe
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1065
2021-02-17 21:42:55 +02:00
Aliaksandr Valialkin
4b110fa21c
docs/CHANGELOG.md: mention that prod binaries are built now with Go1.16
...
This is a follow-up for 4edfe76bef
2021-02-17 21:06:59 +02:00
Aliaksandr Valialkin
bea2f86b7b
docs/CHANGELOG.md: document new per-tenant metrics
2021-02-16 23:34:24 +02:00
Aliaksandr Valialkin
9f5ac603a7
lib/storage: reduce the minimum supported retention for inverted index from one month to one day
2021-02-15 15:12:29 +02:00
Aliaksandr Valialkin
38d7e96602
lib/promscrape/discovery/kubernetes: add __meta_kubernetes_endpoints_label_*
and __meta_kuberntes_endpoints_annotation_*
labels to role: endpoints
...
This syncs kubernetes SD with Prometheus 2.25
See 617c56f55a
2021-02-15 02:51:16 +02:00
Aliaksandr Valialkin
c727d2219b
lib/storage: properly hanle regexp tag filters with dots, which can be converted to full string match filters.
...
For example `{label=~"foo\.bar"}` should be converted to `{label="foo.bar"}`. Previously it has was mistakenly conveted to `{label="foo\.bar"}` .
This could result in missing time series for such tag filters.
2021-02-14 23:38:14 +02:00
Aliaksandr Valialkin
73c95c4e5b
docs/CHANGELOG.md: mention about fixed multiarch build for Docker images
...
Related commit: f9902b3372
2021-02-12 15:23:49 +02:00
Aliaksandr Valialkin
1e38ad6d20
app/vmauth: add ability to route requests from a single users to multiple targets depending on the requested path
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1064
2021-02-11 12:41:16 +02:00
Aliaksandr Valialkin
04faea8b45
app/vmselect: parallelize q1 <binary_op> q2
queries by running q1
and q2
in parallel
...
This should reduce query execution times.
2021-02-10 22:59:39 +02:00
Aliaksandr Valialkin
553016ea99
lib/storage: disable composite index usage when querying old data
2021-02-10 14:57:50 +02:00
Aliaksandr Valialkin
c7dccebaef
lib/storage: optimize search by label filters matching big number of time series
2021-02-10 00:44:54 +02:00
Aliaksandr Valialkin
a4140de9e6
lib/mergeset: unconditionally cache indexdb blocks
...
Production workloads show that indexdb blocks must be cached unconditionally for reducing CPU usage.
This shouldn't increase memory usage too much, since unused blocks are removed from the cache every two minutes.
2021-02-09 00:47:50 +02:00
Aliaksandr Valialkin
4dca03501b
docs/CHANGELOG.md: mention about a bugfix for timezone data from df0cda3ab9
2021-02-08 15:58:25 +02:00
Aliaksandr Valialkin
c5770600a2
lib/cgroup: follow-up after b9bf3cbe3e
2021-02-08 15:54:38 +02:00
Aliaksandr Valialkin
803a00102a
app/{vminsert,vmselect}: accept requests to paths with /graphite
and /prometheus
prefixes
...
This should improve compatibility with path prefixes from VictoriaMetrics cluster.
See https://victoriametrics.github.io/Cluster-VictoriaMetrics.html#url-format
2021-02-04 20:01:18 +02:00
Aliaksandr Valialkin
b44edc7832
docs/CHANGELOG.md: mention recently added changes
2021-02-04 16:41:34 +02:00
Aliaksandr Valialkin
9fb38569eb
lib/httpserver: expose process_open_fds
and process_max_fds
metrics
...
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/402
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1037
2021-02-04 16:40:50 +02:00
Aliaksandr Valialkin
814455138a
docs/CHANGELOG.md: cut v1.53.1
2021-02-03 23:45:21 +02:00
Aliaksandr Valialkin
ea00dac35f
docs/CHANGELOG.md: mention that {__graphite__="foo.*.bar"}
syntax deprecates -search.treatDotsAsIsInRegexps
command-line flag
2021-02-03 20:41:29 +02:00
Aliaksandr Valialkin
a5a1b9bd66
lib/storage: fix a bug, which breaks searching by Graphite wildcard filters
2021-02-03 20:14:22 +02:00
Aliaksandr Valialkin
9d41c06db1
docs/CHANGELOG.md: fix a link to Graphite Render API usage docs
2021-02-03 12:29:37 +02:00
Aliaksandr Valialkin
c7e03f30d8
docs: mention about Graphite render API implementation
2021-02-03 12:12:04 +02:00
Aliaksandr Valialkin
8c568b13b2
docs/CHANGELOG.md: cut v1.53.0
2021-02-03 03:42:31 +02:00
Aliaksandr Valialkin
157c02622b
app/vmselect: add ability to set Graphite-compatible filter via {__graphite__="foo.*.bar"}
syntax
2021-02-03 01:21:54 +02:00
Aliaksandr Valialkin
4068f8d590
lib/promscrape: add vm_promscrape_service_discovery_duration_seconds metric
2021-02-02 16:15:25 +02:00
Aliaksandr Valialkin
bd11fd8f1d
lib/promscrape: add vm_promscrape_scrape_retries_total
, vm_promscrape_discovery_retries_total
and vm_promscrape_discovery_requests_total
metrics
2021-02-01 20:06:27 +02:00
Aliaksandr Valialkin
b43b498fd8
app/vmselect: add ability to pass extra_label=<label>=<value>
query arg to Prometheus Querying API
...
This enforced `{label="value"}` label filter to the query.
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1021
2021-02-01 18:04:17 +02:00
Aliaksandr Valialkin
5d87dbfd65
docs: document ability to query Graphite datasource from vmalert
2021-02-01 15:26:33 +02:00
Aliaksandr Valialkin
b2aa80e74b
app/vmagent: add -remoteWrite.roundDigits command-line option for limiting the number of digits after the point for stored values
...
This commit also adds --vm-round-digits command-line option to vmctl tool.
2021-02-01 14:27:09 +02:00
Aliaksandr Valialkin
d5c180e680
app/vmctl: move vmctl code from github.com/VictoriaMetrics/vmctl
...
It is better developing vmctl tool in VictoriaMetrics repository, so it could be released
together with the rest of vmutils tools such as vmalert, vmagent, vmbackup, vmrestore and vmauth.
2021-02-01 01:10:20 +02:00
Aliaksandr Valialkin
929f09b90d
docs/CHANGELOG.md: typo fixes
2021-01-27 01:18:48 +02:00
Aliaksandr Valialkin
fc5b26d856
lib/promscrape: export vm_promscrape_scrapes_failed_per_url_total
and vm_promscrape_scrapes_skipped_by_sample_limit_per_url_total
metrics
...
These metrics could be useful for determining imporperly working scrape targets.
Note that these metrics are exported only for failing scrape targets. They aren't exposed for normally working targets.
2021-01-27 00:39:26 +02:00
Aliaksandr Valialkin
419ad74269
app/vmagent: add -remoteWrite.rateLimit command-line flag for limiting data rate to remote storage
...
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1035
2021-01-27 00:39:26 +02:00
Aliaksandr Valialkin
3fe848cdd7
lib/logger: add -loggerTimezone
command-line flag for adjusting timezone for timestamps in log messages
2021-01-26 22:51:54 +02:00
Aliaksandr Valialkin
5481906db6
docs/CHANGELOG.md: mention about https://github.com/VictoriaMetrics/VictoriaMetrics/pull/1027
2021-01-26 16:37:36 +02:00
Aliaksandr Valialkin
8cea3c3cc4
lib/promscrape: retry scrape and service discovery requests when the remote server closes http keep-alive connection
2021-01-22 13:22:33 +02:00
Aliaksandr Valialkin
3caac3d12c
docs/CHANGELOG.md: mention about the fix with too big HTTP reconnection rate to targets
...
This has been fixed in 0a45220b0a
2021-01-22 12:09:16 +02:00
Aliaksandr Valialkin
8749c2dd92
app/vmselect: add -search.maxStepForPointsAdjustment
command-line flag, which can be used for disabling adjustment for points returned from /api/v1/query_range
handler if they have timestamps closer than -search.latencyOffset
to the current time
2021-01-19 22:56:32 +02:00
Aliaksandr Valialkin
7ce87ebcb2
docs/CHANGELOG.md: cut v1.52.0
2021-01-13 12:58:51 +02:00
Aliaksandr Valialkin
1051d8aa2d
app/vmselect/promql: add ability to pass multiple labels to sort_by_label
and sort_by_label_desc
functions
...
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/992
2021-01-13 12:44:51 +02:00
Aliaksandr Valialkin
bdd0a1cdb2
lib/backup: increase backup chunk size from 128MB to 1GB
...
This should reduce costs for object storage API calls by 8x. See https://cloud.google.com/storage/pricing#operations-pricing
2021-01-13 12:16:35 +02:00
Aliaksandr Valialkin
acf1a2c72b
app/vmselect/promql: properly parse escaped multibyte utf8 code sequences in metric names and labels names
...
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/990
2021-01-13 10:59:42 +02:00
Aliaksandr Valialkin
89315d719d
docs/CHANGELOG.md: document updated extra_label
query arg behavior
...
Follow-up for dc9d7aedd5
2021-01-13 00:58:20 +02:00
Aliaksandr Valialkin
7373986f9e
docs/CHANGELOG.md: mention that the minimum supported TLS version now is v1.2
...
Follow-up for 7bf5d48315
2021-01-13 00:44:39 +02:00
Aliaksandr Valialkin
fe3444b124
deployment/docker: upgrade base image for Docker packages from Alpine 3.13.1 to Alpine 3.12.3 in order to fix potential security issues
...
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1010
2021-01-12 21:57:01 +02:00
Aliaksandr Valialkin
1837f2f7d3
app/vmselect/promql: add tfirst_over_time(m[d])
and tlast_over_time(m[d])
MetricsQL functions for returning timestamps for the first and the last samples in m
over d
2021-01-12 16:12:12 +02:00
Aliaksandr Valialkin
2140ccbdcc
docs/CHANGELOG.md: document big fixes from the commit 7976c22797
2021-01-12 13:44:17 +02:00
Aliaksandr Valialkin
2c44f9989a
lib/promscrape: properly show scrape duration on /targets
page
...
Previously it has been shown as 0.000s for any scrape duration.
2021-01-11 21:14:46 +02:00
Aliaksandr Valialkin
89611fa48c
docs/CHANGELOG.md: mention about a bugfix for https://github.com/VictoriaMetrics/VictoriaMetrics/issues/989
2021-01-11 13:11:41 +02:00
Aliaksandr Valialkin
24ffad74c1
all: use net.Dial
instead of fasthttp.Dial
, because fasthttp.Dial
limits the number of concurrent dials to 1000
2021-01-11 12:53:30 +02:00
Aliaksandr Valialkin
9dcb18e03d
app/vmstorage: disable final merge by default, since it may result in high disk IO and CPU usage without measurable benefits such as increased query performance and reduced disk space usage
2021-01-08 00:16:05 +02:00
Aliaksandr Valialkin
70df5f4975
docs/CHANGELOG.md: cut v1.51.0
2020-12-27 14:21:29 +02:00
Aliaksandr Valialkin
c86286ec1d
app/vmselect/promql: do not ajdust offset
value provided in the query
...
Previously it could be modified in order to improve response cache hit ratio.
This is unneeded, since cache hit ratio should remain good because the query time range
should be already aligned to multiple of `step` values.
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/976
2020-12-27 14:09:25 +02:00
Aliaksandr Valialkin
59183f66d0
app/vmselect: refactor /api/v1/stats/top_queries
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/907
2020-12-25 16:44:29 +02:00
Aliaksandr Valialkin
932e53522d
docs/CHANGELOG.md: mention that vmalert now properly escapes multi-line queries when passing to Grafana
...
A follow-up for 1de15ad490
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/890
2020-12-25 11:12:06 +02:00
Aliaksandr Valialkin
4eb520a342
docs/CHANGELOG.md: mention about adding missing __meta_kubernetes_service_* labels for endpoints
and endpointslices
roles in kubernetes_sd_config
...
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/982
2020-12-24 11:33:00 +02:00
Aliaksandr Valialkin
820669da69
lib/promscrape: code prettifying for 8dd03ecf19
2020-12-24 10:56:10 +02:00
Aliaksandr Valialkin
9df60518bb
docs: mention that it is possible to set multiple -notifier.tlsInsecureSkipVerify
command-line flags for vmalert
...
See c3a92968343c2b3619f1ab935702d0e9b3a46733
2020-12-22 22:32:13 +02:00
Aliaksandr Valialkin
46dba00756
lib/storage: remove stale parts as soon as they go outside the configured retention
...
Previously such parts could remain undeleted for long durations until they are merged with other parts.
This should help for `-retentionPeriod` values smaller than one month.
2020-12-22 19:54:31 +02:00
Aliaksandr Valialkin
e15b20dde3
docs/CHANGELOG.md: cut v1.50.2
2020-12-19 15:32:34 +02:00
Aliaksandr Valialkin
13804bda8f
docs/CHANGELOG.md: mention about bugfix for populating template variables in vmalert
...
See 404cbd1522
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/974
2020-12-19 14:16:04 +02:00
Aliaksandr Valialkin
88ac4dfc07
app/vmselect: properly parse negative combined offsets such as -1h2m3s
...
Previously such offsets were parsed as `-1h + 2m + 3s`. Now they are parsed as `-(1h + 2m + 3s)`.
2020-12-19 01:23:46 +02:00
Aliaksandr Valialkin
17c2ce18fd
docs/CHANGELOG.md: there is no visible difference for CPU usage and disk IO usage on production workloads
2020-12-19 00:51:41 +02:00
Aliaksandr Valialkin
ebf8da3730
lib/{storage,mergeset}: tune background merge process in order to reduce CPU usage and disk IO usage
2020-12-18 20:01:08 +02:00
Aliaksandr Valialkin
e6666da4e7
docs/CHANGELOG.md: mention that Docker images for vmagent, vmalert, vmauth, vmbackup and vmrestore with tags containing -cluster
suffix are no longer published
...
See 441822c4cc
for details
2020-12-18 20:01:03 +02:00
Aliaksandr Valialkin
9abb2d6c74
lib/protoparser/prometheus: follow-up commit after 7d38627b9f6f212ae602aea6a72f469fe3c70ba2
...
Document the bugfix in docs/CHANGELOG.md and add a test for the bugfix.
2020-12-16 23:40:17 +02:00
Aliaksandr Valialkin
795e32be4a
docs/CHANGELOG.md: cut v1.50.1 release
2020-12-15 21:11:39 +02:00
Aliaksandr Valialkin
e8f645bf52
docs/CHANGELOG.md: mention about bugfix, which properly removes unregistered targets from /targets
page
...
See bugfix at a4c7fcb5e1
2020-12-15 21:06:59 +02:00
Aliaksandr Valialkin
aa56b9217e
app/vmagent: add vmagent_remotewrite_blocks_sent_total
and vmagent_remotewrite_bytes_sent_total
metrics per each -remoteWrite.url
2020-12-15 20:39:57 +02:00
Aliaksandr Valialkin
a13a443bf7
docs/CHANGELOG.md: cut v1.50.0 release
2020-12-15 14:44:42 +02:00
Aliaksandr Valialkin
b730fc2667
lib/promscrape: properly handle scrape errors when stream parsing is enabled
...
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/967
2020-12-15 14:08:28 +02:00
Aliaksandr Valialkin
11fa458e39
app/vmselect/promql: return expected increase()
result for the first point on the graph with value not exceeding 100
2020-12-15 13:40:46 +02:00
Aliaksandr Valialkin
2813d0b1e0
docs/CHANGELOG.md: mention that vmagent now accepts multiple -remoteWrite.sendTimeout
and -remoteWrite.tlsInsecureSkipVerify
command-line flags
2020-12-15 12:58:35 +02:00
Aliaksandr Valialkin
ae3675d3d0
docs/CHANGELOG.md: mention about adding query
, first
and value
functions to alert templates
...
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/539
2020-12-14 20:17:11 +02:00
Aliaksandr Valialkin
5d426dfe0a
docs/CHANGELOG.md: mention bugfix for proper recovering from incorrectly stored persistent queue
...
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/964
2020-12-14 19:28:47 +02:00
Aliaksandr Valialkin
069c9ade52
app/{vmagent,vminsert}: follow-up for ce8c2dd1f1
: return /targets
page in HTML when requested via web browser
2020-12-14 14:06:00 +02:00
Aliaksandr Valialkin
5ebfc275e6
app/victoria-metrics: automatically reset response cache when samples with too timestamps older than now - search.cacheTimestampOffset
are ingested
2020-12-14 13:08:28 +02:00
Aliaksandr Valialkin
081aa4ad68
docs/CHANGELOG.md: mention https://github.com/VictoriaMetrics/VictoriaMetrics/issues/955
2020-12-11 17:48:26 +02:00
Aliaksandr Valialkin
ba8ac08739
app/vmselect/graphite: properly handle wildcards and charsets inside curly braces
...
For example, `foo{bar*,[a-f]a*b}` should match `foobar`, `foobar123`, `foofab`, etc.
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/952
2020-12-11 17:24:12 +02:00
Aliaksandr Valialkin
3ca1ed0fde
docs/CHANGELOG.md: mention about memory leak fix in vmagent when big number of targets is discovered via service discovery
2020-12-09 10:35:26 +02:00
Aliaksandr Valialkin
fd9fd191b9
lib/promscrape/discovery/consul: properly pass Datacenter filter to Consul API server
...
Previously it has been passed as `sdc` query arg, while it should be passed as `dc` query arg.
See https://www.consul.io/api-docs/health#list-nodes-for-service for details.
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/574#issuecomment-740454170
2020-12-08 21:52:42 +02:00
Aliaksandr Valialkin
4146fc4668
all: properly handle CPU limits set on the host system/container
...
This can reduce memory usage on systems with enabled CPU limits.
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/946
2020-12-08 21:07:29 +02:00
Aliaksandr Valialkin
0f1b969aa6
lib/promscrape: export vm_promscrape_scrapers_{started|stopped}_total
metrics for monitoring target churn rate
2020-12-08 11:57:52 +02:00
Aliaksandr Valialkin
05813259dc
lib/promscrape: export vm_promscrape_active_scrapers{type="<sd_type>"}
metric for tracking the number of active scrapers per each service discovery type
2020-12-08 01:54:23 +02:00
Aliaksandr Valialkin
9c1c9d8e76
lib/promscrape: do not enable strict config parsing when -promscrape.config.dryRun
command-line flag is passed
...
Strict parsing for -promscrape.config can be enabled by passing `-promscrape.config.strictParse` command-line flag.
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/944
2020-12-07 13:18:50 +02:00
Aliaksandr Valialkin
007dbf273d
app/vmselect/graphite: remove duplicate name
tag from /tags/autoComplete/tags
handler
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/942
2020-12-07 01:08:52 +02:00
Aliaksandr Valialkin
b0e4b234cb
lib/protoparser/influx: allow multiple whitespace chars between measurement, fields and timestamp in Influx line protocol
2020-12-06 12:01:27 +02:00
Aliaksandr Valialkin
ae1db8fa08
docs/CHANGELOG.md: cut v1.49.0
2020-12-05 13:49:04 +02:00
Aliaksandr Valialkin
d305cc2017
deployment/docker: update Go builder from v1.15.5 to v1.15.6
...
This fixes issues found in Go since v1.15.5 - see https://github.com/golang/go/issues?q=milestone%3AGo1.15.6+label%3ACherryPickApproved
2020-12-05 12:37:34 +02:00
Aliaksandr Valialkin
e2e8ef86d9
app/vmselect/promql: add count_eq_over_time(m[d], N)
and count_ne_over_time(m[d], N)
for calculating the number of samples in m
over d
that are equal / not equal to N
2020-12-05 12:30:46 +02:00
Aliaksandr Valialkin
9d787f9edd
all: do not print usage info for all the flags when incorrect command-line flag is passed
...
This should improve usability for VictoriaMetrics apps that have big number of command-line flags,
i.e. all the apps.
2020-12-03 21:47:37 +02:00
Aliaksandr Valialkin
66379cc69f
app/vmselect/promql: add label_uppercase(q, label1, ... labelN)
and label_lowercase(q, label1, ... labelN)
functions
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/936
2020-12-03 21:47:36 +02:00
Aliaksandr Valialkin
5009b25a03
lib/promscrape: code cleanup after c6dee6c52d
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/574
2020-12-03 19:50:53 +02:00
Aliaksandr Valialkin
1982505c2b
app/vmselect/promql: return nan
from minute(m)
when m
equals to nan
...
This aligns VictoriaMetrics behaviour with Prometheus behaviour.
The issue has been spotted in https://promlabs.com/promql-compliance-test-results/2020-12-01/victoriametrics/
2020-12-02 20:16:58 +02:00
Aliaksandr Valialkin
91a4c279cc
app/vmselect: return metric
values from time() cmp_op metric
query when cmp_op
comparison is true
...
This aligns MetricsQL behavior to Prometheus' one.
The issue has been identified at https://promlabs.com/promql-compliance-test-results/2020-12-01/victoriametrics/
2020-12-02 12:09:34 +02:00
Aliaksandr Valialkin
7590b8477b
vendor: upgrade github.com/VictoriaMetrics/metricsql from v0.7.2 to v0.7.3
...
This fixes parsing of hex numbers in MetricsQL such as 0x3b
The bug has been detected at https://promlabs.com/promql-compliance-test-results/2020-12-01/victoriametrics/
2020-12-02 08:10:43 +02:00
Aliaksandr Valialkin
700bda8e2e
app/vmselect/promql: return nan
from a >bool b
if a
is nan
in the same way as Prometheus does
2020-12-02 00:28:26 +02:00
Aliaksandr Valialkin
a906b3862f
lib/protoparser/prometheus: properly parse OpenMetrics timestamps
...
OpenMetrics timestamps are floating-point numbers, that represent Unix timestamp in seconds.
This differs from Prometheus exposition format, where timestamps are integer numbers representing Unix timestamp in milliseconds.
2020-11-27 14:54:29 +02:00
Aliaksandr Valialkin
2cea4d403f
all: typo fix: thouthand->thousand
2020-11-26 13:33:46 +02:00
Aliaksandr Valialkin
4cb6bcd2d7
docs/CHANGELOG.md: cut v1.48.0 release
2020-11-26 02:05:57 +02:00
Aliaksandr Valialkin
6b1317b6a4
docs/CHANGELOG.md: add a link to Netflix Eureka - https://github.com/Netflix/eureka
2020-11-26 01:36:20 +02:00
Aliaksandr Valialkin
b7fcdb528d
app/{vmagent,victoria-metrics}: add -dryRun
option and make more clear handling for -promscrape.config.dryRun
2020-11-25 22:59:13 +02:00
Aliaksandr Valialkin
8a057e705a
lib/storage: log metric name plus all its labels when the metric timestamp is outside the configured retention
...
This should simplify debugging when the source of the metric with unexpected timestamp must be found.
2020-11-25 14:41:37 +02:00
Aliaksandr Valialkin
ae04378424
lib/protoparser/prometheus: properly parse "infinity" values in OpenMetrics format
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/924
2020-11-24 19:03:38 +02:00
Aliaksandr Valialkin
d0ffb49ee2
docs/CHANGELOG.md: mention that /tags/delSeries
handler is supported after f0c207fae2
2020-11-24 12:34:56 +02:00
Aliaksandr Valialkin
b7f4fc6e0d
lib/protoparser/prometheus: properly parse metrics with exemplars
...
Examplars have been introduced in OpenMetrics - see https://github.com/OpenObservability/OpenMetrics/blob/master/OpenMetrics.md#exemplars-1
Previously VictoriaMetrics couldn't parse the following metric
foo{bar="baz"} 123 # exemplar here
This commit fixes this. Note that VictoriaMetrics ignores the exemplar as for now.
2020-11-24 12:34:56 +02:00
Aliaksandr Valialkin
aa90b93778
lib/promscrape: expose __meta_ec2_ipv6_addresses
label for ec2_sd_config
like Prometheus will do in the next release
2020-11-23 16:56:42 +02:00
Aliaksandr Valialkin
de523c81b9
lib/promscrape: add filters
option to dockerswarm_sd_config
like Prometheus did in v2.23.0
2020-11-23 16:27:40 +02:00
Aliaksandr Valialkin
e614a14b21
docs: sync with cluster branch
2020-11-23 00:42:04 +02:00
Nikolay
eec76718e9
Adds eureka service discovery ( #913 )
...
* Adds eureka service discovery
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/851
Netflix service discovery for AWS
* Apply suggestions from code review
Co-authored-by: Aliaksandr Valialkin <valyala@gmail.com>
2020-11-20 13:38:12 +02:00
Aliaksandr Valialkin
de7f315231
docs/CHANGELOG.md: mention that slow query log now contains remote client address
2020-11-19 12:41:17 +02:00
Aliaksandr Valialkin
97a0c80904
lib/logger: follow-up for 09105ff49c
2020-11-19 12:37:00 +02:00
Aliaksandr Valialkin
170e2f54ab
docs/CHANGELOG.md: mention about snap install victoriametrics
2020-11-18 19:49:54 +02:00
Aliaksandr Valialkin
8b116b619a
docs/CHANGELOG.md: sync with cluster branch
2020-11-18 19:46:05 +02:00
Aliaksandr Valialkin
1c477bc2fc
docs/CHANGELOG.md: cut v1.47.0 release
2020-11-16 21:00:06 +02:00
Aliaksandr Valialkin
30a922f383
docs/CHANGELOG.md: mention about Graphite Tags API implementation
2020-11-16 15:34:20 +02:00
Aliaksandr Valialkin
27a417bcd3
app/vmselect/prometheus: improve performance for /api/v1/labels
and /api/v1/label/<labelName>/values
on time ranges exceeding one day when match[]
query arg is set
2020-11-16 13:51:59 +02:00
Aliaksandr Valialkin
5d6d2ef3a6
docs/CHANGELOG.md: mention about improved performance for /api/v1/series
on a time range exceeding one day
2020-11-16 13:21:13 +02:00
Aliaksandr Valialkin
6c0d36e4a9
app/vmselect: propagate errors from vmstorage
to response to the client if -search.denyPartialResponse
command-line flag is set
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/891
This commit also adds `"isPartial":{true|false}` field to `/api/v1/*` responses. `"isPartial":true` is set when the response
is based on a partial data because some of vmstorage nodes weren't available during query processing.
2020-11-14 12:47:48 +02:00
Aliaksandr Valialkin
ef9a8989fd
docs/Single-server-VictoriaMetrics.md: document /internal/force_flush
endpoint
2020-11-13 18:43:10 +02:00
Aliaksandr Valialkin
67b41c080d
docs/CHANGELOG.md: mentioned that Go builder has been updated from v1.15.4 to v1.15.5
...
See 3fa9ab4a49
for details.
2020-11-13 16:22:12 +02:00
Aliaksandr Valialkin
1f19c167a4
app/vmselect/promql: assume that time series value doesnt change during gaps when calculating increase() and delta()
...
This should remove unexpected spikes at the end of gaps.
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/894
2020-11-13 14:59:24 +02:00
Aliaksandr Valialkin
57a4af98fa
lib/protoparser/promremotewrite: forward errors, which can occur during data ingestion, to the caller of ParseStream, so it could properly return HTTP 503 status code on non-nil error
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/896
2020-11-13 11:01:07 +02:00
Aliaksandr Valialkin
47a038401b
all: consistently return text-based HTTP responses with charset=utf-8
...
This is a follow-up for https://github.com/VictoriaMetrics/VictoriaMetrics/pull/897
2020-11-13 10:35:41 +02:00
Aliaksandr Valialkin
348edd92fe
app/vmselect: add -search.treatDotsAsIsInRegexps
command-line flag for automatic escaping of dots in regexp label filters
2020-11-11 12:39:07 +02:00
Aliaksandr Valialkin
e205975716
lib/promscrape: make a copy of ScrapeWork from discovered []ScrapeWork slice instead of referring to an item in this slice
...
This should prevent from holding previously discovered []ScrapeWork slices when a part of discovered targets changes over time.
This should reduce memory usage for the case when big number of discovered scrape targets changes over time.
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/825
2020-11-10 16:13:05 +02:00
Aliaksandr Valialkin
47390d8947
app/vmselect/promql: do not return data points in the end of the selected time range for time series ending in the middle of the selected time range
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/887
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/845
2020-11-10 14:51:44 +02:00
Aliaksandr Valialkin
0d7a3f4eb3
docs/CHANGELOG.md: mention abot explicit setting of extra labels in alert entities (see 3adf8c5a6f
)
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/870
2020-11-10 00:40:51 +02:00
Aliaksandr Valialkin
bcd139362b
lib/promscrape: add -promscrape.dropOriginalLabels
command-line flag for reducing memory usage when discovering big number of scrape targets
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/878
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/825
2020-11-10 00:19:57 +02:00
Aliaksandr Valialkin
41813eb87a
CHANGELOG.md: cut v1.46.0
2020-11-07 17:52:45 +02:00
Aliaksandr Valialkin
bb3b513bdd
docs/CHANGELOG.md: make docs-sync
2020-11-07 16:30:58 +02:00
Aliaksandr Valialkin
55e98e265e
docs/CHANGELOG.md: add CHANGELOG
header
2020-11-07 01:15:00 +02:00
Aliaksandr Valialkin
dd6bfa50e9
app/vmselect/promql: code cleanup after 43823addea
2020-11-06 01:30:50 +02:00
Aliaksandr Valialkin
944c5ea331
docs: add CHANGELOG.md
2020-11-05 14:58:22 +02:00