Commit graph

3293 commits

Author SHA1 Message Date
Aliaksandr Valialkin
bc7d67cee2 lib/decimal: properly calibrate scale for blocks with Inf values
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/805
2020-10-05 14:52:44 +03:00
Aliaksandr Valialkin
59c26feefa app/vmselect/promql: fill gaps on graphs for range_* and running_* functions
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/806
2020-10-02 13:59:45 +03:00
Aliaksandr Valialkin
9fbb84d5c2 app/vmselect/promql: fill gaps on graphs for range_* and running_* functions
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/806
2020-10-02 13:58:19 +03:00
Aliaksandr Valialkin
bdaa9a91f3 Merge branch 'public-single-node' into pmm-6401-read-prometheus-data-files 2020-10-01 19:31:26 +03:00
Aliaksandr Valialkin
764dc2499f lib/storage: code cleanup after 10f2eedee0
Remove the code that uses metricIDs caches for the current and the previous hour during metricIDs search,
since this code became unused after implementing per-day inverted index almost a year ago.

While at it, fix a bug, which could prevent from finding time series with names containing dots (aka Graphite-like names
such as `foo.bar.baz`).
2020-10-01 19:06:23 +03:00
Aliaksandr Valialkin
10f2eedee0 lib/storage: imrpove cache effectiveness for time series ids matching the given filters
Previously the maximum cache lifetime has been limited by 10 seconds. Now it is extended up to a day.
This should reduce CPU usage in the following cases:

* when querying recently added data with small churn rate for time series
* when querying historical data
2020-10-01 14:38:25 +03:00
Aliaksandr Valialkin
1a91da35be Merge branch 'public-single-node' into pmm-6401-read-prometheus-data-files 2020-09-30 09:58:18 +03:00
Aliaksandr Valialkin
d25dd7fdb6 docs: make docs-sync 2020-09-30 09:50:29 +03:00
Roman Khavronenko
daa2d1c065
vmalert: make maxIdleConnections configurable for datasource HTTP client (#797)
Address https://github.com/VictoriaMetrics/VictoriaMetrics/issues/795
2020-09-30 09:49:45 +03:00
Aliaksandr Valialkin
a44e0c6153 vendor: make vendor-update 2020-09-30 08:59:20 +03:00
Aliaksandr Valialkin
f85be226bb vendor: make vendor-update 2020-09-30 08:58:52 +03:00
Aliaksandr Valialkin
8df5a3c5f6 Merge branch 'public-single-node' into pmm-6401-read-prometheus-data-files 2020-09-30 08:55:57 +03:00
Aliaksandr Valialkin
a897cf2ec3 docs/Release-Guide.md: mention that CHANGELOG.md must be updated before release 2020-09-30 08:53:17 +03:00
Aliaksandr Valialkin
58465bb29b CHANGELOG.md: release v1.42.0 2020-09-30 08:45:31 +03:00
Aliaksandr Valialkin
e59de98384 CHANGELOG.md: initial commit
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/788
2020-09-30 00:12:32 +03:00
Aliaksandr Valialkin
bec9b31b81 lib/storage: allow set values higher than 1 for vm_merge_need_free_disk_space if there are multiple partitions with deferred merges due to disk space shortage 2020-09-29 22:51:43 +03:00
Aliaksandr Valialkin
44bcda81ab app/vmstorage: rename vm_{big|small}_merge_need_free_disk_space to vm_merge_need_free_disk_space
This simplifies alerting.
2020-09-29 22:44:19 +03:00
Aliaksandr Valialkin
a9db81c4ab app/vmstorage: add metrics for determining whether background merges need additional disk space to complete
These metrics are:

* vm_small_merge_need_free_disk_space
* vm_big_merge_need_free_disk_space

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/686
2020-09-29 21:48:33 +03:00
Aliaksandr Valialkin
dbf9402329 docs/Single-server-VictoriaMetrics.md: typo fix 2020-09-29 20:29:46 +03:00
Aliaksandr Valialkin
1137bdec66 docs/Single-server-VictoriaMetrics.md: typo fix: compations -> compactions 2020-09-29 20:27:05 +03:00
Aliaksandr Valialkin
127537d631 app/vmagent/remotewrite: do not show -remoteWrite.url in logs if -remoteWrite.showURL isn't set
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/773
2020-09-29 19:49:12 +03:00
Aliaksandr Valialkin
f7636b0342 app/vmselect/graphite: do not substitute path and path. with path.. in /metrics/find/?format=completer output 2020-09-29 18:03:26 +03:00
Aliaksandr Valialkin
76b244cfcf lib/cgroup: do not adjust the number of detected CPU cores via /sys/devices/system/cpu/online
The adjustement increases the resulting GOMAXPROC by 1, which looks confusing to users
as outlined at https://github.com/VictoriaMetrics/VictoriaMetrics/issues/685#issuecomment-698595309
2020-09-29 13:55:26 +03:00
Aliaksandr Valialkin
7dc67cd883 docs/{vmbackup,vmrestore}: formatting fixes 2020-09-29 13:19:07 +03:00
Aliaksandr Valialkin
efdefbc1cb docs/vmbackup.md: make docs about minio config more prominent 2020-09-29 13:16:04 +03:00
Aliaksandr Valialkin
1659135752 lib/storage: fix tests for 32-bit arches such as GOARCH=386 and GOARCH=arm 2020-09-29 13:10:22 +03:00
Aliaksandr Valialkin
9945b8c98d docs: improve readability a bit 2020-09-29 13:03:38 +03:00
Nikolay Khramchikhin
1e679f3e0d
update vmbackup/vmrestore README usage (#794)
* update vmbackup/vmrestore README usage

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/381

with minio and configuration file examples.

* Apply suggestions from code review

Co-authored-by: Roman Khavronenko <hagen1778@gmail.com>

* added backup/restore docs changes

* added example for relabelConfig flag

* Apply suggestions from code review

Co-authored-by: Aliaksandr Valialkin <valyala@gmail.com>
Co-authored-by: Roman Khavronenko <hagen1778@gmail.com>
2020-09-29 12:53:10 +03:00
Aliaksandr Valialkin
38789e4aa0 lib/storage: fix 32-bit builds for GOARH=386 or GOARCH=arm 2020-09-29 12:40:35 +03:00
Aliaksandr Valialkin
19c0b6f3ef lib/protoparser/prometheus: sort rows before comparing them in TestParseStream, since the order for callback calls is non-deterministic 2020-09-29 12:30:04 +03:00
Aliaksandr Valialkin
7cde336b33 lib/protoparser/prometheus: fix TestParseStream after 124f78857b 2020-09-29 12:11:17 +03:00
Aliaksandr Valialkin
96ee276e6e app/vmselect/prometheus: check for errors returned from bufferedwriter.Write
This makes `make errcheck` happy
2020-09-29 11:37:01 +03:00
Aliaksandr Valialkin
6fdfc67620 app/vmselect/graphite: properly handle case when /metrics/find finds both leaf and node for the given query=prefix.*
In this case only node must be returned with stripped dot in the end of id as carbonapi does
2020-09-29 11:01:59 +03:00
Aliaksandr Valialkin
165c9c6371 .github/workflows: verify builds for vmagent, vmalert, vmbackup and vmrestore 2020-09-29 00:49:20 +03:00
Aliaksandr Valialkin
41f24cdb64 .github/workflows: verify that VictoriaMetrics can be built for GOOS=openbsd 2020-09-29 00:44:44 +03:00
Aliaksandr Valialkin
7673839228 lib/{fs,filestream}: small consistency-related updates after cc90a548b1 2020-09-29 00:42:43 +03:00
Nikolay Khramchikhin
cc90a548b1
added openbsd implementations (#790)
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/785

removed fadvise for openbsd, added freespace implemenation for openbsd
2020-09-29 00:29:04 +03:00
Aliaksandr Valialkin
8d5df13c7c vendor: make vendor-update 2020-09-28 21:59:58 +03:00
Aliaksandr Valialkin
7500146321 lib/protoparser: avoid copying of buffer read from the network to unmarshal buffer 2020-09-28 17:19:16 +03:00
Aliaksandr Valialkin
124f78857b app/{vminsert,vmagent}: improve data ingestion speed over a single connection
Process data obtianed from a single connection on all the available CPU cores.
2020-09-28 04:13:08 +03:00
Aliaksandr Valialkin
978c6b4ba9 docs/Cluster-VictoriaMetrics.md: sync with cluster branch 2020-09-28 02:07:55 +03:00
Aliaksandr Valialkin
5cdad60a6f lib/protoparser: use 64KB read buffer instead of default 4KB buffer provided by net/http.Server
This should reduce syscall overhead when reading big amounts of data
2020-09-28 02:07:10 +03:00
Aliaksandr Valialkin
1b3efccb24 app/vmselect: stop /api/v1/export/* execution if client disconnects 2020-09-27 23:53:13 +03:00
Aliaksandr Valialkin
95688cbfc5 all: add native format for data export and import
The data can be exported via [/api/v1/export/native](https://victoriametrics.github.io/#how-to-export-data-in-native-format) handler
and imported via [/api/v1/import/native](https://victoriametrics.github.io/#how-to-import-data-in-native-format) handler.
2020-09-27 19:54:07 +03:00
Aliaksandr Valialkin
b4bf722d8f lib/protoparser: use all the available CPU cores for processing ingested data from a single /api/v1/import stream
Previously a single data ingestion stream to /api/v1/import could load only a single CPU core.
2020-09-26 04:21:32 +03:00
Aliaksandr Valialkin
c00627c103 app/vminsert: code prettifying 2020-09-26 04:13:18 +03:00
Aliaksandr Valialkin
b6a976b98d app/vmagent: reduce memory usage when importing data via /api/v1/import
Previously vmagent could use big amounts of RAM when each ingested JSON line
contained many samples.
2020-09-26 04:10:24 +03:00
Aliaksandr Valialkin
82973f8ae7 Revert "lib/storage: remove unused fetchData arg from BlockRef.MustReadBlock"
This reverts commit bab6a15ae0.

Reason for revert: the `fetchData` arg is used in cluster branch.
Leaving this arg in master branch makes smaller the diff with cluster branch.
2020-09-24 22:44:23 +03:00
Aliaksandr Valialkin
9d3eb3f4b8 Merge branch 'public-single-node' into pmm-6401-read-prometheus-data-files 2020-09-24 20:49:16 +03:00
Aliaksandr Valialkin
bab6a15ae0 lib/storage: remove unused fetchData arg from BlockRef.MustReadBlock
This arg became unused after 23bdc1f107
2020-09-24 20:48:40 +03:00