1143 commits

Author	SHA1	Message	Date
Aliaksandr Valialkin	3e8890e71b	lib/promscrape/discovery/dockerswarm: properly set __meta_dockerswarm_container_label_* labels instead of __meta_dockerswarm_task_label_* labels ... See https://github.com/prometheus/prometheus/issues/9187	2022-08-05 16:11:28 +03:00
Aliaksandr Valialkin	68de1f4e4a	lib/promscrape/discovery/consul: allow stale responses from Consul service discovery by default ... This aligns with Prometheus behaviour. See `allow_stale` option description at https://prometheus.io/docs/prometheus/latest/configuration/configuration/#consul_sd_config	2022-08-05 14:41:40 +03:00
Aliaksandr Valialkin	0c95d87abd	docs: fixes after 83a4abda3f	2022-08-05 10:15:00 +03:00
Aliaksandr Valialkin	83a4abda3f	lib/promscrape/discovery/yandexcloud: follow-up after 6e5ac32fba ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1386	2022-08-04 22:26:43 +03:00
Aliaksandr Valialkin	75c7170624	docs/CHANGELOG.md: document v1.79.1 security fix	2022-08-02 13:30:50 +03:00
Aliaksandr Valialkin	6b0550c023	app/{vmselect,vmalert}: properly generate http redirects if -http.pathPrefix command-line flag is set ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2918	2022-08-02 12:59:07 +03:00
Aliaksandr Valialkin	de6dd1cd5a	lib/mergeset: optimize mergeInmemoryBlocks() function ... Do not spend CPU time on converting inmemoryBlock structs to inmemoryPart structs. Just merge inmemoryBlock structs directly. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2249	2022-07-27 23:58:05 +03:00
Aliaksandr Valialkin	1f6f883016	docs/CHANGELOG.md: clarify docs about the ability to configure multiple sections for vmauth with identical username and different password values	2022-07-26 19:38:46 +03:00
Aliaksandr Valialkin	92630c1ab4	app/vmselect/netstorage: improve the speed of queries over big number of time series on multi-CPU system ... Reduce inter-CPU communications when processing the query over big number of time series. This should improve performance for queries over big number of time series on systems with many CPU cores. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2896 Based on b596ac3745 Thanks to @zqyzyq for the idea.	2022-07-25 09:18:44 +03:00
Aliaksandr Valialkin	4c2f9a1a2e	lib/promscrape: set up=0 for partially failed scrape in stream parsing mode ... This behaviour aligns with Prometheus behavior	2022-07-22 13:29:44 +03:00
Aliaksandr Valialkin	f2326f953b	docs/CHANGELOG.md: document bugfix for https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2874 ... This is a follow-up for edecd2493c	2022-07-21 20:57:04 +03:00
Aliaksandr Valialkin	2095421905	docs/CHANGELOG.md: follow-up after 88edb3f6cf	2022-07-21 20:53:06 +03:00
Aliaksandr Valialkin	4ce5875fa8	all: add ability to push internal metrics to remote storage system specified via -pushmetrics.url	2022-07-21 20:36:27 +03:00
Roman Khavronenko	88edb3f6cf	vmalert: allow configuring custom headers per group (#2901) ... See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2860 Signed-off-by: hagen1778 <roman@victoriametrics.com>	2022-07-21 15:59:55 +02:00
Roman Khavronenko	70a822f3a0	vmalert: allow configuring custom headers for URLs (#2897) ... See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2860 Signed-off-by: hagen1778 <roman@victoriametrics.com>	2022-07-21 13:57:53 +02:00
Nikolay	88029c521c	app/vmauth: allow duplicate usernames (#2888) ... Usernames could be duplicate if it has uniq password. vmauth makes routing based on auth token and username + password combination must be unique for this case.	2022-07-19 19:33:17 +03:00
Aliaksandr Valialkin	cc7d499bbd	app/vmselect/promql: execute q1 and q2 from q1 op q2 in parallel if labels pushdown cannot be applied ... This should improve query performance if VictoriaMetrics has enough resources for processing `q1` and `q2` in parallel. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2886	2022-07-19 14:27:48 +03:00
Aliaksandr Valialkin	0fd86e2364	lib/promscrape: reload all the scrape configs when the global section is changed inside -promscrape.config ... See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2884	2022-07-18 17:15:07 +03:00
Aliaksandr Valialkin	b5da47bfaf	app/vmselect/promql: properly return q1 series from q1 ifnot q2 when q2 returns nothing	2022-07-18 14:24:54 +03:00
Aliaksandr Valialkin	a4424174bb	docs/CHANGELOG.md: document 2f9668eba5	2022-07-18 12:36:53 +03:00
Aliaksandr Valialkin	429369f028	docs/CHANGELOG.md: mention that v1.79.0 changes binary names to $(APP_NAME)-$(GOOS)-$(GOARCH)... naming	2022-07-18 11:59:45 +03:00
Aliaksandr Valialkin	d3116d9862	docs/CHANGELOG.md: cut v1.79.0	2022-07-14 16:15:24 +03:00
Roman Khavronenko	f07bfcf0c9	vmalert: drop support of deprecated extra_filter_labels param (#2870) ... Signed-off-by: hagen1778 <roman@victoriametrics.com>	2022-07-14 15:45:08 +03:00
Aliaksandr Valialkin	cb79c3e765	docs/CHANGELOG.md: formatting fixes for update notes	2022-07-14 12:47:19 +03:00
Yury Molodov	17c33132df	vmui: optimize table view (#2867) ... * feat: optimize table view * fix: add column display setting * app/vmselect: `make vmui-update` Also document the change at docs/CHANGELOG.md Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2022-07-14 00:15:43 +03:00
Dmytro Kozlov	a0d0ba7219	vmui: update time picker behavior (#2847) ... * vmui: update time picker behavior * docs/CHANGELOG.md: document the change Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2022-07-14 00:05:47 +03:00
Yury Molodov	e4efebf4a4	vmui: change selection from autocomplete (#2862) ... * fix: change selection from autocomplete * update docs/CHANELOG.md Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2022-07-13 23:54:56 +03:00
Nikolay	7301aa678c	lib/promscrape: adds azure service discovery (#2743) ... * lib/promscrape: adds azure service discovery Adds azure service discovery mechanism implements authorization with oauth and msi lists virtual machines and virtual machines managed by scaleSet https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1364 * makes linter happy * Apply suggestions from code review Co-authored-by: Roman Khavronenko <roman@victoriametrics.com> * wip Co-authored-by: Roman Khavronenko <roman@victoriametrics.com> Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2022-07-13 23:43:18 +03:00
Aliaksandr Valialkin	ed93330e66	all: follow-up for d99ba3481b	2022-07-13 16:44:39 +03:00
Aliaksandr Valialkin	765278243b	docs/CHANGELOG.md: document 91faa152a5	2022-07-13 12:40:35 +03:00
Aliaksandr Valialkin	f7eda4a73c	docs/CHANGELOG.md: mention that the communication protocol between vmselect and vmstorage nodes is updated in the new release	2022-07-13 12:16:00 +03:00
Aliaksandr Valialkin	cad471037a	app/vmselect/prometheus: follow-up after 3efe33b917	2022-07-11 20:35:28 +03:00
Dmytro Kozlov	3efe33b917	vmselect/prometeus: Add limit param to api/v1/series api endpoint (#2851) ... * issue-2841: Add limit param to api/v1/series api endpoint * issue-2841: add change log * issue-2841: update logic * issue-2841: simplify logic * issue-2841: simplify logic, add information to documentation	2022-07-11 20:18:30 +03:00
Aliaksandr Valialkin	743ff84863	app/vmselect/netstorage: optimize mergeSortBlocks function ... - Use binary search instead of linear scan when locating the run of smallest timestamps in blocks with intersected time ranges. This should improve performance when merging blocks with big number of samples - Skip samples with duplicate timestamps. This should increase query performance in cluster version of VictoriaMetrics with the enabled replication.	2022-07-09 00:34:42 +03:00
Roman Khavronenko	e1a41cfab5	metricsql: properly evaluate timezone_offset over time interval (#2842) ... * metricsql: properly evaluate `timezone_offset` over time interval https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2771 Signed-off-by: hagen1778 <roman@victoriametrics.com> * Update docs/CHANGELOG.md Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2022-07-08 14:03:56 +03:00
Aliaksandr Valialkin	2e9ae40d56	app/vmselect/vmui: follow-up after 0bf6841140 ... * Document the bugfix in docs/CHANGELOG.md * Run `make vmui-update` for updating static js files for vmui, which are included into vmselect	2022-07-08 13:14:17 +03:00
Aliaksandr Valialkin	126e32f79a	docs/CHANGELOG.md: clarifications after e9b977859b	2022-07-08 13:02:41 +03:00
Aliaksandr Valialkin	08db70fa3e	docs/CHANGELOG.md: recommend clearing caches after the upgrade from v1.78.0 to v1.78.1	2022-07-08 12:49:42 +03:00
Roman Khavronenko	e9b977859b	vmalert: deprecate alert's status link (#2840) ... * vmalert: deprecate alert's status link Deprecate alert's status link `/api/v1/<groupID>/<alertID>/status` in favour of `api/v1/alerts?group_id=<group_id>&alert_id=<alert_id>"`. The change was needed for simplifying logic in vmselect for proxying vmalert's requests. The old alert's status link will be still supported for a few versions but will be removed in the future. https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2825 Signed-off-by: hagen1778 <roman@victoriametrics.com> * vmalert: fix review comments Signed-off-by: hagen1778 <roman@victoriametrics.com>	2022-07-08 10:26:13 +02:00
Aliaksandr Valialkin	0713da9e7a	docs/CHANGELOG.md: link to the issue related to multi-level vmselect	2022-07-08 05:23:18 +03:00
Aliaksandr Valialkin	b206bd0aee	docs/CHANGELOG.md: recommend upgrading from v1.78.0 to v1.78.1 because of the bug in v1.78.0	2022-07-08 03:44:26 +03:00
Aliaksandr Valialkin	138ae99602	docs/CHANGELOG.md: cut v1.78.1	2022-07-08 01:02:42 +03:00
Aliaksandr Valialkin	1f1be61b78	docs/CHANGELOG.md: typo fixes	2022-07-07 20:34:08 +03:00
Aliaksandr Valialkin	1828665a64	docs/CHANGELOG.md: link another bugreport related to the bug with per-day inverted index in v1.78.0	2022-07-07 02:34:48 +03:00
Aliaksandr Valialkin	f97355d9fb	lib/promscrape: properly set Host header when sending requests via http proxy ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2794	2022-07-07 02:27:52 +03:00
Aliaksandr Valialkin	893ca6f87e	docs/CHANGELOG.md: link to the issue about incorrect per-day index handling in v1.78.0	2022-07-07 02:00:07 +03:00
Aliaksandr Valialkin	10cb67adb5	app/{vmagent,vminsert}: follow-up after d19e46de55 ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/2839	2022-07-07 01:30:58 +03:00
Aliaksandr Valialkin	01f55bc66b	lib/promscrape/discovery/kubernetes: properly populate service-level labels for role: endpointslice targets ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2823	2022-07-07 00:32:26 +03:00
Aliaksandr Valialkin	b186b63e07	lib/promscrape/discovery/kubernetes: allow attaching node-level labels to role: endpoints and role: endpointlice targets in the same way as Prometheus does ... See https://github.com/prometheus/prometheus/pull/10759	2022-07-06 23:18:59 +03:00
Aliaksandr Valialkin	bd5b20445e	app/vmselect: add ability to query vmselect from another vmselect	2022-07-06 13:30:12 +03:00
Aliaksandr Valialkin	17de8a41c2	all: follow-up after ed89106274	2022-07-06 12:44:46 +03:00
Aliaksandr Valialkin	c66f676f3b	lib/promscrape: push scrape_samples_limit metric to remote storage if sample_limit option is set in scrape_config for this target ... See https://github.com/VictoriaMetrics/operator/issues/497	2022-07-06 12:37:55 +03:00
Roman Khavronenko	ed89106274	vmselect: allow proxying requests to vmalert from single-node (#2834) ... The change allows to proxy requests with prefix `/vmalert` to the vmalert component if `-vmalert.proxyURL` is set. See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2825 and https://github.com/VictoriaMetrics/VictoriaMetrics/pull/2831 Signed-off-by: hagen1778 <roman@victoriametrics.com>	2022-07-06 10:47:26 +02:00
Aliaksandr Valialkin	edc76286ac	lib/storage: put the (date, metricID) entry in dateMetricIDCache just after the corresponding series is registered in the per-day inverted index ... Previously the time series could be put into dateMetricIDCache without registering in the per-day inverted index if GetOrCreateTSIDByName finds TSID entry in the global index. This could lead to missing series in query results. The issue has been introduced in the commit 55e7afae3a, which has been included in VictoriaMetrics v1.78.0	2022-07-05 14:54:03 +03:00
Aliaksandr Valialkin	f3ece83e67	app/vmselect/promql: properly calculate histogram_quantile over unexpected le buckets ... See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2819	2022-07-05 13:19:24 +03:00
Roman Khavronenko	c7acf36e39	docs: warn about potential issue with read queries for 1.78.0 (#2818) ... * docs: warn about potential issue with read queries for 1.78.0 Signed-off-by: hagen1778 <roman@victoriametrics.com> * docs: warn about potential issue with read queries for 1.78.0 Signed-off-by: hagen1778 <roman@victoriametrics.com>	2022-07-04 11:03:27 +03:00
Aliaksandr Valialkin	84e373e5c7	app/vmselect/promql: properly handle partial counter resets in rate(), irate(), increase() and remove_resets() functions	2022-06-30 22:39:38 +03:00
Aliaksandr Valialkin	c392d6d173	app/vmagent/remotewrite: add -remoteWrite.header command-line flag for setting additional http headers to send to -remoteWrite.url ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2805	2022-06-30 20:00:23 +03:00
Aliaksandr Valialkin	e40b40afe6	Revert "lib/promscrape, vmagent: fix path to files (#2801)" ... This reverts commit 0a8e35835c. Reason for revert: it incorrectly fixes the https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2799 See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2799#issuecomment-1171392005	2022-06-30 18:23:56 +03:00
Dmytro Kozlov	0a8e35835c	lib/promscrape, vmagent: fix path to files (#2801) ... vmagent: respect `-pathPrefix` flag for static files and links	2022-06-30 16:22:54 +02:00
Aliaksandr Valialkin	836c19f7ba	Revert "follow-up after bdf9f4669a (#2803)" ... This reverts commit ec5d3253ff, because there was a similar commit already - 119dc333e1	2022-06-30 13:54:33 +03:00
Aliaksandr Valialkin	119dc333e1	docs/CHANGELOG.md: document bdf9f4669a	2022-06-30 13:53:59 +03:00
Aliaksandr Valialkin	56622bff73	docs: add Troubleshooting doc ... This doc contains troubleshooting guides for typical problems with VictoriaMetrics.	2022-06-30 13:53:59 +03:00
Roman Khavronenko	ec5d3253ff	follow-up after bdf9f4669a (#2803) ... Signed-off-by: hagen1778 <roman@victoriametrics.com>	2022-06-30 12:46:57 +02:00
Aliaksandr Valialkin	a14188dd8e	app/vmselect: expose additional histograms at /metrics page, which may help get more insights for the query workload ... This commit is based on https://github.com/VictoriaMetrics/VictoriaMetrics/pull/2792	2022-06-28 20:18:13 +03:00
Aliaksandr Valialkin	a350d1e81c	lib/storage: return marshaled metric names from SearchMetricNames ... Previously SearchMetricNames was returning unmarshaled metric names. This wasn't great for vmstorage, which should spend additional CPU time for marshaling the metric names before sending them to vmselect. While at it, remove possible duplicate metric names, which could occur when multiple samples for new time series are ingested via concurrent requests. Also sort the metric names before returning them to the client. This simplifies debugging of the returned metric names across repeated requests to /api/v1/series	2022-06-28 18:17:15 +03:00
Aliaksandr Valialkin	741dd47273	docs/CHANGELOG.md: document 45f20ad1aa	2022-06-27 13:52:59 +03:00
Aliaksandr Valialkin	a963b2a0aa	all: show timeRange in traces in human-readable format instead of timestamps in milliseconds	2022-06-27 13:45:51 +03:00
Aliaksandr Valialkin	94445e8bd1	docs/CHANGELOG.md: update after e40d015e9a	2022-06-24 18:04:55 +03:00
Aliaksandr Valialkin	d2bbbf147c	all: limit the maximum memory usage for regexp cache, which stores parsed regular expressions in MetricsQL queries ... Previously the cache could store 10K unique regexps. When every regexp is huge (e.g. hundreds of kilobytes), then the total cache size could grow to multiples of gigabytes. Now the cache size is limited by the total length of all cached regexps. So huge regexps won't result in high memory usage for the cache.	2022-06-24 17:57:43 +03:00
Dmytro Kozlov	bb7f31541f	vmui: added query tracing (#2748) ... * vmui: added query tracing * vmui: updated ui * vmui: update tracing logic, fix bugs, disable tracing by default * vmui: use empty message as props * vmui: fixed ui, added delete for each tacing data, show query in header * vmui: added timelines * vmui: speedup render * vmui: use memo for sorting * vmui: use Trace model, remove unused functions, simplify part of code * vmui: update recursive logic * vmui: fix set query to header * vmui: code cleanup, remove unused code * vmui: remove unused type, rename component * wip * wip Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2022-06-23 22:59:20 +03:00
Nikolay	7f1c73bdaf	app/vmselect: fixes partial response with replicationFactor (#2777) ... * app/vmselect: fixes partial response with replicationFactor Allow partial response if it meets replicationFactor configured at vmselect https://t.me/VictoriaMetrics_ru1/38490 * docs/CHANGELOG.md: document this change Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2022-06-23 20:19:35 +03:00
Yurii Kravets	86e80428d5	docs: Update CHANGELOG Update notes (#2776) ... * docs: Update CHANGELOG Update notes Specified the reason why `vmselect` and `vmstorage` nodes may log communication errors.	2022-06-23 15:46:51 +02:00
Aliaksandr Valialkin	52eadb729e	lib/promscrape: always send stale markers with the real scrape timestamp ... This guarantees that query won't return data just after the series is disappeared.	2022-06-23 11:34:18 +03:00
Aliaksandr Valialkin	1c4f67c5d2	lib/promauth: add ability to send additional http headers in requests to scrape targets ... This solves https://stackoverflow.com/questions/66032498/prometheus-scrape-metric-with-custom-header	2022-06-22 20:39:43 +03:00
Aliaksandr Valialkin	51362f9333	app/vmselect: add -search.setLookbackToStep command-line flag for making the gap filling algorithm similar to InfluxDB data model ... This option should override `-search.maxStalenessInterval` for most cases when users migrate from InfluxDB to VictoriaMetrics	2022-06-22 14:19:30 +03:00
Aliaksandr Valialkin	091408be62	docs/CHANGELOG.md: cut v1.78.0	2022-06-20 18:10:38 +03:00
Aliaksandr Valialkin	3837b50f37	lib/netutil.ConnPool: skip dialing remote address if the previous dial attempt was unsuccessful ... If the previous dial attempt was unsuccessful, then all the new dial attempts are skipped until the background goroutine determines that the given address can be successfully dialed. This reduces query latency when some of vmstorage nodes are unavailable and dialing them is slow. This should help with https://github.com/VictoriaMetrics/VictoriaMetrics/issues/711 This commit is based on ideas from the https://github.com/VictoriaMetrics/VictoriaMetrics/pull/2756 The main differences are: - The check for healthy/unhealthy storage nodes is moved one level lower from app/vmselect/netstorage to lib/netutil.ConnPool. This makes possible re-using this feature everywhere lib/netutil.ConnPool is used. - The check doesn't take into account handshake errors for already established connections. Handshake errors usually mean improperly configured VictoriaMetrics cluster, so they shouldn't be ignored.	2022-06-20 17:36:41 +03:00
Aliaksandr Valialkin	49586566a3	docs: follow-up after e4d6b750f6	2022-06-20 17:14:43 +03:00
Nikolay	e4d6b750f6	lib/httpserver: adds flagsAuthKey command-line flag (#2758) ... * lib/httpserver: adds flagsAuthKey command-line flag It protects /flags endpoint with authKey. https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2753O * Apply suggestions from code review Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2022-06-20 17:09:32 +03:00
Artem Navoiev	954a7a6fc6	docs: replace bash code block type with console (#2746)	2022-06-19 22:57:53 +03:00
Aliaksandr Valialkin	c022c4af0a	docs/CHANGELOG.md: document ef7f52e0e6	2022-06-19 22:48:39 +03:00
Aliaksandr Valialkin	5fb45173ae	lib/storage: do not register new series if -storage.maxHourlySeries or -storage.maxDailySeries limits are exceeded ... Previously samples for new series weren't added as expected when series limits were reached, but new series were still registered in indexdb.	2022-06-19 22:42:09 +03:00
Aliaksandr Valialkin	62e2371a67	lib/storage: reset metric id caches for the previous and the current hour ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2698	2022-06-19 22:42:09 +03:00
Dmytro Kozlov	10454d1735	vmui: added focusLabel, enable cardinality app configuratior (#2736) ... * vmui: added focusLabel, enable app configuratior * vmui: set focusLabel if {labelName!=""} * wip * docs/CHANGELOG.md: mention about focusLabel feature in cardinality explorer Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2730 Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2022-06-17 13:03:02 +03:00
Roman Khavronenko	723d90536c	vmselect: limit end param max value by 2d in future (#2729) ... * vmselect: limit `end` param max value by 2d in future The change is applied only to service handlers like `/labels` or `/series` and limits the `end` param by max value <= now() + 2 days. The same limit is applied for the ingested data, so no reason to allow to request data in future far than that. The change is also needed for corner cases like https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2669 where too high `end` value triggers inefficient global index search. Signed-off-by: hagen1778 <roman@victoriametrics.com> * docs/CHANGELOG.md: document the bugfix Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2022-06-16 20:46:31 +03:00
Aliaksandr Valialkin	c18f8cccfa	lib/promrelabel: support action: graphite relabeling ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2737	2022-06-16 20:24:22 +03:00
Aliaksandr Valialkin	0f889497b5	docs/CHANGELOG.md: document dd327bfa9e2c69fe21ab1d92c14636733d7c5620	2022-06-15 18:40:59 +03:00
Aliaksandr Valialkin	e8214ed4e8	docs/CHANGELOG.md: document 00719e5779a3e4eeedb74cb3d25a9ecfe0e16063	2022-06-15 18:09:04 +03:00
Aliaksandr Valialkin	de2be31275	docs/CHANGELOG.md: document 99dbe7f9d4	2022-06-13 10:01:48 +03:00
Yury Molodov	879670418f	vmui: enhancements (#2638) (#2717) ... * feat: make datepicker to be set to last 30 min by default * fix: correct spinner while loading data * feat: change legend style * app/vmselect: `make vmui-update` Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2022-06-13 09:43:37 +03:00
Aliaksandr Valialkin	55a0d34be5	docs/CHANGELOG.md: refer to the issue, which should be solved after the optimization to /api/v1/labels and /api/v1/label/.../values is added ... The optimization has been added in 374beb350e Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1533	2022-06-12 14:32:45 +03:00
Aliaksandr Valialkin	374beb350e	app/vmselect: optimize /api/v1/labels and /api/v1/label/.../values handlers when match[] query arg is passed to them	2022-06-12 04:32:13 +03:00
Aliaksandr Valialkin	89b778902b	app/vmselect: add optional limit query arg to /api/v1/labels and /api/v1/label_values endpoints ... This arg allows limiting the number of sample values returned from these APIs	2022-06-10 09:50:33 +03:00
Roman Khavronenko	48a60eb593	vmalert: followup for 76f05f8670 (#2706) ... Signed-off-by: hagen1778 <roman@victoriametrics.com>	2022-06-09 08:58:25 +02:00
Aliaksandr Valialkin	8888e2b955	docs: add a link to cardinality explorer playground	2022-06-08 19:39:36 +03:00
Dmytro Kozlov	018d2303c4	Cardinality explorer (#2625) ... * Cardinality explorer * vmui, vmselect: updated field name, added description to spinner * make vmui-update * updated const name, make vmui-update * lib/storage: changes calculation for totalSeries values * added static files * wip * wip * wip * wip * docs/CHANGELOG.md: document cardinality explorer feature See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2233 Co-authored-by: f41gh7 <nik@victoriametrics.com> Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2022-06-08 18:43:05 +03:00
Aliaksandr Valialkin	46d8fb03d1	docs/CHANGELOG.md: document 63b538ecd1	2022-06-07 15:52:32 +03:00
Aliaksandr Valialkin	a93deb307f	docs/CHANGELOG.md: document https://github.com/VictoriaMetrics/VictoriaMetrics/pull/2685	2022-06-07 15:39:13 +03:00
Aliaksandr Valialkin	cbb64c824d	docs/CHANGELOG.md: document backwards-incompatible changes in communication protocol between vmselect and vmstorage ... The changes are related to the added query tracing in afced37c0b See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1403	2022-06-07 15:25:35 +03:00
Aliaksandr Valialkin	638ba4614a	docs/CHANGELOG.md: document e755d0ec3f	2022-06-07 15:16:48 +03:00
Aliaksandr Valialkin	b022f1f113	docs/CHANGELOG.md: document 1ee1e986da	2022-06-07 15:02:22 +03:00
Aliaksandr Valialkin	8608dd093c	all: follow-up after 8edb390e21 ... - Remove unused js bloatware from /targets page. This strips down binary size by more than 100Kb - Add /service-discovery page for API compatibility with Prometheus - Properly load bootstrap.min.css from /prometheus/targets - Serve static contents for /targets page from app/vminsert instead of app/vmselect, because /targets page is served from there	2022-06-07 00:57:09 +03:00
Aliaksandr Valialkin	6f0a0e3072	lib/promscrape/discovery/kubernetes: follow-up after 006b8c7534 ... - make more clear error logs - simplify testing for newKubeConfig by passing only the path to kube_config file instead of SDConfig struct	2022-06-06 14:40:52 +03:00
Aliaksandr Valialkin	0922ed2b7e	lib/promscrape: add -promscrape.cluster.name command-line flag ... This flag is used for proper data de-duplication when the same target is scraped from multiple vmagent clusters. See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2679	2022-06-04 00:37:01 +03:00
Dmytro Kozlov	8edb390e21	lib/promscrape: adds service discovery visualization for /targets page(#2675) ... * lib/promscrape: updated template * lib/promscrape: fixed click on unhealthy and all btns * app/vmselect: jquery scripts into static folder Co-authored-by: f41gh7 <nik@victoriametrics.com>	2022-06-03 15:38:45 +02:00
Nikolay	a18914abee	lib/promscrape/discovery/kubernetes: follow-up after 0b5c874911 (#2672)	2022-06-01 20:44:45 +02:00
Aliaksandr Valialkin	ca689fec54	docs/CHANGELOG.md: follow-up after 2177089f94	2022-06-01 14:51:26 +03:00
Aliaksandr Valialkin	ea06d2fd3c	lib/storage: stop background merge when storage enters read-only mode ... This should prevent from `no space left on device` errors when VictoriaMetrics under-estimates the additional disk space needed for background merge. See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2603	2022-06-01 14:36:45 +03:00
Roman Khavronenko	642eb1c534	lib/storage: make indexdb/tagFilters cache size configurable (#2667) ... The default size of `indexdb/tagFilters` now can be overridden via `storage.cacheSizeIndexDBTagFilters` flag. Please, be careful with changing default size since it may lead to inefficient work of the vmstorage or OOM exceptions. https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2663 Signed-off-by: hagen1778 <roman@victoriametrics.com> Co-authored-by: Nikolay <nik@victoriametrics.com>	2022-06-01 10:07:53 +02:00
Roman Khavronenko	2177089f94	promrelabel: add support of lowercase and uppercase relabeling actions (#2665) ... * promrelabel: add support of `lowercase` and `uppercase` relabeling actions https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2664 Signed-off-by: hagen1778 <roman@victoriametrics.com> * lib/storage: make golangci-lint happy Signed-off-by: hagen1778 <roman@victoriametrics.com> Co-authored-by: Nikolay <nik@victoriametrics.com>	2022-06-01 10:02:37 +02:00
Aliaksandr Valialkin	41958ed5dd	all: add initial support for query tracing ... See https://docs.victoriametrics.com/Single-server-VictoriaMetrics.html#query-tracing Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1403	2022-06-01 02:29:23 +03:00
Aliaksandr Valialkin	89c0172778	docs/CHANGELOG.md: follow-up after 11f91532c5 ... See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2594	2022-05-31 12:27:56 +03:00
Dmytro Kozlov	11f91532c5	issue-2594: use embedded for static files (#2650) ... embed static js and css files from CDN into vmalert, vmagent and vmsingle binaries. Co-authored-by: f41gh7 <nik@victoriametrics.com> https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2594	2022-05-31 01:55:28 +02:00
Aliaksandr Valialkin	796804e4b0	lib/promscrape: add -promscrape.suppressScrapeErrorsDelay command-line flag ... This flag can be used for reducing the amounts of logs when scraping unreliable scrape targets. See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2575 The patch is based on https://github.com/VictoriaMetrics/VictoriaMetrics/pull/2576 . Thanks to @jelmd .	2022-05-25 22:59:36 +03:00
Aliaksandr Valialkin	f6d11a49aa	lib/storage: add ability to change the indexdb rotation time offset with -retentionTimezoneOffset command-line flag ... This is a follow-up for 0fbf59199a See https://github.com/VictoriaMetrics/VictoriaMetrics/pull/2574	2022-05-25 16:05:29 +03:00
Aliaksandr Valialkin	bfe96a3cb4	docs/CHANGELOG.md: document 9e343faa41	2022-05-25 15:03:46 +03:00
Aliaksandr Valialkin	7747708ca7	app: expose /api/v1/status/config endpoint in the same way as Prometheus does ... This endpoint is needed for third-party tools. See https://prometheus.io/docs/prometheus/latest/querying/api/#config	2022-05-25 09:57:13 +03:00
Aliaksandr Valialkin	2d2d15b0d0	docs/CHANGELOG.md: cut v1.77.2	2022-05-21 02:26:15 +03:00
Aliaksandr Valialkin	9854fc4dd5	docs/CHANGELOG.md: group vmalert features together	2022-05-21 01:52:01 +03:00
Aliaksandr Valialkin	cf05750d40	docs/CHANGELOG.md: document 2cf586da78	2022-05-21 01:12:32 +03:00
Roman Khavronenko	d5eb6afe26	lib/promscrape/discovery/kubernetes: fixes kubernetes service discovery (#2615) ... * lib/promscrape/discovery/kubernetes: properly updates discovered scrape works previously, added or updated scrapeworks may override previuosly discovered. it happens because swosByKey may contain small subset of kubernetes objects with it's labels. It happens for objectsUpdated and objectsAdded maps, which include only changed elements * Properly calculate vm_promscrape_discovery_kubernetes_scrape_works Co-authored-by: f41gh7 <nik@victoriametrics.com> Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2022-05-21 01:01:37 +03:00
Aliaksandr Valialkin	6b5979cd76	docs/CHANGELOG.md: document 3df8caca15	2022-05-20 14:22:48 +03:00
Aliaksandr Valialkin	667e018a7e	docs/CHANGELOG.md: formatting fixes	2022-05-20 14:22:47 +03:00
Aliaksandr Valialkin	832623516b	docs/CHANGELOG.md: formatting fix for the issue url https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2607	2022-05-20 14:22:47 +03:00
Aliaksandr Valialkin	65227b88a6	docs/CHANGELOG.md: link to the feature request about reusable templates in vmalert ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2510	2022-05-20 14:22:46 +03:00
Roman Khavronenko	34116882b4	vmalert: support scalar type in response (#2610) ... See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2607 Signed-off-by: hagen1778 <roman@victoriametrics.com>	2022-05-18 09:50:46 +02:00
Roman Khavronenko	1fad4dc919	vmalert: support strings in humanize.* templates (#2606) ... See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2569 Signed-off-by: hagen1778 <roman@victoriametrics.com>	2022-05-17 15:38:54 +02:00
Roman Khavronenko	b74c001c92	vmalert: support /rules path for Grafana's ngalert requests (#2593) ... Unexpectedly, Grafana makes an extra request to `/rules` handler in addition to `/api/v1/rules` calls in alerts UI. This happens only for Grafana versions older than 8.5.*. Apparently, this is related to support of other monitoring systems. Prometheus responds with `text/html` content for UI page `/rules` to such requests. Actually, returning just a blank page with SC=200 works as well. Returning actual response of `/api/v1/rules` results in error in Grafana since it expects a `yaml` (?) in response. So we add a placeholder to `vmalert`. https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2583 Signed-off-by: hagen1778 <roman@victoriametrics.com>	2022-05-16 10:00:24 +02:00
Roman Khavronenko	9bc03f6b04	vmalert: follow-up after 0ac1cdfff5 (#2586) ... Signed-off-by: hagen1778 <roman@victoriametrics.com>	2022-05-14 18:56:31 +02:00
Aliaksandr Valialkin	9d7da130b5	docs/CHANGELOG.md: document 3f0ecee128 ... See https://github.com/VictoriaMetrics/VictoriaMetrics/pull/2577	2022-05-13 16:56:38 +03:00
Aliaksandr Valialkin	c448d2fcbb	app/vmalert: apply -remoteRead.disablePathAppend to -datasource.url in the same way as for the -remoteRead.url ... This is a follow-up for 0e2486df56 The related pull requests: - https://github.com/VictoriaMetrics/VictoriaMetrics/pull/1536 - https://github.com/VictoriaMetrics/VictoriaMetrics/pull/1712	2022-05-13 16:44:43 +03:00
Aliaksandr Valialkin	3f6a7bff85	docs/CHANGELOG.md: document c8af625bcc ... See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1322#issuecomment-1120276146	2022-05-11 14:33:27 +03:00
Aliaksandr Valialkin	19f019d0d5	lib/netutil: limit the number of concurrently established connections when calling ConnPool.Get() ... This should reduce potential spikes in the number of established connections in the following cases: - when the connection establishing procedure becomes temporarily slow - after a temporary spike in the rate of ConnPool.Get() calls See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2552	2022-05-11 14:17:22 +03:00
Denys Holius	3dbdd4ef8a	docs: fixed typos in CHANGELOG.md (#2565)	2022-05-10 13:16:17 +02:00
Aliaksandr Valialkin	a7f18f8cb2	app/vmselect/promql: do not return values from label_value() if the original time series has no values at the selected timestamps	2022-05-09 17:57:39 +03:00
Aliaksandr Valialkin	f901788c7f	docs/CHANGELOG.md: document 8f4f5f1d68	2022-05-09 17:32:51 +03:00
Aliaksandr Valialkin	da9e96bc55	docs/CHANGELOG.md: cut v1.77.1	2022-05-07 01:46:27 +03:00
Aliaksandr Valialkin	f26daecc8d	app/vminsert/netstorage: re-route samples from readonly vmstorage nodes to healthy nodes if -dropSamplesOnOverload command-line flag is set	2022-05-07 01:41:00 +03:00
Aliaksandr Valialkin	d6ad8d090d	app/vmstorage: do not allow to set -retentionPeriod smaller than one day ... VictoriaMetrics doesn't support retention periods smaller than one day, so do not allow to set it to small values. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2496	2022-05-07 00:53:01 +03:00
Aliaksandr Valialkin	123aa4c79e	lib/promscrape: properly implement ScrapeConfig.clone() ... Previously ScrapeConfig.clone() was improperly copying promauth.Secret fields - their contents was replaced with `<secret>` value. This led to inability to use passwords and secrets in `-promscrape.config` file. The bug has been introduced in v1.77.0 in the commit 67b10896d2 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2551	2022-05-07 00:05:40 +03:00
Aliaksandr Valialkin	38629b53ef	docs/CHANGELOG.md: document e726340914	2022-05-06 18:10:40 +03:00
Dmytro Kozlov	9a63f6c1b8	vmbackup: Prevent save backups to the same folder where TSDB data is (#2547) ... * {vmbackup, vmbackup/snapshot}: validate snapshot name * vmbackup/snapshot: added another checks * backup/actions: added check that we ignore backup_complete.ignore file * vmbackup: moved snapshot to lib directory * lib/snapshot: added functions description * lib/snapshot: fixed typo * vmbackup: code cleanup * wip * vmbackup: Prevent save backups to the same folder where TSDB data is * Apply suggestions from code review * wip * wip * wip Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2022-05-06 18:04:09 +03:00
Aliaksandr Valialkin	8d81703635	app/vmagent: add missing _total suffix to vmagent_remotewrite_global_rows_pushed_before_relabel_total counter ... This is a follow up for c536139d0b	2022-05-06 15:50:57 +03:00
Aliaksandr Valialkin	1dc4cc243b	lib/promscrape: rename promscrape_stale_samples_created_total metric to vm_promscrape_stale_samples_created_total, so its name is consistent with the rest of vm_promscrape_ metrics	2022-05-06 15:33:13 +03:00
Aliaksandr Valialkin	c536139d0b	app/vmagent: expose vmagent_remotewrite_global_rows_pushed_before_relabel and vmagent_remotewrite_rows_pushed_after_relabel_total metrics	2022-05-06 15:28:59 +03:00
Aliaksandr Valialkin	51e36fd533	app/vmagent: rename vmagent_remote_write_rate_limit_reached_total to vmagent_remotewrite_rate_limit_reached_total for the sake of consistency with other vmagent_remotewrite_ metrics	2022-05-06 15:01:54 +03:00
Aliaksandr Valialkin	d5b55fe22d	lib/promscrape/discovery/ec2: add ability to filter Availability Zones in ec2_sd_config via az_filters section	2022-05-06 12:43:29 +03:00
Aliaksandr Valialkin	ca4ca4630b	app/vmselect/vmui: make vmui-update after 450d879eaa	2022-05-05 21:26:01 +03:00
Aliaksandr Valialkin	be76d49150	docs/CHANGELOG.md: document bf5e3774cc	2022-05-05 13:38:17 +03:00
Aliaksandr Valialkin	6bb32ab9de	docs/CHANGELOG.md: cut v1.77.0	2022-05-05 00:16:16 +03:00
Aliaksandr Valialkin	381e2de59c	app/vmalert: run make quicktemplate-gen from the root directory after the commit f6dcfbcdd6	2022-05-04 20:27:36 +03:00
Nikolay	d289ecded1	{lib/promscrape,app/vmagent}: adds sigv4 support for vmagent remoteWrite (#2458) ... * {lib/promscrape,app/vmagent}: adds sigv4 support for vmagent remoteWrite moves aws related code into separate lib from lib/promscrape it allows to write data from vmagent to the AWS managed prometheus (cortex) https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1287 * Apply suggestions from code review * wip Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2022-05-04 20:24:19 +03:00
Dmytro Kozlov	f6dcfbcdd6	vmalert/tpl: fixed truncating alerts expression in table (#2494) ... vmalert: improve `/groups` UI visual The change also fixes truncated rules expressions in UI https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2484	2022-05-04 18:02:18 +02:00
Aliaksandr Valialkin	bab9670d69	docs/CHANGELOG.md: yet another typo fix: present -> pressed	2022-05-04 18:20:39 +03:00
Aliaksandr Valialkin	554008bb4e	docs/CHANGELOG.md: typo fixes	2022-05-04 18:18:37 +03:00
Aliaksandr Valialkin	2e6827ff04	docs/CHANGELOG.md: document 8639e79d38	2022-05-04 10:46:03 +03:00
Aliaksandr Valialkin	2ced6746a7	docs/CHANGELOG.md: document 3575aabeaf	2022-05-03 14:01:15 +03:00
Aliaksandr Valialkin	b5fedfd3bb	all: add -cluster.tlsInsecureSkipVerify command-line option to vminsert, vmselect and vmstorage components in order to be able to disable TLS certificate verification in mTLS mode ... See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2490	2022-05-03 13:12:50 +03:00
Aliaksandr Valialkin	53cef612b0	docs/CHANGELOG.md: document 488c34f5e1	2022-05-03 11:01:02 +03:00
Aliaksandr Valialkin	aa02719d86	docs/CHANGELOG.md: document d0706c8c95	2022-05-02 22:24:45 +03:00
Aliaksandr Valialkin	0d86644d65	lib/storage: leave the last sample per each discrete interval during the deduplicaton ... This aligns better with staleness logic in Prometheus - https://prometheus.io/docs/prometheus/latest/querying/basics/#staleness	2022-05-02 21:50:45 +03:00
Aliaksandr Valialkin	a186434b50	app/vmui: execute query by pressing enter in the same way as Prometheus does ... Multi-line query can be entered via `shift-enter` in the query input field	2022-05-02 19:49:29 +03:00
Aliaksandr Valialkin	bae7e8b16b	docs/CHANGELOG.md: document 3616337812 ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2514	2022-05-02 15:36:00 +03:00
Aliaksandr Valialkin	70d9e7346b	docs/CHANGELOG.md: document 32a6b67e6c ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1761	2022-05-02 15:26:21 +03:00
Aliaksandr Valialkin	6039640a26	docs/CHANGELOG.md: document b2294d1cf1	2022-05-02 15:21:24 +03:00
Aliaksandr Valialkin	7bc6595b45	lib/netutil: close connections in ConnPool if they are idle for more than 30 seconds ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2508	2022-05-02 15:14:05 +03:00
Artem Navoiev	37cf509c3a	lib/{storage,flagutil} - Add option for snapshot autoremoval (#2487) ... * lib/{storage,flagutil} - Add option for snapshot autoremoval - add prometheus-like duration as command flag - add option to delete stale snapshots - update duration.go flag to re-use own code * wip * lib/flagutil: re-use Duration.Set() call in NewDuration * wip Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2022-05-02 11:00:15 +03:00
Dima Lazerka	ed8e88af11	Export "null" in jsonl instead of NaN (#2518) ... * Export "null" in jsonl instead of NaN The NaN appeared because of staleness markers that were added for compatibility. I think it's better to use json `null`, implemented here. Also maybe it also makes sense to add a flag like `?skip-staleness-markers=true` to `/export`, to skip nulls at all? * Update app/vmselect/prometheus/export.qtpl * app/vmselect/prometheus/export.qtpl.go: `make quicktemplate-gen` * docs/CHANGELOG.md: document the change Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2022-04-29 12:49:47 +03:00
Aliaksandr Valialkin	f5635d6920	docs/CHANGELOG.md: document c7aad8d441	2022-04-29 11:39:23 +03:00
Aliaksandr Valialkin	cce1b6d7f9	app/vmselect/promql: add tlast_change_over_time(m[d]) function, which returns the timestamp for the last change of m on the given lookbehind window d	2022-04-27 10:59:03 +03:00
Yury Molodov	9b4bff67e0	vmui: add support relative time (#2504) ... * feat: add support relative time * app/vmselect: `make vmui-update` * docs/CHANGELOG.md: document the change Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2022-04-26 15:46:06 +03:00
Aliaksandr Valialkin	8594609385	docs/CHANGELOG.md: document 4c1fbcd6b0 ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2368	2022-04-26 15:10:00 +03:00
Aliaksandr Valialkin	5d87744ba2	docs/CHANGELOG.md: typo fix: may result -> could result	2022-04-23 00:31:22 +03:00
Aliaksandr Valialkin	eb5d7ad089	lib/promauth: add support for min_version option at tls_config section in the same way as Prometheus does	2022-04-23 00:16:39 +03:00
Aliaksandr Valialkin	174431e31b	lib/promauth: add support for proxy_url option at oauth2 section in the same way as Prometheus does	2022-04-23 00:00:44 +03:00
Aliaksandr Valialkin	18b14aad8e	lib/promauth: add support for tls_config section at oauth2 config in the same way as Prometheus does	2022-04-22 23:51:07 +03:00
Aliaksandr Valialkin	15190fcdae	lib/promscrape/discovery/kubernetes: allow attaching node-level labels and annotations to discovered pod targets in the same way as Prometheus 2.35 does ... See https://github.com/prometheus/prometheus/issues/9510 and https://github.com/prometheus/prometheus/pull/10080	2022-04-22 20:15:41 +03:00
Aliaksandr Valialkin	25fe83577d	app/vmselect/promql: properly handle scalar default vector, scalar if vector and scalar ifnot vector queries ... Previously `vector` time series could be unexpectedly returned from such queries	2022-04-21 15:34:36 +03:00
Aliaksandr Valialkin	de892239a9	app/vmselect/promql: add drop_common_labels() function	2022-04-21 14:20:20 +03:00
Aliaksandr Valialkin	98129d4a8e	app/vmstorage: expose vm_indexdb_items_added_total and vm_indexdb_items_added_size_bytes_total counters at /metrics page ... These counters can be used for monitoring the rate of addition of new entries in indexdb (aka inverted index). See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2471	2022-04-21 13:18:39 +03:00
Aliaksandr Valialkin	d335cc886c	app/vmselect/promql: fix duplicate time series error on joins against time series filtered by values ... This should prevent from `duplicate time series` errors when executing the following query: kube_pod_container_resource_requests{resource="cpu"} * on (namespace,pod) group_left() (kube_pod_status_phase{phase=~"Pending|Running"}==1) where `kube_pod_status_phase{phase=~"Pending|Running"}==1` filters out diplicate time series	2022-04-20 22:18:44 +03:00
Aliaksandr Valialkin	694887cea8	docs/CHANGELOG.md: document that the service discovery speed now scales with the number of CPU cores	2022-04-20 16:22:18 +03:00
Aliaksandr Valialkin	3d0549c982	lib/promscrape: optimize getScrapeWork() function ... Reduce the number of memory allocations in this function. This improves its performance by up to 50%. This should improve service discovery speed when big number of potential targets with big number of meta-labels are generated by service discovery. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2270	2022-04-20 15:37:00 +03:00
Aliaksandr Valialkin	f6d0e5e74a	all: typo fix: Kuberntes -> Kubernetes	2022-04-20 10:50:49 +03:00
Dmytro Kozlov	a3ee275149	lib/promscrape: Enable filters for endpoint and labels (#2466) ... * lib/promscrape: Enable filters for endpoint and labels * lib/promscrape: cleanup * lib/promscrape: update template * lib/promscrape: move logic filter logic to backend * lib/promscrape: updated placeholder * lib/promscrape: updated placeholder * lib/promscrape: use two different fields for filters, updated form, added error on parsing queries * lib/promscrape: rename functions * lib/promscrape: removed unused values * wip * wip * wip Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2022-04-19 18:26:21 +03:00
Nikolay	26b78ad707	lib/promscrape: adds job restart method (#2455) ... * lib/promscrape: adds job restart method it must restart only ScrapeConfig with changed content this change greatly reduce time, that needed for job restart and it should decrease possible data loss when config frequently changed at kubernetes based deployments Apply suggestions from code review Co-authored-by: Roman Khavronenko <roman@victoriametrics.com> * wip Co-authored-by: Roman Khavronenko <roman@victoriametrics.com> Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2022-04-16 20:28:46 +03:00
Yury Molodov	514e3660e2	fix: prevent graph hiding without data (#2456) ... * fix: prevent graph hiding without data * fix: add yaxis labels default * app/vmselect: `make vmui-update` * docs/CHANGELOG.md: document the change Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2022-04-16 17:15:31 +03:00
Aliaksandr Valialkin	cad488fe7e	app/vmstorage: add support for mTLS cipher suites via -cluster.tlsCipherSuites command-line flag ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2404	2022-04-16 16:39:21 +03:00
Aliaksandr Valialkin	7e4bdf31ba	lib/httpserver: follow up after def0032c7d	2022-04-16 15:27:21 +03:00
Aliaksandr Valialkin	ebaa1c7ad5	lib/promscrape: follow-up after baa1c24b36	2022-04-16 14:25:54 +03:00
Aliaksandr Valialkin	e6535a75f7	docs/CHANGELOG.md: document 45fcaa33e8	2022-04-13 14:12:17 +03:00
Aliaksandr Valialkin	77ffa4e447	docs/CHANGELOG.md: document f7e4c5a628	2022-04-13 14:09:11 +03:00
Anton Bystrov	9307fe3c04	Update CHANGELOG.md (#2463) ... May be mispint here?	2022-04-13 11:18:42 +03:00
Aliaksandr Valialkin	f8de318bfc	docs/CHANGELOG.md: cut v1.76.1	2022-04-12 16:20:55 +03:00
Aliaksandr Valialkin	ef66b048c9	app/vmui: further improvements for number display on graphs ... This is a follow-up for c4d2cd8336 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2409	2022-04-12 16:01:27 +03:00
Aliaksandr Valialkin	52cb80ed4f	docs/CHANGELOG.md: link to the bug related to improper handling of maxSeries limit passed from vmselect to vmstorage	2022-04-12 16:00:25 +03:00
Aliaksandr Valialkin	f3d4671bb6	lib/promscrape: follow-up after 7e79adfb55	2022-04-12 12:36:17 +03:00
Aliaksandr Valialkin	54de0531a4	app/vmstorage: properly handle maxSeries limit passed from vmselect to vmstorage	2022-04-12 11:23:04 +03:00
Aliaksandr Valialkin	deaa8c1ffa	lib/protoparser/native: follow-up after fe01f4803d	2022-04-11 19:27:07 +03:00
Aliaksandr Valialkin	d7bf0a7348	vendor: update github.com/VictoriaMetrics/metricsql from v0.40.0 to v0.41.0 ... This allows using built-in function names as with template names	2022-04-11 18:31:44 +03:00
Aliaksandr Valialkin	b89e846ce3	docs/CHANGELOG.md: document ed364a42e3	2022-04-11 12:11:32 +03:00
Aliaksandr Valialkin	fbd71f3083	docs/CHANGELOG.md: document backwards-incompatible changes in cluster version of v1.76.0	2022-04-08 12:05:45 +03:00
Aliaksandr Valialkin	dc60e99e94	docs/CHANGELOG.md: document the bugfix in hitCount function	2022-04-08 11:31:52 +03:00
Aliaksandr Valialkin	978f6d0f89	docs/CHANGELOG.md: typo fix	2022-04-07 17:19:59 +03:00
Aliaksandr Valialkin	ef690932ee	docs/CHANGELOG.md: cut v1.76.0	2022-04-07 15:33:55 +03:00
Aliaksandr Valialkin	7da20a4b3f	app/vmagent: reduce the probability of TLS handshake timeout when dialing the remote storage ... The following actions are taken: - Increase the TLS hashdshake timeout from 5 seconds to 10 seconds - Increase dial timeout from 5 seconds to 30 seconds - Specify DialContext instead of Dial in http.Transport. This allows properly handling the Context arg during dialing the remote storage Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1699	2022-04-06 12:34:25 +03:00
Aliaksandr Valialkin	7ecb72648d	docs/CHANGELOG.md: document 0c0efc7781	2022-04-05 19:21:49 +03:00
Aliaksandr Valialkin	4ecb86c179	app/vminsert: reduce the max packet size, which vminsert can send to vmstorage ... This reduces the max memory usage for vminsert and vmstorage under heavy ingestion rate by up to 50% on production workload	2022-04-05 15:43:07 +03:00
Aliaksandr Valialkin	d011446f6f	docs/CHANGELOG.md: document 70bb0d2708	2022-04-04 13:02:27 +03:00
Aliaksandr Valialkin	43df19a742	docs/CHANGELOG.md: document 173073364e1bb1e0259ddc873dbd96ce62b07543	2022-04-04 12:55:43 +03:00
Aliaksandr Valialkin	f977ca8eaf	docs/CHANGELOG.md: document a57e3807537914396ee3eb378648a464fa9e1b97	2022-04-01 12:24:49 +03:00
Aliaksandr Valialkin	1c38ff6f48	docs/CHANGELOG.md: document 0989649ad0	2022-04-01 12:01:34 +03:00
Aliaksandr Valialkin	e210384f7e	docs/CHANGELOG.md: cut v1.75.1	2022-03-28 12:28:48 +03:00
Yury Molodov	c8d29ed78e	vmui: predefined panels (#2243) ... * feat: add basic components for predefined dashboards * fix: change display alert * feat: add autosize and unit for axes * feat: add component for CircularProgress * feat: change layout for predefined dashboards * feat: add override step for predefined panels * feat: add override step for predefined panels * feat: change yaxis limits for predefined panels * fix: rename flag for hide legend * feat: add formatted panel description * feat: add README.md for dashboard setup * feat: validate dashboard settings * feat: add unit for y-ticks * fix: correct display error for dashboards * fix: disable auto refresh after route change * update package-lock.json * fix: add basename for BrowserRouter * fix: add dynamic basename for routing * update packages * feat: add a pre-defined dashboard "per-job resource usage" * feat: display unit in the hover-tooltip * fix: change routing and home layout * fix: change axis width calc * updated packages * app/vmselect: `make vmui-update` Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2022-03-26 13:03:11 +02:00
Aliaksandr Valialkin	6e364e19ef	app/vmselect: add fine-grained limits for the number of returned/scanned time series for various APIs	2022-03-26 11:29:49 +02:00
Dima Lazerka	1fa0f3ec89	VMAnomaly docs fixes (#2361) ... * Added docs for vmanomaly * Add example images * Stylistic fixes * Move images to root * Update docs/vmanomaly.md * Update docs/vmanomaly.md Co-authored-by: Roman Khavronenko <roman@victoriametrics.com> * Squeeze vmanomaly after vmbackupmanager before Case Studies Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com> Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>	2022-03-25 12:08:17 +02:00
Arash Hatami	a8de1ab000	A good change for MD files (#2353) ... * Lint YAML * Remove extra comment * Fix command problem * Format MD files * Format & fix problem of MD files for docs * Another fix for MD files	2022-03-22 13:40:55 +02:00
Roman Khavronenko	f367ff086c	docs: update release notes (#2349) ... Warn about memory issue introduced in releases 1.73 - 1.74	2022-03-21 15:40:50 +02:00
Aliaksandr Valialkin	5ab6c350ec	docs/CHANGELOG.md: document a1e17e91f8	2022-03-21 15:34:49 +02:00
hagen1778	82659ab5b6	docs: add update note to v1.75.0 release note ... Signed-off-by: hagen1778 <roman@victoriametrics.com>	2022-03-19 17:55:38 +01:00
Aliaksandr Valialkin	87d356348b	docs/CHANGELOG.md: cut v1.75.0	2022-03-18 19:53:45 +02:00
Aliaksandr Valialkin	e78f3ac8ac	app/vmselect/bufferedwriter: suppress trivial network errors, which can be generated by remote side ... These errors include `broken pipe` and `reset by peer`. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2334	2022-03-18 19:28:02 +02:00
Aliaksandr Valialkin	ec03dec72d	app/vmagent/remotewrite: prevent from infinite recursion panic when pushing a time series with big number of samples to remote storage ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2335	2022-03-18 19:06:22 +02:00
Aliaksandr Valialkin	620b605786	docs: document 20bb5e703c	2022-03-18 18:41:19 +02:00
Aliaksandr Valialkin	2ae3a9a8a3	lib/storage: reduce the interval for checking for free disk space from 30 seconds to 1 second ... This should reduce the probability of out of disk space panics when -storage.minFreeDiskSpaceBytes is set to low values. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2305	2022-03-18 16:52:27 +02:00
Aliaksandr Valialkin	88605a7ea2	lib/blockcache: properly release memory occupied by deleted entries ... Proviously the deleted entries could remain referenced via lastAccessHeap for long time. This could lead to increased memory usage for the following caches starting from v1.73.0: * indexdb/indexBlocks * indexdb/dataBlocks * storage/indexBlocks Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2242 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2007	2022-03-18 16:52:27 +02:00
Aliaksandr Valialkin	db781a9342	docs/CHANGELOG.md: document e5868b9c29 ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/546 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2255	2022-03-18 13:07:32 +02:00
Aliaksandr Valialkin	e6acd16daf	docs/CHANGELOG.md: document 11ae1ae924	2022-03-17 20:08:05 +02:00
Aliaksandr Valialkin	11869a8307	docs: document the addition of mTLS communication between cluster components	2022-03-17 20:00:56 +02:00
Aliaksandr Valialkin	8ae9825bb4	docs/CHANGELOG.md: document c1d07e7c52f0a2ab892921b0639cd42677aa33a8	2022-03-16 14:25:44 +02:00
Aliaksandr Valialkin	54ec080bbc	docs/CHANGELOG.md: document changes from fb6eab03a2	2022-03-16 13:21:57 +02:00
Aliaksandr Valialkin	370024c7ed	docs/CHANGELOG.md: document 565bd08c43 ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1824	2022-03-16 12:50:49 +02:00
hagen1778	894416b4ca	docs: add update details for some releases ... Some of the releases could negatively affect performance for a limited period of time due to some changes in core. Update details are meant to warn users about expected changes in peformance after the update. Signed-off-by: hagen1778 <roman@victoriametrics.com>	2022-03-15 07:06:48 +03:00
Roman Khavronenko	0fa7effc4b	docs: fix broken links (#2303) ... Signed-off-by: hagen1778 <roman@victoriametrics.com>	2022-03-13 15:56:01 +02:00
Aliaksandr Valialkin	74bb9ea734	docs/CHANGELOG.md: cut v1.74.0	2022-03-03 19:30:41 +02:00
Aliaksandr Valialkin	ce8d28f8f4	docs/CHANGELOG.md: document performance improvements when registering new time series	2022-03-03 17:11:30 +02:00
Aliaksandr Valialkin	e757ebc58b	app/vmselect/netstorage: report vmstorage errors to vmselect clients even if partial responses are allowed ... If a vmstorage is reachable and returns an application-level error to vmselect, then such error must be returned to the caller even if partial responses are allowed, since it usually means cluster mis-configuration. Partial responses may be returned only if some vmstorage nodes are temporarily unavailable. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1941 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/678	2022-02-25 13:56:42 +02:00
Nikolay	9fe2e4e2c2	fixes incorrect step for calculation for MovingWindow functions (#283) ... * fixes incorrect step for calculation for MovingWindow functions https://victoriametrics.zendesk.com/agent/tickets/99 * wip * wip Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2022-02-25 13:54:03 +02:00
Aliaksandr Valialkin	0d79c8cbef	docs/CHANGELOG.md: document cfc6c14dc48ae9dd35e65f1a6e5c7af8ccb9f029	2022-02-25 13:52:48 +02:00
Aliaksandr Valialkin	a16f1ae565	lib/storage: properly handle series selector matching multiple metric names plus a negative filter ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2238 This is a follow-up for 00cbb099b6	2022-02-24 12:15:54 +02:00
Aliaksandr Valialkin	3f49bdaeff	lib/promrelabel: add support for conditional relabeling via if filter ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1998	2022-02-24 02:27:26 +02:00
Nikolay	fbac1a9dad	fixes jwt token parse with correct base64Url decoding (#281) ... * fixes jwt token parse with correct base64Url decoding it must be applied according to jwt RFC that requires token to be URL safe added slow path for decoding tokens with std base64 decoding adds error logging for vmgateway * docs/CHANGELOG.md: document the bugfix Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2022-02-23 13:57:36 +02:00
Aliaksandr Valialkin	62b46007c5	lib/workingsetcache: reduce the default cache rotation period from hour to 20 minutes ... This should reduce memory usage under high time series churn rate	2022-02-23 13:41:45 +02:00
Aliaksandr Valialkin	acbea6c1ee	docs/CHANGELOG.md: cut v1.73.1	2022-02-22 21:11:42 +02:00
Aliaksandr Valialkin	0a38542a45	docs/CHANGELOG.md: link to the feature request for X-Influxdb-Version response header ... Follow-up for 71ef3155c8 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2209	2022-02-22 20:35:53 +02:00
Roman Khavronenko	69d1893f4c	Consul SD - update services on the watcher's start (#2202) ... * lib/discovery/consul: update services on the watcher's start Previously, watcher's start was only initing goroutines for discovery but not waiting for the first iteration to end. It means first Consul discovery wasn't returning discovered targets until the next iteration. The change makes the watcher's start blocking until we get first discovery iteration done and all registries updated. Signed-off-by: hagen1778 <roman@victoriametrics.com> * vmalert: remove workarounds for consul SD Now when consul SD lib properly updates services on the first start, we don't need workarounds in vmalert. Signed-off-by: hagen1778 <roman@victoriametrics.com> * lib/discovery/consul: update after review Signed-off-by: hagen1778 <roman@victoriametrics.com> * wip Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2022-02-21 15:32:45 +02:00
Roman Khavronenko	b6ed9afd6d	lib: allow to configure cache size by type (#2206) ... * lib: allow to configure cache size by type https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1940 Signed-off-by: hagen1778 <roman@victoriametrics.com> * Apply suggestions from code review * wip Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2022-02-21 13:50:34 +02:00
Aliaksandr Valialkin	71ef3155c8	app/vminsert: add X-Influxdb-Version response header for InfluxDB API requests ... This is needed for some clients, which expect this header. See https://github.com/ntop/ntopng/issues/5449#issuecomment-1005347597	2022-02-17 12:47:43 +02:00
Aliaksandr Valialkin	3c3805865b	docs: document 3d19fa6932	2022-02-16 23:30:17 +02:00
Aliaksandr Valialkin	b71be42d90	lib/storage: use binary search instead of full scan for skipping artificial tags when searching for tag names or tag values ... This should improve performance for /api/v1/labels and /api/v1/label/<label_name>/values See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2200	2022-02-16 18:15:41 +02:00
Aliaksandr Valialkin	424121c126	docs/CHANGELOG.md: document 2efa46a11c	2022-02-15 21:11:56 +02:00
Aliaksandr Valialkin	0f9e107e36	docs/CHANGELOG.md: document ad6bdd78d0	2022-02-15 12:47:55 +02:00
Aliaksandr Valialkin	ac502785b6	docs/CHANGELOG.md: cut v1.73.0	2022-02-14 17:54:32 +02:00
Aliaksandr Valialkin	1215f51043	docs/CHANGELOG.md: document 3d890e89f1	2022-02-14 17:39:12 +02:00
Nikolay	75e84144c7	adds release build for macos darwin amd64 and arm64 (#2185) ... * adds release build for macos darwin amd64 and arm64 https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1896 https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1851 * wip Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2022-02-14 17:28:56 +02:00
Aliaksandr Valialkin	578a37aa14	docs/CHANGELOG.md: document c90c1c4d54	2022-02-14 13:09:12 +02:00
Aliaksandr Valialkin	f10c38b827	lib/promscrape: add expand all and collapse all buttons to /targets page	2022-02-12 18:41:29 +02:00
Aliaksandr Valialkin	b1f94f7f0e	app/vmselect/promql: return at most one time series from absent_over_time() in the same way as Prometheus does ... See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2130	2022-02-12 15:45:09 +02:00
Aliaksandr Valialkin	d8ffbf55a2	docs/CHANGELOG.md: document ea153e5f90	2022-02-12 00:48:06 +02:00
Roman Khavronenko	cf1a8bce6b	lib/index: reduce read/write load after indexDB rotation (#2177) ... * lib/index: reduce read/write load after indexDB rotation IndexDB in VM is responsible for storing TSID - ID's used for identifying time series. The index is stored on disk and used by both ingestion and read path. IndexDB is stored separately to data parts and is global for all stored data. It can't be deleted partially as VM deletes data parts. Instead, indexDB is rotated once in `retention` interval. The rotation procedure means that `current` indexDB becomes `previous`, and new freshly created indexDB struct becomes `current`. So in any time, VM holds indexDB for current and previous retention periods. When time series is ingested or queried, VM checks if its TSID is present in `current` indexDB. If it is missing, it checks the `previous` indexDB. If TSID was found, it gets copied to the `current` indexDB. In this way `current` indexDB stores only series which were active during the retention period. To improve indexDB lookups, VM uses a cache layer called `tsidCache`. Both write and read path consult `tsidCache` and on miss the relad lookup happens. When rotation happens, VM resets the `tsidCache`. This is needed for ingestion path to trigger `current` indexDB re-population. Since index re-population requires additional resources, every index rotation event may cause some extra load on CPU and disk. While it may be unnoticeable for most of the cases, for systems with very high number of unique series each rotation may lead to performance degradation for some period of time. This PR makes an attempt to smooth out resource usage after the rotation. The changes are following: 1. `tsidCache` is no longer reset after the rotation; 2. Instead, each entry in `tsidCache` gains a notion of indexDB to which they belong; 3. On ingestion path after the rotation we check if requested TSID was found in `tsidCache`. Then we have 3 branches: 3.1 Fast path. It was found, and belongs to the `current` indexDB. Return TSID. 3.2 Slow path. It wasn't found, so we generate it from scratch, add to `current` indexDB, add it to `tsidCache`. 3.3 Smooth path. It was found but does not belong to the `current` indexDB. In this case, we add it to the `current` indexDB with some probability. The probability is based on time passed since the last rotation with some threshold. The more time has passed since rotation the higher is chance to re-populate `current` indexDB. The default re-population interval in this PR is set to `1h`, during which entries from `previous` index supposed to slowly re-populate `current` index. The new metric `vm_timeseries_repopulated_total` was added to identify how many TSIDs were moved from `previous` indexDB to the `current` indexDB. This metric supposed to grow only during the first `1h` after the last rotation. https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1401 Signed-off-by: hagen1778 <roman@victoriametrics.com> * wip * wip Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2022-02-12 00:30:08 +02:00
Roman Khavronenko	e3adcbec6e	lib/promscrape: support prometheus-like duration in scrape configs (#2169) ... * lib/promscrape: support prometheus-like duration in scrape configs The change allows to specify duration values like `1d`, `1w` for fields `scrape_interval`, `scrape_timeout`, etc. https://github.com/VictoriaMetrics/VictoriaMetrics/issues/817#issuecomment-1033384766 Signed-off-by: hagen1778 <roman@victoriametrics.com> * lib/blockcache: make linter happy Signed-off-by: hagen1778 <roman@victoriametrics.com> * lib/promscrape: support prometheus-like duration in scrape configs * add support for extra fields `scrape_align_interval` and `scrape_offset`; * support Prometheus duration parsing for `__scrape_interval__` and `__scrape_duration__` labels; Signed-off-by: hagen1778 <roman@victoriametrics.com> * wip * wip * docs/CHANGELOG.md: document the feature Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2022-02-11 16:17:00 +02:00
Aliaksandr Valialkin	3cb72ccc2a	lib/promscrape/discovery/kubernetes: add __meta_kubernetes_endpointslice_{label,annotation}* labels to be consistent with other role values for Kubernetes service discovery	2022-02-11 14:54:47 +02:00
Nikolay	4e7f7f3302	fixes service discovery for kubernetes (#2173) ... * fixes service discovery for kubernetes now it must take in account all pods that belong to the discovered endpoint and endpointslice adds simple test for endpoints https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2134 * wip * docs/CHANGELOG.md: document the change Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2022-02-11 13:34:22 +02:00
Aliaksandr Valialkin	8f2d03fdc7	docs/CHANGELOG.md: document 4e722c459b ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2167	2022-02-10 12:20:12 +02:00
Aliaksandr Valialkin	0028b2c6d1	docs/CHANGELOG.md: add instructions on how to build VictoriaMetrics components from source code in order to test tip changes	2022-02-08 16:44:08 +02:00
Nikolay	a8acad7453	adds CGO build for arm64 (#2102) ... * adds CGO build for arm64 it must improve performance for arm64 based deployments of vmstorage and vmsingle for 15-20% it depends on gozstd package update for correct musl gozstd vendoring * typo fixes * docs/CHANGELOG.md: document the change Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2022-02-08 16:25:59 +02:00
Aliaksandr Valialkin	9bb60ab00f	lib/promscrape: set -promscrape.config.strictParse to true by default ... This allows detecting long-living silent errors in -promscrape.config	2022-02-08 15:41:43 +02:00
Aliaksandr Valialkin	8b36044c93	docs/CHANGELOG.md: add links to issues, which could benefit from improved re-routing algorithm	2022-02-07 16:55:28 +02:00
Aliaksandr Valialkin	865f09ecbb	docs: sync with cluster branch	2022-02-07 14:54:16 +02:00
Aliaksandr Valialkin	b5b3c585b3	lib/promscrape: show the total number of scrapes and the total number of scrape errors per target at /targets page ... This information may be useful when debugging unreliable scrape targets	2022-02-03 20:22:41 +02:00
Aliaksandr Valialkin	2968779f16	lib/promscrape: provide the ability to fetch target responses on behalf of vmagent or single-node VictoriaMetrics ... This feature may be useful when debugging metrics for the given target located in isolated environment	2022-02-03 19:00:55 +02:00
Aliaksandr Valialkin	4b850c2a59	app/vmselect/promql: do not push down filters, which enumerate more than 10k unique values ... Such filters may slow down time series search, so just skip them. This is a follow-up for e7f1ceeb84 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1827	2022-02-02 23:40:02 +02:00
Aliaksandr Valialkin	4ef32df4fa	docs/CHANGELOG.md: document 55e3bbd4cc ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1567	2022-02-02 23:32:35 +02:00
Aliaksandr Valialkin	6530bcedec	docs: updates after 5da71eb685 ... * Mention about the ability to configure vmalert notifiers via files in docs/CHANGELOG.md * Mention about the ability to use Consul service discovery for vmalert notifiers in docs/CHANGELOG.md * Run `make docs-sync` in order to sync app/vmalert/README.md to docs/vmalert.md	2022-02-02 23:26:18 +02:00
Aliaksandr Valialkin	ead66155ef	lib/cgroup: expose process_cpu_cores_available metric ... This metric shows the number of CPU cores available to the process. This allows creating alerting rules on CPU saturation with the following query: rate(process_cpu_seconds_total[5m]) / process_cpu_cores_available > 0.9 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2107	2022-01-31 20:24:41 +02:00
Aliaksandr Valialkin	e7f1ceeb84	app/vmselect/promql: optimize queries, which join on _info metrics. ... Automatically add common filters from one side of binary operation to the other side before sending the query to storage subsystem. See https://grafana.com/blog/2021/08/04/how-to-use-promql-joins-for-more-effective-queries-of-prometheus-metrics-at-scale/ and https://www.robustperception.io/exposing-the-software-version-to-prometheus	2022-01-31 19:32:36 +02:00
Aliaksandr Valialkin	3d8a4bf023	docs/CHANGELOG.md: document 6a519896db	2022-01-31 12:41:57 +02:00
Aliaksandr Valialkin	e02e0508da	vendor: update github.com/VictoriaMetrics/metricsql from v0.37.0 to v0.38.0 ... This adds more optimization cases for https://utcc.utoronto.ca/~cks/space/blog/sysadmin/PrometheusLabelNonOptimization For example: * Multi-level transform functions. For example, abs(round(foo{a="b"})) + bar{x="y"} is now optimized to abs(round(foo{a="b",x="y"})) + bar{a="b",x="y"} * Binary operations with `on()`, `without()`, `group_left()` and `group_right()` modifiers. For example, foo{a="b"} on (a) + bar is now optimized to foo{a="b"} on (a) + bar{a="b"} * Multi-level binary operations. For example, foo{a="b"} + bar{x="y"} + baz{z="q"} is now optimized to foo{a="b",x="y",z="q"} + bar{a="b",x="y",z="q"} + baz{a="b",x="y",z="q"} * Aggregate functions. For example, sum(foo{a="b"}) by (c) + bar{c="d"} is now optimized to sum(foo{a="b",c="d"}) by (c) + bar{c="d"}	2022-01-27 19:03:54 +02:00
Aliaksandr Valialkin	746ee191e8	lib/logger/throttler.go: show the original location of the error and warning message ... Previously the location inside LogThrottler implementation was shown. This could complicate debugging.	2022-01-23 13:55:00 +02:00
Yury Molodov	ad5059f2d3	vmui: fixed display type switching (#2088) ... * fix: correct switch display type * docs/CHANGELOG.md: document the bugfix Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2022-01-21 16:56:22 +02:00
Yury Molodov	adbb821eac	vmui: fix time range selector (#2085) ... * fix: add date validate for time range * app/vmselect/vmui: `make vmui-update` * docs/CHANGELOG.md: document the bugfix Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2022-01-21 12:02:38 +02:00
Aliaksandr Valialkin	145337792d	lib/{mergeset,storage}: properly limit cache sizes for indexdb ... Previously these caches could exceed limits set via `-memory.allowedPercent` and/or `-memory.allowedBytes`, since limits were set independently per each data part. If the number of data parts was big, then limits could be exceeded, which could result to out of memory errors. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2007	2022-01-20 18:37:17 +02:00
Aliaksandr Valialkin	84f6b3014c	docs/CHANGELOG.md: document the bugfix for highestMax() function is Graphite render API	2022-01-20 12:16:18 +02:00
Aliaksandr Valialkin	109363de49	docs/CHANGELOG.md: add missing parens in example for @ modifier	2022-01-19 13:04:51 +02:00
Aliaksandr Valialkin	98edeac7b7	docs/CHANGELOG.md: fix incorrect link to the issue ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1911	2022-01-19 00:06:44 +02:00
Aliaksandr Valialkin	8f5902dfcf	docs/CHANGELOG.md: cut v1.72.0	2022-01-18 22:43:18 +02:00
Yury Molodov	8bdc45ba00	fix: remove buffer period (#2078) ... * fix: remove buffer period * app/vmselect/vmui: `make vmui-update` * docs/CHANGELOG.md: document the implemented feature Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2064 Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2022-01-18 21:42:56 +02:00
Yury Molodov	70737ea4ac	vmui: correct url encoding (#2067) ... * fix: correct encode multi-line queries * fix: change autocomplete for correct arrows work * app/vmselect/vmui: `make vmui-update` * docs/CHANGELOG.md: document the bugfix for https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2039 Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2022-01-18 21:31:46 +02:00
Aliaksandr Valialkin	dcadec65b6	docs/CHANGELOG.md: document 8e3f9c1fbb	2022-01-18 21:24:49 +02:00
Aliaksandr Valialkin	e933e3150d	docs/CHANGELOG.md: document fcd33fc409	2022-01-18 12:46:33 +02:00
Aliaksandr Valialkin	c2a3911bb5	docs/CHANGELOG.md: document that vmgateway now supports extra_filters option ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1863	2022-01-18 12:39:25 +02:00
Aliaksandr Valialkin	1bdc71d917	app/vmselect/promql: implement keep_metric_names modifier for transform and rollup functions ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/949	2022-01-14 04:14:59 +02:00
Aliaksandr Valialkin	f41846d002	app/vmselect/promql: add stale_samples_over_time() function	2022-01-14 01:48:04 +02:00
Aliaksandr Valialkin	96707223db	docs/CHANGELOG.md: yet another attempt to fix formatting for yaml snippet	2022-01-14 01:14:55 +02:00
Aliaksandr Valialkin	d7d83d6d93	docs/CHANGELOG.md: fix formatting for scrape_configs example	2022-01-14 01:02:32 +02:00
Aliaksandr Valialkin	1d05444b33	lib/promscrape: expose promscrape_stale_samples_created_total metric for monitoring the number of created stale samples	2022-01-14 01:00:46 +02:00
Aliaksandr Valialkin	831b93a755	app/vmalert: add parseDuration function in the same way as Prometheus does ... See https://github.com/prometheus/prometheus/pull/8817	2022-01-13 23:30:41 +02:00
Aliaksandr Valialkin	80f03177c4	lib/promscrape/discovery/kubernetes: add __meta_kubernetes_node_provider_id label for discovered Kubernetes nodes in the same way as Prometheus does ... See https://github.com/prometheus/prometheus/pull/9603	2022-01-13 23:16:02 +02:00
Aliaksandr Valialkin	80f966b80c	app/vmalert: add stripPort template function in the same way as Prometheus does ... See https://github.com/prometheus/prometheus/pull/10002	2022-01-13 22:53:42 +02:00
Aliaksandr Valialkin	355a63733d	lib/promscrape/discovery/kubernetes: add the ability to limit service discovery to the current namespace ... See https://github.com/prometheus/prometheus/issues/9782 and https://github.com/prometheus/prometheus/pull/9881	2022-01-13 22:44:35 +02:00
Aliaksandr Valialkin	c883c15878	app/vmselect/promql: add support for @ modifier ... Add support for `@` modifier in MetricsQL according to https://prometheus.io/docs/prometheus/latest/querying/basics/#modifier Extend the support with the following features: * Allow using `@` modifier everywhere in the query. For example, `sum(foo) @ end()` * Allow using arbitrary expression as `@` modifier. For example, `foo @ (end() - 1h)` returns `foo` value at `end - 1 hour` timestamp on the selected time range `[start ... end]` Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1348	2022-01-13 22:12:06 +02:00
Aliaksandr Valialkin	17eb86a689	lib/promscrape/discovery/dockerswarm: follow up after 68a117a25a ... - Document the bugfix at docs/CHANGELOG.md - Set __address__ field after copying commonLabels to the resulting map of discovered labels. This makes sure that the correct __address__ label is used.	2022-01-11 09:20:10 +02:00
Aliaksandr Valialkin	7a2c46d951	docs/CHANGELOG.md: document 77bfa8181d	2022-01-11 08:53:49 +02:00
Aliaksandr Valialkin	bdba50432b	docs/CHANGELOG.md: add release dates for every release	2022-01-07 12:27:32 +02:00
Aliaksandr Valialkin	e4e36383e2	lib/promscrape: do not send staleness markers on graceful shutdown ... This follows Prometheus behavior. See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2013#issuecomment-1006994079	2022-01-07 01:17:57 +02:00
Denys Holius	ae89b4e818	Old links replaced for newest (#2033) ... * replaced old links to the website * fixed deletion main README.md file * fix: added docs files after docs-sync	2022-01-05 16:30:13 +02:00
Aliaksandr Valialkin	dbe592597f	docs/CHANGELOG.md: clarify the issue description, which is fixed by 38bf5fc136	2022-01-05 16:19:06 +02:00
Aliaksandr Valialkin	178dd87e26	lib/storage: follow-up for 38bf5fc136	2022-01-05 16:00:11 +02:00
Aliaksandr Valialkin	e1d7cbfc77	docs/CHANGELOG.md: document 60266078ca	2022-01-04 11:44:57 +02:00
Aliaksandr Valialkin	5ce94e1dd3	docs/CHANGELOG.md: document ac47733044	2022-01-03 21:15:44 +02:00
Aliaksandr Valialkin	b5ce35dfc8	docs/CHANGELOG.md: document 543bd0ea0c ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1999	2021-12-23 12:08:06 +02:00
Aliaksandr Valialkin	cbaa2af280	lib/promscrape: scrape replicated targets at different offsets in vmagent replicated clustering mode ... This guarantees that the deduplication consistently leaves samples from the same vmagent replica. See https://docs.victoriametrics.com/vmagent.html#scraping-big-number-of-targets	2021-12-23 00:20:39 +02:00
Nikolay	8ff7da7202	adds restore.lock (#1988) ... * adds restore.lock it must prevent from running storage after incomplete restore process https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1958 * return back flock file deletion * Apply suggestions from code review * wip * docs/CHANGELOG.md: document https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1958 Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2021-12-22 13:10:15 +02:00
Aliaksandr Valialkin	3cfb90b227	docs/CHANGELOG.md: document 34fdc8881b ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1911	2021-12-21 16:40:39 +02:00
Aliaksandr Valialkin	4701c108ff	docs/CHANGELOG.md: cut v1.71.0	2021-12-20 19:10:24 +02:00
Aliaksandr Valialkin	974d9c0eee	app/vmselect/promql: follow-up after 177e345d8a ... * Document changes_prometheus(), increase_prometheus() and delta_prometheus() functions. * Simplify their implementation * Mention these functions in docs/CHANGELOG.md	2021-12-20 13:19:44 +02:00
Aliaksandr Valialkin	16636a458f	docs/CHANGELOG.md: document 6814cc6809 ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1943	2021-12-17 20:16:22 +02:00
Aliaksandr Valialkin	193331d522	app/vmselect: de-duplicate data exported via /api/v1/export/csv by default ... Previously the exported data wasn't de-duplicated. Now it is possible to export the raw data without deduplication by passing reduce_mem_usage=1 query arg to /api/v1/export/csv See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1837	2021-12-17 10:57:39 +02:00
Aliaksandr Valialkin	a3adf24527	lib/promscrape: allow up to 5 redirects when scraping a target by default ... See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1945	2021-12-16 00:14:14 +02:00
Aliaksandr Valialkin	5efe377a26	app/vmselect/promql: add timestamp_with_name(m[d]) function ... This function works the same as `timestamp()`, but doesn't remove source time series names. See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/949#issuecomment-995222388	2021-12-15 23:37:07 +02:00
Aliaksandr Valialkin	8e1a87491a	docs: document the added dowsnampling support in VictoriaMetrics enterprise ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/36	2021-12-15 16:25:51 +02:00
Aliaksandr Valialkin	4ff647137a	lib/storage: deduplicate samples more thoroughly ... Previously some duplicate samples may be left on disk for time series with high churn rate. This may result in higher disk space usage.	2021-12-15 15:59:58 +02:00
Aliaksandr Valialkin	e1a715b0f5	lib/storage: convert alternate regexps into Graphite wildcards inside __graphite__ pseudo-label ... For example, `{__graphite__=~"foo.(bar|baz)"}` is automatically converted to `{__graphite__=~"foo.{bar,baz}"}` before execution. This allows using multi-value Grafana template variables such as `{__graphite__=~"foo.($app)"}`.	2021-12-14 19:51:49 +02:00
Aliaksandr Valialkin	d456af7499	docs/CHANGELOG.md: link to the issue about unaligned 64-bit atomic opertion panic on 32-bit architectures	2021-12-14 15:00:10 +02:00
Aliaksandr Valialkin	80996d916b	docs/CHANGELOG.md: an attempt to properly show $labels.alertname at https://docs.victoriametrics.com/CHANGELOG.html	2021-12-14 14:56:57 +02:00
Yury Molodov	49e6a921df	vmui: custom step (#1942) ... * feat: add a label for the Query field * fix: change zoom position * fix: add description and error code to alerts * fix: correct logic query history * fix: correct update query history * feat: add custom step * update package-lock.json * docs: document that VMUI now supports overriding of `step` query arg, which is passed to `/api/v1/query_range` Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2021-12-14 14:51:45 +02:00
Yury Molodov	b5b701d590	vmui: minor fixes (#1936) ... * feat: add a label for the Query field * fix: change zoom position * fix: add description and error code to alerts * fix: correct logic query history * fix: correct update query history * app/vmselect/vmui: `make vmui-update` * docs/CHANGELOG.md: document bugfixes Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2021-12-13 13:42:37 +02:00
Aliaksandr Valialkin	c82cc9cd11	docs/CHANGELOG.md: document 7c3b6365f0	2021-12-12 19:09:20 +02:00
Aliaksandr Valialkin	d1f8915ed1	app/vmselect/promql: preserve the order of time series passed to limit_offset() function ... Previously time series passed to `limit_offset()` were shuffled according to hash for their labels. This was unexpected behaviour for most users. See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1920 and https://github.com/VictoriaMetrics/VictoriaMetrics/issues/951	2021-12-12 18:04:58 +02:00
Aliaksandr Valialkin	3d4349343d	docs/CHANGELOG.md: document 2851709745 ... See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1921	2021-12-10 12:13:34 +02:00
Aliaksandr Valialkin	52eb9c99e2	docs: document the ability to investigate correlation between two queries at /vmui ... This is a follow-up for c1fd93e8a0	2021-12-08 17:24:01 +02:00
Aliaksandr Valialkin	ff15a752c1	app/vmselect: accept optional extra_filters[] query args for all the supported Prometheus querying APIs ... See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1863	2021-12-06 17:07:09 +02:00
Aliaksandr Valialkin	45d082bbe2	app/vminsert: add -maxLabelValueLen command-line flag ... See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1908	2021-12-06 11:40:34 +02:00
Aliaksandr Valialkin	da402fbdfa	lib/workingsetcache: fix unaligned 64-bit atomic operation panic on 32-bit architectures ... The panic has been introduced in 7275ebf91a	2021-12-03 01:21:51 +02:00
Aliaksandr Valialkin	06642d97f5	app: allow specifying http and https urls in the following command-line flags ... * -promscrape.config * -relabelConfig * -remoteWrite.relabelConfig * -remoteWrite.urlRelabelConfig	2021-12-03 00:10:02 +02:00
Aliaksandr Valialkin	62b4efb3e7	app/vmauth: follow-up for 13368bed18 ... * Document the ability to specify http or https urls in `-auth.config` at docs/CHANGELOG.md * Move the ReadFileOrHTTP to lib/fs, so it can be re-used in other places where a file should be read from the given path. For example, in `-promscrape.config` at `vmagent`.	2021-12-02 23:32:05 +02:00
Aliaksandr Valialkin	7f2f26b25f	docs/CHANGELOG.md: cut v1.70.0	2021-12-02 15:01:25 +02:00
Aliaksandr Valialkin	ed9ef7733b	docs/CHANGELOG.md: document 0afd14a14a	2021-12-02 14:47:20 +02:00
Aliaksandr Valialkin	394a345ae0	lib/httpserver: expose /-/healthy and /-/ready endpoints as Prometheus does ... This improves integration with third-party solutions, which rely on these endpoints. See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1833	2021-12-02 14:36:58 +02:00
Aliaksandr Valialkin	03f5ad3060	lib/protoparser/graphite: allow multiple separators between metric name, value and timestamp	2021-12-02 13:43:49 +02:00
Aliaksandr Valialkin	49a18b8660	lib/protoparser/graphite: properly parse Graphite line with whitespace after the timestamp ... See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1865	2021-12-02 13:33:26 +02:00
Aliaksandr Valialkin	c0cbf0de2a	app/{vmbackup,vmrestore}: export internal metrics at /metrics http handler	2021-12-02 11:55:58 +02:00
Aliaksandr Valialkin	7275ebf91a	app/vmstorage: export vm_cache_size_max_bytes metrics for determining capacity of various caches ... The vm_cache_size_max_bytes metric can be used for determining caches which reach their capacity via the following query: vm_cache_size_bytes / vm_cache_size_max_bytes > 0.9	2021-12-02 10:30:43 +02:00
Aliaksandr Valialkin	2f63dec2e3	lib/fs: add vm_filestream_read_duration_seconds_total and vm_filestream_write_duration_seconds_total metrics ... These metrics help determining persistent disk saturation with `rate(vm_filestream_read_duration_seconds_total) > 0.9`	2021-12-02 10:30:42 +02:00
Aliaksandr Valialkin	ed54b27b85	docs/CHANGELOG.md: document 06eff5a72c ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/487	2021-12-01 12:28:06 +02:00
Aliaksandr Valialkin	10bd8b1d86	docs/CHANGELOG.md: document 852a895b70	2021-11-30 01:22:37 +02:00
Aliaksandr Valialkin	ca6fc0265e	docs/CHANGELOG.md: document f05cddd2fc ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1830	2021-11-30 01:15:41 +02:00
Aliaksandr Valialkin	841647643a	docs/CHANGELOG.md: document 9dd650f67f	2021-11-30 01:09:10 +02:00
Aliaksandr Valialkin	7e79fc6e3c	docs/CHANGELOG.md: document 624ad73705	2021-11-30 01:05:05 +02:00