121 commits

Author	SHA1	Message	Date
Nikolay	0c0efc7781	vmctl verify-blocks command (#2390) ... * lib/protoparser: changes ParseStream for native format uses reader instead of http.Request updates app/vmagent and app/vmagent method usage * app/vmctl: add verify-block subcommand it allows to check exported from VictoriaMetrics data block in native format https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2362 Update app/vmctl/README.md Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>	2022-04-05 16:01:32 +02:00
Aliaksandr Valialkin	71ef3155c8	app/vminsert: add X-Influxdb-Version response header for InfluxDB API requests ... This is needed for some clients, which expect this header. See https://github.com/ntop/ntopng/issues/5449#issuecomment-1005347597	2022-02-17 12:47:43 +02:00
Eugene	3d19fa6932	Add -influxDBLabel parameter (#2203)	2022-02-16 23:27:06 +02:00
Aliaksandr Valialkin	2968779f16	lib/promscrape: provide the ability to fetch target responses on behalf of vmagent or single-node VictoriaMetrics ... This feature may be useful when debugging metrics for the given target located in isolated environment	2022-02-03 19:00:55 +02:00
Aliaksandr Valialkin	45d082bbe2	app/vminsert: add -maxLabelValueLen command-line flag ... See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1908	2021-12-06 11:40:34 +02:00
Aliaksandr Valialkin	06642d97f5	app: allow specifying http and https urls in the following command-line flags ... * -promscrape.config * -relabelConfig * -remoteWrite.relabelConfig * -remoteWrite.urlRelabelConfig	2021-12-03 00:10:02 +02:00
Aliaksandr Valialkin	e5ac9d8e57	all: consistently return application/json content-type without charset=utf-8 ... The `application/json` content-type has utf-8 encoding by default. See https://stackoverflow.com/questions/9254891/what-does-content-type-application-json-charset-utf-8-really-mean Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/897	2021-11-09 18:04:44 +02:00
Aliaksandr Valialkin	d1eb87c831	app/{vmagent,vminsert}: add ability to restrict access to /config page with authKey query arg ... The authKey can be configured via `-configAuthKey` command-line flag. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1764	2021-11-01 16:44:54 +02:00
Aliaksandr Valialkin	5a58c041c2	app/vmagent: expose -promscrape.config contents at /config page as Prometheus does ... See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1695	2021-10-12 16:25:37 +03:00
Aliaksandr Valialkin	6d9f1d4227	app/vminsert: document that -relabelConfig is reloaded on SIGHUP signal	2021-09-29 21:18:58 +03:00
Aliaksandr Valialkin	91b3c601bc	app/{vminsert,vmagent}: add ability to ingest data via DataDog "submit metrics" API ... See https://docs.datadoghq.com/api/latest/metrics/#submit-metrics Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/206	2021-09-29 00:13:08 +03:00
Nikolay	ad08d9dfc0	changes protoparser apis for accepting reading from io.Reader (#1624) ... adds InsertHandlerForReader apis to vmagent	2021-09-20 14:49:28 +03:00
Aliaksandr Valialkin	7f0a8d4bdb	docs: consistency renaming: Influx -> InfluxDB	2021-09-13 17:05:16 +03:00
Aliaksandr Valialkin	ed944313b0	app/{vminsert,vmselect}: export vminsert_request_duration_seconds and vmselect_request_duration_seconds histograms	2021-07-07 13:25:21 +03:00
Aliaksandr Valialkin	766edbc421	lib/httpserver: print full requestURI in httpserver.Errorf ... This should simplify debugging.	2021-07-07 13:09:40 +03:00
Aliaksandr Valialkin	78f83dc5ad	app/{vmagent,vminsert}: follow-up after 2fe045e2a4 ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1343	2021-06-04 20:27:58 +03:00
jelmd	2fe045e2a4	new feature: debug relabeling (#1344) ... * new feature: relabel logging Use scrape_configs[x].relabel_debug = true to log metric names inkl. labels before and after relabeling. After relabeling related metrics get dropped, i.e. not submitted to servers. * vminsert wants relabel logging, too.	2021-06-04 17:50:23 +03:00
Aliaksandr Valialkin	c54bb73867	all: do not skip SIGHUP signal during service initialization ... This can lead to stale or incomplete configs like in the https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1240	2021-05-21 16:34:06 +03:00
Aliaksandr Valialkin	12d733dd5d	app/vminsert: add support for data ingestion via other vminsert nodes	2021-05-08 19:52:57 +03:00
Aliaksandr Valialkin	6bc52fe41a	all: rename https://victoriametrics.github.io to https://docs.victoriametrics.com	2021-04-20 20:16:17 +03:00
Aliaksandr Valialkin	dc9eafcd02	app/{vminsert,vmagent}: add -sortLabels command-line option for sorting time series labels before ingesting them in the storage ... This option can be useful when samples for the same time series are ingested with distinct order of labels. For example, metric{k1="v1",k2="v2"} and metric{k2="v2",k1="v1"}.	2021-03-31 23:27:58 +03:00
Aliaksandr Valialkin	44a6cc5eca	app/{vminsert,vmagent}: use Influx field as metric name if measurement is empty and -influxSkipSingleField command-line is set ... See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1139	2021-03-22 13:53:53 +02:00
Aliaksandr Valialkin	85a95bf60c	all: various fixes in command-line flag descriptions	2021-03-15 21:59:25 +02:00
Aliaksandr Valialkin	7c37e9aea9	app/{vminsert,vmagent}: a follow-up for b1aa8c3d8f ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1124	2021-03-15 21:37:55 +02:00
Nikolay	b1aa8c3d8f	adds fake response for telegraph queries (#1130) ... https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1124	2021-03-15 21:10:47 +02:00
Aliaksandr Valialkin	d136081040	lib/promrelabel: add more optimizations for relabeling for common cases	2021-02-22 16:33:55 +02:00
Aliaksandr Valialkin	803a00102a	app/{vminsert,vmselect}: accept requests to paths with /graphite and /prometheus prefixes ... This should improve compatibility with path prefixes from VictoriaMetrics cluster. See https://victoriametrics.github.io/Cluster-VictoriaMetrics.html#url-format	2021-02-04 20:01:18 +02:00
Nikolay	dc9d7aedd5	adds extra_label to all import apis (#1007) ... * adds extra_label to all import apis, changes priority for extra_label - now it has priority over original labels * Update README.md Co-authored-by: Aliaksandr Valialkin <valyala@gmail.com> * Update README.md Co-authored-by: Aliaksandr Valialkin <valyala@gmail.com> * adds extra labels to vmagent import api changes order for adding labels, now its added after user values * adds tests for extra_label * import fix Co-authored-by: Aliaksandr Valialkin <valyala@gmail.com>	2021-01-13 00:52:50 +02:00
Nikolay	7976c22797	Fixes error handling for promscrape.streamParse (#1009) ... properly return error if client cannot read data, properly suppress scraper errors	2021-01-12 13:31:47 +02:00
Aliaksandr Valialkin	069c9ade52	app/{vmagent,vminsert}: follow-up for ce8c2dd1f1: return /targets page in HTML when requested via web browser	2020-12-14 14:06:00 +02:00
Nikolay	ce8c2dd1f1	Changes targets api (#961) ... * changes /targets api adds html response if requester accepts text/html, adds quick template for /targets api, fixes pathPrefix for / requests * changes namings * renamed targets file * Update app/victoria-metrics/main.go Co-authored-by: Aliaksandr Valialkin <valyala@gmail.com> * adds trimspace to qtpl, moves content-type for targets response closer to writer * fixes bug with prefix Co-authored-by: Aliaksandr Valialkin <valyala@gmail.com>	2020-12-14 13:36:48 +02:00
Aliaksandr Valialkin	4146fc4668	all: properly handle CPU limits set on the host system/container ... This can reduce memory usage on systems with enabled CPU limits. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/946	2020-12-08 21:07:29 +02:00
BigFish	0acdab3ab9	Update main.go (#922) ... fix spelling mistake	2020-11-23 17:33:17 +02:00
Aliaksandr Valialkin	d3794eb994	app/{vminsert,vmselect}: move /tags/tagSeries and /tags/tagMultiSeries api from vminsert to vmselect ... This is needed for consistency, since all the `/tags*` api handlers are located in vmselect.	2020-11-23 12:33:19 +02:00
Aliaksandr Valialkin	48d033a198	app/vminsert: add /tags/tagSeries and /tags/tagMultiSeries handlers from Graphite Tags API ... See https://graphite.readthedocs.io/en/stable/tags.html#adding-series-to-the-tagdb	2020-11-16 02:39:58 +02:00
Aliaksandr Valialkin	47a038401b	all: consistently return text-based HTTP responses with charset=utf-8 ... This is a follow-up for https://github.com/VictoriaMetrics/VictoriaMetrics/pull/897	2020-11-13 10:35:41 +02:00
Aliaksandr Valialkin	83df20b5b5	lib/promscrape: clean references to label name and label value strings after applying per-target relabeling ... This should reduce memory usage when per-target relabeling creates big number of temporary labels with long names and/or values. See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/825	2020-11-07 16:19:59 +02:00
Nikolay	4c808d58bf	Adds ready probe (#874) ... * adds leading forward slash check for scrapeURL path https://github.com/VictoriaMetrics/VictoriaMetrics/issues/835 * adds ready probe for scrape config initialization, it should prevent metrics loss during vmagent rolling update, /ready api will return 425 http code, if some scrape config still waits for initialization. * updates docs * Update app/vmagent/README.md * renames var * Update app/vmagent/README.md Co-authored-by: Aliaksandr Valialkin <valyala@gmail.com>	2020-11-04 20:29:18 +02:00
Aliaksandr Valialkin	b7638f04a7	app/vmagent: expose /api/v1/targets page according to https://prometheus.io/docs/prometheus/latest/querying/api/#targets ... This page is exposed by vmagent and by a single-node VictoriaMetrics Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/643	2020-11-01 23:35:06 +02:00
Aliaksandr Valialkin	84227ea2fc	app/{vminsert,vmagent}: take into account all the inserted rows before relabeling in vm_rows_inserted_total and vmagent_rows_inserted_total metrics	2020-10-09 13:29:51 +03:00
Aliaksandr Valialkin	71ea4935de	lib/promscrape: add -promscrape.suppressDuplicateScrapeTargetErrors command-line flag in order to suppress duplicate scrape target errors ... Show also original labels for duplicate targets in error message in order to simplify debugging the issue. Now `/targets` endpoint accepts optional `show_original_labels=1` query arg, which shows original labels for each target. This may simplify debugging for target relabeling. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/651	2020-10-08 18:58:30 +03:00
Aliaksandr Valialkin	124f78857b	app/{vminsert,vmagent}: improve data ingestion speed over a single connection ... Process data obtianed from a single connection on all the available CPU cores.	2020-09-28 04:13:08 +03:00
Aliaksandr Valialkin	95688cbfc5	all: add native format for data export and import ... The data can be exported via [/api/v1/export/native](https://victoriametrics.github.io/#how-to-export-data-in-native-format) handler and imported via [/api/v1/import/native](https://victoriametrics.github.io/#how-to-import-data-in-native-format) handler.	2020-09-27 19:54:07 +03:00
Aliaksandr Valialkin	c00627c103	app/vminsert: code prettifying	2020-09-26 04:13:18 +03:00
Aliaksandr Valialkin	2380e9b017	app/{vminsert,vmagent}: allow passing timestamp via timestamp query arg when ingesting data to /api/v1/import/prometheus ... See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/750	2020-09-11 13:27:14 +03:00
Aliaksandr Valialkin	f41b36bb9a	app/{vminsert,vmagent}: allow adding extra labels when importing data via Prometheus, CSV and JSON line formats ... Extra labels may be added to the imported data by passing `extra_label=name=value` query args. Multiple query args may be passed in order to add multiple extra labels. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/719	2020-09-02 19:43:21 +03:00
Aliaksandr Valialkin	576da0fe46	app/{vminsert,vmagent}: improve documentation for -influxListenAddr command-line flag	2020-08-14 18:04:44 +03:00
Aliaksandr Valialkin	0f63da3698	app/{vmagent,vminsert}: properly preserve db tag from query string passed to Influx line protocol query ... Previously `db` tag from the query string wasn't added to metrics after encountering `db` tag in the Influx line Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/653	2020-07-28 21:25:19 +03:00
Aliaksandr Valialkin	215eba0b82	app/vminsert: flush bufs if needed after the current row is added ... Previously the data for the added row could be overwritten by the flush before the row addition is complete.	2020-07-26 12:10:11 +03:00
Aliaksandr Valialkin	a090627059	app/vminsert: limit memory usage when ingesting data in big packets	2020-07-24 23:32:40 +03:00