github-mirrors/VictoriaMetrics
1
0
Fork 0
mirror of https://github.com/VictoriaMetrics/VictoriaMetrics.git synced 2024-11-21 14:44:00 +00:00
Code Issues Projects Releases Packages Wiki Activity
2035 commits 150 branches 748 tags 347 MiB
Commit graph

746 commits

Author SHA1 Message Date
Aliaksandr Valialkin
05813259dc lib/promscrape: export vm_promscrape_active_scrapers{type="<sd_type>"} metric for tracking the number of active scrapers per each service discovery type 2020-12-08 01:54:23 +02:00
Aliaksandr Valialkin
9c1c9d8e76 lib/promscrape: do not enable strict config parsing when -promscrape.config.dryRun command-line flag is passed 
Strict parsing for -promscrape.config can be enabled by passing `-promscrape.config.strictParse` command-line flag.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/944
 2020-12-07 13:18:50 +02:00
Aliaksandr Valialkin
82972a8f2a lib/promscrape: mention in scrape error message that scrape errors can be disabled by -promscrape.suppressScrapeErrors command-line flag 2020-12-06 23:27:58 +02:00
Aliaksandr Valialkin
299a35948c lib/promscrape: clarify error message on failed connection to scrape target when -enableTCP6 command-line flag isn't set 2020-12-06 13:18:39 +02:00
Aliaksandr Valialkin
b0e4b234cb lib/protoparser/influx: allow multiple whitespace chars between measurement, fields and timestamp in Influx line protocol 2020-12-06 12:01:27 +02:00
Aliaksandr Valialkin
52915c8f7e lib/promscrape/discoveryutils: remove limit on the number of concurrently running blocking queries 
Too low limit could result in unexpected errors when performing big number of blocking queries.
 2020-12-05 12:15:52 +02:00
Aliaksandr Valialkin
eb27dbde13 lib/flagutil: make golangci-lint happy by using strings.TrimPrefix instead of manual prefix removal via strings.HasPrefix 2020-12-03 22:07:57 +02:00
Aliaksandr Valialkin
9d787f9edd all: do not print usage info for all the flags when incorrect command-line flag is passed 
This should improve usability for VictoriaMetrics apps that have big number of command-line flags,
i.e. all the apps.
 2020-12-03 21:47:37 +02:00
Aliaksandr Valialkin
de0643fab5 lib/promscrape/discovery/consul: log the time needed for stoppig Consul service watcher 2020-12-03 20:14:55 +02:00
Aliaksandr Valialkin
9cd8eb92f1 lib/promscrape/discovery/consul: make sure that block response contains X-Consul-Index header 2020-12-03 20:05:23 +02:00
Aliaksandr Valialkin
5009b25a03 lib/promscrape: code cleanup after c6dee6c52d 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/574
 2020-12-03 19:50:53 +02:00
Nikolay
c6dee6c52d
Changes consul discovery api (#921) 
* adds consul watch api,
it must reduce load on consul service with blocking wait requests,
changed discoveryClient api with fetchResponseMeta callback.

* small fix

* fix after master merge

* adds watch client at discovery utils

* fixes consul watcher,
changes namings,
fixes data race

* small typo fix

* sanity fix

* fix naming and service node update
 2020-12-03 19:47:40 +02:00
Aliaksandr Valialkin
357f886f97 Revert "lib/storage: add missing (AccountID, ProjectID) in MetricName.String() test" 
This reverts commit f3e80eb70e493620e42a0cc22a62c9af75076c77, since it isn't needed for single-node version of VictoriaMetrics
 2020-11-29 01:27:25 +02:00
Aliaksandr Valialkin
ace969d595 lib/storage: add missing (AccountID, ProjectID) in MetricName.String() test 2020-11-29 01:26:04 +02:00
Aliaksandr Valialkin
32869e4c0f lib/promscrape: fix failing tests after a906b3862f 2020-11-29 01:26:03 +02:00
Aliaksandr Valialkin
a906b3862f lib/protoparser/prometheus: properly parse OpenMetrics timestamps 
OpenMetrics timestamps are floating-point numbers, that represent Unix timestamp in seconds.
This differs from Prometheus exposition format, where timestamps are integer numbers representing Unix timestamp in milliseconds.
 2020-11-27 14:54:29 +02:00
Aliaksandr Valialkin
3a32789352 lib/promscrape: reduce memory allocations when unpacking gzipped responses received from scrape targets 2020-11-26 18:32:06 +02:00
Aliaksandr Valialkin
2cea4d403f all: typo fix: thouthand->thousand 2020-11-26 13:33:46 +02:00
Aliaksandr Valialkin
b0a5c382ee lib/promscrape: release http response non-200 status code 2020-11-26 13:25:17 +02:00
Aliaksandr Valialkin
b7fcdb528d app/{vmagent,victoria-metrics}: add -dryRun option and make more clear handling for -promscrape.config.dryRun 2020-11-25 22:59:13 +02:00
Aliaksandr Valialkin
1c669a69a8 lib/mergeset: tune the number of rawItemsBlocks to merge at once 
512 blocks give higher ingestion performance and slightly lower memory usage
 2020-11-25 21:52:52 +02:00
Aliaksandr Valialkin
7119f294f3 lib/mergeset: help GC by removing refereces to slices in inmemoryBlock.Reset 2020-11-25 21:19:43 +02:00
Aliaksandr Valialkin
8a057e705a lib/storage: log metric name plus all its labels when the metric timestamp is outside the configured retention 
This should simplify debugging when the source of the metric with unexpected timestamp must be found.
 2020-11-25 14:41:37 +02:00
Aliaksandr Valialkin
b65236530c lib/storage: typo fix in error message: allowd->allowed 2020-11-25 14:15:42 +02:00
Aliaksandr Valialkin
ae04378424 lib/protoparser/prometheus: properly parse "infinity" values in OpenMetrics format 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/924
 2020-11-24 19:03:38 +02:00
Aliaksandr Valialkin
bf95fbfc1d lib/logger: disable rate limiting for error and warn logs by default 2020-11-24 12:42:25 +02:00
Aliaksandr Valialkin
78d2715d04 all: spelling fix: superflouos->superfluous. This is a follow-up for 0acdab3ab9 2020-11-24 12:42:22 +02:00
Aliaksandr Valialkin
b7f4fc6e0d lib/protoparser/prometheus: properly parse metrics with exemplars 
Examplars have been introduced in OpenMetrics - see https://github.com/OpenObservability/OpenMetrics/blob/master/OpenMetrics.md#exemplars-1
Previously VictoriaMetrics couldn't parse the following metric

    foo{bar="baz"} 123 # exemplar here

This commit fixes this. Note that VictoriaMetrics ignores the exemplar as for now.
 2020-11-24 12:34:56 +02:00
Aliaksandr Valialkin
aa90b93778 lib/promscrape: expose __meta_ec2_ipv6_addresses label for ec2_sd_config like Prometheus will do in the next release 2020-11-23 16:56:42 +02:00
Aliaksandr Valialkin
de523c81b9 lib/promscrape: add filters option to dockerswarm_sd_config like Prometheus did in v2.23.0 2020-11-23 16:27:40 +02:00
Aliaksandr Valialkin
f765985947 lib/fs: replace fs.OpenReaderAt with fs.MustOpenReaderAt 
All the callers for fs.OpenReaderAt expect that the file will be opened.
So it is better to log fatal error inside fs.MustOpenReaderAt instead of leaving this to the caller.
 2020-11-23 09:57:21 +02:00
Aliaksandr Valialkin
9d160f9048 lib/promscrape: hint that -enableTCP6 command-line flag can be used for connecting to IPv6 addresses 2020-11-21 14:39:00 +02:00
Aliaksandr Valialkin
d7932775cc lib/promscrape/discovery/eureka: follow-up after eec76718e9 2020-11-20 14:00:12 +02:00
Nikolay
eec76718e9
Adds eureka service discovery (#913) 
* Adds eureka service discovery
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/851
Netflix service discovery for AWS

* Apply suggestions from code review

Co-authored-by: Aliaksandr Valialkin <valyala@gmail.com>
 2020-11-20 13:38:12 +02:00
Aliaksandr Valialkin
97a0c80904 lib/logger: follow-up for 09105ff49c 2020-11-19 12:37:00 +02:00
Nikolay
09105ff49c
Adds log suppression per caller (#908) 
* Adds log suppression per caller
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/905

* fixes style and report message
 2020-11-19 12:17:23 +02:00
Aliaksandr Valialkin
a6b2b2c005 lib/logger: add -loggerWarnsPerSecondLimit command-line flag for rate limiting of WARN log messages 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/905
 2020-11-18 03:43:37 +02:00
Aliaksandr Valialkin
266788be14 app/vmselect: use storage.NewSearchQuery() instead of constructing storage.SearchQuery in-place 
This should prevent from bugs when AccountID and ProjectID aren't set in storage.SearchQuery.
 2020-11-16 18:24:00 +02:00
Aliaksandr Valialkin
0208d8c103 lib/storage: add a test for Storage.SearchMetricNames 2020-11-16 13:15:16 +02:00
Aliaksandr Valialkin
465923b181 app/vmselect/graphite: add /tags/findSeries handler from Graphite Tags API 
See https://graphite.readthedocs.io/en/stable/tags.html#exploring-tags
 2020-11-16 12:53:13 +02:00
Aliaksandr Valialkin
48d033a198 app/vminsert: add /tags/tagSeries and /tags/tagMultiSeries handlers from Graphite Tags API 
See https://graphite.readthedocs.io/en/stable/tags.html#adding-series-to-the-tagdb
 2020-11-16 02:39:58 +02:00
Aliaksandr Valialkin
4aaee33860 lib/storage: do not show artifically created label for reverse Graphite labels at /api/v1/labels page 2020-11-16 00:44:35 +02:00
Aliaksandr Valialkin
0deabbbb4a lib/protoparser/promremotewrite: log the time spent on unsuccessful data read from the network 
This should help with debugging `connection timed out` errors.
 2020-11-13 17:49:12 +02:00
Vasily
6fcbd17bdd Add omitempty for DisableCompression and DisableKeepAlive fields in ScrapeConfig (#796) 
* Add omitempty for DisableCompression and DisableKeepAlive fields in ScrapeConfig

* Add omitempty annotation to all the default/optional values

* Fix annotations after review
 2020-11-13 16:19:05 +02:00
Aliaksandr Valialkin
cdf1e6684b lib/protoparser/opentsdbhttp: increment errors counter on unmarshal errors 
This is a follow-up for 149c0c4a6d
 2020-11-13 13:23:17 +02:00
Aliaksandr Valialkin
149c0c4a6d lib/protoparser: propagate callback error to the caller of ParseStream for every supported data ingestion protocols 
The caller of ParseStream then can generate HTTP 503 responses for non-nil errors occured in callbacks when processing incoming requests.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/896
 2020-11-13 13:05:24 +02:00
Aliaksandr Valialkin
4f8a3af061 lib/protoparser/promremotewrite: synchronously process Prometheus remote_write requests 
There is no reason in processing these requests asynchronously in the face of https://github.com/VictoriaMetrics/VictoriaMetrics/issues/896
Synchronous processing code is easier to read and understand than the previous async code
 2020-11-13 12:17:25 +02:00
Aliaksandr Valialkin
57a4af98fa lib/protoparser/promremotewrite: forward errors, which can occur during data ingestion, to the caller of ParseStream, so it could properly return HTTP 503 status code on non-nil error 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/896
 2020-11-13 11:01:07 +02:00
Aliaksandr Valialkin
47a038401b all: consistently return text-based HTTP responses with charset=utf-8 
This is a follow-up for https://github.com/VictoriaMetrics/VictoriaMetrics/pull/897
 2020-11-13 10:35:41 +02:00
immerrr again
51c529a2b6
app/vmstorage: add "/internal/force_flush" endpoint (#893) 2020-11-11 14:40:27 +02:00