Hui Wang
69f4a58f76
do not print redundant error logs when failed to scrape consul or no… ( #5239 )
...
* do not print redundant error logs when failed to scrape consul or nomad target
prometheus performs the same because it uses consul lib which just drops the error(1806bcb38c/api/api.go (L1134)
2023-10-27 14:18:47 +02:00
Aliaksandr Valialkin
1f30f53df2
lib/promscrape/discovery: close unused HTTP connections to service discovery servers
...
This should prevent from connection leaks
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4724
2023-07-27 14:47:55 -07:00
Aliaksandr Valialkin
5b8095a30a
lib/promscrape: disable support for service discovery and metrics scrape via http2
...
Reasons for disabling http2:
- http2 is used very rarely comparing to http for Prometheus metrics exposition and service discovery
- http2 is much harder to debug than http
- http2 has very bad security record because of its complexity - see https://portswigger.net/research/http2
VictoriaMetrics components are compiled with nethttpomithttp2 tag because of these issues.
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4283
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4274
This is a follow-up for 72c3cd47eb
2023-07-06 16:04:31 -07:00
Haleygo
6edf94c4b9
vmagent:scrape config support enable_http2 ( #4295 )
...
app/vmagent: support `enable_http2` in scrape config
This change adds HTTP2 support for scrape config
and improves compatibility with Prometheus config.
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4283
(cherry picked from commit 72c3cd47eb
2023-06-09 10:40:17 +02:00
Haleygo
73a8f763a0
vmagent:support follow_redirects on SD level ( #4286 )
...
* vmagent:support follow_redirects on SD level
* fix follow_redirects on sd level
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4282
(cherry picked from commit b3d0ff463a
2023-06-02 13:19:35 +02:00
Aliaksandr Valialkin
a47b9e55ac
lib/promscrape/discovery/consulagent: substitute metaPrefix with the __meta_consulagent_ plaintext string
...
This simplifies future code navigation and search for the specific meta-label starting from __meta_consulagent_* prefix.
For example, `grep __meta_consulagent_namespace` finds the exact place where this label is defined.
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3953
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/4217
2023-05-09 22:58:08 -07:00
Alexander Marshalov
26fc4afff8
added new consulagent service discovery ( #3953 ) ( #4217 )
2023-05-08 23:43:59 -07:00
Oleksandr Redko
0e1c395609
app,lib: fix typos in comments ( #3804 )
2023-02-13 09:32:35 -08:00
Zakhar Bessarab
40d524edb8
discovery/{consul,nomad}: fix cancelling serviceWatcher in-flight requests ( #3658 )
...
* lib/promscrape/discovery/{consul,nomad}: fix background service update watches not canceling requests on serviceWatcher stop
Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
* lib/promscrape/discovery/{consul,nomad}: fix closing serviseWatcher during scrape job restart
Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
* wip
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3468
Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2023-01-17 21:47:51 -08:00
Aliaksandr Valialkin
43a4dcdaf8
lib/promscrape/discovery/nomad: sync nomad_sd_configs fields with the Prometheus implementation
...
See the list of configs supported by Prometheus at f88a0a7d83/discovery/nomad/nomad.go (L76-L84)865ee8d37c/api/api.go (L297)865ee8d37c/api/api.go (L555-L556)865ee8d37c/api/api.go (L294-L296)8aee209c53https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3367
2023-01-09 21:30:19 -08:00
Zakhar Bessarab
de5aad2cde
lib/promscrape/discoveryutils: switch to native http client from fasthttp ( #3568 )
2023-01-05 21:23:15 -08:00
Aliaksandr Valialkin
750d309f63
lib/promscrape/discovery/nomad: follow-up after 48f371a46c
...
- Remove undocumented `username` and `password` config options from `nomad_sd_config`.
TODO: probably, remove these options from `consul_sd_config` too?
These options exist there for backwards compatibility purposes.
- Add __meta_nomad_service_alloc_id and __meta_nomad_service_job_id meta-labels
These labels contain AllocID and JobID fields for the discovered Nomad services.
- Various typo fixes.
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3367
2023-01-05 18:09:23 -08:00
Aliaksandr Valialkin
06f6de6d47
all: use os.{Read|Write}File instead of ioutil.{Read|Write}File
...
The ioutil.{Read|Write}File is deprecated since Go1.16 -
see https://tip.golang.org/doc/go1.16#ioutil
VictoriaMetrics needs at least Go1.18, so it is safe to remove ioutil usage
from source code.
This is a follow-up for 02ca2342ab
2022-08-21 23:55:20 +03:00
Aliaksandr Valialkin
847004fa77
app/{vminsert,vmagent}: hide passwords and auth tokens by default at /config page
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1764
2021-11-05 14:42:13 +02:00
Aliaksandr Valialkin
f10fa0d1d7
lib/promscrape/discovery/consul: properly pass namespace to Consul watcher
...
Follow-up for 58a2989fe7
2021-06-22 17:43:20 +03:00
Aliaksandr Valialkin
3ab3902f17
lib/promscrape/discovery: support generic auth configs in Consul service discovery in the same way as Prometheus 2.28 does
2021-06-22 13:18:51 +03:00
Nikolay
827a2396d2
adds consul enterprise namespace support ( #1400 )
...
* adds consul enterprise namespace support
* Update lib/promscrape/discovery/consul/consul.go
Co-authored-by: Aliaksandr Valialkin <valyala@gmail.com>
2021-06-22 12:56:11 +03:00
Nikolay
2780d6dbcd
basic OAuth2 support for remoteWrite and scrape targets ( #1316 )
...
* adds OAuth2 support for remoteWrite and scrapping
* adds tests
changes init
2021-05-22 18:02:01 +03:00
Aliaksandr Valialkin
ab9e1eb41f
lib/promscrape: support for simple HTTP proxies without CONNECT method support such as https://github.com/prometheus-community/PushProx
...
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1179
2021-04-04 00:40:58 +03:00
Aliaksandr Valialkin
87700f1259
lib/promscrape: add support for authorization config in -promscrape.config as Prometheus 2.26 does
...
See https://github.com/prometheus/prometheus/pull/8512
2021-04-02 21:20:37 +03:00
Aliaksandr Valialkin
6b1f807418
app/vmagent: add -promscrape.consul.waitTime command-line flag for configuring Consul service discovery wait time
...
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1144
2021-03-23 19:34:12 +02:00
Aliaksandr Valialkin
c3bf72992f
lib/promscrape: explicitly stop and cleanup service discovery routines when new config is read from -promscrape.config
...
This should reduce memory usage when `-promscrape.config` file frequently changes
2021-03-01 14:15:16 +02:00
Aliaksandr Valialkin
367fc17933
lib/promscrape: code prettifying for 8dd03ecf19
2020-12-24 10:57:20 +02:00
Nikolay
b00f7816e2
adds proxy_url support, ( #980 )
...
* adds proxy_url support,
adds proxy_url to the dockerswarm, eureka, kubernetes and consul service discovery,
adds proxy_url to the scrape_config for targets scrapping,
http based proxy is supported atm,
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/503
* fixes imports
2020-12-24 10:57:19 +02:00
Aliaksandr Valialkin
c80d38f00c
lib/promscrape/discovery/consul: reduce load on Consul API server by increasing timeout for blocking requests from 50 seconds to 9 minutes
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/574
2020-12-11 17:26:34 +02:00
Aliaksandr Valialkin
4e4a93c586
lib/promscrape/discovery/consul: make sure that block response contains X-Consul-Index header
2020-12-03 20:05:54 +02:00
Aliaksandr Valialkin
7a889f6850
lib/promscrape: code cleanup after c6dee6c52d
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/574
2020-12-03 19:52:09 +02:00
Nikolay
0b302d33cb
Changes consul discovery api ( #921 )
...
* adds consul watch api,
it must reduce load on consul service with blocking wait requests,
changed discoveryClient api with fetchResponseMeta callback.
* small fix
* fix after master merge
* adds watch client at discovery utils
* fixes consul watcher,
changes namings,
fixes data race
* small typo fix
* sanity fix
* fix naming and service node update
2020-12-03 19:52:08 +02:00
Aliaksandr Valialkin
d962568e93
all: use %w instead of %s for wrapping errors in fmt.Errorf
...
This will simplify examining the returned errors such as httpserver.ErrorWithStatusCode .
See https://blog.golang.org/go1.13-errors for details.
2020-06-30 23:33:46 +03:00
Aliaksandr Valialkin
50aa34bcbe
lib/promscrape/discovery/consul: reduce load on Consul when discovering big number of targets by using background caching
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/574
2020-06-20 18:20:07 +03:00
Aliaksandr Valialkin
054457d1f4
lib/promscrape: allow explicitly setting empty token via token: "" in consul_sd_config
2020-05-05 07:49:54 +03:00
Aliaksandr Valialkin
89aa6dbf56
lib/promscrape: add Prometheus-compatible service discovery for Consul aka consul_sd_configs
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/330
2020-05-04 20:53:06 +03:00
