VictoriaMetrics/lib
Aliaksandr Valialkin d5a599badc
lib/promauth: follow-up for e16d3f5639
- Make sure that invalid/missing TLS CA file or TLS client certificate files at vmagent startup
  don't prevent from processing the corresponding scrape targets after the file becomes correct,
  without the need to restart vmagent.
  Previously scrape targets with invalid TLS CA file or TLS client certificate files
  were permanently dropped after the first attempt to initialize them, and they didn't
  appear until the next vmagent reload or the next change in other places of the loaded scrape configs.

- Make sure that TLS CA is properly re-loaded from file after it changes without the need to restart vmagent.
  Previously the old TLS CA was used until vmagent restart.

- Properly handle errors during http request creation for the second attempt to send data to remote system
  at vmagent and vmalert. Previously failed request creation could result in nil pointer dereferencing,
  since the returned request is nil on error.

- Add more context to the logged error during AWS sigv4 request signing before sending the data to -remoteWrite.url at vmagent.
  Previously it could miss details on the source of the request.

- Do not create a new HTTP client per second when generating OAuth2 token needed to put in Authorization header
  of every http request issued by vmagent during service discovery or target scraping.
  Re-use the HTTP client instead until the corresponding scrape config changes.

- Cache error at lib/promauth.Config.GetAuthHeader() in the same way as the auth header is cached,
  e.g. the error is cached for a second now. This should reduce load on CPU and OAuth2 server
  when auth header cannot be obtained because of temporary error.

- Share tls.Config.GetClientCertificate function among multiple scrape targets with the same tls_config.
  Cache the loaded certificate and the error for one second. This should significantly reduce CPU load
  when scraping big number of targets with the same tls_config.

- Allow loading TLS certificates from HTTP and HTTPs urls by specifying these urls at `tls_config->cert_file` and `tls_config->key_file`.

- Improve test coverage at lib/promauth

- Skip unreachable or invalid files specified at `scrape_config_files` during vmagent startup, since these files may become valid later.
  Previously vmagent was exitting in this case.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4959
2023-10-25 23:19:37 +02:00
..
appmetrics all: replace atomic.Value with atomic.Pointer[T] 2023-07-19 17:42:06 -07:00
auth lib/auth: add NewTokenPossibleMultitenant() for parsing auth token, which can be multitenant 2023-08-30 14:17:55 +02:00
awsapi .golangci.yml: properly enable revive linter and fix all the warnings it detects 2023-02-26 12:18:59 -08:00
backup fixed error when creating a full backup using the -origin flag (#5180) 2023-10-16 12:02:51 +02:00
blockcache Makefile: update golangci-lint from v1.51.2 to v1.54.2 2023-09-01 10:16:42 +02:00
bloomfilter all: replace atomic.Value with atomic.Pointer[T] 2023-07-19 17:42:06 -07:00
bufferedwriter app/vmselect: move common http functionality from app/vmselect/searchutils to lib/httputils 2023-06-19 22:34:20 -07:00
buildinfo all: open-sourcing single-node version 2019-05-23 00:18:06 +03:00
bytesutil Makefile: update golangci-lint from v1.51.2 to v1.54.2 2023-09-01 10:16:42 +02:00
cgroup lib/cgroup: add SetGOGC() function 2023-06-19 22:39:00 -07:00
decimal lib/decimal: use consistent randomizer in tests 2023-01-23 19:23:39 -08:00
encoding lib/encoding/zstd: switch back from atomic.Pointer to atomic.Value for map[...]... 2023-07-20 20:56:11 -07:00
envflag lib/envflag: do not allow unsupported form for boolean command-line flags in the form -boolFlag value 2023-08-17 13:26:53 +02:00
envtemplate allowed using dashes and dots in environment variables names (#4009) 2023-03-24 15:43:05 -07:00
fastnum Makefile: add build and test rules with enabled race detector. These rules have -race suffix 2020-03-05 12:03:38 +02:00
fasttime lib: extract common code for returning fast unix timestamp into lib/fasttime 2020-05-14 23:02:07 +03:00
filestream lib/filestream: add -filestream.disableFadvise syscall for unconditional disabling of fadvise syscall 2023-10-04 16:19:46 +02:00
flagutil lib/flagutil: add Duration.Milliseconds() convenience function after 0c7d46d637 2023-09-03 10:56:44 +02:00
formatutil app/vmbackupmanager: add metrics for better observability (#488) 2022-12-20 14:18:06 -08:00
fs all: consistently use %w instead of %s in when error is passed to fmt.Errorf() 2023-10-25 21:24:03 +02:00
htmlcomponents app/vmselect: remove dependency on lib/promscrape from app/vmselect 2023-01-03 23:28:27 -08:00
httpserver all: consistently use %w instead of %s in when error is passed to fmt.Errorf() 2023-10-25 21:24:03 +02:00
httputils lib/httputils: fix test after b49d04b3dc 2023-07-04 09:40:12 -07:00
influxutils lib/flagutil: rename Array to ArrayString 2022-10-01 18:26:36 +03:00
ingestserver lib/netutil: init implimentation of proxy protocol (#3687) 2023-01-26 23:08:35 -08:00
leveledbytebufferpool all: make fmt via the upcoming Go1.19 2022-07-11 19:22:15 +03:00
logger lib/logger: increase the maximum log arg size from 200 to 500 2023-09-07 16:11:08 +02:00
logjson app/vlinsert/jsonline: code prettifying 2023-06-21 19:39:22 -07:00
logstorage all: consistently use %w instead of %s in when error is passed to fmt.Errorf() 2023-10-25 21:24:03 +02:00
lrucache Makefile: update golangci-lint from v1.51.2 to v1.54.2 2023-09-01 10:16:42 +02:00
memory docs: follow-up after aec4b5db81 (#4638) 2023-07-19 10:10:51 +02:00
mergeset lib/{mergeset,storage}: consistently reset isInMerge field in parts passed to mergeParts() before returning from the function 2023-10-02 08:05:29 +02:00
metricsql all: make fmt via the upcoming Go1.19 2022-07-11 19:22:15 +03:00
netutil all: consistently use %w instead of %s in when error is passed to fmt.Errorf() 2023-10-25 21:24:03 +02:00
persistentqueue Makefile: update golangci-lint from v1.51.2 to v1.54.2 2023-09-01 10:16:42 +02:00
procutil lib/procutil: stop immediately after receiving the second SIGINT or SIGTERM signal 2022-10-20 21:40:20 +03:00
promauth lib/promauth: follow-up for e16d3f5639 2023-10-25 23:19:37 +02:00
prompb app/vminsert: moved -maxInsertRequestSize command-line flag out of lib/prompb in order to prevent its inclusion in vmselect and vmstorage apps 2020-01-28 23:02:08 +02:00
prompbmarshal all: use %w instead of %s for wrapping errors in fmt.Errorf 2020-06-30 23:05:11 +03:00
promrelabel all: consistently use %w instead of %s in when error is passed to fmt.Errorf() 2023-10-25 21:24:03 +02:00
promscrape lib/promauth: follow-up for e16d3f5639 2023-10-25 23:19:37 +02:00
promutils lib/promutils, app/vmalert-tool/unittest: move promutils.Duration.ParseTime() to app/vmalert-tool/unittest.durationToTime() 2023-10-16 14:19:31 +02:00
protoparser lib/promauth: follow-up for e16d3f5639 2023-10-25 23:19:37 +02:00
proxy lib/promauth: follow-up for e16d3f5639 2023-10-25 23:19:37 +02:00
pushmetrics fixed typos in documentation and commandline flags descriptions (#4275) 2023-05-10 09:50:41 +02:00
querytracer all: consistently use %w instead of %s in when error is passed to fmt.Errorf() 2023-10-25 21:24:03 +02:00
regexutil all: consistently use %w instead of %s in when error is passed to fmt.Errorf() 2023-10-25 21:24:03 +02:00
snapshot app/vmbackup: prevent password leaks (#3672) 2023-01-18 11:35:21 -08:00
storage all: consistently use %w instead of %s in when error is passed to fmt.Errorf() 2023-10-25 21:24:03 +02:00
streamaggr lib/streamaggr: respect streamAgg.dropInput with empty stream aggr config (#5213) 2023-10-20 15:55:58 +02:00
syncwg all: open-sourcing single-node version 2019-05-23 00:18:06 +03:00
tenantmetrics lib/encoding/zstd: switch back from atomic.Pointer to atomic.Value for map[...]... 2023-07-20 20:56:11 -07:00
timerpool lib/timerpool: use timer pool in concurrency limiters 2019-05-28 17:20:10 +03:00
uint64set lib/uint64set: use repeatable randomizer in tests 2023-01-23 19:22:58 -08:00
workingsetcache lib/workingsetcache: fix outdated comments for Load() and New() functions 2023-10-25 21:04:20 +02:00
writeconcurrencylimiter lib/writeconcurrencylimiter: initialize concurrencyLimitCh before exporting vm_concurrent_insert_capacity and vm_concurrent_insert_current metrics 2023-02-07 11:08:17 -08:00