From 4cd173bbaab2726dcdbb81df7ddfc3c425df2dfb Mon Sep 17 00:00:00 2001 From: Aliaksandr Valialkin Date: Thu, 20 Oct 2022 21:40:17 +0300 Subject: [PATCH] lib/procutil: stop immediately after receiving the second SIGINT or SIGTERM signal Previously VictoriaMetrics apps could stop responding to SIGINT and SIGTERM signals if they hang for some reason in graceful shutdown procedure. --- docs/CHANGELOG.md | 3 ++- lib/procutil/signal.go | 4 ++++ 2 files changed, 6 insertions(+), 1 deletion(-) diff --git a/docs/CHANGELOG.md b/docs/CHANGELOG.md index da96ba4c9..d0e96e32c 100644 --- a/docs/CHANGELOG.md +++ b/docs/CHANGELOG.md @@ -39,7 +39,8 @@ The following tip changes can be tested by building VictoriaMetrics components f * FEATURE: [vmagent](https://docs.victoriametrics.com/vmagent.html): allow controlling staleness tracking on a per-[scrape_config](https://docs.victoriametrics.com/sd_configs.html#scrape_configs) basis by specifying `no_stale_markers: true` or `no_stale_markers: false` option in the corresponding [scrape_config](https://docs.victoriametrics.com/sd_configs.html#scrape_configs). * FEATURE: [vmui](https://docs.victoriametrics.com/#vmui): limit the number of plotted series. This should prevent from browser crashes or hangs when the query returns big number of time series. See [this feature request](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3155). -* FEATURE: log error if some environment variables referred at `-promscrape.config` via `%{ENV_VAR}` aren't found. This should prevent from using incorrect config files. +* FEATURE: log error if some environment variables referred at `-promscrape.config` via `%{ENV_VAR}` aren't found. This should prevent from silent using incorrect config files. +* FEATURE: immediately shut down VictoriaMetrics apps on the second SIGINT or SIGTERM signal if they couldn't be finished gracefully for some reason after receiving the first signal. * BUGFIX: [MetricsQL](https://docs.victoriametrics.com/MetricsQL.html): properly merge buckets with identical `le` values, but with different string representation of these values when calculating [histogram_quantile](https://docs.victoriametrics.com/MetricsQL.html#histogram_quantile) and [histogram_share](https://docs.victoriametrics.com/MetricsQL.html#histogram_share). For example, `http_request_duration_seconds_bucket{le="5"}` and `http_requests_duration_seconds_bucket{le="5.0"}`. Such buckets may be returned from distinct targets. Thanks to @647-coder for the [pull request](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/3225). * BUGFIX: [vmalert](https://docs.victoriametrics.com/vmalert.html): change severity level for log messages about failed attempts for sending data to remote storage from `error` to `warn`. The message for about all failed send attempts remains at `error` severity level. diff --git a/lib/procutil/signal.go b/lib/procutil/signal.go index 58a863461..97439850e 100644 --- a/lib/procutil/signal.go +++ b/lib/procutil/signal.go @@ -26,6 +26,10 @@ func WaitForSigterm() os.Signal { // Prevent from the program stop on SIGHUP continue } + // Stop listening for SIGINT and SIGTERM signals, + // so the app could be interrupted by sending these signals again + // in the case if the caller doesn't finish the app gracefully. + signal.Stop(ch) return sig } }