lib/{httpserver,netutil}: allow to define min and max TLS version of the http server (#3109)

* lib/{httpserver,netutil}: allow to define min and max TLS version of the http server

* lib/httpserver: added descriptions about tls supported versions

* lib/netutil: check minimal tls version, added supported tls versions to error

* wip

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
This commit is contained in:
Dmytro Kozlov 2022-09-26 17:35:45 +03:00 committed by GitHub
parent d63410bf6f
commit 6ab1cede62
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
23 changed files with 137 additions and 40 deletions

View file

@ -2342,6 +2342,8 @@ Pass `-help` to VictoriaMetrics in order to see the list of supported command-li
Supports an array of values separated by comma or specified via multiple flags. Supports an array of values separated by comma or specified via multiple flags.
-tlsKeyFile string -tlsKeyFile string
Path to file with TLS key if -tls is set. The provided key file is automatically re-read every second, so it can be dynamically updated Path to file with TLS key if -tls is set. The provided key file is automatically re-read every second, so it can be dynamically updated
-tlsMinVersion string
Optional minimum TLS version to use for incoming requests over HTTPS if -tls is set. Supported values: TLS10, TLS11, TLS12, TLS13
-usePromCompatibleNaming -usePromCompatibleNaming
Whether to replace characters unsupported by Prometheus with underscores in the ingested metric names and label names. For example, foo.bar{a.b='c'} is transformed into foo_bar{a_b='c'} during data ingestion if this flag is set. See https://prometheus.io/docs/concepts/data_model/#metric-names-and-labels Whether to replace characters unsupported by Prometheus with underscores in the ingested metric names and label names. For example, foo.bar{a.b='c'} is transformed into foo_bar{a_b='c'} during data ingestion if this flag is set. See https://prometheus.io/docs/concepts/data_model/#metric-names-and-labels
-version -version

View file

@ -1269,6 +1269,8 @@ See the docs at https://docs.victoriametrics.com/vmagent.html .
Supports an array of values separated by comma or specified via multiple flags. Supports an array of values separated by comma or specified via multiple flags.
-tlsKeyFile string -tlsKeyFile string
Path to file with TLS key if -tls is set. The provided key file is automatically re-read every second, so it can be dynamically updated Path to file with TLS key if -tls is set. The provided key file is automatically re-read every second, so it can be dynamically updated
-tlsMinVersion string
Optional minimum TLS version to use for incoming requests over HTTPS if -tls is set. Supported values: TLS10, TLS11, TLS12, TLS13
-usePromCompatibleNaming -usePromCompatibleNaming
Whether to replace characters unsupported by Prometheus with underscores in the ingested metric names and label names. For example, foo.bar{a.b='c'} is transformed into foo_bar{a_b='c'} during data ingestion if this flag is set. See https://prometheus.io/docs/concepts/data_model/#metric-names-and-labels Whether to replace characters unsupported by Prometheus with underscores in the ingested metric names and label names. For example, foo.bar{a.b='c'} is transformed into foo_bar{a_b='c'} during data ingestion if this flag is set. See https://prometheus.io/docs/concepts/data_model/#metric-names-and-labels
-version -version

View file

@ -1069,6 +1069,8 @@ The shortlist of configuration flags is the following:
Supports an array of values separated by comma or specified via multiple flags. Supports an array of values separated by comma or specified via multiple flags.
-tlsKeyFile string -tlsKeyFile string
Path to file with TLS key if -tls is set. The provided key file is automatically re-read every second, so it can be dynamically updated Path to file with TLS key if -tls is set. The provided key file is automatically re-read every second, so it can be dynamically updated
-tlsMinVersion string
Optional minimum TLS version to use for incoming requests over HTTPS if -tls is set. Supported values: TLS10, TLS11, TLS12, TLS13
-version -version
Show VictoriaMetrics version Show VictoriaMetrics version
``` ```

View file

@ -308,6 +308,8 @@ See the docs at https://docs.victoriametrics.com/vmauth.html .
Supports an array of values separated by comma or specified via multiple flags. Supports an array of values separated by comma or specified via multiple flags.
-tlsKeyFile string -tlsKeyFile string
Path to file with TLS key if -tls is set. The provided key file is automatically re-read every second, so it can be dynamically updated Path to file with TLS key if -tls is set. The provided key file is automatically re-read every second, so it can be dynamically updated
-tlsMinVersion string
Optional minimum TLS version to use for incoming requests over HTTPS if -tls is set. Supported values: TLS10, TLS11, TLS12, TLS13
-version -version
Show VictoriaMetrics version Show VictoriaMetrics version
``` ```

View file

@ -266,6 +266,8 @@ See [this article](https://medium.com/@valyala/speeding-up-backups-for-big-time-
Supports an array of values separated by comma or specified via multiple flags. Supports an array of values separated by comma or specified via multiple flags.
-tlsKeyFile string -tlsKeyFile string
Path to file with TLS key if -tls is set. The provided key file is automatically re-read every second, so it can be dynamically updated Path to file with TLS key if -tls is set. The provided key file is automatically re-read every second, so it can be dynamically updated
-tlsMinVersion string
Optional minimum TLS version to use for incoming requests over HTTPS if -tls is set. Supported values: TLS10, TLS11, TLS12, TLS13
-version -version
Show VictoriaMetrics version Show VictoriaMetrics version
``` ```

View file

@ -72,14 +72,14 @@ Backup manager launched with the following configuration:
```console ```console
export NODE_IP=192.168.0.10 export NODE_IP=192.168.0.10
export VMSTORAGE_ENDPOINT=http://127.0.0.1:8428 export VMSTORAGE_ENDPOINT=http://127.0.0.1:8428
./vmbackupmanager -dst=gs://vmstorage-data/$NODE_IP -credsFilePath=credentials.json -storageDataPath=/vmstorage-data -snapshot.createURL=$VMSTORAGE_ENDPOINT/snapshot/create -eula ./vmbackupmanager -dst=gs://vmstorage-data/$NODE_IP -credsFilePath=credentials.json -storageDataPath=/vmstorage-data -snapshot.createURL=$VMSTORAGE_ENDPOINT/snapshot/create -eula
``` ```
Expected logs in vmbackupmanager: Expected logs in vmbackupmanager:
```console ```console
info lib/backup/actions/backup.go:131 server-side copied 81 out of 81 parts from GCS{bucket: "vmstorage-data", dir: "192.168.0.10//latest/"} to GCS{bucket: "vmstorage-data", dir: "192.168.0.10//weekly/2020-34/"} in 2.549833008s info lib/backup/actions/backup.go:131 server-side copied 81 out of 81 parts from GCS{bucket: "vmstorage-data", dir: "192.168.0.10//latest/"} to GCS{bucket: "vmstorage-data", dir: "192.168.0.10//weekly/2020-34/"} in 2.549833008s
info lib/backup/actions/backup.go:169 backed up 853315 bytes in 2.882 seconds; deleted 0 bytes; server-side copied 853315 bytes; uploaded 0 bytes info lib/backup/actions/backup.go:169 backed up 853315 bytes in 2.882 seconds; deleted 0 bytes; server-side copied 853315 bytes; uploaded 0 bytes
``` ```
Expected logs in vmstorage: Expected logs in vmstorage:
@ -93,7 +93,7 @@ info VictoriaMetrics/lib/storage/storage.go:319 deleted snapshot "/vmstora
The result on the GCS bucket The result on the GCS bucket
* The root folder * The root folder
![root](vmbackupmanager_root_folder.png) ![root](vmbackupmanager_root_folder.png)
* The latest folder * The latest folder
@ -265,6 +265,8 @@ vmbackupmanager performs regular backups according to the provided configs.
Supports an array of values separated by comma or specified via multiple flags. Supports an array of values separated by comma or specified via multiple flags.
-tlsKeyFile string -tlsKeyFile string
Path to file with TLS key if -tls is set. The provided key file is automatically re-read every second, so it can be dynamically updated Path to file with TLS key if -tls is set. The provided key file is automatically re-read every second, so it can be dynamically updated
-tlsMinVersion string
Optional minimum TLS version to use for incoming requests over HTTPS if -tls is set. Supported values: TLS10, TLS11, TLS12, TLS13
-version -version
Show VictoriaMetrics version Show VictoriaMetrics version
``` ```

View file

@ -149,7 +149,7 @@ cat << EOF > limit.yaml
limits: limits:
- type: queries - type: queries
value: 100 value: 100
- type: rows_inserted - type: rows_inserted
value: 100000 value: 100000
- type: new_series - type: new_series
value: 1000 value: 1000
@ -168,7 +168,7 @@ curl 'http://localhost:8431/api/v1/import/prometheus' -X POST -d 'foo{bar="baz1
# read metric from tenant 1:5 # read metric from tenant 1:5
curl 'http://localhost:8431/api/v1/labels' -H 'Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE2MjAxNjIwMDAwMDAsInZtX2FjY2VzcyI6eyJ0ZW5hbnRfaWQiOnsiYWNjb3VudF9pZCI6MTV9fX0.PB1_KXDKPUp-40pxOGk6lt_jt9Yq80PIMpWVJqSForQ' curl 'http://localhost:8431/api/v1/labels' -H 'Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE2MjAxNjIwMDAwMDAsInZtX2FjY2VzcyI6eyJ0ZW5hbnRfaWQiOnsiYWNjb3VudF9pZCI6MTV9fX0.PB1_KXDKPUp-40pxOGk6lt_jt9Yq80PIMpWVJqSForQ'
# check rate limit # check rate limit
``` ```
## Configuration ## Configuration
@ -199,11 +199,11 @@ The shortlist of configuration flags include the following:
-datasource.maxIdleConnections int -datasource.maxIdleConnections int
Defines the number of idle (keep-alive connections) to each configured datasource. Consider setting this value equal to the value: groups_total * group.concurrency. Too low a value may result in a high number of sockets in TIME_WAIT state. (default 100) Defines the number of idle (keep-alive connections) to each configured datasource. Consider setting this value equal to the value: groups_total * group.concurrency. Too low a value may result in a high number of sockets in TIME_WAIT state. (default 100)
-datasource.oauth2.clientID string -datasource.oauth2.clientID string
Optional OAuth2 clientID to use for -datasource.url. Optional OAuth2 clientID to use for -datasource.url.
-datasource.oauth2.clientSecret string -datasource.oauth2.clientSecret string
Optional OAuth2 clientSecret to use for -datasource.url. Optional OAuth2 clientSecret to use for -datasource.url.
-datasource.oauth2.clientSecretFile string -datasource.oauth2.clientSecretFile string
Optional OAuth2 clientSecretFile to use for -datasource.url. Optional OAuth2 clientSecretFile to use for -datasource.url.
-datasource.oauth2.scopes string -datasource.oauth2.scopes string
Optional OAuth2 scopes to use for -datasource.url. Scopes must be delimited by ';' Optional OAuth2 scopes to use for -datasource.url. Scopes must be delimited by ';'
-datasource.oauth2.tokenUrl string -datasource.oauth2.tokenUrl string
@ -315,6 +315,8 @@ The shortlist of configuration flags include the following:
Supports an array of values separated by comma or specified via multiple flags. Supports an array of values separated by comma or specified via multiple flags.
-tlsKeyFile string -tlsKeyFile string
Path to file with TLS key if -tls is set. The provided key file is automatically re-read every second, so it can be dynamically updated Path to file with TLS key if -tls is set. The provided key file is automatically re-read every second, so it can be dynamically updated
-tlsMinVersion string
Optional minimum TLS version to use for incoming requests over HTTPS if -tls is set. Supported values: TLS10, TLS11, TLS12, TLS13
-version -version
Show VictoriaMetrics version Show VictoriaMetrics version
-write.url string -write.url string

View file

@ -166,6 +166,8 @@ i.e. the end result would be similar to [rsync --delete](https://askubuntu.com/q
Supports an array of values separated by comma or specified via multiple flags. Supports an array of values separated by comma or specified via multiple flags.
-tlsKeyFile string -tlsKeyFile string
Path to file with TLS key if -tls is set. The provided key file is automatically re-read every second, so it can be dynamically updated Path to file with TLS key if -tls is set. The provided key file is automatically re-read every second, so it can be dynamically updated
-tlsMinVersion string
Optional minimum TLS version to use for incoming requests over HTTPS if -tls is set. Supported values: TLS10, TLS11, TLS12, TLS13
-version -version
Show VictoriaMetrics version Show VictoriaMetrics version
``` ```

View file

@ -25,6 +25,7 @@ The following tip changes can be tested by building VictoriaMetrics components f
* FEATURE: check the correctess of raw sample timestamps stored on disk when reading them. This reduces the probability of possible silent corruption of the data stored on disk. This should help [this](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2998) and [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3011). * FEATURE: check the correctess of raw sample timestamps stored on disk when reading them. This reduces the probability of possible silent corruption of the data stored on disk. This should help [this](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2998) and [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3011).
* FEATURE: atomically delete directories with snapshots, parts and partitions at [storage level](https://docs.victoriametrics.com/#storage). Previously such directories can be left in partially deleted state when the deletion operation was interrupted by unclean shutdown. This may result in `cannot open file ...: no such file or directory` error on the next start. The probability of this error was quite high when NFS or EFS was used as persistent storage for VictoriaMetrics data. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3038). * FEATURE: atomically delete directories with snapshots, parts and partitions at [storage level](https://docs.victoriametrics.com/#storage). Previously such directories can be left in partially deleted state when the deletion operation was interrupted by unclean shutdown. This may result in `cannot open file ...: no such file or directory` error on the next start. The probability of this error was quite high when NFS or EFS was used as persistent storage for VictoriaMetrics data. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3038).
* FEATURE: set the `start` arg to `end - 5 minutes` if isn't passed explicitly to [/api/v1/labels](https://docs.victoriametrics.com/url-examples.html#apiv1labels) and [/api/v1/label/.../values](https://docs.victoriametrics.com/url-examples.html#apiv1labelvalues). See [this pull request](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/3052). * FEATURE: set the `start` arg to `end - 5 minutes` if isn't passed explicitly to [/api/v1/labels](https://docs.victoriametrics.com/url-examples.html#apiv1labels) and [/api/v1/label/.../values](https://docs.victoriametrics.com/url-examples.html#apiv1labelvalues). See [this pull request](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/3052).
* FEATURE: allow to define the minimum TLS version to use when accepting https requests to VictoriaMetrics components if `-tls` command-line flag is set. The minimum TLS version can be set via `-tlsMinVersion` command-line flag. See [this feature request](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3090).
* FEATURE: [vmctl](https://docs.victoriametrics.com/vmctl.html): add `vm-native-step-interval` command line flag for `vm-native` mode. New option allows splitting the import process into chunks by time interval. This helps migrating data sets with high churn rate and provides better control over the process. See [feature request](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2733). * FEATURE: [vmctl](https://docs.victoriametrics.com/vmctl.html): add `vm-native-step-interval` command line flag for `vm-native` mode. New option allows splitting the import process into chunks by time interval. This helps migrating data sets with high churn rate and provides better control over the process. See [feature request](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2733).
* FEATURE: [vmui](https://docs.victoriametrics.com/#vmui): add `top queries` tab, which shows various stats for recently executed queries. See [these docs](https://docs.victoriametrics.com/#top-queries) and [this feature request](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2707). * FEATURE: [vmui](https://docs.victoriametrics.com/#vmui): add `top queries` tab, which shows various stats for recently executed queries. See [these docs](https://docs.victoriametrics.com/#top-queries) and [this feature request](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2707).
* FEATURE: [vmalert](https://docs.victoriametrics.com/vmalert.html): add `debug` mode to the alerting rule settings for printing additional information into logs during evaluation. See `debug` param in [alerting rule config](https://docs.victoriametrics.com/vmalert.html#alerting-rules). * FEATURE: [vmalert](https://docs.victoriametrics.com/vmalert.html): add `debug` mode to the alerting rule settings for printing additional information into logs during evaluation. See `debug` param in [alerting rule config](https://docs.victoriametrics.com/vmalert.html#alerting-rules).

View file

@ -777,6 +777,8 @@ Below is the output for `/path/to/vminsert -help`:
Supports an array of values separated by comma or specified via multiple flags. Supports an array of values separated by comma or specified via multiple flags.
-tlsKeyFile string -tlsKeyFile string
Path to file with TLS key if -tls is set. The provided key file is automatically re-read every second, so it can be dynamically updated Path to file with TLS key if -tls is set. The provided key file is automatically re-read every second, so it can be dynamically updated
-tlsMinVersion string
Optional minimum TLS version to use for incoming requests over HTTPS if -tls is set. Supported values: TLS10, TLS11, TLS12, TLS13
-usePromCompatibleNaming -usePromCompatibleNaming
Whether to replace characters unsupported by Prometheus with underscores in the ingested metric names and label names. For example, foo.bar{a.b='c'} is transformed into foo_bar{a_b='c'} during data ingestion if this flag is set. See https://prometheus.io/docs/concepts/data_model/#metric-names-and-labels Whether to replace characters unsupported by Prometheus with underscores in the ingested metric names and label names. For example, foo.bar{a.b='c'} is transformed into foo_bar{a_b='c'} during data ingestion if this flag is set. See https://prometheus.io/docs/concepts/data_model/#metric-names-and-labels
-version -version
@ -981,6 +983,8 @@ Below is the output for `/path/to/vmselect -help`:
Supports an array of values separated by comma or specified via multiple flags. Supports an array of values separated by comma or specified via multiple flags.
-tlsKeyFile string -tlsKeyFile string
Path to file with TLS key if -tls is set. The provided key file is automatically re-read every second, so it can be dynamically updated Path to file with TLS key if -tls is set. The provided key file is automatically re-read every second, so it can be dynamically updated
-tlsMinVersion string
Optional minimum TLS version to use for incoming requests over HTTPS if -tls is set. Supported values: TLS10, TLS11, TLS12, TLS13
-version -version
Show VictoriaMetrics version Show VictoriaMetrics version
-vmalert.proxyURL string -vmalert.proxyURL string
@ -1141,6 +1145,8 @@ Below is the output for `/path/to/vmstorage -help`:
Supports an array of values separated by comma or specified via multiple flags. Supports an array of values separated by comma or specified via multiple flags.
-tlsKeyFile string -tlsKeyFile string
Path to file with TLS key if -tls is set. The provided key file is automatically re-read every second, so it can be dynamically updated Path to file with TLS key if -tls is set. The provided key file is automatically re-read every second, so it can be dynamically updated
-tlsMinVersion string
Optional minimum TLS version to use for incoming requests over HTTPS if -tls is set. Supported values: TLS10, TLS11, TLS12, TLS13
-version -version
Show VictoriaMetrics version Show VictoriaMetrics version
-vminsertAddr string -vminsertAddr string

View file

@ -2342,6 +2342,8 @@ Pass `-help` to VictoriaMetrics in order to see the list of supported command-li
Supports an array of values separated by comma or specified via multiple flags. Supports an array of values separated by comma or specified via multiple flags.
-tlsKeyFile string -tlsKeyFile string
Path to file with TLS key if -tls is set. The provided key file is automatically re-read every second, so it can be dynamically updated Path to file with TLS key if -tls is set. The provided key file is automatically re-read every second, so it can be dynamically updated
-tlsMinVersion string
Optional minimum TLS version to use for incoming requests over HTTPS if -tls is set. Supported values: TLS10, TLS11, TLS12, TLS13
-usePromCompatibleNaming -usePromCompatibleNaming
Whether to replace characters unsupported by Prometheus with underscores in the ingested metric names and label names. For example, foo.bar{a.b='c'} is transformed into foo_bar{a_b='c'} during data ingestion if this flag is set. See https://prometheus.io/docs/concepts/data_model/#metric-names-and-labels Whether to replace characters unsupported by Prometheus with underscores in the ingested metric names and label names. For example, foo.bar{a.b='c'} is transformed into foo_bar{a_b='c'} during data ingestion if this flag is set. See https://prometheus.io/docs/concepts/data_model/#metric-names-and-labels
-version -version

View file

@ -2346,6 +2346,8 @@ Pass `-help` to VictoriaMetrics in order to see the list of supported command-li
Supports an array of values separated by comma or specified via multiple flags. Supports an array of values separated by comma or specified via multiple flags.
-tlsKeyFile string -tlsKeyFile string
Path to file with TLS key if -tls is set. The provided key file is automatically re-read every second, so it can be dynamically updated Path to file with TLS key if -tls is set. The provided key file is automatically re-read every second, so it can be dynamically updated
-tlsMinVersion string
Optional minimum TLS version to use for incoming requests over HTTPS if -tls is set. Supported values: TLS10, TLS11, TLS12, TLS13
-usePromCompatibleNaming -usePromCompatibleNaming
Whether to replace characters unsupported by Prometheus with underscores in the ingested metric names and label names. For example, foo.bar{a.b='c'} is transformed into foo_bar{a_b='c'} during data ingestion if this flag is set. See https://prometheus.io/docs/concepts/data_model/#metric-names-and-labels Whether to replace characters unsupported by Prometheus with underscores in the ingested metric names and label names. For example, foo.bar{a.b='c'} is transformed into foo_bar{a_b='c'} during data ingestion if this flag is set. See https://prometheus.io/docs/concepts/data_model/#metric-names-and-labels
-version -version

View file

@ -1273,6 +1273,8 @@ See the docs at https://docs.victoriametrics.com/vmagent.html .
Supports an array of values separated by comma or specified via multiple flags. Supports an array of values separated by comma or specified via multiple flags.
-tlsKeyFile string -tlsKeyFile string
Path to file with TLS key if -tls is set. The provided key file is automatically re-read every second, so it can be dynamically updated Path to file with TLS key if -tls is set. The provided key file is automatically re-read every second, so it can be dynamically updated
-tlsMinVersion string
Optional minimum TLS version to use for incoming requests over HTTPS if -tls is set. Supported values: TLS10, TLS11, TLS12, TLS13
-usePromCompatibleNaming -usePromCompatibleNaming
Whether to replace characters unsupported by Prometheus with underscores in the ingested metric names and label names. For example, foo.bar{a.b='c'} is transformed into foo_bar{a_b='c'} during data ingestion if this flag is set. See https://prometheus.io/docs/concepts/data_model/#metric-names-and-labels Whether to replace characters unsupported by Prometheus with underscores in the ingested metric names and label names. For example, foo.bar{a.b='c'} is transformed into foo_bar{a_b='c'} during data ingestion if this flag is set. See https://prometheus.io/docs/concepts/data_model/#metric-names-and-labels
-version -version

View file

@ -1073,6 +1073,8 @@ The shortlist of configuration flags is the following:
Supports an array of values separated by comma or specified via multiple flags. Supports an array of values separated by comma or specified via multiple flags.
-tlsKeyFile string -tlsKeyFile string
Path to file with TLS key if -tls is set. The provided key file is automatically re-read every second, so it can be dynamically updated Path to file with TLS key if -tls is set. The provided key file is automatically re-read every second, so it can be dynamically updated
-tlsMinVersion string
Optional minimum TLS version to use for incoming requests over HTTPS if -tls is set. Supported values: TLS10, TLS11, TLS12, TLS13
-version -version
Show VictoriaMetrics version Show VictoriaMetrics version
``` ```

View file

@ -312,6 +312,8 @@ See the docs at https://docs.victoriametrics.com/vmauth.html .
Supports an array of values separated by comma or specified via multiple flags. Supports an array of values separated by comma or specified via multiple flags.
-tlsKeyFile string -tlsKeyFile string
Path to file with TLS key if -tls is set. The provided key file is automatically re-read every second, so it can be dynamically updated Path to file with TLS key if -tls is set. The provided key file is automatically re-read every second, so it can be dynamically updated
-tlsMinVersion string
Optional minimum TLS version to use for incoming requests over HTTPS if -tls is set. Supported values: TLS10, TLS11, TLS12, TLS13
-version -version
Show VictoriaMetrics version Show VictoriaMetrics version
``` ```

View file

@ -270,6 +270,8 @@ See [this article](https://medium.com/@valyala/speeding-up-backups-for-big-time-
Supports an array of values separated by comma or specified via multiple flags. Supports an array of values separated by comma or specified via multiple flags.
-tlsKeyFile string -tlsKeyFile string
Path to file with TLS key if -tls is set. The provided key file is automatically re-read every second, so it can be dynamically updated Path to file with TLS key if -tls is set. The provided key file is automatically re-read every second, so it can be dynamically updated
-tlsMinVersion string
Optional minimum TLS version to use for incoming requests over HTTPS if -tls is set. Supported values: TLS10, TLS11, TLS12, TLS13
-version -version
Show VictoriaMetrics version Show VictoriaMetrics version
``` ```

View file

@ -76,14 +76,14 @@ Backup manager launched with the following configuration:
```console ```console
export NODE_IP=192.168.0.10 export NODE_IP=192.168.0.10
export VMSTORAGE_ENDPOINT=http://127.0.0.1:8428 export VMSTORAGE_ENDPOINT=http://127.0.0.1:8428
./vmbackupmanager -dst=gs://vmstorage-data/$NODE_IP -credsFilePath=credentials.json -storageDataPath=/vmstorage-data -snapshot.createURL=$VMSTORAGE_ENDPOINT/snapshot/create -eula ./vmbackupmanager -dst=gs://vmstorage-data/$NODE_IP -credsFilePath=credentials.json -storageDataPath=/vmstorage-data -snapshot.createURL=$VMSTORAGE_ENDPOINT/snapshot/create -eula
``` ```
Expected logs in vmbackupmanager: Expected logs in vmbackupmanager:
```console ```console
info lib/backup/actions/backup.go:131 server-side copied 81 out of 81 parts from GCS{bucket: "vmstorage-data", dir: "192.168.0.10//latest/"} to GCS{bucket: "vmstorage-data", dir: "192.168.0.10//weekly/2020-34/"} in 2.549833008s info lib/backup/actions/backup.go:131 server-side copied 81 out of 81 parts from GCS{bucket: "vmstorage-data", dir: "192.168.0.10//latest/"} to GCS{bucket: "vmstorage-data", dir: "192.168.0.10//weekly/2020-34/"} in 2.549833008s
info lib/backup/actions/backup.go:169 backed up 853315 bytes in 2.882 seconds; deleted 0 bytes; server-side copied 853315 bytes; uploaded 0 bytes info lib/backup/actions/backup.go:169 backed up 853315 bytes in 2.882 seconds; deleted 0 bytes; server-side copied 853315 bytes; uploaded 0 bytes
``` ```
Expected logs in vmstorage: Expected logs in vmstorage:
@ -97,7 +97,7 @@ info VictoriaMetrics/lib/storage/storage.go:319 deleted snapshot "/vmstora
The result on the GCS bucket The result on the GCS bucket
* The root folder * The root folder
![root](vmbackupmanager_root_folder.png) ![root](vmbackupmanager_root_folder.png)
* The latest folder * The latest folder
@ -269,6 +269,8 @@ vmbackupmanager performs regular backups according to the provided configs.
Supports an array of values separated by comma or specified via multiple flags. Supports an array of values separated by comma or specified via multiple flags.
-tlsKeyFile string -tlsKeyFile string
Path to file with TLS key if -tls is set. The provided key file is automatically re-read every second, so it can be dynamically updated Path to file with TLS key if -tls is set. The provided key file is automatically re-read every second, so it can be dynamically updated
-tlsMinVersion string
Optional minimum TLS version to use for incoming requests over HTTPS if -tls is set. Supported values: TLS10, TLS11, TLS12, TLS13
-version -version
Show VictoriaMetrics version Show VictoriaMetrics version
``` ```

View file

@ -153,7 +153,7 @@ cat << EOF > limit.yaml
limits: limits:
- type: queries - type: queries
value: 100 value: 100
- type: rows_inserted - type: rows_inserted
value: 100000 value: 100000
- type: new_series - type: new_series
value: 1000 value: 1000
@ -172,7 +172,7 @@ curl 'http://localhost:8431/api/v1/import/prometheus' -X POST -d 'foo{bar="baz1
# read metric from tenant 1:5 # read metric from tenant 1:5
curl 'http://localhost:8431/api/v1/labels' -H 'Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE2MjAxNjIwMDAwMDAsInZtX2FjY2VzcyI6eyJ0ZW5hbnRfaWQiOnsiYWNjb3VudF9pZCI6MTV9fX0.PB1_KXDKPUp-40pxOGk6lt_jt9Yq80PIMpWVJqSForQ' curl 'http://localhost:8431/api/v1/labels' -H 'Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE2MjAxNjIwMDAwMDAsInZtX2FjY2VzcyI6eyJ0ZW5hbnRfaWQiOnsiYWNjb3VudF9pZCI6MTV9fX0.PB1_KXDKPUp-40pxOGk6lt_jt9Yq80PIMpWVJqSForQ'
# check rate limit # check rate limit
``` ```
## Configuration ## Configuration
@ -203,11 +203,11 @@ The shortlist of configuration flags include the following:
-datasource.maxIdleConnections int -datasource.maxIdleConnections int
Defines the number of idle (keep-alive connections) to each configured datasource. Consider setting this value equal to the value: groups_total * group.concurrency. Too low a value may result in a high number of sockets in TIME_WAIT state. (default 100) Defines the number of idle (keep-alive connections) to each configured datasource. Consider setting this value equal to the value: groups_total * group.concurrency. Too low a value may result in a high number of sockets in TIME_WAIT state. (default 100)
-datasource.oauth2.clientID string -datasource.oauth2.clientID string
Optional OAuth2 clientID to use for -datasource.url. Optional OAuth2 clientID to use for -datasource.url.
-datasource.oauth2.clientSecret string -datasource.oauth2.clientSecret string
Optional OAuth2 clientSecret to use for -datasource.url. Optional OAuth2 clientSecret to use for -datasource.url.
-datasource.oauth2.clientSecretFile string -datasource.oauth2.clientSecretFile string
Optional OAuth2 clientSecretFile to use for -datasource.url. Optional OAuth2 clientSecretFile to use for -datasource.url.
-datasource.oauth2.scopes string -datasource.oauth2.scopes string
Optional OAuth2 scopes to use for -datasource.url. Scopes must be delimited by ';' Optional OAuth2 scopes to use for -datasource.url. Scopes must be delimited by ';'
-datasource.oauth2.tokenUrl string -datasource.oauth2.tokenUrl string
@ -319,6 +319,8 @@ The shortlist of configuration flags include the following:
Supports an array of values separated by comma or specified via multiple flags. Supports an array of values separated by comma or specified via multiple flags.
-tlsKeyFile string -tlsKeyFile string
Path to file with TLS key if -tls is set. The provided key file is automatically re-read every second, so it can be dynamically updated Path to file with TLS key if -tls is set. The provided key file is automatically re-read every second, so it can be dynamically updated
-tlsMinVersion string
Optional minimum TLS version to use for incoming requests over HTTPS if -tls is set. Supported values: TLS10, TLS11, TLS12, TLS13
-version -version
Show VictoriaMetrics version Show VictoriaMetrics version
-write.url string -write.url string

View file

@ -170,6 +170,8 @@ i.e. the end result would be similar to [rsync --delete](https://askubuntu.com/q
Supports an array of values separated by comma or specified via multiple flags. Supports an array of values separated by comma or specified via multiple flags.
-tlsKeyFile string -tlsKeyFile string
Path to file with TLS key if -tls is set. The provided key file is automatically re-read every second, so it can be dynamically updated Path to file with TLS key if -tls is set. The provided key file is automatically re-read every second, so it can be dynamically updated
-tlsMinVersion string
Optional minimum TLS version to use for incoming requests over HTTPS if -tls is set. Supported values: TLS10, TLS11, TLS12, TLS13
-version -version
Show VictoriaMetrics version Show VictoriaMetrics version
``` ```

View file

@ -35,6 +35,8 @@ var (
tlsCertFile = flag.String("tlsCertFile", "", "Path to file with TLS certificate if -tls is set. Prefer ECDSA certs instead of RSA certs as RSA certs are slower. The provided certificate file is automatically re-read every second, so it can be dynamically updated") tlsCertFile = flag.String("tlsCertFile", "", "Path to file with TLS certificate if -tls is set. Prefer ECDSA certs instead of RSA certs as RSA certs are slower. The provided certificate file is automatically re-read every second, so it can be dynamically updated")
tlsKeyFile = flag.String("tlsKeyFile", "", "Path to file with TLS key if -tls is set. The provided key file is automatically re-read every second, so it can be dynamically updated") tlsKeyFile = flag.String("tlsKeyFile", "", "Path to file with TLS key if -tls is set. The provided key file is automatically re-read every second, so it can be dynamically updated")
tlsCipherSuites = flagutil.NewArray("tlsCipherSuites", "Optional list of TLS cipher suites for incoming requests over HTTPS if -tls is set. See the list of supported cipher suites at https://pkg.go.dev/crypto/tls#pkg-constants") tlsCipherSuites = flagutil.NewArray("tlsCipherSuites", "Optional list of TLS cipher suites for incoming requests over HTTPS if -tls is set. See the list of supported cipher suites at https://pkg.go.dev/crypto/tls#pkg-constants")
tlsMinVersion = flag.String("tlsMinVersion", "", "Optional minimum TLS version to use for incoming requests over HTTPS if -tls is set. "+
"Supported values: TLS10, TLS11, TLS12, TLS13")
pathPrefix = flag.String("http.pathPrefix", "", "An optional prefix to add to all the paths handled by http server. For example, if '-http.pathPrefix=/foo/bar' is set, "+ pathPrefix = flag.String("http.pathPrefix", "", "An optional prefix to add to all the paths handled by http server. For example, if '-http.pathPrefix=/foo/bar' is set, "+
"then all the http requests will be handled on '/foo/bar/*' paths. This may be useful for proxied requests. "+ "then all the http requests will be handled on '/foo/bar/*' paths. This may be useful for proxied requests. "+
@ -95,9 +97,9 @@ func Serve(addr string, rh RequestHandler) {
logger.Infof("pprof handlers are exposed at %s://%s/debug/pprof/", scheme, hostAddr) logger.Infof("pprof handlers are exposed at %s://%s/debug/pprof/", scheme, hostAddr)
var tlsConfig *tls.Config var tlsConfig *tls.Config
if *tlsEnable { if *tlsEnable {
tc, err := netutil.GetServerTLSConfig(*tlsCertFile, *tlsKeyFile, *tlsCipherSuites) tc, err := netutil.GetServerTLSConfig(*tlsCertFile, *tlsKeyFile, *tlsMinVersion, *tlsCipherSuites)
if err != nil { if err != nil {
logger.Fatalf("cannot load TLS cert from -tlsCertFile=%q, -tlsKeyFile=%q: %s", *tlsCertFile, *tlsKeyFile, err) logger.Fatalf("cannot load TLS cert from -tlsCertFile=%q, -tlsKeyFile=%q, -tlsMinVersion=%q: %s", *tlsCertFile, *tlsKeyFile, *tlsMinVersion, err)
} }
tlsConfig = tc tlsConfig = tc
} }

View file

@ -10,7 +10,7 @@ import (
) )
// GetServerTLSConfig returns TLS config for the server. // GetServerTLSConfig returns TLS config for the server.
func GetServerTLSConfig(tlsCertFile, tlsKeyFile string, tlsCipherSuites []string) (*tls.Config, error) { func GetServerTLSConfig(tlsCertFile, tlsKeyFile, minTLSVersion string, tlsCipherSuites []string) (*tls.Config, error) {
var certLock sync.Mutex var certLock sync.Mutex
var certDeadline uint64 var certDeadline uint64
var cert *tls.Certificate var cert *tls.Certificate
@ -22,10 +22,15 @@ func GetServerTLSConfig(tlsCertFile, tlsKeyFile string, tlsCipherSuites []string
if err != nil { if err != nil {
return nil, fmt.Errorf("cannot use TLS cipher suites from tlsCipherSuites=%q: %w", tlsCipherSuites, err) return nil, fmt.Errorf("cannot use TLS cipher suites from tlsCipherSuites=%q: %w", tlsCipherSuites, err)
} }
minVersion, err := ParseTLSVersion(minTLSVersion)
if err != nil {
return nil, fmt.Errorf("cannnot use TLS min version from minTLSVersion=%q. Supported TLS versions (TLS10, TLS11, TLS12, TLS13): %w", minTLSVersion, err)
}
cert = &c cert = &c
cfg := &tls.Config{ cfg := &tls.Config{
MinVersion: tls.VersionTLS12, MinVersion: minVersion,
PreferServerCipherSuites: true, // Do not set MaxVersion, since this has no sense from security PoV.
// This can only result in lower security level if improperly set.
GetCertificate: func(info *tls.ClientHelloInfo) (*tls.Certificate, error) { GetCertificate: func(info *tls.ClientHelloInfo) (*tls.Certificate, error) {
certLock.Lock() certLock.Lock()
defer certLock.Unlock() defer certLock.Unlock()
@ -63,3 +68,22 @@ func cipherSuitesFromNames(cipherSuiteNames []string) ([]uint16, error) {
} }
return cipherSuites, nil return cipherSuites, nil
} }
// ParseTLSVersion returns tls version from the given string s.
func ParseTLSVersion(s string) (uint16, error) {
switch strings.ToUpper(s) {
case "":
// Special case - use default TLS version provided by tls package.
return 0, nil
case "TLS13":
return tls.VersionTLS13, nil
case "TLS12":
return tls.VersionTLS12, nil
case "TLS11":
return tls.VersionTLS11, nil
case "TLS10":
return tls.VersionTLS10, nil
default:
return 0, fmt.Errorf("unsupported TLS version %q", s)
}
}

View file

@ -1,6 +1,7 @@
package netutil package netutil
import ( import (
"crypto/tls"
"reflect" "reflect"
"testing" "testing"
) )
@ -76,3 +77,44 @@ func TestCipherSuitesFromNames(t *testing.T) {
}) })
} }
} }
func TestParseTLSVersionSuccess(t *testing.T) {
f := func(s string, want uint16) {
t.Helper()
got, err := ParseTLSVersion(s)
if err != nil {
t.Fatalf("unexpected error for ParseTLSVersion(%q): %s", s, err)
}
if got != want {
t.Fatalf("unexpected value got from ParseTLSVersion(%q); got %d; want %d", s, got, want)
}
}
// lowercase tlsName
f("tls10", tls.VersionTLS10)
f("tls11", tls.VersionTLS11)
f("tls12", tls.VersionTLS12)
f("tls13", tls.VersionTLS13)
// uppercase tlsName
f("TLS10", tls.VersionTLS10)
f("TLS11", tls.VersionTLS11)
f("TLS12", tls.VersionTLS12)
f("TLS13", tls.VersionTLS13)
// empty tlsName
f("", 0)
}
func TestParseTLSVersionFailure(t *testing.T) {
f := func(s string) {
t.Helper()
_, err := ParseTLSVersion(s)
if err == nil {
t.Fatalf("expecting non-nil error for ParseTLSVersion(%q)", s)
}
}
// incorrect tlsName
f("123")
// incorrect tlsName with correct prefix
f("TLS1")
// incorrect tls version in tlsName
f("TLS14")
}

View file

@ -15,6 +15,7 @@ import (
"github.com/VictoriaMetrics/VictoriaMetrics/lib/fasttime" "github.com/VictoriaMetrics/VictoriaMetrics/lib/fasttime"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/fs" "github.com/VictoriaMetrics/VictoriaMetrics/lib/fs"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/logger" "github.com/VictoriaMetrics/VictoriaMetrics/lib/logger"
"github.com/VictoriaMetrics/VictoriaMetrics/lib/netutil"
"github.com/VictoriaMetrics/fasthttp" "github.com/VictoriaMetrics/fasthttp"
"github.com/cespare/xxhash/v2" "github.com/cespare/xxhash/v2"
"golang.org/x/oauth2" "golang.org/x/oauth2"
@ -79,6 +80,8 @@ type TLSConfig struct {
ServerName string `yaml:"server_name,omitempty"` ServerName string `yaml:"server_name,omitempty"`
InsecureSkipVerify bool `yaml:"insecure_skip_verify,omitempty"` InsecureSkipVerify bool `yaml:"insecure_skip_verify,omitempty"`
MinVersion string `yaml:"min_version,omitempty"` MinVersion string `yaml:"min_version,omitempty"`
// Do not define MaxVersion field (max_version), since this has no sense from security PoV.
// This can only result in lower security level if improperly set.
} }
// String returns human-readable representation of tc // String returns human-readable representation of tc
@ -399,6 +402,8 @@ func (ac *Config) NewTLSConfig() *tls.Config {
tlsCfg.ServerName = ac.TLSServerName tlsCfg.ServerName = ac.TLSServerName
tlsCfg.InsecureSkipVerify = ac.TLSInsecureSkipVerify tlsCfg.InsecureSkipVerify = ac.TLSInsecureSkipVerify
tlsCfg.MinVersion = ac.TLSMinVersion tlsCfg.MinVersion = ac.TLSMinVersion
// Do not set tlsCfg.MaxVersion, since this has no sense from security PoV.
// This can only result in lower security level if improperly set.
return tlsCfg return tlsCfg
} }
@ -713,27 +718,10 @@ func (tctx *tlsContext) initFromTLSConfig(baseDir string, tc *TLSConfig) error {
return fmt.Errorf("cannot parse data from `ca_file` %q", tc.CAFile) return fmt.Errorf("cannot parse data from `ca_file` %q", tc.CAFile)
} }
} }
if tc.MinVersion != "" { v, err := netutil.ParseTLSVersion(tc.MinVersion)
v, err := parseTLSVersion(tc.MinVersion) if err != nil {
if err != nil { return fmt.Errorf("cannot parse `min_version`: %w", err)
return fmt.Errorf("cannot parse `min_version`: %w", err)
}
tctx.minVersion = v
} }
tctx.minVersion = v
return nil return nil
} }
func parseTLSVersion(s string) (uint16, error) {
switch strings.ToUpper(s) {
case "TLS13":
return tls.VersionTLS13, nil
case "TLS12":
return tls.VersionTLS12, nil
case "TLS11":
return tls.VersionTLS11, nil
case "TLS10":
return tls.VersionTLS10, nil
default:
return 0, fmt.Errorf("unsupported TLS version %q", s)
}
}