vmauth: support for reading remote auth config file (#1898)

* add support for reading remote auth_config file via http

* fix lint

* fix defer on close body

Co-authored-by: Tiago Magalhães <tmagalhaes@wavecom.pt>

app/vmauth/README.md

@@ -4,7 +4,6 @@
 It reads auth credentials from `Authorization` http header ([Basic Auth](https://en.wikipedia.org/wiki/Basic_access_authentication) and `Bearer token` is supported),
 matches them against configs pointed by [-auth.config](#auth-config) command-line flag and proxies incoming HTTP requests to the configured per-user `url_prefix` on successful match.
 
-
 ## Quick start
 
 Just download `vmutils-*` archive from [releases page](https://github.com/VictoriaMetrics/VictoriaMetrics/releases), unpack it
@@ -26,12 +25,10 @@ Pass `-help` to `vmauth` in order to see all the supported command-line flags wi
 Feel free [contacting us](mailto:info@victoriametrics.com) if you need customized auth proxy for VictoriaMetrics with the support of LDAP, SSO, RBAC, SAML,
 accounting and rate limiting such as [vmgateway](https://docs.victoriametrics.com/vmgateway.html).
 
-
 ## Load balancing
 
 Each `url_prefix` in the [-auth.config](#auth-config) may contain either a single url or a list of urls. In the latter case `vmauth` balances load among the configured urls in a round-robin manner. This feature is useful for balancing the load among multiple `vmselect` and/or `vminsert` nodes in [VictoriaMetrics cluster](https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html).
 
-
 ## Auth config
 
 `-auth.config` is represented in the following simple `yml` format:
@@ -124,7 +121,6 @@ users:
 The config may contain `%{ENV_VAR}` placeholders, which are substituted by the corresponding `ENV_VAR` environment variable values.
 This may be useful for passing secrets to the config.
 
-
 ## Security
 
 Do not transfer Basic Auth headers in plaintext over untrusted networks. Enable https. This can be done by passing the following `-tls*` command-line flags to `vmauth`:
@@ -142,7 +138,6 @@ Alternatively, [https termination proxy](https://en.wikipedia.org/wiki/TLS_termi
 
 It is recommended protecting `/-/reload` endpoint with `-reloadAuthKey` command-line flag, so external users couldn't trigger config reload.
 
-
 ## Monitoring
 
 `vmauth` exports various metrics in Prometheus exposition format at `http://vmauth-host:8427/metrics` page. It is recommended setting up regular scraping of this page
@@ -161,7 +156,6 @@ users:
 
 It is recommended using [binary releases](https://github.com/VictoriaMetrics/VictoriaMetrics/releases) - `vmauth` is located in `vmutils-*` archives there.
 
-
 ### Development build
 
 1. [Install Go](https://golang.org/doc/install). The minimum supported version is Go 1.16.
@@ -187,7 +181,6 @@ by setting it via `<ROOT_IMAGE>` environment variable. For example, the followin
 ROOT_IMAGE=scratch make package-vmauth
 ```
 
-
 ## Profiling
 
 `vmauth` provides handlers for collecting the following [Go profiles](https://blog.golang.org/profiling-go-programs):
@@ -208,7 +201,6 @@ The command for collecting CPU profile waits for 30 seconds before returning.
 
 The collected profiles may be analyzed with [go tool pprof](https://github.com/google/pprof).
 
-
 ## Advanced usage
 
 Pass `-help` command-line arg to `vmauth` in order to see all the configuration options:
@@ -221,7 +213,7 @@ vmauth authenticates and authorizes incoming requests and proxies them to Victor
 See the docs at https://docs.victoriametrics.com/vmauth.html .
 
   -auth.config string
-    	Path to auth config. See https://docs.victoriametrics.com/vmauth.html for details on the format of this auth config
+    	Http URL or File Path to auth config. See https://docs.victoriametrics.com/vmauth.html for details on the format of this auth config
   -enableTCP6
     	Whether to enable IPv6 for listening and dialing. By default only IPv4 TCP and UDP is used
   -envflag.enable

app/vmauth/auth_config.go

@@ -5,6 +5,7 @@ import (
 	"flag"
 	"fmt"
 	"io/ioutil"
+	"net/http"
 	"net/url"
 	"os"
 	"regexp"
@@ -237,7 +238,19 @@ var authConfigWG sync.WaitGroup
 var stopCh chan struct{}
 
 func readAuthConfig(path string) (map[string]*UserInfo, error) {
-	data, err := ioutil.ReadFile(path)
+	var data []byte
+	var err error
+	// reads remote file via http, if url is given
+	if isHTTPURL(path) {
+		httpPath, err := http.Get(path)
+		if err != nil {
+			return nil, fmt.Errorf("cannot read %q: %w", path, err)
+		}
+		defer func() { _ = httpPath.Body.Close() }()
+		data, err = ioutil.ReadAll(httpPath.Body)
+	} else {
+		data, err = ioutil.ReadFile(path)
+	}
 	if err != nil {
 		return nil, fmt.Errorf("cannot read %q: %w", path, err)
 	}
@@ -372,3 +385,10 @@ func sanitizeURLPrefix(urlPrefix *url.URL) (*url.URL, error) {
 	}
 	return urlPrefix, nil
 }
+
+// isHTTPURL checks if a given targetURL is valid and contains a valid http scheme
+func isHTTPURL(targetURL string) bool {
+	parsed, err := url.Parse(targetURL)
+	return err == nil && (parsed.Scheme == "http" || parsed.Scheme == "https") && parsed.Host != ""
+
+}

app/vmauth/auth_config_test.go

@@ -368,3 +368,19 @@ func mustParseURLs(us []string) *URLPrefix {
 		urls: pus,
 	}
 }
+
+func TestIsHTTPURLSuccess(t *testing.T) {
+	assert := func(s string, expected bool) {
+		t.Helper()
+		res := isHTTPURL(s)
+		if res != expected {
+			t.Fatalf("expecting %t, got %t", expected, res)
+		}
+	}
+
+	assert("http://isvalid:8000/filepath", true)  // test http
+	assert("https://isvalid:8000/filepath", true) // test https
+	assert("tcp://notvalid:8000/filepath", false) // test tcp
+	assert("0/filepath", false)                   // something invalid
+	assert("filepath.extension", false)           // something invalid
+}