From 5d8596865999f3eca4547f0fdadf80c79473efb2 Mon Sep 17 00:00:00 2001 From: andriibeee <154226341+andriibeee@users.noreply.github.com> Date: Fri, 15 Nov 2024 13:28:25 +0200 Subject: [PATCH 1/3] app/vmauth: fix unauthorized_user routing inconsistency This commit makes vmauth respect the routing config for unauthorized requests for requests that despite having Authorization header failed to authorize successfully. It covers the following use-cases: - vmauth is used at load-balanacer and must forward requests as is. There is no any authorization configs. - vmauth has authorization config, but it must forward requests with invalid credential tokens to some other backend. related issue: https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7543 --------- Signed-off-by: Andrii --- app/vmauth/main.go | 6 ++++++ app/vmauth/main_test.go | 14 ++++++++++++++ docs/changelog/CHANGELOG.md | 1 + 3 files changed, 21 insertions(+) diff --git a/app/vmauth/main.go b/app/vmauth/main.go index 466147bf2..bcf296b4f 100644 --- a/app/vmauth/main.go +++ b/app/vmauth/main.go @@ -123,6 +123,12 @@ func requestHandler(w http.ResponseWriter, r *http.Request) bool { ui := getUserInfoByAuthTokens(ats) if ui == nil { + uu := authConfig.Load().UnauthorizedUser + if uu != nil { + processUserRequest(w, r, uu) + return true + } + invalidAuthTokenRequests.Inc() if *logInvalidAuthTokens { err := fmt.Errorf("cannot authorize request with auth tokens %q", ats) diff --git a/app/vmauth/main_test.go b/app/vmauth/main_test.go index 6414ef803..9078de8a4 100644 --- a/app/vmauth/main_test.go +++ b/app/vmauth/main_test.go @@ -90,6 +90,20 @@ User-Agent: vmauth X-Forwarded-For: 12.34.56.78, 42.2.3.84` f(cfgStr, requestURL, backendHandler, responseExpected) + // routing of all failed to authorize requests to unauthorized_user (issue #7543) + cfgStr = ` +unauthorized_user: + url_prefix: "{BACKEND}/foo" + keep_original_host: true` + requestURL = "http://foo:invalid-secret@some-host.com/abc/def" + backendHandler = func(w http.ResponseWriter, r *http.Request) { + fmt.Fprintf(w, "requested_url=http://%s%s", r.Host, r.URL) + } + responseExpected = ` +statusCode=200 +requested_url=http://some-host.com/foo/abc/def` + f(cfgStr, requestURL, backendHandler, responseExpected) + // keep_original_host cfgStr = ` unauthorized_user: diff --git a/docs/changelog/CHANGELOG.md b/docs/changelog/CHANGELOG.md index 315e3da9e..03039e448 100644 --- a/docs/changelog/CHANGELOG.md +++ b/docs/changelog/CHANGELOG.md @@ -20,6 +20,7 @@ See also [LTS releases](https://docs.victoriametrics.com/lts-releases/). * SECURITY: upgrade Go builder from Go1.23.1 to Go1.23.3. See the list of issues addressed in [Go1.23.2](https://github.com/golang/go/issues?q=milestone%3AGo1.23.2+label%3ACherryPickApproved) and [Go1.23.3](https://github.com/golang/go/issues?q=milestone%3AGo1.23.3+label%3ACherryPickApproved). +* BUGFIX: [vmauth](https://docs.victoriametrics.com/vmauth/): fixed unauthorized routing behavior inconsistency. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7543) for details. * BUGFIX: [vmctl](https://docs.victoriametrics.com/vmctl/): drop rows that do not belong to the current series during import. The dropped rows should belong to another series whose tags are a superset of the current series. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7301) and [this pull request](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/7330). Thanks to @dpedu for reporting and cooperating with the test. * BUGFIX: [vmsingle](https://docs.victoriametrics.com/single-server-victoriametrics/), `vmselect` in [VictoriaMetrics cluster](https://docs.victoriametrics.com/cluster-victoriametrics/): keep the order of resulting time series when `limit_offset` is applied. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7068). * BUGFIX: [graphite](https://docs.victoriametrics.com/#graphite-render-api-usage): properly handle xFilesFactor=0 for `transformRemoveEmptySeries` function. See [this PR](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/7337) for details. From 47a52f5f409d5c1b534b14072f02c5ad8ff4c105 Mon Sep 17 00:00:00 2001 From: Aliaksandr Valialkin Date: Fri, 15 Nov 2024 14:15:06 +0100 Subject: [PATCH 2/3] docs/Articles.md: fix broken link to `Open-sourcing VictoriaMetrics` article --- docs/Articles.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/Articles.md b/docs/Articles.md index 7a976abc0..ae0e81ced 100644 --- a/docs/Articles.md +++ b/docs/Articles.md @@ -92,7 +92,7 @@ See also [case studies](https://docs.victoriametrics.com/casestudies/). ### Announcements * [Open-source strategy at VictoriaMetrics](https://www.youtube.com/watch?v=-DbbIZzFHIY) -* [Open-sourcing VictoriaMetrics](https://blog.usejournal.com/open-sourcing-victoriametrics-f31e34485c2b) +* [Open-sourcing VictoriaMetrics](https://valyala.medium.com/open-sourcing-victoriametrics-f31e34485c2b) * [VictoriaMetrics — creating the best remote storage for Prometheus](https://faun.pub/victoriametrics-creating-the-best-remote-storage-for-prometheus-5d92d66787ac) * [Anomaly Detection in VictoriaMetrics](https://victoriametrics.medium.com/anomaly-detection-in-victoriametrics-9528538786a7) From fc0b6c62fe99b47b5e48b053ca1a35c3ca37ee4d Mon Sep 17 00:00:00 2001 From: Aliaksandr Valialkin Date: Fri, 15 Nov 2024 14:29:01 +0100 Subject: [PATCH 3/3] docs: refer to the https://itnext.io/how-do-open-source-solutions-for-logs-work-elasticsearch-loki-and-victorialogs-9f7097ecbc2f in the appropriate places --- docs/Articles.md | 1 + docs/VictoriaLogs/FAQ.md | 2 +- docs/VictoriaLogs/README.md | 2 +- 3 files changed, 3 insertions(+), 2 deletions(-) diff --git a/docs/Articles.md b/docs/Articles.md index ae0e81ced..f80c41492 100644 --- a/docs/Articles.md +++ b/docs/Articles.md @@ -119,6 +119,7 @@ See also [case studies](https://docs.victoriametrics.com/casestudies/). * [Improving histogram usability for Prometheus and Grafana](https://valyala.medium.com/improving-histogram-usability-for-prometheus-and-grafana-bc7e5df0e350) * [Why irate from Prometheus doesn't capture spikes](https://valyala.medium.com/why-irate-from-prometheus-doesnt-capture-spikes-45f9896d7832) * [VictoriaMetrics: PromQL compliance](https://medium.com/@romanhavronenko/victoriametrics-promql-compliance-d4318203f51e) +* [How do open source solutions for logs work: Elasticsearch, Loki and VictoriaLogs](https://itnext.io/how-do-open-source-solutions-for-logs-work-elasticsearch-loki-and-victorialogs-9f7097ecbc2f) ### Tutorials, guides and how-to articles diff --git a/docs/VictoriaLogs/FAQ.md b/docs/VictoriaLogs/FAQ.md index 9dbafc9ae..98f8df3b0 100644 --- a/docs/VictoriaLogs/FAQ.md +++ b/docs/VictoriaLogs/FAQ.md @@ -34,7 +34,7 @@ VictoriaLogs is optimized specifically for logs. So it provides the following fe - Easy to setup and operate. There is no need in tuning configuration for optimal performance or in creating any indexes for various log types. Just run VictoriaLogs on the most suitable hardware, ingest logs into it via [supported data ingestion protocols](https://docs.victoriametrics.com/victorialogs/data-ingestion/) and get the best available performance out of the box. -- Up to 30x less RAM usage than Elasticsearch for the same workload. +- Up to 30x less RAM usage than Elasticsearch for the same workload. See [this article](https://itnext.io/how-do-open-source-solutions-for-logs-work-elasticsearch-loki-and-victorialogs-9f7097ecbc2f) for details. - Up to 15x less disk space usage than Elasticsearch for the same amounts of stored logs. - Ability to work efficiently with hundreds of terabytes of logs on a single node. - Easy to use query language optimized for typical log analysis tasks - [LogsQL](https://docs.victoriametrics.com/victorialogs/logsql/). diff --git a/docs/VictoriaLogs/README.md b/docs/VictoriaLogs/README.md index 36e259cc0..68c779354 100644 --- a/docs/VictoriaLogs/README.md +++ b/docs/VictoriaLogs/README.md @@ -15,7 +15,7 @@ VictoriaLogs provides the following features: - VictoriaLogs' capacity and performance scales linearly with the available resources (CPU, RAM, disk IO, disk space). It runs smoothly on Raspberry PI and on servers with hundreds of CPU cores and terabytes of RAM. - It can handle up to 30x bigger data volumes than Elasticsearch and Grafana Loki when running on the same hardware. - See [these docs](#benchmarks). + See [these docs](#benchmarks) and [this article](https://itnext.io/how-do-open-source-solutions-for-logs-work-elasticsearch-loki-and-victorialogs-9f7097ecbc2f) for details. - It provides fast full-text search out of the box for [log fields](https://docs.victoriametrics.com/victorialogs/keyconcepts/#data-model) with high cardinality (e.g. high number of unique values) such as `trace_id`, `user_id` and `ip`. - It supports multitenancy - see [these docs](#multitenancy).