Commit graph

25 commits

Author SHA1 Message Date
Aliaksandr Valialkin
8b1c38abde
app/vmauth: follow-up for 3a45bbb4e0
- Move the test for SRV discovery into a separate function. This allows verifying round-robin discovery across SRV records.
- Restore the original netutil.Resolver after the test finishes, so it doesn't interfere with other tests.
- Move the description of the bugfix into the correct place at docs/CHANGELOG.md - it should be placed under v1.102.0-rc2
  instead of v1.102.0-rc1.
- Remove unneeded code in URLPrefix.sanitizeAndInitialize(), since it is expected this function is called only once
  for finishing URLPrefix initializiation. In this case URLPrefix.nextDiscoveryDeadline and URLPrefix.n are equal to 0
  according to https://pkg.go.dev/sync/atomic#Uint64
- Properly fix the bug at URLPrefix.discoverBackendAddrsIfNeeded() - it is expected that hostToAddrs map uses
  the original hostname keys, including 'srv+' prefix, so it shouldn't be removed when looping over up.busOriginal.
  Instead, the 'srv+' prefix must be removed from the hostname only locally before passing the hostname to netutil.Resolver.LookupSRV.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6401
2024-07-16 10:41:08 +02:00
LHHDZ
41e4135371
app/vmauth: fix discovering backend IPs when url_prefix contains hostname with srv+ prefix (#6401)
This change fixes the following panic:
```
2024-06-04T11:16:52.899Z        warn    app/vmauth/auth_config.go:353   cannot discover backend SRV records for http://srv+localhost:8080: lookup localhost on 10.100.10.4:53: server misbehaving; use it literally
panic: runtime error: integer divide by zero

goroutine 9 [running]:
github.com/VictoriaMetrics/VictoriaMetrics/lib/httpserver.handlerWrapper.func1()
        /Users/lhhdz/wd/projects/go/VictoriaMetrics/lib/httpserver/httpserver.go:291 +0x58
panic({0x103115100?, 0x10338d700?})
        /Users/lhhdz/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.3.darwin-arm64/src/runtime/panic.go:770 +0x124
main.getLeastLoadedBackendURL({0x0?, 0x22?, 0x1400014757b?}, 0x1400013c120?)
        /Users/lhhdz/wd/projects/go/VictoriaMetrics/app/vmauth/auth_config.go:473 +0x210
main.(*URLPrefix).getBackendURL(0x140000aa080)
        /Users/lhhdz/wd/projects/go/VictoriaMetrics/app/vmauth/auth_config.go:312 +0xb8
```

---------

Co-authored-by: Haley Wang <haley@victoriametrics.com>
2024-06-12 11:47:44 +02:00
Aliaksandr Valialkin
6cdf01297e
app/vmauth: follow-up for b155b20de4
- Use exact matching by default for the query arg value provided via arg=value syntax at src_query_args.
  Regex matching can be enabled by using =~ instead of = . For example, arg=~regex.
  This ensures that the exact matching works as expected without the need to escape special regex chars.

- Add helper functions for creating QueryArg, Header and Regex structs in tests.
  This improves maintainability of the tests.

- Remove url.QueryUnescape() call on the url in TestCreateTargetURLSuccess(), since this is bogus approach.
  The url.QueryUnescape() must be applied to individual query args, and it mustn't be applied to the whole url,
  since in this case it may perform invalid unescaping in the context of the url, or make the resulting url invalid.

While at it, properly marshal all the fields inside UserInfo config to yaml in tests.
Previously Header and QueryArg structs were improperly marshaled because the custom MarshalYAML
is called only on pointers to Header and QueryArg structs. This improves test coverage.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6070
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6115
2024-04-17 14:53:20 +02:00
Roman Khavronenko
256c561005
app/vmauth: support regex matching in src_query_args (#6115)
Support regex matching when routing incoming requests based on HTTP query args
via `src_query_args` option at `url_map`.

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6070

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-04-17 14:52:02 +02:00
Aliaksandr Valialkin
93ef217857
app/vmauth: allow discovering backend ips behind shared hostname and spreading load among the discovered ips
This is done with the `discover_backend_ips` option at `user` and `url_map` level.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5707
2024-03-07 01:02:38 +02:00
Aliaksandr Valialkin
b2e182ca62
app/vmauth: add src_headers option at url_map, which allows routing incoming requests to different backends depending on request headers 2024-03-06 21:56:42 +02:00
Aliaksandr Valialkin
87d59cdfa5
app/vmauth: add ability to route requests based on HTTP query args via src_query_args option
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5878
2024-03-06 20:52:42 +02:00
Aliaksandr Valialkin
4ee42e9e73
app/vmauth: allow specifying an empty retry_status_codes and and zero drop_src_path_prefix_parts in order to override user-level setting
Previously `retry_status_codes: []` and `drop_src_path_prefix_parts: 0` at `url_map` were equivalent to missing values.
This was resulting in using the user-level values instead.
2023-12-14 01:06:50 +02:00
Aliaksandr Valialkin
51acf0179c
app/vmauth: add ability to route requests to different backends depending on the request host 2023-12-14 00:47:00 +02:00
Aliaksandr Valialkin
49552eaa15
app/vmauth: add support for hot standby mode via first_available load balancing policy
vmauth in `hot standby` mode sends requests to the first url_prefix while it is available.
If the first url_prefix becomes unavailable, then vmauth falls back to the next url_prefix.
This allows building highly available setup as described at https://docs.victoriametrics.com/vmauth.html#high-availability

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4893
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4792
2023-12-08 23:32:10 +02:00
Aliaksandr Valialkin
c1f651a9f9
app/vmauth: add ability to drop the specified number of /-delimited prefix parts from request path
This can be done via `drop_src_path_prefix_parts` option at `url_map` and `user` levels.

See https://docs.victoriametrics.com/vmauth.html#dropping-request-path-prefix
2023-11-13 22:34:40 +01:00
Aliaksandr Valialkin
23bb38ef80
app/vmauth: add ability to specify response status codes for retrying requests during load-balancing
Response status codes for retrying can be specified via retry_status_codes list

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4893
2023-09-08 23:26:36 +02:00
Aliaksandr Valialkin
953699d59a
app/vmauth: add tests for ResponseHeaders
This is a follow-up for b18eed3427

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4825
2023-09-01 09:22:01 +02:00
Alexander Marshalov
9ba352da0f
vmauth: added ability to set and remove response headers (#4825) (#4914)
* added ability to set and clear response headers (#4825)

Signed-off-by: Alexander Marshalov <_@marshalov.org>

* added ability to set and clear response headers (#4825)

Signed-off-by: Alexander Marshalov <_@marshalov.org>

* fix review comment

Signed-off-by: Alexander Marshalov <_@marshalov.org>

---------

Signed-off-by: Alexander Marshalov <_@marshalov.org>
2023-09-01 08:57:52 +02:00
Alexander Marshalov
402d906d40
added default_url field in vmauth users config (#4084) (#4156)
* added default url field in vmauth users config (#4084)

---------

Signed-off-by: Alexander Marshalov <_@marshalov.org>
2023-05-08 15:13:46 -07:00
Aliaksandr Valialkin
9ed5b872df
app/vmauth: improve load balancing by sending incoming requests to backends with the lowest number of concurrent requests
While at it, stop sending requests to unavailable backend for 3 seconds
before the next attempt. This should reduce the amounts of useless work
and the number of useless network packets when the backend is temporarily unavailable.
2023-02-11 00:32:56 -08:00
Aliaksandr Valialkin
769d7da25a
app/vmauth: automatically retry failing GET requests on the remaining backends 2023-02-09 21:06:09 -08:00
Aliaksandr Valialkin
7dc38fba72
app/vmauth: consistency renaming: UserInfo.URLMap -> UserInfo.URLMaps
This is based on https://github.com/VictoriaMetrics/VictoriaMetrics/pull/3486
2023-01-27 00:19:46 -08:00
Aliaksandr Valialkin
559dd996c4
app/vmauth: add ability to specify http headers to send in requests to backends
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1736
2021-10-22 19:11:35 +03:00
benclive
76816a0193 Remove trailing slash for URLPrefixes with specific path (#1554) 2021-08-25 13:32:04 +03:00
Aliaksandr Valialkin
e7c4fde756 app/vmauth: parse url_prefix only once during config load 2021-04-21 10:57:17 +03:00
Nikolay
7d249d787d adds query params support for vmauth urlPrefix (#1226)
* adds query params support for vmauth urlPrefix

* Update app/vmauth/example_config.yml

* Update app/vmauth/example_config.yml

Co-authored-by: Aliaksandr Valialkin <valyala@gmail.com>
2021-04-20 10:59:20 +03:00
Aliaksandr Valialkin
26cb6f8861 app/vmauth: allow using regexps in url_map paths
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1112
2021-03-05 18:53:40 +02:00
Aliaksandr Valialkin
9e88ff3075 app/vmauth: add ability to route requests from a single users to multiple targets depending on the requested path
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1064
2021-02-11 12:41:38 +02:00
Aliaksandr Valialkin
2b403d3f42 app/vmauth: prevent from attacks with .. in path for accessing resources outside the configured url_prefix 2020-05-07 12:55:04 +03:00