Aliaksandr Valialkin
c080443fef
app/vmagent: automatically detect whether the remote storage supports VictoriaMetrics remote write protocol
...
Substitute -remoteWrite.useVMProto with -remoteWrite.forcePromProto command-line flag,
which can be used for forcing Prometheus remote write protocol in cases when the remote storage
supports VictoriaMetrics remote write protocol.
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3847
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1225
2023-02-23 17:36:55 -08:00
Artem Navoiev
6b4ccc17c2
docs: fix typo OpentTSDB -> OpenTSDB ( #3854 )
...
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2023-02-22 11:24:26 +01:00
Aliaksandr Valialkin
76f2c70be3
app/vmagent: add support for VictoriaMetrics remote write protocol, which allows saving up to 10x on network bandwidth costs under high load
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1225
2023-02-20 19:11:30 -08:00
Aliaksandr Valialkin
5c4f5b83fc
all: rename ParseStream -> stream.Parse
...
This is a follow-up for 057698f7fb
2023-02-13 10:52:05 -08:00
Aliaksandr Valialkin
ccdddf7996
lib/protoparser/promremotewrite: extract stream parsing code into a separate stream package
...
This is a follow-up for 057698f7fb
2023-02-13 10:46:54 -08:00
Aliaksandr Valialkin
9be1398b92
lib/protoparser/native: extract stream parsing code into a separate stream package
...
This is a follow-up for 057698f7fb
2023-02-13 10:43:05 -08:00
Aliaksandr Valialkin
8830607021
lib/protoparser/graphite: extract stream parsing code into a separate stream package
2023-02-13 10:32:36 -08:00
Aliaksandr Valialkin
a646841c07
lib/protoparser/csvimport: extract stream parsing code into a separate stream package
...
This is a follow-up for 057698f7fb
2023-02-13 10:25:46 -08:00
Aliaksandr Valialkin
7568658c19
lib/protoparser/vmimport: extract stream parsing code into a separate stream package
...
This is a follow-up for 057698f7fb
2023-02-13 10:20:19 -08:00
Aliaksandr Valialkin
af37717108
lib/protoparser/opentsdbhttp: extract stream parsing code into a separate stream package
...
This is a follow-up for 057698f7fb
2023-02-13 10:16:03 -08:00
Aliaksandr Valialkin
7720d403c0
lib/protoparser/opentsdb: extract stream parsing code into a separate stream package
...
This is a follow-up for 057698f7fb
2023-02-13 10:03:16 -08:00
Aliaksandr Valialkin
fe196e0b7a
lib/protoparser/influx: extract stream parsing code into a separate stream package
...
This is a follow-up for 057698f7fb
2023-02-13 09:58:52 -08:00
Aliaksandr Valialkin
f83d6d69b2
lib/protoparser/datadog: extract stream parsing code into a separate stream package
...
This is a follow-up for 057698f7fb
2023-02-13 09:51:47 -08:00
Roman Khavronenko
057698f7fb
lib/protoparser/prometheus: move streamparser to subpackage ( #3814 )
...
`lib/protoparser/prometheus` is used by various applications,
such as `app/vmalert`. The recent change to the
`lib/protoparser/prometheus` package introduced a new dependency
of `lib/writeconcurrencylimiter` which exposes some metrics.
Because of the dependency, now all applications which have this
dependency also expose these metrics.
Creating a new `lib/protoparser/prometheus/stream` package helps
to remove these metrics from apps which use `lib/protoparser/prometheus`
as dependency.
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3761
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-02-13 09:26:07 -08:00
Oleksandr Redko
9fff48c3e3
app,lib: fix typos in comments ( #3804 )
2023-02-13 13:27:13 +01:00
Nikolay
73256fe438
lib/netutil: init implimentation of proxy protocol ( #3687 )
...
* lib/netutil: init implimentation of proxy protocol
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3335
* wip
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2023-01-26 23:08:35 -08:00
Aliaksandr Valialkin
d655d6b047
lib/streamaggr: add ability to de-duplicate input samples before aggregation
2023-01-25 09:14:49 -08:00
Aliaksandr Valialkin
f7acdb13db
app/{vmagent,vminsert}: follow-up for 1cfa183c2b
...
- Call httpserver.GetQuotedRemoteAddr() and httpserver.GetRequestURI() only when the error occurs.
This saves CPU time on fast path when there are no parsing errors.
- Create a helper function - httpserver.LogError() - for logging the error with the request uri and remote addr context.
2023-01-23 22:26:53 -08:00
Artem Navoiev
1cfa183c2b
add error handler for parsing prometheus text format to vmagent and v… ( #3693 )
...
* add error handler for parsing prometheus text format to vmagent and vminsert
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
* fix typo
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
* typo
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
* fix variables naming and error message
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2023-01-23 22:14:34 -08:00
Aliaksandr Valialkin
cdb6d651e9
app/vminsert: return 200 OK status code when importing data in pushgateway format
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1415
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3636
2023-01-15 13:29:53 -08:00
Aliaksandr Valialkin
95ce1ba6ce
lib/httpserver: directly pass flag value to CheckAuthFlag()
...
There is no sense in passing a pointer to flag value there.
This is a follow-up for 4225a0bd75
2023-01-10 15:52:23 -08:00
Zakhar Bessarab
4225a0bd75
Use httpAuth.* flags as a fallback for endpoints protected by *AuthKey flags ( #3582 )
...
* {lib/server, app/}: use `httpAuth.*` flag as fallback for `*AuthKey` if it is not set
* lib/ingestserver/opentsdbhttp: fix opentdb HTTP handler not respecting `httpAuth.*` flags
* Apply suggestions from code review
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2023-01-10 15:46:13 -08:00
Aliaksandr Valialkin
293e4dc77b
app/{vminsert,vmstorage}: add comments on why storage.AddRows() is called without limiting the number of concurrent calls
2023-01-06 22:40:07 -08:00
Aliaksandr Valialkin
c63755c316
lib/writeconcurrencylimiter: improve the logic behind -maxConcurrentInserts limit
...
Previously the -maxConcurrentInserts was limiting the number of established client connections,
which write data to VictoriaMetrics. Some of these connections could be idle.
Such connections do not consume big amounts of CPU and RAM, so there is a little sense in limiting
the number of such connections. So now the -maxConcurrentInserts command-line option
limits the number of concurrently executed insert requests, not including idle connections.
It is recommended removing -maxConcurrentInserts command-line option, since the default value
for this option should work good for most cases.
2023-01-06 22:20:19 -08:00
Aliaksandr Valialkin
fd175ad80b
docs: update -help outputs for vm* tools
2023-01-03 23:27:06 -08:00
Aliaksandr Valialkin
fa13bbc48a
app/{vmagent,vminsert}: add support for streaming aggregation
...
See https://docs.victoriametrics.com/stream-aggregation.html
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3460
2023-01-03 22:19:21 -08:00
Aliaksandr Valialkin
88597f187b
app/{vmagent,vminsert}/datadog: make the host label optional in DataDog data ingestion protocol
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3432
2022-12-10 23:32:31 -08:00
Aliaksandr Valialkin
a30ae502ef
lib/promscrape: allow editing relabeling configs and labels at /target-relabel-debug page
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3407
2022-12-10 12:44:45 -08:00
Aliaksandr Valialkin
3e7276639e
app/{vminsert,vmselect}: move the handler for /metric-relabel-debug from vminsert to vmselect to be consistent with the cluster version
2022-12-10 02:45:40 -08:00
Aliaksandr Valialkin
a8b8e23d68
lib/promscrape: implement target-level and metric-level relabel debugging
...
Target-level debugging is performed by clicking the 'debug' link at the corresponding target
on either http://vmagent:8429/targets page or on http://vmagent:8428/service-discovery page.
Metric-level debugging is perfromed at http://vmagent:8429/metric-relabel-debug page.
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3407
See https://docs.victoriametrics.com/vmagent.html#relabel-debug
2022-12-10 02:09:44 -08:00
Pedro Gonçalves
1e0666abb4
Datadog - Add device as a tag if it's present as a field in the series object ( #3431 )
...
* Datadog - Add device as a tag if it's present as a field in the series object
* address PR comments
2022-12-05 23:06:03 -08:00
Aliaksandr Valialkin
58d459e8a8
app/{vminsert,vmagent}: follow-up after 53a63c6c4c
...
Extend /api/v1/import/prometheus with the support for Pushgateway way of specifying additional labels.
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1415
2022-11-25 16:48:14 -08:00
Aliaksandr Valialkin
9498f871e7
app/vminsert: add missing vm_relabel_config_* metrics after 03d88bc066
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3345
2022-11-22 00:47:49 +02:00
Aliaksandr Valialkin
50f5eae0e0
lib/promrelabel: remove unconditional sorting of the labels in ParsedConfigs.Apply(), since the sorting isnt needed in many places
...
Sort labels explicitly after calling the ParsedConfigs.Apply() when needed.
This reduces CPU usage when performing metric-level relabeling, where labels' sorting isn't needed.
2022-10-09 14:51:16 +03:00
Aliaksandr Valialkin
ef435f8cc4
lib/promrelabel: add SanitizeName() function for sanitizing Prometheus metric names and label names
...
Optimize this function by using results cache for input strings.
Use this function all over the code.
This is a follow-up for fcffdba9dchttps://github.com/VictoriaMetrics/VictoriaMetrics/issues/3113
2022-09-28 10:40:59 +03:00
Aliaksandr Valialkin
fcffdba9dc
app/{vmagent,vminsert}: add -usePromCompatibleNaming command-line flag for normalizing metric names and label names in the ingested samples
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3113
Thanks to @erkexzcx for the idea and the initial pull request at https://github.com/VictoriaMetrics/VictoriaMetrics/pull/3146
2022-09-26 13:11:39 +03:00
Aliaksandr Valialkin
46d7792b72
lib/promscrape: follow-up after 2c553d5a2f
...
- fix broken tests
- cosmetic code cleanup
- document the change at https://docs.victoriametrics.com/vmagent.html#multitenancy
- document the change at https://docs.victoriametrics.com/CHANGELOG.html
2022-08-08 14:46:26 +03:00
Aliaksandr Valialkin
5ced032d66
all: follow-up after 46f803fa7a
...
Add -pushmetrics.* command-line flags to all the VictoriaMetrics apps
2022-07-21 20:36:27 +03:00
Aliaksandr Valialkin
4ce5875fa8
all: add ability to push internal metrics to remote storage system specified via -pushmetrics.url
2022-07-21 20:36:27 +03:00
Aliaksandr Valialkin
10cb67adb5
app/{vmagent,vminsert}: follow-up after d19e46de55
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/2839
2022-07-07 01:30:58 +03:00
Pedro Gonçalves
d19e46de55
app/vminsert: allow to ingest datadog metrics with simpler tags - not enforcing key:value ( #2839 )
2022-07-07 01:18:00 +03:00
Dmytro Kozlov
4583ed23a8
Added a stub for datadog endpoint ( #2710 )
...
* Added a stub for datadog endpoint
* Update app/vmagent/main.go
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-06-13 09:52:13 +03:00
Aliaksandr Valialkin
638ba4614a
docs/CHANGELOG.md: document e755d0ec3f
2022-06-07 15:16:48 +03:00
elProxy
e755d0ec3f
Support legacy datadog agent ( #2670 )
...
dd-agent v5 can issue some requests with trailing slashes.
(e.g.
526559be73/ddagent.py (L303)
2022-06-07 15:06:18 +03:00
Aliaksandr Valialkin
8608dd093c
all: follow-up after 8edb390e21
...
- Remove unused js bloatware from /targets page. This strips down binary size by more than 100Kb
- Add /service-discovery page for API compatibility with Prometheus
- Properly load bootstrap.min.css from /prometheus/targets
- Serve static contents for /targets page from app/vminsert instead of app/vmselect, because /targets page is served from there
2022-06-07 00:57:09 +03:00
Aliaksandr Valialkin
7747708ca7
app: expose /api/v1/status/config endpoint in the same way as Prometheus does
...
This endpoint is needed for third-party tools.
See https://prometheus.io/docs/prometheus/latest/querying/api/#config
2022-05-25 09:57:13 +03:00
Aliaksandr Valialkin
ce47d18052
app/{vmagent,vminsert}: mention port 8089 instead of 8189 in the description for -influxListenAddr flag
...
InfluxDB uses 8089 port for sending plain Influx line protocol data over TCP and UDP.
See https://docs.influxdata.com/influxdb/v1.8/administration/ports/
This is a follow-up for 20cef877a1
2022-05-13 16:50:45 +03:00
Nikolay
0c0efc7781
vmctl verify-blocks command ( #2390 )
...
* lib/protoparser: changes ParseStream for native format
uses reader instead of http.Request
updates app/vmagent and app/vmagent method usage
* app/vmctl: add verify-block subcommand
it allows to check exported from VictoriaMetrics data block in native format
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2362
Update app/vmctl/README.md
Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
2022-04-05 16:01:32 +02:00
Aliaksandr Valialkin
71ef3155c8
app/vminsert: add X-Influxdb-Version response header for InfluxDB API requests
...
This is needed for some clients, which expect this header.
See https://github.com/ntop/ntopng/issues/5449#issuecomment-1005347597
2022-02-17 12:47:43 +02:00
Eugene
3d19fa6932
Add -influxDBLabel parameter ( #2203 )
2022-02-16 23:27:06 +02:00
