Commit graph

301 commits

Author SHA1 Message Date
Aliaksandr Valialkin
38dc127275
app/{vmagent,vminsert}: follow-up for 434a5803e7
Document the /opentelemetry/v1/metrics endpoint instead of /opentelemetry/api/v1/push,
since the /v1/metrics suffix is hardcoded in OpenTelemetry protocol specification.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5871
2024-02-29 18:10:40 +02:00
Aliaksandr Valialkin
f2ba311c80
all: upgrade Go builder from Go1.21.7 to Go1.22.0
See https://go.dev/doc/go1.22
2024-02-12 22:46:51 +02:00
Aliaksandr Valialkin
7537d75ce7
go.mod: increase the minimum supported Go version from Go1.19 to Go1.20 2023-11-02 21:29:24 +01:00
Roman Khavronenko
53b2e6aa50
app/vmselect: limit the number of parallel workers by 32 (#5195)
* app/vmselect: limit the number of parallel workers by 32

The change should improve performance and memory usage during query processing
on machines with big number of CPU cores. The number of parallel workers for
query processing is controlled via `-search.maxWorkersPerQuery` command-line flag.
By default, the number of workers is limited by the number of available CPU cores,
but not more than 32. The limit can be increased via `-search.maxWorkersPerQuery`.

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* wip

- The `-search.maxWorkersPerQuery` command-line flag doesn't limit resource usage,
  so move it from the `resource usage limits` to `troubleshooting` chapter at docs/Single-server-VictoriaMetrics.md

- Make more clear the description for the `-search.maxWorkersPerQuery` command-line flag

- Add the description of `-search.maxWorkersPerQuery` to docs/Cluster-VictoriaMetrics.md

- Limit the maximum value, which can be passed to `-search.maxWorkersPerQuery`, to GOMAXPROCS,
  because bigger values may worsen query performance and increase CPU usage

- Improve the the description of the change at docs/CHANGELOG.md. Mark it as FEATURE instead of BUGFIX,
  since it is closer to a feature than to a bugfix.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5087

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2023-10-26 11:50:23 +02:00
Aliaksandr Valialkin
cd9826d3dd
lib/license: cleanups and prettifications for log messages and docs related to licensing
- Make more clear the docs at docs/enterprise.md, so readers could figure out faster
  on how to obtain enterprise key and how to pass it to VictoriaMetrics Enterprise components.

- Fix examples at docs/enterprise.md, which were referring to non-existing `-license-file` command-line flag.
  The `-licenseFile` command-line flag must be used instead.

- Improve the description of `-license*` command-line flags, so users could understand
  faster how to use them.

- Improve the warning message, which is emitted when the deprecated -eula command-line flag is passed,
  so the user could figure out how to switch faster to -license* command-line flags.

- Disallow running VictoriaMetrics components with both -license and -licenseFile command-line flags.

- Disallow running VictoriaMetrics components when -licensFile points to an empty file.

- Consistently use the phrase "This flag is available only in Enterprise binaries" across
  all the enterprise-specific command-line flags.

- Remove unneeded level of indirection for `noLicenseMessage` and `expiredMessage` string contants
  in order to improve code readability and maintainability.

- Remove unneded `return` statements after `logger.Fatalf()` calls, since these calls exit the app and never return.

- Make sure that the info log message about successful license verification is emitted
  when the license is verified successfully. Previously the error message could be logged
  when the license payload is invalid or if it misses some required features.
2023-10-16 13:17:20 +02:00
Aliaksandr Valialkin
e69580fe97
docs/stream-aggregation.md: clarify the usage of -remoteWrite.label after the fix at a27c2f3773
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4247
2023-08-17 15:19:52 +02:00
hagen1778
be7001351c
alerts: add docs section for the full list of alerting rules
The change also includes update of all references in other docs
to the alerting rules.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-08-11 04:38:30 -07:00
Jan Kielmann
24515715c4
Fix spelling mistake in opentelemetry ingestion API path 2023-08-11 04:34:41 -07:00
Artem Navoiev
3a8842cdcc
docs: add Naver Case Study (#4755)
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
2023-08-11 04:13:38 -07:00
Nikolay
85de94e85c
lib/protoparser: adds opentelemetry parser (#2570)
* lib/protoparser: adds opentelemetry parser
app/{vmagent,vminsert}: adds opentelemetry ingestion path

Adds ability to ingest data with opentelemetry protocol
protobuf and json encoding is supported
data converted into prometheus protobuf timeseries
each data type has own converter and it may produce multiple timeseries
from single datapoint (for summary and histogram).
only cumulative aggregationFamily is supported for sum(prometheus
counter) and histogram.

Apply suggestions from code review

Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>

updates deps

fixes tests

wip

wip

wip

wip

lib/protoparser/opentelemetry: moves to vtprotobuf generator

go mod vendor

lib/protoparse/opentelemetry: reduce memory allocations

* wip

- Remove support for JSON parsing, since it is too fragile and is rarely used in practice.
  The most clients send OpenTelemetry metrics in protobuf.
  The JSON parser can be added in the future if needed.
- Remove unused code from lib/protoparser/opentelemetry/pb and lib/protoparser/opentelemetry/proto
- Do not re-use protobuf message between ParseStream() calls, since there is high chance
  of high fragmentation of the re-used message because of too complex nested structure of the message.

* wip

* wip

* wip

---------

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2023-07-27 13:37:15 -07:00
Aliaksandr Valialkin
5466183fd9
docs/Single-server-VictoriaMetrics.md: use 1. instead of N. in numbered bullets, so they are automatically adjusted by Github Markdown engine
See https://docs.github.com/en/get-started/writing-on-github/getting-started-with-writing-and-formatting-on-github/basic-writing-and-formatting-syntax#lists
2023-07-26 14:12:45 -07:00
Aliaksandr Valialkin
c049778ad1
lib/streamaggr: follow-up for 736197179e
- Use a byte slice instead of a map for tracking indexes for matching series.
  This improves performance, since access by slice index is faster than access by map key.
- Re-use the byte slice for tracking indexes for matching series.
  This removes unnecessary memory allocations and improves stream aggregation performance a bit.
- Add an ability to return to the previous behvaiour by specifying -remoteWrite.streamAggr.dropInput command-line flag.
  In this case all the input samples are dropped when stream aggregation is enabled.
- Backport the new stream aggregation behaviour from vmagent to single-node VictoriaMetrics when -streamAggr.config
  option is set.
- Improve docs regarding this change at docs/CHANGELOG.md
- Document the new behavior at docs/stream-aggregation.md

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4243
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/4575
2023-07-24 17:06:09 -07:00
hagen1778
42af241d59
docs: update datadog config recommendations
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-07-24 16:17:35 -07:00
Dmytro Kozlov
41f0227af2
docs: update information about tsdb usage in cluster version (#4678)
update information about tsdb usage in cluster version
2023-07-20 21:50:58 -07:00
hagen1778
5b963e1971
docs: mention streaming aggregation as way to reduce cardinality
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-07-20 21:48:09 -07:00
Aliaksandr Valialkin
29c6bcbd5b
app/vmselect: follow-up after 6a96fd8ed5
- Add `Active queries` chapter to VMUI docs
- Set `Content-Type: json` header inside promql.WriteActiveQueries() handler,
  in order to be consistent with other request handlers called at app/vmselect/main.go
- Pass the request to promql.WriteActiveQueries() handler, so it can change its output
  depending on the provided request params. This also improves consistency of
  promql.WriteActiveQueries() args with other request hanlers at app/vmselect/main.go

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/4653
2023-07-19 16:31:26 -07:00
Roman Khavronenko
80768d53dd
docs: follow-up after aec4b5db81 (#4638)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-07-19 14:48:17 -07:00
Aliaksandr Valialkin
81fe089546
docs/Single-server-VictoriaMetrics.md: mention how to use Prometheus config file with unsupported options
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4274
2023-07-09 12:37:44 -07:00
Zakhar Bessarab
ddd918b93c
docs: make httpAuth.* flags description less ambiguous (#4588)
* docs: make `httpAuth.*` flags description less ambiguous

Currently, it may confuse users whether `httpAuth.*` flags are used by HTTP client or server configuration(see https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4586 for example).

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* docs: fix a typo

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

---------

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2023-07-09 12:36:14 -07:00
Aliaksandr Valialkin
65d7194588
docs/Single-server-VictoriaMetrics.md: update -help output 2023-07-06 22:45:58 -07:00
Roman Khavronenko
690f58c016
docs: explicitly mention errors processing for import APIs (#4583)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-07-06 22:26:04 -07:00
Denys Holius
97a5bdf4f0
docs: adds curl commands to clear the query cache (#4468)
adds curl commands to clear query cache on vmselect/VM Single
2023-07-06 22:25:21 -07:00
Aliaksandr Valialkin
7cf5efc5b8
README.md: add a link to VictoriaLogs 2023-07-06 22:23:54 -07:00
Aliaksandr Valialkin
5b94246d92
docs: add Roblox case study 2023-07-06 22:23:13 -07:00
Aliaksandr Valialkin
ff96e9cfc7
docs/Single-server-VictoriaMetrics.md: fix link to Storage section after the ab2d184e42 2023-07-06 22:22:42 -07:00
Roman Khavronenko
9cfd8d6b86
Docs retention (#4568)
* docs: mention parts and partitions in Retention section

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-07-06 22:19:43 -07:00
Zakhar Bessarab
b801360075
docs: clarify downsampling periods requirements (#4542)
It is required for periods to be multiplies, but it was not stated clearly in documentation.

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2023-07-06 21:58:40 -07:00
Artem Navoiev
28301d1b41
remove deleted repo from the docs
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2023-07-06 21:47:07 -07:00
Aliaksandr Valialkin
0edfa06daa
docs/Single-server-VictoriaMetrics.md: update link to JSON stream format, since the previous link became broken 2023-07-06 21:20:32 -07:00
Aliaksandr Valialkin
61f9b25782
docs/Single-server-VictoriaMetrics.md: refer to Slack chat at the top of the docs 2023-07-06 21:18:59 -07:00
Aliaksandr Valialkin
46210c4d5e
lib/promutils.ParseTime(): add support for timestamps in milliseconds
See https://stackoverflow.com/questions/76437098/how-to-handle-time-unit-and-step-while-ingesting-or-querying-in-victoriametrics/76438405

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4459
2023-07-06 17:11:54 -07:00
Roman Khavronenko
0e3e045f5b
docs: mention errors processing for /api/v1/import API (#4448)
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4446

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-07-06 17:01:34 -07:00
Roman Khavronenko
2e3c3cf7ea
docs: make docs-sync (#4430)
docs: fix typo

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-07-06 16:47:10 -07:00
Dmytro Kozlov
88ac6116bd
docs: clarify -retentionPeriod flag usage (#4417)
app/vmstorage: clarify the min value for `-retentionPeriod` flag

Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>

(cherry picked from commit 24f34347f1)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-06-09 10:44:22 +02:00
Roman Khavronenko
3544e66a95
docs: mention checksums and data corruption (#4404)
Signed-off-by: hagen1778 <roman@victoriametrics.com>

(cherry picked from commit 96b40b044c)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-06-09 10:40:48 +02:00
Dmytro Kozlov
c5debee3f4
app/{graphite,netstorage,prometheus}: fix graphite search tags api limits, remove redudant limit from SeriesHandler handler (#4352)
* app/{graphite,netstorage,prometheus}: fix graphite search tags api limits, remove unused limit from SeriesHandler handler,

* app/{graphite,netstorage,prometheus}: use search.maxTagValues for Graphite

* app/{graphite,netstorage,prometheus}: update CHANGELOG.md

* app/{graphite,netstorage,prometheus}: use own flags for Graphite API

* app/{graphite,netstorage,prometheus}: cleanup

* app/{graphite,netstorage,prometheus}: cleanup

* app/{graphite,netstorage,prometheus}: update docs

---------

Co-authored-by: Nikolay <nik@victoriametrics.com>

(cherry picked from commit c7884f8686)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-06-09 10:39:12 +02:00
Roman Khavronenko
7da0c37d2b
docs: clarify deduplication docs (#4371)
The purpose of the change is too highlight what HA pair is
and how deduplication needs identical labels to be present
in raw samples.

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4367

Signed-off-by: hagen1778 <roman@victoriametrics.com>

(cherry picked from commit 8185c2466c)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-06-02 13:29:45 +02:00
Roman Khavronenko
c862952cb0
docs: mention disk space requirements for downsampling (#4369)
Signed-off-by: hagen1778 <roman@victoriametrics.com>

(cherry picked from commit a0c040ea58)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-06-02 13:29:14 +02:00
Roman Khavronenko
a0825ca5cd
docs: mention multi-tenancy in docs (#4357)
The update should make understanding of multi-tenancy more clear
for influxdb users.

Signed-off-by: hagen1778 <roman@victoriametrics.com>

(cherry picked from commit 1c3f50f791)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-06-02 13:28:02 +02:00
Artem Navoiev
3185b8048c
update docs-sync Makefile command, add hugo front-matter
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>

(cherry picked from commit f791811b15)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-06-02 13:24:41 +02:00
Roman Khavronenko
ddc3f0c5c5
docs: mention link to public relabeling playground (#4306)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-05-12 16:57:17 -07:00
Roman Khavronenko
f613d20466
docs: update docs about VMUI pages (#4305)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-05-12 16:56:00 -07:00
Aliaksandr Valialkin
d21a244641
lib/promutils: properly parse time strings with timezones at ParseTime() 2023-05-11 13:36:00 -07:00
Dmytro Kozlov
fd35f023dd
docs: Add information about datasource plugin (#4266)
docs: Add information about datasource plugin
2023-05-10 02:24:33 -07:00
Alexander Marshalov
d321ea91f2
fixed typos in documentation and commandline flags descriptions (#4275) 2023-05-10 02:22:06 -07:00
Aliaksandr Valialkin
e55ddaa962
docs: document IP filters functionality in vmauth
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3491

This is a follow-up for 2f08ed3be2
2023-05-09 22:39:47 -07:00
Roman Khavronenko
f247160699
docs: note automatic conversion to ms for influx protocol (#4224)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-05-08 21:55:58 -07:00
Artem Navoiev
283a75afba
prepare static docs to migration
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2023-05-08 21:54:20 -07:00
Roman Khavronenko
c6511bc2d0
Revert "http server: limit max concurrent requests (#4185)" (#4215)
This reverts commit 77f76371

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-05-08 17:22:27 -07:00
Roman Khavronenko
20b025dc88
http server: limit max concurrent requests (#4185)
* lib/httpserver: introduce `-http.maxConcurrentRequests` command-line flag

Introduce `-http.maxConcurrentRequests` command-line flag to protect
VM components from resource exhaustion during unexpected spikes of HTTP requests.
By default, the new flag's value is set to 0 which means no limits are applied.

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* lib/httpserver: mention http.maxConcurrentRequests in docs

Signed-off-by: hagen1778 <roman@victoriametrics.com>

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-05-08 13:13:58 -07:00