Commit graph

487 commits

Author SHA1 Message Date
Aliaksandr Valialkin
037652d5ae
app/vlinsert: support _time field without timezone information during data ingestion
Use local timezone of the host server in this case. The timezone can be overridden
with TZ environment variable if needed.

While at it, allow using whitespace instead of T as a delimiter between data and time
in the ingested _time field. For example, '2024-09-20 10:20:30' is now accepted
during data ingestion. This is valid ISO8601 format, which is used by some log shippers,
so it should be supported. This format is also known as SQL datetime format.

Also assume local time zone when time without timezone information is passed to querying APIs.
Previously such a time was parsed in UTC timezone. Add `Z` to the end of the time string
if the old behaviour is preferred.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6721
2024-09-26 12:49:35 +02:00
Roman Khavronenko
9a0f697622
docs: update CONTRIBUTING.md with practical requirements (#7087)
The change supposed to have more practical recommendations and reflect
the real processes for maintaining the project.


Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-09-24 18:22:18 +02:00
rtm0
e7f1297517
docs: document IndexDB (#6840)
### Describe Your Changes

This is an attempt to document IndexDB. I guess I was trying to touch
the important points that might be of interest for the end users while
refraining from making it too detailed (such as I did not enumerate and
describe all the specific record types).

Please take a look and any suggestions are very welcome.

### Checklist

The following checks are **mandatory**:

- [x ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Signed-off-by: Artem Fetishev <wwctrsrx@gmail.com>
2024-08-27 13:52:46 +02:00
hagen1778
74e8d6ad55
docs: rm mentions of google group as it was deprecated
See https://docs.victoriametrics.com/#community-and-contributions

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-08-21 14:08:27 +02:00
Andrii Chubatiuk
2e16732fdb
docs: updated docs titles and links (#6741)
The changes are based on SEO report and supposed to improve
ranking and indexation by search engines by using prompt and unique titles
and by updating unreachable links.

It also updates links to have a simplified form and replaces relative links with absolute links
according to https://docs.victoriametrics.com/#documentation

---------

Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
2024-08-06 15:54:52 +02:00
Andrii Chubatiuk
58e667c895
docs: grouped changelog docs, removed old make commands, replaced docs in root README with official docs links (#6727)
### Describe Your Changes

- replace docs in root README with a link to official documentation
- remove old make commands for documentation
- remove redundant "VictoriaMetrics" from document titles
- merge changelog docs into a section
- rm content of Single-server-VictoriaMetrics.md as it can be included from docs/README
- add basic information to README in the root folder, so it will be useful for github users
- rm `picture` tag from docs/README as it was needed for github only, we don't display VM logo at docs.victoriametrics.com
- update `## documentation` section in docs/README to reflect the changes
- rename DD pictures, as they now belong to docs/README

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
2024-07-31 16:03:18 +02:00
Nikolay
135a9fdf1d
docs: mention graphite incompatibilities (#6664)
updates
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5810
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2969

---------

Signed-off-by: f41gh7 <nik@victoriametrics.com>
Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
2024-07-29 14:24:37 +02:00
Aliaksandr Valialkin
d5809f8e12
Revert "replaced global http refs with relative markdown ones (#6692)"
This reverts commit 537266363a.

Reason for revert: relative links in docs are much harder to maintain in consistent state
comparing to absolute links:

- It is non-trivial to figure out the proper relative link path when creating and editing docs.
- Relative links break after moving the doc files to another paths, and it is non-trivial
  to figure which links are broken after that.
- The updated relative links do not work properly right now in the docs.
  For example, the https://docs.victoriametrics.com/victorialogs/quickstart.md#building-from-source-code
  link at https://docs.victoriametrics.com/victorialogs/changelog/ leads to 404 page.

This is documented at https://docs.victoriametrics.com/#images-in-documentation .
2024-07-25 12:48:12 +02:00
Andrii Chubatiuk
537266363a
replaced global http refs with relative markdown ones (#6692)
### Describe Your Changes

Replaced global http links in docs with relative markdown ones

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-07-24 07:50:09 -07:00
Andrii Chubatiuk
ce4cc4cbb2
view documentation locally (#6677)
### Describe Your Changes

- moved files from root to VictoriaMetrics folder to be able to mount
operator docs and VictoriaMetrics docs independently
- added ability to run website locally

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-07-24 01:00:31 -07:00
Andrii Chubatiuk
332b7788fb
added _index.md for hugo, which point to README.md (#6686)
### Describe Your Changes

Please provide a brief description of the changes you made. Be as
specific as possible to help others understand the purpose and impact of
your modifications.

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-07-23 17:30:49 +03:00
Aliaksandr Valialkin
cda6a31ae8
docs/Single-server-VictoriaMetrics.md: recommend periodic running of make spellcheck command
This is a follow-up for fabf0b928e
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6562
2024-07-18 11:51:53 +02:00
Andrii Chubatiuk
ef01df10bd
Code tooltip updates (#6657)
### Describe Your Changes

recently added custom `promtextmetric` and `influxtextmetric` prismjs
plugins to vmdocs to convert this
<img width="1081" alt="Screenshot 2024-07-17 at 12 13 29"
src="https://github.com/user-attachments/assets/66d4ea18-10fe-45ef-80b4-989d0eb3bd92">
to this
<img width="1087" alt="Screenshot 2024-07-17 at 12 24 34"
src="https://github.com/user-attachments/assets/60d4ab44-79e5-4c63-b966-54b989ead1aa">




### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-07-18 02:50:49 -07:00
Aliaksandr Valialkin
aefc4538bd
docs/Single-server-VictoriaMetrics.md: explain why images must have the same filepath as the corresponding docs at docs/ folder
This is a follow-up for f2c1d30011
2024-07-17 18:26:16 +02:00
Aliaksandr Valialkin
74affa3aec
lib/protoparser/graphite: follow-up for 476faf5578
- Clarify the description of -graphite.sanitizeMetricName command-line flag at README.md
- Do not sanitize tag values - only metric names and tag names must be sanitized,
  since they are treated specially by Grafana. Grafana doesn't apply any restrictions on tag values.
- Properly replace more than two consecutive dots with a single dot.
- Disallow unicode letters in metric names and tag names, since neither Prometheus nor Grafana
  do not support them.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6489
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6077
2024-07-17 12:41:55 +02:00
Aliaksandr Valialkin
9cf187c83b
docs: remove the remaining mentioning about snap packages after the commit 0a42c8fd8b
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6543
2024-07-16 18:35:38 +02:00
Aliaksandr Valialkin
233e5f0a9e
lib/httpserver: skip basic auth check for additional request paths, which should call httpserver.CheckAuthFlag()
This is a follow-up for 61dce6f2a1

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6338
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6329
2024-07-16 01:00:45 +02:00
Aliaksandr Valialkin
e3d5714f6f
app/vminsert: increase default value for -maxLabelValueLen command-line flag from 1KiB to 4KiB
It has been appeared that the standard Kubernetes monitoring can generate labels with sizes up to 4KiB

This is a follow-up for a5d1013042
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6176
2024-07-15 23:32:36 +02:00
hagen1778
f2c1d30011
docs: mention assets naming policy and placement
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-07-12 11:12:24 +02:00
hagen1778
1c155a0cd1
docs: follow-up after c341369fc1
* account for `source` tag in `docs-sync` command
* run `make docs-sync`

c341369fc1
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-07-10 12:25:21 +02:00
Phuong Le
c341369fc1
docs: use white version of logo for dark theme (#6610)
The logo on the README is hard to recognize in dark mode on GitHub, so I
made it responsive based on user preferences.
2024-07-09 11:04:00 +02:00
hagen1778
ee66fb4387
docs: update refereneces to victoriametrics-datasource plugin
The plugin was renamed in https://github.com/VictoriaMetrics/victoriametrics-datasource/pull/178

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-07-05 09:46:17 +02:00
Aliaksandr Valialkin
2da7dfc754
Revert c6c5a5a186 and b2765c45d0
Reason for revert:

There are many statsd servers exist:

- https://github.com/statsd/statsd - classical statsd server
- https://docs.datadoghq.com/developers/dogstatsd/ - statsd server from DataDog built into DatDog Agent ( https://docs.datadoghq.com/agent/ )
- https://github.com/avito-tech/bioyino - high-performance statsd server
- https://github.com/atlassian/gostatsd - statsd server in Go
- https://github.com/prometheus/statsd_exporter - statsd server, which exposes the aggregated data as Prometheus metrics

These servers can be used for efficient aggregating of statsd data and sending it to VictoriaMetrics
according to https://docs.victoriametrics.com/#how-to-send-data-from-graphite-compatible-agents-such-as-statsd (
the https://github.com/prometheus/statsd_exporter can be scraped as usual Prometheus target
according to https://docs.victoriametrics.com/#how-to-scrape-prometheus-exporters-such-as-node-exporter ).

Adding support for statsd data ingestion protocol into VictoriaMetrics makes sense only if it provides
significant advantages over the existing statsd servers, while has no significant drawbacks comparing
to existing statsd servers.

The main advantage of statsd server built into VictoriaMetrics and vmagent - getting rid of additional statsd server.
The main drawback is non-trivial and inconvenient streaming aggregation configs, which must be used for the ingested statsd metrics (
see https://docs.victoriametrics.com/stream-aggregation/ ). These configs are incompatible with the configs for standalone statsd servers.
So you need to manually translate configs of the used statsd server to stream aggregation configs when migrating
from standalone statsd server to statsd server built into VictoriaMetrics (or vmagent).

Another important drawback is that it is very easy to shoot yourself in the foot when using built-in statsd server
with the -statsd.disableAggregationEnforcement command-line flag or with improperly configured streaming aggregation.
In this case the ingested statsd metrics will be stored to VictoriaMetrics as is without any aggregation.
This may result in high CPU usage during data ingestion, high disk space usage for storing all the unaggregated
statsd metrics and high CPU usage during querying, since all the unaggregated metrics must be read, unpacked and processed
during querying.

P.S. Built-in statsd server can be added to VictoriaMetrics and vmagent after figuring out more ergonomic
specialized configuration for aggregating of statsd metrics. The main requirements for this configuration:

- easy to write, read and update (ideally it should work out of the box for most cases without additional configuration)
- hard to misconfigure (e.g. hard to shoot yourself in the foot)

It would be great if this configuration will be compatible with the configuration of the most widely used statsd server.

In the mean time it is recommended continue using external statsd server.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6265
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5053
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5052
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/206
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4600
2024-07-03 23:51:56 +02:00
Andrii Chubatiuk
476faf5578
lib/protoparser/graphite: added -graphite.sanitizeMetricName flag (#6489)
### Describe Your Changes

Added flag to sanitize graphite metrics
fixes #6077

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
2024-07-02 14:56:41 +02:00
Andrii Chubatiuk
0a42c8fd8b
deployment: remove snap packages support (#6543)
### Describe Your Changes

Removed snap packages support as it requires time for maintenance and
it's not popular at all

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
2024-07-01 15:51:32 +02:00
Aliaksandr Valialkin
87f1c8bd6c
lib/logstorage: work-in-progress 2024-06-27 14:20:43 +02:00
James Rhoat
6652fb630f
Updating documentation around OTEL (#6519)
Updating documentation around the opentelemetry endpoint for metrics and
the "How to use OpenTelemetry metrics with VictoriaMetrics" guide so
that it shows not only how to directly write but also how to write to
the otel collector and view metrics in vmui.

---------

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
Co-authored-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2024-06-26 13:16:21 +04:00
hagen1778
6775a00622
docs: rm mentions of vmrecover as it still doesn't exist
While there, add a hint about truncated error messages.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-06-26 11:10:42 +02:00
hagen1778
cc6a1da0d7
docs: make docs-sync
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-06-25 10:17:55 +02:00
Hui Wang
2259e3cac7
doc: add missing cmd flag description (#6504) 2024-06-18 15:24:38 +02:00
Zakhar Bessarab
34071ac660
lib/promscrape: increase default value for promscrape.maxDroppedTargets to 10_000 (#6459)
### Describe Your Changes
This limit can be increased since after
4513893ead
tracking of dropped targets uses much less memory per entry.

See:
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6381#issuecomment-2156708228


### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2024-06-12 16:34:18 +02:00
Hui Wang
61dce6f2a1
lib/httpserver: allow reloadAuthKey and configAuthKey to override htt… (#6338)
…pAuth.*

address https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6329, 
makes `reloadAuthKey`, `configAuthKey`, `flagsAuthKey`, `pprofAuthKey`
behavior the same way,
but keys like `-snapshotAuthKey`, `-forceMergeAuthKey` are still
protected by httpAuth.*. All the available key are listed in
https://docs.victoriametrics.com/single-server-victoriametrics/#security.

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
2024-06-10 12:09:47 +02:00
Aliaksandr Valialkin
f8f9a58163
docs: run make docs-sync after c740a8042e
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6362
2024-06-07 16:45:16 +02:00
Arkadii Yakovets
c740a8042e
docs: fix docs/ and README.md spelling errors (#6362)
Fixes `docs/` and `README.md` typos and errors.

Signed-off-by: Arkadii Yakovets <ark@victoriametrics.com>
2024-06-03 10:04:13 +02:00
Aliaksandr Valialkin
e2590f0485
lib/logstorage: work-in-progress 2024-05-25 00:30:58 +02:00
Nikolay
a5d1013042
lib/storage: change default value for maxLabelValueLen to 1024 (#6313)
* It must reduce memory usage for misbehaving clients. Since
VictoriaMetrics stores sparse index inmemory.
* Reduce disk space usage for indexdb.
* Prevent possible indexDB items drops.
* It may trigger slow insert and new timeseries registration due to
default value for flag change

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6176

---------

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2024-05-22 21:53:53 +02:00
Nikolay
b2765c45d0
follow-up for c6c5a5a186 (#6265)
* adds datadog extensions for statsd:
  - multiple packed values (v1.1)
  - additional types distribution, histogram

* adds type check and append metric type to the labels with special tag
name `__statsd_metric_type__`. It simplifies streaming aggregation
config.

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
2024-05-16 09:25:42 +02:00
Aliaksandr Valialkin
de7fc743ca
README.md: mention that -tlsCertFile and -tlsKeyFile options aren't needed when automatic issuing of tls certificates is enabled with -tlsAutoCertHosts flag 2024-05-12 09:48:40 +02:00
Aliaksandr Valialkin
f4051dd1e0
docs/Single-server-VictoriaMetrics.md and docs/README.md: sync with README.md with make docs-sync after the commit c6c5a5a186 2024-05-12 09:40:27 +02:00
Oleg
c6c5a5a186
Statsd protocol compatibility (#5053)
In this PR I added compatibility with [statsd
protocol](https://github.com/b/statsd_spec) with tags to be able to send
metrics directly from statsd clients to vmagent or directly to VM.
For example its compatible with
[statsd-instrument](https://github.com/Shopify/statsd-instrument) and
[dogstatsd-ruby](https://github.com/DataDog/dogstatsd-ruby) gems

Related issues: #5052, #206, #4600
2024-05-07 21:46:08 +02:00
hagen1778
6aaf1768f4
Revert "docs: removed code-style highlighting for commanad-line flags of VM components (#6147)"
This reverts commit 9bedbcfa2f.
2024-04-25 11:23:02 +02:00
Denys Holius
9bedbcfa2f
docs: removed code-style highlighting for commanad-line flags of VM components (#6147)
Using `sh` or `console` formatting doesn't do word-breaking on render. This makes flags description
harder to read, as users need to scroll the web page horizontally.
Removing the formatting renders the description with normal word-breaking.
2024-04-22 14:53:26 +02:00
hagen1778
bae3874e6a
app/streamaggr: follow-up after c0e4ccb7b5
* rm vmagent mentions from vminsert flags
* improve documentation wording, add links to related sections
* mention `ignore_first_intervals` in the stream aggr options
* update flags description
* add basic test for config parsing validation

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-04-22 14:22:59 +02:00
Aliaksandr Valialkin
0e75581dd9
docs: remove duplicate information regarding contribution guidelines to VictoriaMetrics
Substitute duplicate information with the link to https://docs.victoriametrics.com/contributing/ across all the docs.
2024-04-21 00:00:50 +02:00
Aliaksandr Valialkin
b2ce0657b2
all: replace old https://docs.victoriametrics.com/Quick-Start.html url with the new one - https://docs.victoriametrics.com/quick-start/ 2024-04-18 03:32:41 +02:00
Aliaksandr Valialkin
59d495d469
all: replace old https://docs.victoriametrics.com/Troubleshooting.html url with the new one - https://docs.victoriametrics.com/troubleshooting/ 2024-04-18 03:26:36 +02:00
Aliaksandr Valialkin
2e3580905f
all: replace old https://docs.victoriametrics.com/relabeling.html url with the new one - https://docs.victoriametrics.com/relabeling/ 2024-04-18 03:22:22 +02:00
Aliaksandr Valialkin
e6027e043a
all: replace old https://docs.victoriametrics.com/Articles.html url with the new one - https://docs.victoriametrics.com/articles/ 2024-04-18 03:19:31 +02:00
Aliaksandr Valialkin
a196370837
all: replace old https://docs.victoriametrics.com/CaseStudies.html url with the new one - https://docs.victoriametrics.com/casestudies/ 2024-04-18 03:16:05 +02:00
Aliaksandr Valialkin
e9642e99f2
all: replace old https://docs.victoriametrics.com/Single-server-VictoriaMetrics.html url with the new one - https://docs.victoriametrics.com/single-server-victoriametrics/ 2024-04-18 03:11:03 +02:00