Commit graph

8866 commits

Author SHA1 Message Date
Zakhar Bessarab
cb00b4b00f
lib/backup/s3remote: add retryer configuration (#6747)
### Describe Your Changes

This helps to improve reliability of performing backups in environments
with unreliable connection and tolerate temporary errors at S3 provider
side.

See: https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6732

Default retry timeout is up to 3 minutes to make this consistent with
the same configuration for GCS:
a05317f61f/lib/backup/gcsremote/gcs.go (L70-L76)

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2024-08-07 16:55:29 +02:00
Roman Khavronenko
f28f496a9d
lib/bytesutil: smooth buffer growth rate (#6761)
Before, buffer growth was always x2 of its size, which could lead to
excessive memory usage when processing big amount of data.
For example, scraping a target with hundreds of MBs in response could
result into hih memory spikes in vmagent because buffer has to double
its size to fit the response. See
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6759

The change smoothes out the growth rate, trading higher allocation rate
for lower mem usage at certain conditions.

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-08-07 16:49:43 +02:00
Andrii Chubatiuk
35d77a3bed
docs: updated guides structure, removed deprecated sort option (#6767)
### Describe Your Changes

* `sort` param is unused by the current website engine, and was present only for compatibility
with previous website engine. It is time to remove it as it makes no effect
* re-structure guides content into folders to simplify assets management

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-08-07 16:48:08 +02:00
hagen1778
1154f90d2d
lib/mergeset: fix typos in comments
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-08-07 15:54:15 +02:00
Fred Navruzov
264e9caf5a
docs/vmanomaly: fix typos after v1.15.0 (#6774)
### Describe Your Changes

Fixing remaining typos and missing words after v.1.15.0 updates

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-08-07 15:15:11 +03:00
Fred Navruzov
b670fa1a29
docs/vmanomaly: release v1.15.0 (#6768)
### Describe Your Changes

- updated docs on `vmanomaly` with v1.15.0
- additional chapters of FAQ and model pages

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-08-07 14:31:07 +03:00
Anton L
8acb671fdc
app/vmselect/graphite: respect denyPartialResponse for graphite requests (#6748)
VM has different responses to equivalent queries for MetricsQL and
GraphiteQL in case of failed access to one of vmstorage node of the
cluster vmstorage nodes. For GraphiteQL, the denyPartialResponse feature
is not used, it is always true, which is not always correct (depending
on the configuration).

In the PR I have removed the hardcoded denyPartialResponse for
GraphiteQL, just like MetricsQL does.

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

(cherry picked from commit 79008b712f)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-08-07 12:37:41 +02:00
Aliaksandr Valialkin
04981c7a7f
lib/streamaggr: remove resetState arg from aggrState.flushState()
The resetState arg was used only for the BenchmarkAggregatorsFlushInternalSerial benchmark.
This benchmark was testing aggregate state flush performance by keeping the same state across flushes.
The benhmark didn't reflect the performance and scalability of stream aggregation in production,
while it led to non-trivial code changes related to resetState arg handling.

So let's drop the benchmark together with all the code related to resetState handling,
in order to simplify the code at lib/streamaggr a bit.

Thanks to @AndrewChubatiuk for the original idea at https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6314
2024-08-07 11:39:14 +02:00
Aliaksandr Valialkin
86c7afd126
lib/streamaggr: consistently use the same timestamp across all the output aggregated samples in a single aggregation interval
Prevsiously every aggregation output was using its own timestamp for the output aggregated samples
in a single aggregation interval. This could result in unexpected inconsitent timesetamps for the output
aggregated samples.

This commit consistently uses the same timestamp across all the output aggregated samples.
This commit makes sure that the duration between subsequent timestamps strictly equals
the configured aggregation interval.

Thanks to @AndrewChubatiuk for the original idea at https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6314
This commit should help https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4580
2024-08-07 11:39:13 +02:00
Hui Wang
8f5c26d788
app/vmagent/remotewrite: make -remoteWrite.streamAggr.ignoreFirstIntervals of array type (#6744)
Make `-remoteWrite.streamAggr.ignoreFirstIntervals` of array type so it could
 accept multiple values which can be applied to the corresponding`-remoteWrite.url`.

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
2024-08-07 09:53:50 +02:00
Hui Wang
4863605469
app/vmagent/remotewrite: fix -streamAggr.dropInputLabels behavior (#6743)
Fix `-streamAggr.dropInputLabels` behavior  when global deduplication is enabled without `-streamAggr.config`.
Previously, `-remoteWrite.streamAggr.dropInputLabels` is misapplied.

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
2024-08-07 09:48:15 +02:00
hagen1778
3a88553315
docs: mention __sample_limit__ in sd_configs
Follow-up after 994796367b

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-08-07 09:42:46 +02:00
Anzor
994796367b
app/vmagent: read __sample_limit__ from labels (#6665) (#6666)
By introducing this feature, users will have the ability to customize
the sampleLimit parameter on a per-target basis, providing more
flexibility and control over the job execution behavior.
2024-08-07 09:36:14 +02:00
hagen1778
9726e6c1a2
app/vmalert: rm unnecessary err check
The error check was needed before a84491324d
It was kept by mistake and makes no sense to have rn.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-08-07 09:09:24 +02:00
Andrii Chubatiuk
86b473c476
docs: fixed docs build (#6765)
### Describe Your Changes

fixed yaml header in a guide doc, that causes hugo build error

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-08-06 16:32:12 +02:00
Yury Molodov
04c2232e45
vmui/logs: add display top streams in the hits graph (#6647)
### Describe Your Changes

- Adds support for displaying the top 5 log streams in the hits graph,
grouping the remaining streams into an "other" label.
   #6545

- Adds options to customize the graph display with bar, line, stepped
line, and points views.

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-08-06 16:28:44 +02:00
Andrii Chubatiuk
2e16732fdb
docs: updated docs titles and links (#6741)
The changes are based on SEO report and supposed to improve
ranking and indexation by search engines by using prompt and unique titles
and by updating unreachable links.

It also updates links to have a simplified form and replaces relative links with absolute links
according to https://docs.victoriametrics.com/#documentation

---------

Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
2024-08-06 15:54:52 +02:00
Zakhar Bessarab
58b6c54da2
app/vlinsert/elasticsearch: add fake response for logstash requests (#6742)
### Describe Your Changes

This is needed in order to support standard Elasticsearch output in
Logstash pipelines.

See: https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6660

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2024-08-06 15:43:33 +02:00
Tommy
f23650ccf9
docs: fix typo in VictoriaLogs FAQ.md (#6755)
Update VictoriaLogs FQA section to replace VictoriaMetrics with
VictoriaLogs.

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-08-06 15:41:04 +02:00
Hui Wang
c1b54779a2
vmalert: respect HTTP headers defined in notifier configuration file (#6762)
Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
2024-08-06 15:37:25 +02:00
hagen1778
be0b892ce6
docs: data ingestion
* rm extra epmty lines
* rename images in according to https://docs.victoriametrics.com/#images-in-documentation

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-08-06 15:34:07 +02:00
Mathias Palmersheim
a46d554f74
docs: add vmagent and alloy data ingestion docs (#6678)
Adds Prometheus Grafana Alloy and vmagent to the data ingestion
protocols. Grafana Agent was not added since it has been deprecated in
favor of alloy

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
2024-08-06 15:29:42 +02:00
hagen1778
f283126084
fix typos in comments
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-08-06 14:54:49 +02:00
Zhu Jiekun
7551d799f7
docs: Added grafana version requirement for vm+vl datasorce (#6760)
### Describe Your Changes

https://victoriametrics.slack.com/archives/C05UNTPAEDN/p1722833182319299

Sometimes users may use the wrong (lower) version of Grafana when
setting up the VictoriaLogs datasource.

It would be good to document the requirements of Grafana versions to use
VictoriaMetrics/VictoriaLogs datasource.

### Checklist

The following checks are **mandatory**:

- [X] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-08-05 16:19:17 +02:00
Ivan Yatskevich
3c48e5662f
docs: fix typo in quick start guide (#6757)
### Describe Your Changes

Fix a typo in docs.

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-08-05 10:36:16 +02:00
Zakhar Bessarab
9877a5e7d5
app/{vminsert,vmagent}: add healthcheck for influx ingestion endpoints (#6749)
### Describe Your Changes

This is useful for clients which validate InfluxDB is available before
data ingestion can be started.

See: https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6653

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
2024-08-05 09:34:54 +02:00
Dmytro Kozlov
6f401daacb
vmctl: add --backoff-retries, --backoff-factor, --backoff-min-duration global command-line flags (#6639)
### Describe Your Changes

Added `--vm-backoff-retries`, `--vm-backoff-factor`,
`--vm-backoff-min-duration` and `--vm-native-backoff-retries`,
`--vm-native-backoff-factor`, `--vm-native-backoff-min-duration`
command-line flags to the `vmctl` app. Those changes will help to
configure the retry backoff policy for different situations.

Related issue:
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6622
### Checklist

The following checks are **mandatory**:

- [X] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
2024-08-03 19:12:48 +02:00
Yury Molodov
e06a19d85f
vmui/logs: improve UI functionality (#6688)
* add a toggle button to the "Group" tab that allows users to expand or collapse all groups at once
* introduce the ability to select a key for grouping logs within the "Group" tab
* display the number of entries within each log group.
* move the Markdown toggle to the general settings panel in the upper left corner.
2024-08-02 15:48:36 +02:00
f41gh7
3edbebd3ed
docs: mention v1.102.x LTS release line
Signed-off-by: f41gh7 <nik@victoriametrics.com>
2024-08-02 13:54:02 +02:00
hagen1778
f9d6bb91b8
docs: rm reference to step number as it could change in future
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-08-02 11:57:10 +02:00
f41gh7
b85e3f8bbd
deployment: update compose images to v1.102.1 release
Signed-off-by: f41gh7 <nik@victoriametrics.com>
2024-08-02 11:29:12 +02:00
f41gh7
26fa8106c8
add new LTS release v1.102.x
Signed-off-by: f41gh7 <nik@victoriametrics.com>
2024-08-02 11:12:20 +02:00
hagen1778
93e54d65d1
docs: mention step for re-building js static files
Changes to .ts files in vmui or vmui for logs require re-building
static files that will be included into compiled binary afterwards.

We don't update static files on each .ts change PR because it results
in too many changes and complicates review.

So we need to update these static files before the actual release.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-08-02 10:56:04 +02:00
hagen1778
7bf83c7c0e
docs: use absolute links instead of relatives
See https://docs.victoriametrics.com/#documentation

```
Use absolute links. This simplifies moving docs between different files.
```

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-08-02 10:25:37 +02:00
Yury Molodov
a05317f61f
vmui/logs: add fields for tenant configuration (#6661)
Added fields for configuring AccountID and ProjectID
#6631
2024-08-02 09:57:39 +02:00
f41gh7
996b623585
make vmui-update 2024-08-01 14:45:09 +02:00
f41gh7
2222d62aa6
docs/CHANGELOG.md: cut v1.102.1 release
Signed-off-by: f41gh7 <nik@victoriametrics.com>
2024-08-01 14:23:19 +02:00
f41gh7
7b21471f78
vendor: updates metricsql to v0.77.0 with bugfix
Fixes panic if incorrect metricsql expression passed to the prettifier API.

Prettify function had misleading panic for duration expression formatting. It expected all WITH templates to be already parsed.
But WITH expression expand was removed.

Bug was introduced at e712a49898
and present at v1.98.0+ releases

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6736
Signed-off-by: f41gh7 <nik@victoriametrics.com>
2024-08-01 10:50:05 +02:00
hagen1778
77b768089f
docs: follow-up after 58e667c895
* add Logo guidelines to GitHub Readme, as it may have higher chances to be viewed by users
* rm Logo guidelines from cluster version docs, as it makes no sense anymore for this page
* rm `picutre` tag from cluster version docs, as it is not used by GitHub anymore

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-07-31 16:20:17 +02:00
Andrii Chubatiuk
58e667c895
docs: grouped changelog docs, removed old make commands, replaced docs in root README with official docs links (#6727)
### Describe Your Changes

- replace docs in root README with a link to official documentation
- remove old make commands for documentation
- remove redundant "VictoriaMetrics" from document titles
- merge changelog docs into a section
- rm content of Single-server-VictoriaMetrics.md as it can be included from docs/README
- add basic information to README in the root folder, so it will be useful for github users
- rm `picture` tag from docs/README as it was needed for github only, we don't display VM logo at docs.victoriametrics.com
- update `## documentation` section in docs/README to reflect the changes
- rename DD pictures, as they now belong to docs/README

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
2024-07-31 16:03:18 +02:00
Yury Molodov
53919327b2
vmui: fix auto-completion triggers (#6566)
### Describe Your Changes

- Fixes auto-complete triggers according to [these
comments](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5866#issuecomment-2065273421).
- Fixes loading and displaying suggestions when there is no metric in
the expression.
   Related issue: #6153
- Adds quotes when inserting label values. 
   Related issue: #6260

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-07-31 15:00:14 +02:00
hagen1778
b8b6c565f4
docs: rm typo in vmalert docs
typo was added in b9f7c3169a

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-07-30 16:58:08 +02:00
hagen1778
d225a2eb56
dashboards: add Scrape duration 0.99 quantile panel
The new panel will show the 99th quantile of scrape duration in seconds.
This should help identifying vmagent instances that experiences too high scraping durations.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-07-30 14:57:17 +02:00
Dmytro Kozlov
d09182da11
docs: rename managed to cloud (#6689)
### Describe Your Changes

These changes were made as part of the title transition from Managed
VictoriaMetrics to VictoriaMetrics Cloud

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-07-30 09:59:29 +02:00
Andrii Chubatiuk
88ea9c2fb3
docs: removed redundant 'VictoriaLogs' from title (#6715)
### Describe Your Changes

After breadcrumb was added to docs.victoriametrics.com there's no need
to specify parent page name in a title

<img width="1437" alt="Screenshot 2024-07-27 at 10 20 09"
src="https://github.com/user-attachments/assets/733f41f4-a727-4f52-a7c0-6019edf1b803">

Also added vmdocs to gitignore to avoid committing it

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-07-29 14:29:15 +02:00
Nikolay
135a9fdf1d
docs: mention graphite incompatibilities (#6664)
updates
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5810
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2969

---------

Signed-off-by: f41gh7 <nik@victoriametrics.com>
Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
2024-07-29 14:24:37 +02:00
Juraj Bubniak
11c0b05e8a
lib/backup/s3remote: fix typos (#6694)
Fixes a few typos in errors in lib/backup/s3remote package.
2024-07-29 14:18:31 +02:00
jackyin
e5d279bb71
lib/netutil: validate TLS cert and key files immediately (#6621)
Validate files specified via `-tlsKeyFile` and `-tlsCertFile` cmd-line flags on the process start-up. Previously, validation happened on the first connection accepted by HTTP server.

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6608

---------

Co-authored-by: hagen1778 <roman@victoriametrics.com>
2024-07-29 13:58:53 +02:00
Fred Navruzov
9cbf844903
docs/vmanomaly - changelog upd v1.14.1-v1.14.2 (#6718)
### Describe Your Changes

- Doc updates on v1.14.1 - v1.14.2 of `vmanomaly`
-
[Changelog](https://docs.victoriametrics.com/anomaly-detection/changelog/)
page

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-07-27 23:03:16 +03:00
Fred Navruzov
0cd19edce1
docs/vmanomaly-v1.14.1-2-updates (#6717)
### Describe Your Changes

- Doc updates on v1.14.1 - v1.14.2 of `vmanomaly`
-
[Changelog](https://docs.victoriametrics.com/anomaly-detection/changelog/)
page
-
[Reader](https://docs.victoriametrics.com/anomaly-detection/components/reader/#vm-reader)
page (`queries` arg refactor)
- Also, a slight modification of `presets`
[page](https://docs.victoriametrics.com/anomaly-detection/presets/)

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-07-27 22:55:30 +03:00