Commit graph

4455 commits

Author SHA1 Message Date
Hui Wang
62d19369a3
stream aggregation: do not allow to enable -stream.keepInput and `k… (#6723)
…eep_metric_names` options in stream aggregation config together

With aggregated data and raw data under the same metric, results would
be confusing.

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
2024-08-13 08:54:35 -04:00
hagen1778
331573b0bb
docs: mention https://blog.zomato.com/migrating-to-victoriametrics-a-complete-overhaul-for-enhanced-observability
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-08-13 08:26:20 -04:00
Fred Navruzov
a99fcfbf1a
docs/vmanomaly - typos fix & clarity (#6793)
### Describe Your Changes

typos fix & clarity improvement of vmanomaly docs after v1.15.1 release

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-08-10 17:43:05 +03:00
Fred Navruzov
985e4f0b99
docs: vmanomaly - release v1.15.1 (#6782)
### Describe Your Changes

 vmanomaly - release v1.15.1 updates to docs:
- changelog page
- reader page (new arguments docs)
- typos & fixes

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-08-10 16:54:27 +03:00
Zhu Jiekun
9e2bd82376
app/vmagent: fixes azure service discovery pagination
Azure API response with link to the next page was incorrectly validate. Validation used url.Host header to match configure API URL.


https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6784
2024-08-09 15:22:47 +02:00
hagen1778
1b788fab51
docs: mention deduplication change in update notes for 1.100.0
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-08-08 09:28:31 +02:00
Dmytro Kozlov
075f5145c2
docs: update user management guide for cloud (#6738)
### Describe Your Changes

Updated user management guide with new cloud content
This PR should be merged after the cloud PR

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-08-08 09:16:36 +02:00
Zakhar Bessarab
cb00b4b00f
lib/backup/s3remote: add retryer configuration (#6747)
### Describe Your Changes

This helps to improve reliability of performing backups in environments
with unreliable connection and tolerate temporary errors at S3 provider
side.

See: https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6732

Default retry timeout is up to 3 minutes to make this consistent with
the same configuration for GCS:
a05317f61f/lib/backup/gcsremote/gcs.go (L70-L76)

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2024-08-07 16:55:29 +02:00
Roman Khavronenko
f28f496a9d
lib/bytesutil: smooth buffer growth rate (#6761)
Before, buffer growth was always x2 of its size, which could lead to
excessive memory usage when processing big amount of data.
For example, scraping a target with hundreds of MBs in response could
result into hih memory spikes in vmagent because buffer has to double
its size to fit the response. See
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6759

The change smoothes out the growth rate, trading higher allocation rate
for lower mem usage at certain conditions.

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-08-07 16:49:43 +02:00
Andrii Chubatiuk
35d77a3bed
docs: updated guides structure, removed deprecated sort option (#6767)
### Describe Your Changes

* `sort` param is unused by the current website engine, and was present only for compatibility
with previous website engine. It is time to remove it as it makes no effect
* re-structure guides content into folders to simplify assets management

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-08-07 16:48:08 +02:00
Fred Navruzov
264e9caf5a
docs/vmanomaly: fix typos after v1.15.0 (#6774)
### Describe Your Changes

Fixing remaining typos and missing words after v.1.15.0 updates

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-08-07 15:15:11 +03:00
Fred Navruzov
b670fa1a29
docs/vmanomaly: release v1.15.0 (#6768)
### Describe Your Changes

- updated docs on `vmanomaly` with v1.15.0
- additional chapters of FAQ and model pages

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-08-07 14:31:07 +03:00
Anton L
8acb671fdc
app/vmselect/graphite: respect denyPartialResponse for graphite requests (#6748)
VM has different responses to equivalent queries for MetricsQL and
GraphiteQL in case of failed access to one of vmstorage node of the
cluster vmstorage nodes. For GraphiteQL, the denyPartialResponse feature
is not used, it is always true, which is not always correct (depending
on the configuration).

In the PR I have removed the hardcoded denyPartialResponse for
GraphiteQL, just like MetricsQL does.

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

(cherry picked from commit 79008b712f)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-08-07 12:37:41 +02:00
Hui Wang
8f5c26d788
app/vmagent/remotewrite: make -remoteWrite.streamAggr.ignoreFirstIntervals of array type (#6744)
Make `-remoteWrite.streamAggr.ignoreFirstIntervals` of array type so it could
 accept multiple values which can be applied to the corresponding`-remoteWrite.url`.

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
2024-08-07 09:53:50 +02:00
Hui Wang
4863605469
app/vmagent/remotewrite: fix -streamAggr.dropInputLabels behavior (#6743)
Fix `-streamAggr.dropInputLabels` behavior  when global deduplication is enabled without `-streamAggr.config`.
Previously, `-remoteWrite.streamAggr.dropInputLabels` is misapplied.

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
2024-08-07 09:48:15 +02:00
hagen1778
3a88553315
docs: mention __sample_limit__ in sd_configs
Follow-up after 994796367b

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-08-07 09:42:46 +02:00
Anzor
994796367b
app/vmagent: read __sample_limit__ from labels (#6665) (#6666)
By introducing this feature, users will have the ability to customize
the sampleLimit parameter on a per-target basis, providing more
flexibility and control over the job execution behavior.
2024-08-07 09:36:14 +02:00
Andrii Chubatiuk
86b473c476
docs: fixed docs build (#6765)
### Describe Your Changes

fixed yaml header in a guide doc, that causes hugo build error

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-08-06 16:32:12 +02:00
Yury Molodov
04c2232e45
vmui/logs: add display top streams in the hits graph (#6647)
### Describe Your Changes

- Adds support for displaying the top 5 log streams in the hits graph,
grouping the remaining streams into an "other" label.
   #6545

- Adds options to customize the graph display with bar, line, stepped
line, and points views.

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-08-06 16:28:44 +02:00
Andrii Chubatiuk
2e16732fdb
docs: updated docs titles and links (#6741)
The changes are based on SEO report and supposed to improve
ranking and indexation by search engines by using prompt and unique titles
and by updating unreachable links.

It also updates links to have a simplified form and replaces relative links with absolute links
according to https://docs.victoriametrics.com/#documentation

---------

Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
2024-08-06 15:54:52 +02:00
Zakhar Bessarab
58b6c54da2
app/vlinsert/elasticsearch: add fake response for logstash requests (#6742)
### Describe Your Changes

This is needed in order to support standard Elasticsearch output in
Logstash pipelines.

See: https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6660

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2024-08-06 15:43:33 +02:00
Tommy
f23650ccf9
docs: fix typo in VictoriaLogs FAQ.md (#6755)
Update VictoriaLogs FQA section to replace VictoriaMetrics with
VictoriaLogs.

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-08-06 15:41:04 +02:00
Hui Wang
c1b54779a2
vmalert: respect HTTP headers defined in notifier configuration file (#6762)
Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
2024-08-06 15:37:25 +02:00
hagen1778
be0b892ce6
docs: data ingestion
* rm extra epmty lines
* rename images in according to https://docs.victoriametrics.com/#images-in-documentation

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-08-06 15:34:07 +02:00
Mathias Palmersheim
a46d554f74
docs: add vmagent and alloy data ingestion docs (#6678)
Adds Prometheus Grafana Alloy and vmagent to the data ingestion
protocols. Grafana Agent was not added since it has been deprecated in
favor of alloy

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
2024-08-06 15:29:42 +02:00
Zhu Jiekun
7551d799f7
docs: Added grafana version requirement for vm+vl datasorce (#6760)
### Describe Your Changes

https://victoriametrics.slack.com/archives/C05UNTPAEDN/p1722833182319299

Sometimes users may use the wrong (lower) version of Grafana when
setting up the VictoriaLogs datasource.

It would be good to document the requirements of Grafana versions to use
VictoriaMetrics/VictoriaLogs datasource.

### Checklist

The following checks are **mandatory**:

- [X] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-08-05 16:19:17 +02:00
Ivan Yatskevich
3c48e5662f
docs: fix typo in quick start guide (#6757)
### Describe Your Changes

Fix a typo in docs.

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-08-05 10:36:16 +02:00
Zakhar Bessarab
9877a5e7d5
app/{vminsert,vmagent}: add healthcheck for influx ingestion endpoints (#6749)
### Describe Your Changes

This is useful for clients which validate InfluxDB is available before
data ingestion can be started.

See: https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6653

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
2024-08-05 09:34:54 +02:00
Dmytro Kozlov
6f401daacb
vmctl: add --backoff-retries, --backoff-factor, --backoff-min-duration global command-line flags (#6639)
### Describe Your Changes

Added `--vm-backoff-retries`, `--vm-backoff-factor`,
`--vm-backoff-min-duration` and `--vm-native-backoff-retries`,
`--vm-native-backoff-factor`, `--vm-native-backoff-min-duration`
command-line flags to the `vmctl` app. Those changes will help to
configure the retry backoff policy for different situations.

Related issue:
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6622
### Checklist

The following checks are **mandatory**:

- [X] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
2024-08-03 19:12:48 +02:00
Yury Molodov
e06a19d85f
vmui/logs: improve UI functionality (#6688)
* add a toggle button to the "Group" tab that allows users to expand or collapse all groups at once
* introduce the ability to select a key for grouping logs within the "Group" tab
* display the number of entries within each log group.
* move the Markdown toggle to the general settings panel in the upper left corner.
2024-08-02 15:48:36 +02:00
f41gh7
3edbebd3ed
docs: mention v1.102.x LTS release line
Signed-off-by: f41gh7 <nik@victoriametrics.com>
2024-08-02 13:54:02 +02:00
hagen1778
f9d6bb91b8
docs: rm reference to step number as it could change in future
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-08-02 11:57:10 +02:00
f41gh7
26fa8106c8
add new LTS release v1.102.x
Signed-off-by: f41gh7 <nik@victoriametrics.com>
2024-08-02 11:12:20 +02:00
hagen1778
93e54d65d1
docs: mention step for re-building js static files
Changes to .ts files in vmui or vmui for logs require re-building
static files that will be included into compiled binary afterwards.

We don't update static files on each .ts change PR because it results
in too many changes and complicates review.

So we need to update these static files before the actual release.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-08-02 10:56:04 +02:00
hagen1778
7bf83c7c0e
docs: use absolute links instead of relatives
See https://docs.victoriametrics.com/#documentation

```
Use absolute links. This simplifies moving docs between different files.
```

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-08-02 10:25:37 +02:00
Yury Molodov
a05317f61f
vmui/logs: add fields for tenant configuration (#6661)
Added fields for configuring AccountID and ProjectID
#6631
2024-08-02 09:57:39 +02:00
f41gh7
2222d62aa6
docs/CHANGELOG.md: cut v1.102.1 release
Signed-off-by: f41gh7 <nik@victoriametrics.com>
2024-08-01 14:23:19 +02:00
f41gh7
7b21471f78
vendor: updates metricsql to v0.77.0 with bugfix
Fixes panic if incorrect metricsql expression passed to the prettifier API.

Prettify function had misleading panic for duration expression formatting. It expected all WITH templates to be already parsed.
But WITH expression expand was removed.

Bug was introduced at e712a49898
and present at v1.98.0+ releases

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6736
Signed-off-by: f41gh7 <nik@victoriametrics.com>
2024-08-01 10:50:05 +02:00
hagen1778
77b768089f
docs: follow-up after 58e667c895
* add Logo guidelines to GitHub Readme, as it may have higher chances to be viewed by users
* rm Logo guidelines from cluster version docs, as it makes no sense anymore for this page
* rm `picutre` tag from cluster version docs, as it is not used by GitHub anymore

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-07-31 16:20:17 +02:00
Andrii Chubatiuk
58e667c895
docs: grouped changelog docs, removed old make commands, replaced docs in root README with official docs links (#6727)
### Describe Your Changes

- replace docs in root README with a link to official documentation
- remove old make commands for documentation
- remove redundant "VictoriaMetrics" from document titles
- merge changelog docs into a section
- rm content of Single-server-VictoriaMetrics.md as it can be included from docs/README
- add basic information to README in the root folder, so it will be useful for github users
- rm `picture` tag from docs/README as it was needed for github only, we don't display VM logo at docs.victoriametrics.com
- update `## documentation` section in docs/README to reflect the changes
- rename DD pictures, as they now belong to docs/README

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
2024-07-31 16:03:18 +02:00
Yury Molodov
53919327b2
vmui: fix auto-completion triggers (#6566)
### Describe Your Changes

- Fixes auto-complete triggers according to [these
comments](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5866#issuecomment-2065273421).
- Fixes loading and displaying suggestions when there is no metric in
the expression.
   Related issue: #6153
- Adds quotes when inserting label values. 
   Related issue: #6260

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-07-31 15:00:14 +02:00
hagen1778
b8b6c565f4
docs: rm typo in vmalert docs
typo was added in b9f7c3169a

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-07-30 16:58:08 +02:00
hagen1778
d225a2eb56
dashboards: add Scrape duration 0.99 quantile panel
The new panel will show the 99th quantile of scrape duration in seconds.
This should help identifying vmagent instances that experiences too high scraping durations.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-07-30 14:57:17 +02:00
Dmytro Kozlov
d09182da11
docs: rename managed to cloud (#6689)
### Describe Your Changes

These changes were made as part of the title transition from Managed
VictoriaMetrics to VictoriaMetrics Cloud

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-07-30 09:59:29 +02:00
Andrii Chubatiuk
88ea9c2fb3
docs: removed redundant 'VictoriaLogs' from title (#6715)
### Describe Your Changes

After breadcrumb was added to docs.victoriametrics.com there's no need
to specify parent page name in a title

<img width="1437" alt="Screenshot 2024-07-27 at 10 20 09"
src="https://github.com/user-attachments/assets/733f41f4-a727-4f52-a7c0-6019edf1b803">

Also added vmdocs to gitignore to avoid committing it

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-07-29 14:29:15 +02:00
Nikolay
135a9fdf1d
docs: mention graphite incompatibilities (#6664)
updates
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5810
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2969

---------

Signed-off-by: f41gh7 <nik@victoriametrics.com>
Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
2024-07-29 14:24:37 +02:00
jackyin
e5d279bb71
lib/netutil: validate TLS cert and key files immediately (#6621)
Validate files specified via `-tlsKeyFile` and `-tlsCertFile` cmd-line flags on the process start-up. Previously, validation happened on the first connection accepted by HTTP server.

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6608

---------

Co-authored-by: hagen1778 <roman@victoriametrics.com>
2024-07-29 13:58:53 +02:00
Fred Navruzov
9cbf844903
docs/vmanomaly - changelog upd v1.14.1-v1.14.2 (#6718)
### Describe Your Changes

- Doc updates on v1.14.1 - v1.14.2 of `vmanomaly`
-
[Changelog](https://docs.victoriametrics.com/anomaly-detection/changelog/)
page

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-07-27 23:03:16 +03:00
Fred Navruzov
0cd19edce1
docs/vmanomaly-v1.14.1-2-updates (#6717)
### Describe Your Changes

- Doc updates on v1.14.1 - v1.14.2 of `vmanomaly`
-
[Changelog](https://docs.victoriametrics.com/anomaly-detection/changelog/)
page
-
[Reader](https://docs.victoriametrics.com/anomaly-detection/components/reader/#vm-reader)
page (`queries` arg refactor)
- Also, a slight modification of `presets`
[page](https://docs.victoriametrics.com/anomaly-detection/presets/)

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-07-27 22:55:30 +03:00
Aliaksandr Valialkin
8551fbe9f3
Revert "refactor(vmstorage): Refactor the code to reduce the time complexity of MustAddRows and improve readability (#6629)"
This reverts commit e280d90e9a.

Reason for revert: the updated code doesn't improve the performance of table.MustAddRows for the typical case
when rows contain timestamps belonging to ptws[0].

The performance may be improved in theory for the case when all the rows belong to partiton other than ptws[0],
but this partition is automatically moved to ptws[0] by the code at lines
6aad1d43e9/lib/storage/table.go (L287-L298) ,
so the next time the typical case will work.

Also the updated code makes the code harder to follow, since it introduces an additional level of indirection
with non-trivial semantics inside table.MustAddRows - the partition.TimeRangeInPartition() function.
This function needs to be inspected and understood when reading the code at table.MustAddRows().
This function depends on minTsInRows and maxTsInRows vars, which are defined and initialized
many lines above the partition.TimeRangeInPartition() call. This complicates reading and understanding
the code even more.

The previous code was using clearer loop over rows with the clear call to partition.HasTimestamp()
for every timestamp in the row. The partition.HasTimestamp() call is used in the table.MustAddRows()
function multiple times. This makes the use of partition.HasTimestamp() call more consistent,
easier to understand and easier to maintain comparing to the mix of partition.HasTimestamp() and partition.TimeRangeInPartition()
calls.

Aslo, there is no need in documenting some hardcore software engineering refactoring at docs/CHANGLELOG.md,
since the docs/CHANGELOG.md is intended for VictoriaMetrics users, who may not know software engineering.
The docs/CHANGELOG.md must document user-visible changes, and the docs must be concise and clear for VictoriaMetrics users.
See https://docs.victoriametrics.com/contributing/#pull-request-checklist for more details.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6629
2024-07-25 14:32:09 +02:00