github-mirrors/VictoriaMetrics
1
0
Fork 0
mirror of https://github.com/VictoriaMetrics/VictoriaMetrics.git synced 2024-11-21 14:44:00 +00:00
Code Issues Projects Releases Packages Wiki Activity
VictoriaMetrics/docs/VictoriaLogs/CHANGELOG.md

510 lines
67 KiB
Markdown
Raw Normal View History

docs: remove id clashes for pages, properly cover victorialogs changelog page (#5123) Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2023-10-04 13:46:28 +00:00
---
docs: victorialogs change the doc ordering.2 Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2023-10-05 08:40:55 +00:00
weight: 7
docs: removed redundant 'VictoriaLogs' from title (#6715) ### Describe Your Changes After breadcrumb was added to docs.victoriametrics.com there's no need to specify parent page name in a title <img width="1437" alt="Screenshot 2024-07-27 at 10 20 09" src="https://github.com/user-attachments/assets/733f41f4-a727-4f52-a7c0-6019edf1b803"> Also added vmdocs to gitignore to avoid committing it ### Checklist The following checks are **mandatory**: - [ ] My change adheres [VictoriaMetrics contributing guidelines](https://docs.victoriametrics.com/contributing/).
2024-07-29 12:29:15 +00:00
title: CHANGELOG
docs: remove id clashes for pages, properly cover victorialogs changelog page (#5123) Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2023-10-04 13:46:28 +00:00
menu:
docs:
identifier: "victorialogs-changelog"
parent: "victorialogs"
docs: victorialogs change the doc ordering.2 Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2023-10-05 08:40:55 +00:00
weight: 7
docs add docs titles Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2024-01-19 16:02:20 +00:00
title: CHANGELOG
docs: remove id clashes for pages, properly cover victorialogs changelog page (#5123) Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2023-10-04 13:46:28 +00:00
aliases:
- /VictoriaLogs/CHANGELOG.html
---
docs/VictoriaLogs: add CHANGELOG.md
2023-07-18 06:14:05 +00:00
The following `tip` changes can be tested by building VictoriaLogs from the latest commit of [VictoriaMetrics](https://github.com/VictoriaMetrics/VictoriaMetrics/) repository
Revert "removed unneeded ref shortcodes, updated VM changelog to use relative markdown links (#6691)" This reverts commit 2e9b1efeb9422c60a345c67ebbbcaa5f8d37469d. Reason for revert: relative links in docs are much harder to maintain in consistent state comparing to absolute links: - It is non-trivial to figure out the proper relative link path when creating and editing docs. - Relative links break after moving the doc files to another paths, and it is non-trivial to figure which links are broken after that. - The updated relative links do not work properly right now in the docs. For example, the https://docs.victoriametrics.com/victorialogs/quickstart.md#building-from-source-code link at https://docs.victoriametrics.com/victorialogs/changelog/ leads to 404 page. This is documented at https://docs.victoriametrics.com/#images-in-documentation .
2024-07-25 10:47:35 +00:00
according to [these docs](https://docs.victoriametrics.com/victorialogs/quickstart/#building-from-source-code)
docs/VictoriaLogs: add CHANGELOG.md
2023-07-18 06:14:05 +00:00
## tip
vlogs: added basic alerts (#7252) ### Describe Your Changes Added basic VLogs alerts Signed-off-by: hagen1778 <roman@victoriametrics.com> Co-authored-by: hagen1778 <roman@victoriametrics.com>
2024-10-17 09:33:06 +00:00
* FEATURE: add basic [alerting rules](https://github.com/VictoriaMetrics/VictoriaMetrics/blob/master/deployment/docker/alerts-vlogs.yml) for VictoriaLogs process. See details at [monitoring docs](https://docs.victoriametrics.com/victorialogs/index.html#monitoring).
lib/logstorage: optimize performance for `top` pipe when it is applied to a field with millions of unique values - Use parallel merge of per-CPU shard results. This improves merge performance on multi-CPU systems. - Use topN heap sort of per-shard results. This improves performance when results contain millions of entries.
2024-10-17 19:19:16 +00:00
* FEATURE: improve [`top` pipe](https://docs.victoriametrics.com/victorialogs/logsql/#top-pipe) performance on multi-CPU hosts when it is applied to [log fields](https://docs.victoriametrics.com/victorialogs/keyconcepts/#data-model) with big number of unique values. For example, `_time:1d | top 5 (user_id)` should be executed much faster now when `user_id` field contains millions of unique values.
vlogs: added basic alerts (#7252) ### Describe Your Changes Added basic VLogs alerts Signed-off-by: hagen1778 <roman@victoriametrics.com> Co-authored-by: hagen1778 <roman@victoriametrics.com>
2024-10-17 09:33:06 +00:00
docs/VictoriaLogs/CHANGELOG.md: cut v0.36.0-victorialogs release
2024-10-16 18:00:35 +00:00
## [v0.36.0](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.36.0-victorialogs)
Released at 2024-10-16
lib/logstorage: refactor storage format to be more efficient for querying wide events It has been appeared that VictoriaLogs is frequently used for collecting logs with tens of fields. For example, standard Kuberntes setup on top of Filebeat generates more than 20 fields per each log. Such logs are also known as "wide events". The previous storage format was optimized for logs with a few fields. When at least a single field was referenced in the query, then the all the meta-information about all the log fields was unpacked and parsed per each scanned block during the query. This could require a lot of additional disk IO and CPU time when logs contain many fields. Resolve this issue by providing an (field -> metainfo_offset) index per each field in every data block. This index allows reading and extracting only the needed metainfo for fields used in the query. This index is stored in columnsHeaderIndexFilename ( columns_header_index.bin ). This allows increasing performance for queries over wide events by 10x and more. Another issue was that the data for bloom filters and field values across all the log fields except of _msg was intermixed in two files - fieldBloomFilename ( field_bloom.bin ) and fieldValuesFilename ( field_values.bin ). This could result in huge disk read IO overhead when some small field was referred in the query, since the Operating System usually reads more data than requested. It reads the data from disk in at least 4KiB blocks (usually the block size is much bigger in the range 64KiB - 512KiB). So, if 512-byte bloom filter or values' block is read from the file, then the Operating System reads up to 512KiB of data from disk, which results in 1000x disk read IO overhead. This overhead isn't visible for recently accessed data, since this data is usually stored in RAM (aka Operating System page cache), but this overhead may become very annoying when performing the query over large volumes of data which isn't present in OS page cache. The solution for this issue is to split bloom filters and field values across multiple shards. This reduces the worst-case disk read IO overhead by at least Nx where N is the number of shards, while the disk read IO overhead is completely removed in best case when the number of columns doesn't exceed N. Currently the number of shards is 8 - see bloomValuesShardsCount . This solution increases performance for queries over large volumes of newly ingested data by up to 1000x. The new storage format is versioned as v1, while the old storage format is version as v0. It is stored in the partHeader.FormatVersion. Parts with the old storage format are converted into parts with the new storage format during background merge. It is possible to force merge by querying /internal/force_merge HTTP endpoint - see https://docs.victoriametrics.com/victorialogs/#forced-merge .
2024-10-16 14:18:28 +00:00
* FEATURE: optimize [LogsQL queries](https://docs.victoriametrics.com/victorialogs/logsql/), which need to scan big number of logs with big number of [log fields](https://docs.victoriametrics.com/victorialogs/keyconcepts/#data-model) (aka `wide events`). The performance for such queries is improved by 10x and more depending on the number of log fields in the scanned logs. The performance improvement is visible when querying logs ingested after the upgrade to this release.
app/vlstorage: add support for forced merge via /internal/force_merge HTTP endpoint
2024-10-13 20:20:31 +00:00
* FEATURE: add support for forced merge. See [these docs](https://docs.victoriametrics.com/victorialogs/#forced-merge).
lib/logstorage: refactor storage format to be more efficient for querying wide events It has been appeared that VictoriaLogs is frequently used for collecting logs with tens of fields. For example, standard Kuberntes setup on top of Filebeat generates more than 20 fields per each log. Such logs are also known as "wide events". The previous storage format was optimized for logs with a few fields. When at least a single field was referenced in the query, then the all the meta-information about all the log fields was unpacked and parsed per each scanned block during the query. This could require a lot of additional disk IO and CPU time when logs contain many fields. Resolve this issue by providing an (field -> metainfo_offset) index per each field in every data block. This index allows reading and extracting only the needed metainfo for fields used in the query. This index is stored in columnsHeaderIndexFilename ( columns_header_index.bin ). This allows increasing performance for queries over wide events by 10x and more. Another issue was that the data for bloom filters and field values across all the log fields except of _msg was intermixed in two files - fieldBloomFilename ( field_bloom.bin ) and fieldValuesFilename ( field_values.bin ). This could result in huge disk read IO overhead when some small field was referred in the query, since the Operating System usually reads more data than requested. It reads the data from disk in at least 4KiB blocks (usually the block size is much bigger in the range 64KiB - 512KiB). So, if 512-byte bloom filter or values' block is read from the file, then the Operating System reads up to 512KiB of data from disk, which results in 1000x disk read IO overhead. This overhead isn't visible for recently accessed data, since this data is usually stored in RAM (aka Operating System page cache), but this overhead may become very annoying when performing the query over large volumes of data which isn't present in OS page cache. The solution for this issue is to split bloom filters and field values across multiple shards. This reduces the worst-case disk read IO overhead by at least Nx where N is the number of shards, while the disk read IO overhead is completely removed in best case when the number of columns doesn't exceed N. Currently the number of shards is 8 - see bloomValuesShardsCount . This solution increases performance for queries over large volumes of newly ingested data by up to 1000x. The new storage format is versioned as v1, while the old storage format is version as v0. It is stored in the partHeader.FormatVersion. Parts with the old storage format are converted into parts with the new storage format during background merge. It is possible to force merge by querying /internal/force_merge HTTP endpoint - see https://docs.victoriametrics.com/victorialogs/#forced-merge .
2024-10-16 14:18:28 +00:00
* FEATURE: skip empty log fields in query results, since they are treated as non-existing fields in [VictoriaLogs data model](https://docs.victoriametrics.com/victorialogs/keyconcepts/#data-model). This should reduce the level of confusion for end users when they see empty log fields.
docs/VictoriaLogs/CHANGELOG.md: typo fix: refer the correct endpoints for stats results
2024-10-16 18:19:22 +00:00
* FEATURE: allow using [`format` pipe](https://docs.victoriametrics.com/victorialogs/logsql/#format-pipe) for creating output labels from existing log fields at [`/select/logsql/stats_query`](https://docs.victoriametrics.com/victorialogs/querying/#querying-log-stats) and [`/select/logsql/stats_query_range`](https://docs.victoriametrics.com/victorialogs/querying/#querying-log-range-stats) endpoints.
vmui: add the ability to cancel running queries (#7204) ### Describe Your Changes - Added functionality to cancel running queries on the Explore Logs and Query pages. - The loader was changed from a spinner to a top bar within the block. This still indicates loading, but solves the issue of the spinner "flickering," especially during graph dragging. Related issue: #7097 https://github.com/user-attachments/assets/98e59aeb-905b-4b9d-bbb2-688223b22a82 ### Checklist The following checks are **mandatory**: - [ ] My change adheres [VictoriaMetrics contributing guidelines](https://docs.victoriametrics.com/contributing/).
2024-10-15 12:48:40 +00:00
* FEATURE: [web UI](https://docs.victoriametrics.com/victorialogs/querying/#web-ui): add the ability to cancel running queries. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7097).
app/vlstorage: add support for forced merge via /internal/force_merge HTTP endpoint
2024-10-13 20:20:31 +00:00
lib/logstorage: make a copy of s.partitions slice when performing queries over the selected partitions s.partitions can be changed when new partition is registered or when old partition is dropped. This could lead to data races and panics when s.partitions slice is accessed by concurrently executed queries. The fix is to make a copy of the selected partitions under s.partitionsLock before performing the query.
2024-10-13 20:14:31 +00:00
* BUGFIX: avoid possible panic when logs for a new day are ingested during execution of concurrent queries.
docs/VictoriaLogs/CHANGELOG.md: add missing part of the sentence
2024-10-16 18:22:19 +00:00
* BUGFIX: avoid panic at `lib/logstorage.(*blockResultColumn).forEachDictValue()` when the query contains [stats with additional filters](https://docs.victoriametrics.com/victorialogs/logsql/#stats-with-additional-filters). The panic has been introduced in [v0.33.0](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.33.0-victorialogs) in [this commit](https://github.com/VictoriaMetrics/VictoriaMetrics/commit/a350be48b68330ee1a487e1fb09b002d3be45163).
victorialogs: add more checks for stats query APIs (#7254) 1. Verify if field in [fields pipe](https://docs.victoriametrics.com/victorialogs/logsql/#fields-pipe) exists. If not, it generates a metric with illegal float value "" for prometheus metrics protocol. 2. check if multiple time range filters produce conflicted query time range, for instance: ``` query: _time: 5m | stats count(), start:2024-10-08T10:00:00.806Z, end: 2024-10-08T12:00:00.806Z, time: 2024-10-10T10:02:59.806Z ``` must give no result due to invalid final time range. --------- Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2024-10-16 17:25:43 +00:00
* BUGFIX: add more checks for [stats query APIs](https://docs.victoriametrics.com/victorialogs/querying/#querying-log-stats) to avoid invalid results.
vmui: fix alert display with long messages (#7228) ### Describe Your Changes Fix `Alert` component to prevent it from overflowing the screen when displaying long messages. Related issue: #7207 ### Checklist The following checks are **mandatory**: - [x] My change adheres [VictoriaMetrics contributing guidelines](https://docs.victoriametrics.com/contributing/). --------- Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com> Signed-off-by: hagen1778 <roman@victoriametrics.com> Co-authored-by: hagen1778 <roman@victoriametrics.com>
2024-10-15 14:35:57 +00:00
* BUGFIX: [vmui](https://docs.victoriametrics.com/#vmui): fix error messages rendering from overflowing the screen with long messages. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7207).
lib/logstorage: make sure that bs.br is non-nil before checking br.bs.bsw.bh.rowsCount there br.bs may be nil when br contains the block with additional filters applied during pipe calculations. For example, `* | count() if (error) errors`.
2024-10-12 18:49:34 +00:00
docs/VictoriaLogs: cut v0.35.0 release
2024-10-09 13:55:20 +00:00
## [v0.35.0](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.35.0-victorialogs)
docs/VictoriaLogs/CHANGELOG.md: cut v0.34.0 release
2024-10-08 10:21:02 +00:00
docs/VictoriaLogs/CHANGELOG.md: cut v0.36.0-victorialogs release
2024-10-16 18:00:35 +00:00
Released at 2024-10-09
app/vlogscli: add support for live tailing
2024-10-09 10:26:07 +00:00
* FEATURE: [vlogscli](https://docs.victoriametrics.com/victorialogs/querying/vlogscli/): add ability to live tail query results - see [these docs](https://docs.victoriametrics.com/victorialogs/querying/vlogscli/#live-tailing).
* FEATURE: [vlogscli](https://docs.victoriametrics.com/victorialogs/querying/vlogscli/): add compact output mode for query results. It can be enabled by typing `\c` and then pressing `enter`. See [these docs](https://docs.victoriametrics.com/victorialogs/querying/vlogscli/#output-modes).
app/vlogscli: add -accountID and -projectID command-line flags for querying the given tenants
2024-10-09 10:56:49 +00:00
* FEATURE: [vlogscli](https://docs.victoriametrics.com/victorialogs/querying/vlogscli/): add `-accountID` and `-projectID` command-line flags for setting `AccountID` and `ProjectID` values when querying the specific [tenants](https://docs.victoriametrics.com/victorialogs/#multitenancy).
app/vlogscli: add support for live tailing
2024-10-09 10:26:07 +00:00
docs/VictoriaLogs: cut v0.35.0 release
2024-10-09 13:55:20 +00:00
## [v0.34.0](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.34.0-victorialogs)
docs/VictoriaLogs/CHANGELOG.md: cut v0.34.0 release
2024-10-08 10:21:02 +00:00
Released at 2024-10-08
app/vlogscli: add ability to display query results in logfmt, single-line and multi-line json modes
2024-10-07 10:18:18 +00:00
* FEATURE: [vlogscli](https://docs.victoriametrics.com/victorialogs/querying/vlogscli/): add ability to display results in `logfmt` mode, single-line and multi-line JSON modes according [these docs](https://docs.victoriametrics.com/victorialogs/querying/vlogscli/#output-modes).
lib/logstorage: add `len` pipe for calculating byte length of log field values
2024-10-03 14:26:03 +00:00
* FEATURE: [vlogscli](https://docs.victoriametrics.com/victorialogs/querying/vlogscli/): preserve `less` output after the exit from scrolling mode. This should help re-using previous query results in subsequent queries.
* FEATURE: add [`len` pipe](https://docs.victoriametrics.com/victorialogs/logsql/#len-pipe) for calculating the length for the given [log field](https://docs.victoriametrics.com/victorialogs/keyconcepts/#data-model) value in bytes.
docs/VictoriaLogs/CHANGELOG.md: cut v0.33.0-victorialogs release
2024-10-01 11:42:18 +00:00
## [v0.33.0](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.33.0-victorialogs)
Released at 2024-10-01
app/vlogscli: add interactive command-line tool for querying VictoriaLogs
2024-10-01 10:08:17 +00:00
* FEATURE: add interactive command-line tool for querying VictoriaLogs - [`vlogscli`](https://docs.victoriametrics.com/victorialogs/querying/vlogscli/).
lib/logstorage: do not count dictionary values which have no matching logs in `count_uniq` stats function Create blockResultColumn.forEachDictValue* helper functions for visiting matching dictionary values. These helper functions should prevent from counting dictionary values without matching logs in the future. This is a follow-up for 0c0f013a60c812f46df1b8fa2863d1038e0e0208 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7152
2024-10-01 11:29:07 +00:00
* BUGFIX: [`count_uniq` stats function](https://docs.victoriametrics.com/victorialogs/logsql/#count_uniq-stats): do not count field values, which aren't matched by the used filters. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7152).
docs/VictoriaLogs/CHANGELOG.md: cut v0.32.1-victorialogs release
2024-09-30 12:28:34 +00:00
## [v0.32.1](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.32.1-victorialogs)
Released at 2024-09-30
lib/logstorage: skip values with zero hits for 'uniq', 'top' and 'field_values' pipes See https://github.com/VictoriaMetrics/victorialogs-datasource/issues/72#issuecomment-2352078483
2024-09-30 12:06:20 +00:00
* BUGFIX: do not return field values with zero matching logs from [`field_values`](https://docs.victoriametrics.com/victorialogs/logsql/#field_values-pipe), [`top`](https://docs.victoriametrics.com/victorialogs/logsql/#top-pipe) and [`uniq`](https://docs.victoriametrics.com/victorialogs/logsql/#uniq-pipe) pipes. See [this issue](https://github.com/VictoriaMetrics/victorialogs-datasource/issues/72#issuecomment-2352078483).
docs/VictoriaLogs/CHANGELOG.md: cut v0.32.0-victorialogs
2024-09-29 12:47:31 +00:00
## [v0.32.0](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.32.0-victorialogs)
Released at 2024-09-29
* FEATURE: [data ingestion](https://docs.victoriametrics.com/victorialogs/data-ingestion/): accept Unix timestamps in seconds in the ingested logs. This simplifies integration with systems, which prefer Unix timestamps over text-based representation of time.
* FEATURE: [`sort` pipe](https://docs.victoriametrics.com/victorialogs/logsql/#sort-pipe): allow using `order` alias instead of `sort`. For example, `_time:5s | order by (_time)` query works the same as `_time:5s | sort by (_time)`. This simplifies the to [LogsQL](https://docs.victoriametrics.com/victorialogs/logsql/) transition from SQL-like query languages.
lib/logstorage: add non-empty `if (...)` condition to automatically generated result names in `stats` pipe This allows executing queries with `stats` pipe, which calculate multiple results with the same functions, but with different `if (...)` conditions. For example: _time:5m | count(), count() if (error) Previously such queries couldn't be executed becasue automatically generated name for the second result didn't include `if (error)`, so names for both results were identical - `count(*)`.
2024-09-29 07:46:11 +00:00
* FEATURE: [`stats` pipe](https://docs.victoriametrics.com/victorialogs/logsql/#stats-pipe): allow using multiple identical [stats functions](https://docs.victoriametrics.com/victorialogs/logsql/#stats-pipe-functions) with distinct [filters](https://docs.victoriametrics.com/victorialogs/logsql/#stats-with-additional-filters) and automatically generated result names. For example, `_time:5m | count(), count() if (error)` query works as expected now, e.g. it returns two results over the last 5 minutes: the total number of logs and the number of logs with `error` [word](https://docs.victoriametrics.com/victorialogs/logsql/#word). Previously this query couldn't be executed because the `if (...)` condition wasn't included in the automatically generate result name, so both results had the same name - `count(*)`.
app/vlinsert: accept unix timestamp in seconds additionally to milliseconds at ElasticSearch bulk API Timestamps in seconds are sometimes used for data ingestion via ElasticSearch bulk API
2024-09-28 19:17:10 +00:00
lib/logstorage: clear hits slice obtained from encoding.GetUint64s() before updating it with hits for valueTypeDict column encoding.GetUint64s() returns uninitialized slice, which may contain arbitrary values. So values in this slice must be reset to zero before using it for counting hits in `uniq` and `top` pipes.
2024-09-29 08:27:33 +00:00
* BUGFIX: properly calculate [`uniq`](https://docs.victoriametrics.com/victorialogs/logsql/#uniq-pipe) and [`top`](https://docs.victoriametrics.com/victorialogs/logsql/#top-pipe) pipes. Previously they could return invalid results in some cases.
docs/VictoriaLogs/CHANGELOG.md: cut v0.31.0-victorialogs release
2024-09-27 11:54:17 +00:00
## [v0.31.0](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.31.0-victorialogs)
Released at 2024-09-27
vmui/logs: improve graph usability (#7025) ### Describe Your Changes - Show the time range in the tooltip when hovering over staircase graphs. - Use bolder lines for staircase graphs. - Increase the number of steps on the staircase graph to 100. - Reduce the maximum width of the tooltip to 1/3 of the screen. - Insert only the label name under the cursor into the query input field when `Ctrl`-clicking the line legend. See [this comment](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6545#issuecomment-2336805237). ### Checklist The following checks are **mandatory**: - [ ] My change adheres [VictoriaMetrics contributing guidelines](https://docs.victoriametrics.com/contributing/). --------- Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2024-09-27 11:19:46 +00:00
* FEATURE: [web UI](https://docs.victoriametrics.com/victorialogs/querying/#web-ui): improved readability of staircase graphs and tooltip usability. See [this comment](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6545#issuecomment-2336805237).
* FEATURE: [web UI](https://docs.victoriametrics.com/victorialogs/querying/#web-ui): simplify query input by adding only the label name when `ctrl`+clicking the line legend. See [this comment](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6545#issuecomment-2336805237).
vmui: add functionality to preserve selected columns (#7037) ### Describe Your Changes 1) Changed table settings from a popup to a modal window to simplify future functionality additions. 2) Added functionality to save selected columns when data is modified or the page is reloaded. See #7016. <details> <summary>Example screenshots</summary> <img alt="demo-1" width="600" src="https://github.com/user-attachments/assets/a5d9a910-363c-4931-8b12-18ea8b3d97d8"/> </details> ### Checklist The following checks are **mandatory**: - [x] My change adheres [VictoriaMetrics contributing guidelines](https://docs.victoriametrics.com/contributing/). --------- Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
2024-09-27 09:52:01 +00:00
* FEATURE: [web UI](https://docs.victoriametrics.com/victorialogs/querying/#web-ui): keep selected columns in table view on page reloads. Before, selected columns were reset on each update. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7016).
lib/logstorage: support skipping _stream: prefix for stream filters '_stream:{...}' can be written as '{...}' This simplifies writing queries with stream filters, and makes them more familier to Loki users.
2024-09-27 10:05:40 +00:00
* FEATURE: allow skipping `_stream:` prefix in [stream filters](https://docs.victoriametrics.com/victorialogs/logsql/#stream-filter). This simplifies writing queries with stream filters. Now `{foo="bar"}` is the recommended format for stream filters over the `_stream:{foo="bar"}` format.
lib/logstorage: allow using `-` instead of `!` as a shorthand for `NOT` operator in LogsQL
2024-09-27 11:12:34 +00:00
* FEATURE: allow using `-` instead of `!` as `NOT` operator shorthand in [logical filters](https://docs.victoriametrics.com/victorialogs/logsql/#logical-filter). For example, `-info -warn` query is equivalent to `!info !warn`. This simplifies transition from other query languages with full-text search support, which usually use `-` as `NOT` operator.
vmui: add functionality to preserve selected columns (#7037) ### Describe Your Changes 1) Changed table settings from a popup to a modal window to simplify future functionality additions. 2) Added functionality to save selected columns when data is modified or the page is reloaded. See #7016. <details> <summary>Example screenshots</summary> <img alt="demo-1" width="600" src="https://github.com/user-attachments/assets/a5d9a910-363c-4931-8b12-18ea8b3d97d8"/> </details> ### Checklist The following checks are **mandatory**: - [x] My change adheres [VictoriaMetrics contributing guidelines](https://docs.victoriametrics.com/contributing/). --------- Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
2024-09-27 09:52:01 +00:00
docs/VictoriaLogs/CHANGELOG.md: cut v0.30.1-victorialogs release
2024-09-27 09:20:58 +00:00
## [v0.30.1](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.30.1-victorialogs)
Released at 2024-09-27
lib/logstorage: consistently sort stream contexts belonging to different streams by the minimum time seen in the matching logs This should simplify debugging of stream_context output, since it remains stable over repeated requests.
2024-09-27 09:15:43 +00:00
* BUGFIX: consistently return matching log streams sorted by time from [`stream_context` pipe](https://docs.victoriametrics.com/victorialogs/logsql/#stream_context-pipe). Previously log streams could be returned in arbitrary order with every request. This could complicate using `stream_context` pipe.
lib/logstorage: allow using `-` instead of `!` as a shorthand for `NOT` operator in LogsQL
2024-09-27 11:12:34 +00:00
* BUGFIX: [`stream_context` pipe](https://docs.victoriametrics.com/victorialogs/logsql/#stream_context-pipe): add missing `_msg="---"` delimiter between stream contexts belonging to different [log streams](https://docs.victoriametrics.com/victorialogs/keyconcepts/#stream-fields). This should simplify investigating `stream_context` output for multiple matching log streams.
lib/logstorage: consistently sort stream contexts belonging to different streams by the minimum time seen in the matching logs This should simplify debugging of stream_context output, since it remains stable over repeated requests.
2024-09-27 09:15:43 +00:00
docs/VictoriaLogs/CHANGELOG.md: cut v0.30.0-victorialogs release
2024-09-27 07:18:31 +00:00
## [v0.30.0](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.30.0-victorialogs)
Released at 2024-09-27
vmui/logs: add auto refresh (#7038) ### Describe Your Changes Add auto refresh #7017 ![image](https://github.com/user-attachments/assets/20ed1102-d5e4-4d3f-9c24-7d298d93400a) ### Checklist The following checks are **mandatory**: - [ ] My change adheres [VictoriaMetrics contributing guidelines](https://docs.victoriametrics.com/contributing/). --------- Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
2024-09-19 09:11:16 +00:00
* FEATURE: [web UI](https://docs.victoriametrics.com/victorialogs/querying/#web-ui): add button for enabling auto refresh, similarly to VictoriaMetrics vmui. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7017).
feature: [victorialogs] drop logs without non-empty _msg field (#7056) ### Describe Your Changes VictoriaLogs allows logs without `_msg` field or `_msg` field is empty. This lead to incorrect search result. See: https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6785 This pull request search for non-empty `_msg` field before log entry is added to `LogRows`. New counter `vl_rows_dropped_total{reason="msg_not_exist"}` is introduced. Example log output: ``` 2024-09-23T02:33:19.719Z warn app/vlinsert/insertutils/common_params.go:189 dropping log line without _msg field; [{@timestamp 2024-09-18T13:42:16.600000000Z} {Attributes.array.attribute ["many","values"]} {Attributes.boolean.attribute true} {Attributes.double.attribute 637.704} {Attributes.int.attribute 10} {Attributes.map.attribute.some.map.key some value} {Attributes.string.attribute some string} {Body Example ddddddddddlog record} {Resource.service.name my.service} {Scope.my.scope.attribute some scope attribute} {Scope.name my.library} {Scope.version 1.0.0} {SeverityNumber 10} {SeverityText Information} {SpanId eee19b7ec3c1b174} {TraceFlags 0} {TraceId 5b8efff798038103d269b633813fc60c}] ``` ### Checklist The following checks are **mandatory**: - [x] My change adheres [VictoriaMetrics contributing guidelines](https://docs.victoriametrics.com/contributing/). - [ ] Benchmark for potential performance loss. --------- Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2024-09-26 07:35:28 +00:00
* FEATURE: drop logs without [`_msg`](https://docs.victoriametrics.com/victorialogs/keyconcepts/#message-field) field or with empty `_msg` field, since this field is required to be non-empty in [VictoriaLogs data model](https://docs.victoriametrics.com/victorialogs/keyconcepts/#data-model). See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6785).
lib/logstorage: read timestamps column when it is really needed during query execution Previously timestamps column was read unconditionally on every query. This could significantly slow down queries, which do not need reading this column like in https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7070 .
2024-09-25 14:16:53 +00:00
* FEATURE: improve performance of analytical queries, which do not need reading the `_time` field. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7070).
lib/logstorage: add `blocks_count` pipe This pipe is useful for debugging purposes when the number of processed blocks must be calculated for the given query: <query> | blocks_count This helps detecting the root cause of query performance slowdown in cases like https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7070
2024-09-25 17:15:32 +00:00
* FEATURE: add [`blocks_count` pipe](https://docs.victoriametrics.com/victorialogs/logsql/#blocks_count-pipe), which can be used for counting the number of matching blocks for the given query. For example, `_time:5m | blocks_count` returns the number of blocks with logs for the last 5 minutes. This pipe can be useful for debugging purposes.
app/vlinsert: support `_time` field without timezone information during data ingestion Use local timezone of the host server in this case. The timezone can be overridden with TZ environment variable if needed. While at it, allow using whitespace instead of T as a delimiter between data and time in the ingested _time field. For example, '2024-09-20 10:20:30' is now accepted during data ingestion. This is valid ISO8601 format, which is used by some log shippers, so it should be supported. This format is also known as SQL datetime format. Also assume local time zone when time without timezone information is passed to querying APIs. Previously such a time was parsed in UTC timezone. Add `Z` to the end of the time string if the old behaviour is preferred. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6721
2024-09-26 10:43:16 +00:00
* FEATURE: support [ingesting logs](https://docs.victoriametrics.com/victorialogs/data-ingestion/) with `_time` field, which doesn't contain timezone information. For example, `2024-09-20T10:20:30`. In this case the local timezone of the host where VictoriaLogs runs is used. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6721).
lib/logstorage: properly return surrounding logs outside the selected time range by stream_context pipe Previously only logs inside the selected time range could be returned by stream_context pipe. For example, the following query could return up to 10 surrounding logs only for the last 5 minutes, while most users expect this query should return up to 10 surrounding logs without restrictions on the time range. _time:5m panic | stream_context before 10 This enables the ability to implement stream context feature at VictoriaLogs web UI: https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7063 . Reduce memory usage when returning stream context over big log streams with millions of entries. The new logic scans over all the log messages for the selected log stream, while keeping in memory only the given number of surrounding logs. Previously all the logs for the given log stream on the selected time range were loaded in memory before selecting the needed surrounding logs. This should help https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6730 . Reduce the scan performance for big log streams by fetching only the requested fields. For example, the following query should be executed much faster than before if logs contain many fields other than _stream, _msg and _time: panic | stream_context after 30 | fields _stream, _msg, _time
2024-09-26 14:52:55 +00:00
* FEATURE: reduce memory usage when [`stream_context` pipe](https://docs.victoriametrics.com/victorialogs/logsql/#stream_context-pipe) is applied to [log streams](https://docs.victoriametrics.com/victorialogs/keyconcepts/#stream-fields) with big number of messages. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6730).
vmui/logs: add auto refresh (#7038) ### Describe Your Changes Add auto refresh #7017 ![image](https://github.com/user-attachments/assets/20ed1102-d5e4-4d3f-9c24-7d298d93400a) ### Checklist The following checks are **mandatory**: - [ ] My change adheres [VictoriaMetrics contributing guidelines](https://docs.victoriametrics.com/contributing/). --------- Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
2024-09-19 09:11:16 +00:00
docs/VictoriaLogs/CHANGELOG.md: document the fix for Windows build This is a follow-up for 264c2ec6bd78fcf222780e9aea91793563591702 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6998 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6973
2024-09-26 07:15:25 +00:00
* BUGFIX: fix Windows build, which has been broken in [v0.29.0](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.29.0-victorialogs). See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6973).
docs/VictoriaLogs/CHANGELOG.md: typo fix after 255d1d4e139acc0bc66cfbec2ecdaffdf9177e14: returns -> return
2024-09-26 07:00:55 +00:00
* BUGFIX: properly return logs from [`/select/logsql/tail` endpoint](https://docs.victoriametrics.com/victorialogs/querying/#live-tailing) if the query contains [`_time:some_duration` filter](https://docs.victoriametrics.com/victorialogs/logsql/#time-filter) like `_time:5m`. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7028). The bug has been introduced in [v0.29.0](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.29.0-victorialogs).
docs/VictoriaLogs/CHANGELOG.md: typo fix: itentifying -> identifying
2024-09-26 07:41:30 +00:00
* BUGFIX: properly return logs without [`_msg`](https://docs.victoriametrics.com/victorialogs/keyconcepts/#message-field) field when `*` query is passed to [`/select/logsql/query` endpoint](https://docs.victoriametrics.com/victorialogs/querying/#querying-logs) together with positive `limit` arg. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6785). Thanks to @jiekun for identifying the root cause of the issue.
app/vlinsert: support `_time` field without timezone information during data ingestion Use local timezone of the host server in this case. The timezone can be overridden with TZ environment variable if needed. While at it, allow using whitespace instead of T as a delimiter between data and time in the ingested _time field. For example, '2024-09-20 10:20:30' is now accepted during data ingestion. This is valid ISO8601 format, which is used by some log shippers, so it should be supported. This format is also known as SQL datetime format. Also assume local time zone when time without timezone information is passed to querying APIs. Previously such a time was parsed in UTC timezone. Add `Z` to the end of the time string if the old behaviour is preferred. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6721
2024-09-26 10:43:16 +00:00
* BUGFIX: support [ingesting logs](https://docs.victoriametrics.com/victorialogs/data-ingestion/) with `_time` field containing whitespace delimiter between the date and time instead of `T` delimiter. For example, `2024-09-20 10:20:30`. This is valid [ISO8601 format](https://en.wikipedia.org/wiki/ISO_8601) aka `SQL datetime` format, which sometimes is used in production. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6721).
lib/logstorage: properly return surrounding logs outside the selected time range by stream_context pipe Previously only logs inside the selected time range could be returned by stream_context pipe. For example, the following query could return up to 10 surrounding logs only for the last 5 minutes, while most users expect this query should return up to 10 surrounding logs without restrictions on the time range. _time:5m panic | stream_context before 10 This enables the ability to implement stream context feature at VictoriaLogs web UI: https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7063 . Reduce memory usage when returning stream context over big log streams with millions of entries. The new logic scans over all the log messages for the selected log stream, while keeping in memory only the given number of surrounding logs. Previously all the logs for the given log stream on the selected time range were loaded in memory before selecting the needed surrounding logs. This should help https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6730 . Reduce the scan performance for big log streams by fetching only the requested fields. For example, the following query should be executed much faster than before if logs contain many fields other than _stream, _msg and _time: panic | stream_context after 30 | fields _stream, _msg, _time
2024-09-26 14:52:55 +00:00
* BUGFIX: return all the requested surrounding logs for [`stream_context` pipe](https://docs.victoriametrics.com/victorialogs/logsql/#stream_context-pipe). Previously only logs matching the [`_time` filter](https://docs.victoriametrics.com/victorialogs/logsql/#time-filter) were returned. This is needed for [this feature](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7063).
app/vlselect/logsql: call `Query.Optimize()` on the cloned query in order to replace `*` filter with `filterNoop` inside getLastNQueryResults() Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6785
2024-09-18 16:24:14 +00:00
docs/VictoriaLogs/CHANGELOG.md: cut v0.29.0
2024-09-08 19:10:17 +00:00
## [v0.29.0](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.29.0-victorialogs)
Released at 2024-09-08
app/vlselect: add /select/logsql/stats_query endpoint, which is going to be used by vmalert Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6942 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6706
2024-09-06 21:00:55 +00:00
* FEATURE: add [`/select/logsql/stats_query` HTTP API](https://docs.victoriametrics.com/victorialogs/querying/#querying-log-stats), which is going to be used by [vmalert](https://docs.victoriametrics.com/vmalert/) for executing alerting and recording rules against VictoriaLogs. See [this feature request](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6942) for details.
app/vlselect: add /select/logsql/stats_query_range endpoint for building time series panels in VictoriaLogs plugin for Grafana Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6943 Updates https://github.com/VictoriaMetrics/victorialogs-datasource/issues/61
2024-09-06 22:41:44 +00:00
* FEATURE: add [`/select/logsql/stats_query_range` HTTP API](https://docs.victoriametrics.com/victorialogs/querying/#querying-log-range-stats), which is going to be used by [VictoriaLogs plugin for Grafana](https://docs.victoriametrics.com/victorialogs/victorialogs-datasource/) for building time series panels. See [this feature request](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6943) for details.
lib/logstorage: pre-calculate hashes from tokens used in bloom filter search Previously per-token hashes for per-block bloom filters were re-calculated on every scanned block. This could be slow when the number of tokens is big or when the number of blocks to scan is big. Pre-calculate hashes for bloom filters and then use them for searching in bloom filters. This improves performance by 2.5x for in(...) filters with many values to search inside `in()`.
2024-09-05 17:40:50 +00:00
* FEATURE: optimize [multi-exact queries](https://docs.victoriametrics.com/victorialogs/logsql/#multi-exact-filter) with many phrases to search. For example, `ip:in(path:="/foo/bar" | keep ip)` when there are many unique values for `ip` field among log entries with `/foo/bar` path.
vmui/logs: add display top streams in the hits graph (#6647) ### Describe Your Changes - Adds support for displaying the top 5 log streams in the hits graph, grouping the remaining streams into an "other" label. #6545 - Adds options to customize the graph display with bar, line, stepped line, and points views. ### Checklist The following checks are **mandatory**: - [x] My change adheres [VictoriaMetrics contributing guidelines](https://docs.victoriametrics.com/contributing/).
2024-08-06 14:28:44 +00:00
* FEATURE: [web UI](https://docs.victoriametrics.com/victorialogs/querying/#web-ui): add support for displaying the top 5 log streams in the hits graph. The remaining log streams are grouped into an "other" label. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6545).
* FEATURE: [web UI](https://docs.victoriametrics.com/victorialogs/querying/#web-ui): add the ability to customize the graph display with options for bar, line, stepped line, and points.
vmui/logs: add fields for tenant configuration (#6661) Added fields for configuring AccountID and ProjectID #6631
2024-08-02 07:57:39 +00:00
* FEATURE: [web UI](https://docs.victoriametrics.com/victorialogs/querying/#web-ui): add fields for setting AccountID and ProjectID. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6631).
vmui/logs: improve UI functionality (#6688) * add a toggle button to the "Group" tab that allows users to expand or collapse all groups at once * introduce the ability to select a key for grouping logs within the "Group" tab * display the number of entries within each log group. * move the Markdown toggle to the general settings panel in the upper left corner.
2024-08-02 13:48:36 +00:00
* FEATURE: [web UI](https://docs.victoriametrics.com/victorialogs/querying/#web-ui): add a toggle button to the "Group" tab that allows users to expand or collapse all groups at once.
* FEATURE: [web UI](https://docs.victoriametrics.com/victorialogs/querying/#web-ui): introduce the ability to select a key for grouping logs within the "Group" tab.
* FEATURE: [web UI](https://docs.victoriametrics.com/victorialogs/querying/#web-ui): display the number of entries within each log group.
* FEATURE: [web UI](https://docs.victoriametrics.com/victorialogs/querying/#web-ui): move the Markdown toggle to the general settings panel in the upper left corner.
vmui: add column search in table settings (#6804) ### Describe Your Changes Add search functionality to the column display settings in the table #6668 ![image](https://github.com/user-attachments/assets/e9bd52c3-6428-4d4f-8b7f-d83dd80b6912) ### Checklist The following checks are **mandatory**: - [ ] My change adheres [VictoriaMetrics contributing guidelines](https://docs.victoriametrics.com/contributing/). --------- Signed-off-by: hagen1778 <roman@victoriametrics.com> Co-authored-by: hagen1778 <roman@victoriametrics.com>
2024-08-22 14:57:26 +00:00
* FEATURE: [web UI](https://docs.victoriametrics.com/victorialogs/querying/#web-ui): add search functionality to the column display settings in the table. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6668).
* FEATURE: [web UI](https://docs.victoriametrics.com/victorialogs/querying/#web-ui): add the ability to select all columns in the column display settings of the table. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6668). Thanks to @yincongcyincong for the [pull request](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6680).
follow-up after 1731c0eabfe0c69fb0fa9a3dec8b39d9546db30c * updates change log * adds VL-Debug http header * updates doc * extracts only the first value of http headers for VL-Stream-Fields and VL-Ignore-Fields. It makes behaviour the same as Query string args. And allows to easily configure client applications. Since most of the client collectors don't support multi value headers. Signed-off-by: f41gh7 <nik@victoriametrics.com>
2024-09-03 17:16:10 +00:00
* FEATURE: Allow to define ingestion parameters via headers. Supported headers - `VL-Msg-Field`,`VL-Stream-Fields`,`VL-Ignore-Fields`,`VL-Time-Field`, `VL-Debug`. See this [PR](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6443) for details.
docs/changelog: moves victorialogs changes to proper file Signed-off-by: f41gh7 <nik@victoriametrics.com>
2024-09-04 12:44:24 +00:00
* FEATURE: [vlinsert](https://docs.victoriametrics.com/victorialogs/): added OpenTelemetry logs ingestion support. See this [PR](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6218) for details.
vmui/logs: add fields for tenant configuration (#6661) Added fields for configuring AccountID and ProjectID #6631
2024-08-02 07:57:39 +00:00
docs/victorialogs/CHANGELOG.md: mention issue #4750 in the change Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-08-29 13:09:52 +00:00
* BUGFIX: properly handle Logstash requests for Elasticsearch configuration when using `outputs.elasticsearch` in Logstash pipelines. Previously, the requests could be rejected with `400 Bad Request` response. Updates [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4750).
docs: move change from VM changelog to VL changelog The change for https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6764 was added to VM changelog by mistake Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-08-20 15:16:04 +00:00
* BUGFIX: [vmui](https://docs.victoriametrics.com/#vmui): fix `not found index.js` error when loading vmui in VictoriaLogs. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6764). Thanks to @yincongcyincong for the [pull request](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6770).
lib/logstorage: properly fix incorrect extraction of common tokens for `OR` filters at distinct log fields Previously (f1:foo OR f2:bar) was incorrectly returning `foo` token for `f1` and `bar` token for `f2`. These tokens were used for checking against bloom filter for every data block, so the data block, which didn't contain simultaneously `foo` token for `f1` field and `bar` token for `f2` field, was skipped. This was incorrect, since such a block may contain logs matching the original OR filter. The fix is to return common tokens from `OR`-delimted filters only if these tokens exist at EVERY such filter for the given field name. If some `OR`-delimited filter misses the given field name, then `OR`-delimited filters do not contain common tokens, which could be used for checking against bloom filter. While at it, add more tests covering various edge cases for filters delimited by AND and OR. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6554 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6556
2024-09-05 12:14:57 +00:00
* BUGFIX: properly execute queries with `OR` [filters](https://docs.victoriametrics.com/victorialogs/logsql/#logical-filter) for distinct [log fields](https://docs.victoriametrics.com/victorialogs/keyconcepts/#data-model). For example, `field1:foo OR field2:bar`. Previously logs matching these filters may be skipped during querying. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6554) for details. Thanks to @yincongcyincong for the [pull request](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6556).
app/vlinsert/elasticsearch: add fake response for logstash requests (#6742) ### Describe Your Changes This is needed in order to support standard Elasticsearch output in Logstash pipelines. See: https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6660 ### Checklist The following checks are **mandatory**: - [x] My change adheres [VictoriaMetrics contributing guidelines](https://docs.victoriametrics.com/contributing/). --------- Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2024-08-06 13:43:33 +00:00
docs/VictoriaLogs/CHANGELOG.md: cut v0.28.0-victorialogs release
2024-07-10 01:08:16 +00:00
## [v0.28.0](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.28.0-victorialogs)
Released at 2024-07-10
Revert "removed unneeded ref shortcodes, updated VM changelog to use relative markdown links (#6691)" This reverts commit 2e9b1efeb9422c60a345c67ebbbcaa5f8d37469d. Reason for revert: relative links in docs are much harder to maintain in consistent state comparing to absolute links: - It is non-trivial to figure out the proper relative link path when creating and editing docs. - Relative links break after moving the doc files to another paths, and it is non-trivial to figure which links are broken after that. - The updated relative links do not work properly right now in the docs. For example, the https://docs.victoriametrics.com/victorialogs/quickstart.md#building-from-source-code link at https://docs.victoriametrics.com/victorialogs/changelog/ leads to 404 page. This is documented at https://docs.victoriametrics.com/#images-in-documentation .
2024-07-25 10:47:35 +00:00
* FEATURE: [web UI](https://docs.victoriametrics.com/victorialogs/querying/#web-ui): show a spinner on top of bar chart until user's request is finished. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6558).
* FEATURE: [web UI](https://docs.victoriametrics.com/victorialogs/querying/#web-ui): use compact representation of JSON lines at `JSON` tab if only a single [log field](https://docs.victoriametrics.com/victorialogs/keyconcepts/#data-model) is queried. See [this feature request](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6559).
* FEATURE: [web UI](https://docs.victoriametrics.com/victorialogs/querying/#web-ui): properly show the number of matching logs on the selected time range at bar chart for queries with arbitrary [pipes](https://docs.victoriametrics.com/victorialogs/logsql/#pipes), including [`stats` pipe](https://docs.victoriametrics.com/victorialogs/logsql/#stats-pipe) and [`top` pipe](https://docs.victoriametrics.com/victorialogs/logsql/#top-pipe).
vmui/logs: add spinner to bar chart (#6577) Add a spinner to the bar chart https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6558 --------- Signed-off-by: hagen1778 <roman@victoriametrics.com> Co-authored-by: hagen1778 <roman@victoriametrics.com>
2024-07-09 12:58:48 +00:00
docs/VictoriaLogs/CHANGELOG.md: cut v0.27.1-victorialogs
2024-07-04 23:22:06 +00:00
## [v0.27.1](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.27.1-victorialogs)
Released at 2024-07-05
Revert "removed unneeded ref shortcodes, updated VM changelog to use relative markdown links (#6691)" This reverts commit 2e9b1efeb9422c60a345c67ebbbcaa5f8d37469d. Reason for revert: relative links in docs are much harder to maintain in consistent state comparing to absolute links: - It is non-trivial to figure out the proper relative link path when creating and editing docs. - Relative links break after moving the doc files to another paths, and it is non-trivial to figure which links are broken after that. - The updated relative links do not work properly right now in the docs. For example, the https://docs.victoriametrics.com/victorialogs/quickstart.md#building-from-source-code link at https://docs.victoriametrics.com/victorialogs/changelog/ leads to 404 page. This is documented at https://docs.victoriametrics.com/#images-in-documentation .
2024-07-25 10:47:35 +00:00
* BUGFIX: properly JSON-encode strings with special chars in [HTTP querying API](https://docs.victoriametrics.com/victorialogs/querying/#http-api) responses. This fixes the `error decode response: invalid character 'x' in string escape code` error in [VictoriaLogs datasource for Grafana](https://github.com/VictoriaMetrics/victorialogs-datasource/). See [this issue](https://github.com/VictoriaMetrics/victorialogs-datasource/issues/24). The issue has been introduced in the release [v0.9.0](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.9.0-victorialogs).
lib/logstorage: use quicktemplate.AppendJSONString instead of strconv.AppendQuote for encoding JSON strings The strconv.AppendQuote improperly encodes special chars such as \x1b . They must be encoded as \u001b . See https://github.com/VictoriaMetrics/victorialogs-datasource/issues/24
2024-07-04 23:17:03 +00:00
docs/VictoriaLogs/CHANGELOG.md: cut v0.27.0-victorialogs
2024-07-01 23:43:46 +00:00
## [v0.27.0](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.27.0-victorialogs)
Released at 2024-07-02
Revert "removed unneeded ref shortcodes, updated VM changelog to use relative markdown links (#6691)" This reverts commit 2e9b1efeb9422c60a345c67ebbbcaa5f8d37469d. Reason for revert: relative links in docs are much harder to maintain in consistent state comparing to absolute links: - It is non-trivial to figure out the proper relative link path when creating and editing docs. - Relative links break after moving the doc files to another paths, and it is non-trivial to figure which links are broken after that. - The updated relative links do not work properly right now in the docs. For example, the https://docs.victoriametrics.com/victorialogs/quickstart.md#building-from-source-code link at https://docs.victoriametrics.com/victorialogs/changelog/ leads to 404 page. This is documented at https://docs.victoriametrics.com/#images-in-documentation .
2024-07-25 10:47:35 +00:00
* FEATURE: add `-syslog.useLocalTimestamp.tcp` and `-syslog.useLocalTimestamp.udp` command-line flags, which could be used for using the local timestamp as [`_time` field](https://docs.victoriametrics.com/victorialogs/keyconcepts/#time-field) for the logs ingested via the corresponding `-syslog.listenAddr.tcp` / `-syslog.listenAddr.udp`. By default the timestap from the syslog message is used as [`_time` field](https://docs.victoriametrics.com/victorialogs/keyconcepts/#time-field). See [these docs](https://docs.victoriametrics.com/victorialogs/data-ingestion/syslog/).
app/vlinsert/syslog: add an ability to use log ingestion time as the _time field
2024-07-01 22:23:54 +00:00
docs/VictoriaLogs/CHANGELOG.md: cut v0.27.0-victorialogs
2024-07-01 23:43:46 +00:00
* BUGFIX: make slowly ingested logs visible for search as soon as they are ingested into VictoriaLogs. Previously slowly ingested logs could remain invisible for search for long time.
app/vlinsert/insertutils: flush the ingested logs from in-memory buffer to storage every second Previously the in-memory buffer could remain unflushed for long periods of time under low ingestion rate. The ingested logs weren't visible for search during this time.
2024-07-01 23:28:02 +00:00
docs/VictoriaLogs/CHANGELOG.md: cut v0.26.1-victorialogs
2024-07-01 00:32:51 +00:00
## [v0.26.1](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.26.1-victorialogs)
Released at 2024-07-01
Revert "removed unneeded ref shortcodes, updated VM changelog to use relative markdown links (#6691)" This reverts commit 2e9b1efeb9422c60a345c67ebbbcaa5f8d37469d. Reason for revert: relative links in docs are much harder to maintain in consistent state comparing to absolute links: - It is non-trivial to figure out the proper relative link path when creating and editing docs. - Relative links break after moving the doc files to another paths, and it is non-trivial to figure which links are broken after that. - The updated relative links do not work properly right now in the docs. For example, the https://docs.victoriametrics.com/victorialogs/quickstart.md#building-from-source-code link at https://docs.victoriametrics.com/victorialogs/changelog/ leads to 404 page. This is documented at https://docs.victoriametrics.com/#images-in-documentation .
2024-07-25 10:47:35 +00:00
* BUGFIX: return the proper surrounding logs for [`stream_context` pipe](https://docs.victoriametrics.com/victorialogs/logsql/#stream_context-pipe) when additional [pipes](https://docs.victoriametrics.com/victorialogs/logsql/#pipes) are put after the `stream_context` pipe. This has been broken in [v0.26.0](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.26.0-victorialogs).
lib/logstorage: properly search for the surrounding logs in `stream_context` pipe The set of log fields in the found logs may differ from the set of log fields present in the log stream. So compare only the log fields in the found logs when searching for the matching log entry in the log stream. While at it, return _stream field in the delimiter log entry, since this field is used by VictoriaLogs Web UI for grouping logs by log streams.
2024-07-01 00:27:13 +00:00
docs/VictoriaLogs/CHANGELOG.md: cut v0.26.0-victorialogs
2024-06-30 23:49:49 +00:00
## [v0.26.0](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.26.0-victorialogs)
Released at 2024-07-01
Revert "removed unneeded ref shortcodes, updated VM changelog to use relative markdown links (#6691)" This reverts commit 2e9b1efeb9422c60a345c67ebbbcaa5f8d37469d. Reason for revert: relative links in docs are much harder to maintain in consistent state comparing to absolute links: - It is non-trivial to figure out the proper relative link path when creating and editing docs. - Relative links break after moving the doc files to another paths, and it is non-trivial to figure which links are broken after that. - The updated relative links do not work properly right now in the docs. For example, the https://docs.victoriametrics.com/victorialogs/quickstart.md#building-from-source-code link at https://docs.victoriametrics.com/victorialogs/changelog/ leads to 404 page. This is documented at https://docs.victoriametrics.com/#images-in-documentation .
2024-07-25 10:47:35 +00:00
* FEATURE: add ability to return log position (aka rank) after sorting logs with [`sort` pipe](https://docs.victoriametrics.com/victorialogs/logsql/#sort-pipe). This can be done by adding `rank as <fieldName>` to the end of `| sort ...` pipe. For example, `_time:5m | sort by (_time) rank as position` instructs storing position of every sorted log line into `position` [field name](https://docs.victoriametrics.com/victorialogs/keyconcepts/#data-model).
* FEATURE: add delimiter log with `---` message between log chunks returned by [`stream_context` pipe](https://docs.victoriametrics.com/victorialogs/logsql/#stream_context-pipe). This should simplify investigation of the returned logs.
* FEATURE: reduce memory usage when big number of context logs are requested from [`stream_context` pipe](https://docs.victoriametrics.com/victorialogs/logsql/#stream_context-pipe).
lib/logstorage: add delimiter between log chunks returned from `| stream_context` pipe
2024-06-30 22:49:31 +00:00
docs/VictoriaLogs/CHANGELOG.md: cut v0.25.0-victorialogs
2024-06-28 17:17:49 +00:00
## [v0.25.0](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.25.0-victorialogs)
Released at 2024-06-28
Revert "removed unneeded ref shortcodes, updated VM changelog to use relative markdown links (#6691)" This reverts commit 2e9b1efeb9422c60a345c67ebbbcaa5f8d37469d. Reason for revert: relative links in docs are much harder to maintain in consistent state comparing to absolute links: - It is non-trivial to figure out the proper relative link path when creating and editing docs. - Relative links break after moving the doc files to another paths, and it is non-trivial to figure which links are broken after that. - The updated relative links do not work properly right now in the docs. For example, the https://docs.victoriametrics.com/victorialogs/quickstart.md#building-from-source-code link at https://docs.victoriametrics.com/victorialogs/changelog/ leads to 404 page. This is documented at https://docs.victoriametrics.com/#images-in-documentation .
2024-07-25 10:47:35 +00:00
* FEATURE: add ability to select surrounding logs in front and after the selected logs via [`stream_context` pipe](https://docs.victoriametrics.com/victorialogs/logsql/#stream_context-pipe). This functionality may be useful for investigating stacktraces, panics or some correlated log messages. This functionality is similar to `grep -A` and `grep -B`.
* FEATURE: add ability to return top `N` `"fields"` groups from [`/select/logsql/hits` HTTP endpoint](https://docs.victoriametrics.com/victorialogs/querying/#querying-hits-stats), by specifying `fields_limit=N` query arg. This query arg is going to be used in [this feature request](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6545).
app/vlselect/logsql: add optional fields_limit query arg to /select/logsql/hits HTTP endpoint This query arg is needed for https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6545 in order to return top N groups with the biggest number of hits.
2024-06-28 01:08:37 +00:00
Revert "removed unneeded ref shortcodes, updated VM changelog to use relative markdown links (#6691)" This reverts commit 2e9b1efeb9422c60a345c67ebbbcaa5f8d37469d. Reason for revert: relative links in docs are much harder to maintain in consistent state comparing to absolute links: - It is non-trivial to figure out the proper relative link path when creating and editing docs. - Relative links break after moving the doc files to another paths, and it is non-trivial to figure which links are broken after that. - The updated relative links do not work properly right now in the docs. For example, the https://docs.victoriametrics.com/victorialogs/quickstart.md#building-from-source-code link at https://docs.victoriametrics.com/victorialogs/changelog/ leads to 404 page. This is documented at https://docs.victoriametrics.com/#images-in-documentation .
2024-07-25 10:47:35 +00:00
* BUGFIX: fix `runtime error: index out of range [0] with length 0` panic when empty lines are ingested via [Syslog format](https://docs.victoriametrics.com/victorialogs/data-ingestion/syslog/) by Cisco controllers. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6548).
app/vlinsert/syslog: properly skip empty lines in Syslog protocol Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6548
2024-06-28 12:09:26 +00:00
lib/logstorage: work-in-progress
2024-06-27 12:18:42 +00:00
## [v0.24.0](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.24.0-victorialogs)
Released at 2024-06-27
Revert "removed unneeded ref shortcodes, updated VM changelog to use relative markdown links (#6691)" This reverts commit 2e9b1efeb9422c60a345c67ebbbcaa5f8d37469d. Reason for revert: relative links in docs are much harder to maintain in consistent state comparing to absolute links: - It is non-trivial to figure out the proper relative link path when creating and editing docs. - Relative links break after moving the doc files to another paths, and it is non-trivial to figure which links are broken after that. - The updated relative links do not work properly right now in the docs. For example, the https://docs.victoriametrics.com/victorialogs/quickstart.md#building-from-source-code link at https://docs.victoriametrics.com/victorialogs/changelog/ leads to 404 page. This is documented at https://docs.victoriametrics.com/#images-in-documentation .
2024-07-25 10:47:35 +00:00
* FEATURE: add `/select/logsql/tail` HTTP endpoint, which can be used for live tailing of [LogsQL query](https://docs.victoriametrics.com/victorialogs/logsql/) results. See [these docs](https://docs.victoriametrics.com/victorialogs/querying/#live-tailing) for details.
* FEATURE: add `/select/logsql/stream_ids` HTTP endpoint, which can be used for returning [`_stream_id` values](https://docs.victoriametrics.com/victorialogs/keyconcepts/#stream-fields) with the number of hits for the given [LogsQL query](https://docs.victoriametrics.com/victorialogs/logsql/). See [these docs](https://docs.victoriametrics.com/victorialogs/querying/#querying-stream_ids) for details.
* FEATURE: add `-retention.maxDiskSpaceUsageBytes` command-line flag, which allows limiting disk space usage for [VictoriaLogs data](https://docs.victoriametrics.com/victorialogs/#storage) by automatic dropping the oldest per-day partitions if the storage disk space usage becomes bigger than the `-retention.maxDiskSpaceUsageBytes`. See [these docs](https://docs.victoriametrics.com/victorialogs/#retention-by-disk-space-usage).
app/vlstorage: add -retention.maxDiskSpaceUsageBytes command-line flag for limiting the retention at VictoriaLogs by disk space usage
2024-06-25 15:30:02 +00:00
docs: add spellcheck command (#6562) ### Describe Your Changes Implement spellcheck command: - add cspell configuration files - dockerize spellchecking process - add Makefile targets This PR adds a standalone `make spellcheck` target to check `docs/*.md` files for spelling errors. The target process is dockerized to be run in a separate npm environment. Some `docs/` typo fixes also included. ### Checklist The following checks are **mandatory**: - [x] My change adheres [VictoriaMetrics contributing guidelines](https://docs.victoriametrics.com/contributing/). --------- Signed-off-by: Arkadii Yakovets <ark@victoriametrics.com> Signed-off-by: hagen1778 <roman@victoriametrics.com> Co-authored-by: hagen1778 <roman@victoriametrics.com>
2024-07-11 10:39:42 +00:00
* BUGFIX: properly take into account query timeout specified via `-search.maxQueryDuration` command-line flag and/or via `timeout` query arg. Previously these timeouts could be ignored during query execution.
Revert "removed unneeded ref shortcodes, updated VM changelog to use relative markdown links (#6691)" This reverts commit 2e9b1efeb9422c60a345c67ebbbcaa5f8d37469d. Reason for revert: relative links in docs are much harder to maintain in consistent state comparing to absolute links: - It is non-trivial to figure out the proper relative link path when creating and editing docs. - Relative links break after moving the doc files to another paths, and it is non-trivial to figure which links are broken after that. - The updated relative links do not work properly right now in the docs. For example, the https://docs.victoriametrics.com/victorialogs/quickstart.md#building-from-source-code link at https://docs.victoriametrics.com/victorialogs/changelog/ leads to 404 page. This is documented at https://docs.victoriametrics.com/#images-in-documentation .
2024-07-25 10:47:35 +00:00
* BUGFIX: [web UI](https://docs.victoriametrics.com/victorialogs/querying/#web-ui): fix the update of the relative time range when `Execute Query` is clicked. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6345).
vmui/logs: fix the update of the relative time range (#6517) ### Describe Your Changes - Fixed the update of the relative time range when `Execute Query` is clicked - Optimized server requests: now, if an error occurs in the `/query` request, the `/hits` request will not be executed. #6345 (duplicates: #6440, #6312)
2024-06-26 09:23:22 +00:00
docs/VictoriaLogs/CHANGELOG.md: cut v0.23.0-victorialogs release
2024-06-25 12:59:11 +00:00
## [v0.23.0](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.23.0-victorialogs)
Released at 2024-06-25
Revert "removed unneeded ref shortcodes, updated VM changelog to use relative markdown links (#6691)" This reverts commit 2e9b1efeb9422c60a345c67ebbbcaa5f8d37469d. Reason for revert: relative links in docs are much harder to maintain in consistent state comparing to absolute links: - It is non-trivial to figure out the proper relative link path when creating and editing docs. - Relative links break after moving the doc files to another paths, and it is non-trivial to figure which links are broken after that. - The updated relative links do not work properly right now in the docs. For example, the https://docs.victoriametrics.com/victorialogs/quickstart.md#building-from-source-code link at https://docs.victoriametrics.com/victorialogs/changelog/ leads to 404 page. This is documented at https://docs.victoriametrics.com/#images-in-documentation .
2024-07-25 10:47:35 +00:00
* FEATURE: [syslog data ingestion](https://docs.victoriametrics.com/victorialogs/data-ingestion/syslog/): parse [STRUCTURED-DATA](https://datatracker.ietf.org/doc/html/rfc5424#section-6.3) into `SD-ID.field1=value1`, `SD-ID.field2=value2`, ..., `SD-ID.fieldN=valueN` [log fields](https://docs.victoriametrics.com/victorialogs/keyconcepts/#data-model). Previously the `STRUCTURED-DATA` was parsed into a single log field with the `SD-ID` name and `field1=value1 field2=value2 ... fieldN=valueN` value. This could complicate querying of such data.
lib/logstorage: parse syslog structured data into separate fields in order to simplify further querying of this data
2024-06-25 12:52:43 +00:00
Revert "removed unneeded ref shortcodes, updated VM changelog to use relative markdown links (#6691)" This reverts commit 2e9b1efeb9422c60a345c67ebbbcaa5f8d37469d. Reason for revert: relative links in docs are much harder to maintain in consistent state comparing to absolute links: - It is non-trivial to figure out the proper relative link path when creating and editing docs. - Relative links break after moving the doc files to another paths, and it is non-trivial to figure which links are broken after that. - The updated relative links do not work properly right now in the docs. For example, the https://docs.victoriametrics.com/victorialogs/quickstart.md#building-from-source-code link at https://docs.victoriametrics.com/victorialogs/changelog/ leads to 404 page. This is documented at https://docs.victoriametrics.com/#images-in-documentation .
2024-07-25 10:47:35 +00:00
* BUGFIX: properly parse timestamps with timezones during [data ingestion](https://docs.victoriametrics.com/victorialogs/data-ingestion/) and [querying](https://docs.victoriametrics.com/victorialogs/querying/). This has been broken in [v0.20.0](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.20.0-victorialogs). See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6508).
lib/logstorage: properly parse timezone offset at TryParseTimestampRFC3339Nano() The TryParseTimestampRFC3339Nano() must properly parse RFC3339 timestamps with timezone offsets. While at it, make tryParseTimestampISO8601 function private in order to prevent from improper usage of this function from outside the lib/logstorage package. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6508
2024-06-25 12:08:01 +00:00
lib/logstorage: work-in-progress
2024-06-24 21:27:12 +00:00
## [v0.22.0](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.22.0-victorialogs)
Released at 2024-06-24
Revert "removed unneeded ref shortcodes, updated VM changelog to use relative markdown links (#6691)" This reverts commit 2e9b1efeb9422c60a345c67ebbbcaa5f8d37469d. Reason for revert: relative links in docs are much harder to maintain in consistent state comparing to absolute links: - It is non-trivial to figure out the proper relative link path when creating and editing docs. - Relative links break after moving the doc files to another paths, and it is non-trivial to figure which links are broken after that. - The updated relative links do not work properly right now in the docs. For example, the https://docs.victoriametrics.com/victorialogs/quickstart.md#building-from-source-code link at https://docs.victoriametrics.com/victorialogs/changelog/ leads to 404 page. This is documented at https://docs.victoriametrics.com/#images-in-documentation .
2024-07-25 10:47:35 +00:00
* FEATURE: allow specifying multiple `_stream_id` values in [`_stream_id` filter](https://docs.victoriametrics.com/victorialogs/logsql/#_stream_id-filter) via `_stream_id:in(id1, ..., idN)` syntax.
* FEATURE: allow specifying subquery for searching for `_stream_id` values inside [`_stream_id` filter](https://docs.victoriametrics.com/victorialogs/logsql/#_stream_id-filter). For example, `_stream_id:in(_time:5m error | fields _stream_id)` returns logs for [logs streams](https://docs.victoriametrics.com/victorialogs/keyconcepts/#stream-fields) with the `error` word across logs for the last 5 minutes.
lib/logstorage: work-in-progress
2024-06-24 21:27:12 +00:00
lib/logstorage: work-in-progress
2024-06-20 01:08:07 +00:00
## [v0.21.0](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.21.0-victorialogs)
Released at 2024-06-20
Revert "removed unneeded ref shortcodes, updated VM changelog to use relative markdown links (#6691)" This reverts commit 2e9b1efeb9422c60a345c67ebbbcaa5f8d37469d. Reason for revert: relative links in docs are much harder to maintain in consistent state comparing to absolute links: - It is non-trivial to figure out the proper relative link path when creating and editing docs. - Relative links break after moving the doc files to another paths, and it is non-trivial to figure which links are broken after that. - The updated relative links do not work properly right now in the docs. For example, the https://docs.victoriametrics.com/victorialogs/quickstart.md#building-from-source-code link at https://docs.victoriametrics.com/victorialogs/changelog/ leads to 404 page. This is documented at https://docs.victoriametrics.com/#images-in-documentation .
2024-07-25 10:47:35 +00:00
* FEATURE: [web UI](https://docs.victoriametrics.com/victorialogs/querying/#web-ui): add a bar chart displaying the number of log entries over a time range. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6404).
* FEATURE: expose `_stream_id` field, which uniquely identifies [log streams](https://docs.victoriametrics.com/victorialogs/keyconcepts/#stream-fields). This field can be used for quick obtaining of all the logs belonging to a particular stream via [`_stream_id` filter](https://docs.victoriametrics.com/victorialogs/logsql/#_stream_id-filter).
vmui/logs: add bar chart (#6461) - Added a bar chart displaying the number of log entries over a time range. #6404 - When `_msg` is empty, all fields are displayed in a single line. - Added double quotes when copying pairs: `key: "value"`. - Minor style adjustments.
2024-06-18 13:23:21 +00:00
app/vlinsert: properly parse timestamps with nanosecond precision at /insert/jsonline HTTP endpoint This has been broken in 2b6a634ec0676c6f8bc71bcf51fab488dd4ab2e1
2024-06-17 22:23:17 +00:00
## [v0.20.2](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.20.2-victorialogs)
docs/VictoriaLogs/CHANGELOG.md: typo fix in the release date for v0.20.2-victorialogs
2024-06-17 22:28:40 +00:00
Released at 2024-06-18
app/vlinsert: properly parse timestamps with nanosecond precision at /insert/jsonline HTTP endpoint This has been broken in 2b6a634ec0676c6f8bc71bcf51fab488dd4ab2e1
2024-06-17 22:23:17 +00:00
Revert "removed unneeded ref shortcodes, updated VM changelog to use relative markdown links (#6691)" This reverts commit 2e9b1efeb9422c60a345c67ebbbcaa5f8d37469d. Reason for revert: relative links in docs are much harder to maintain in consistent state comparing to absolute links: - It is non-trivial to figure out the proper relative link path when creating and editing docs. - Relative links break after moving the doc files to another paths, and it is non-trivial to figure which links are broken after that. - The updated relative links do not work properly right now in the docs. For example, the https://docs.victoriametrics.com/victorialogs/quickstart.md#building-from-source-code link at https://docs.victoriametrics.com/victorialogs/changelog/ leads to 404 page. This is documented at https://docs.victoriametrics.com/#images-in-documentation .
2024-07-25 10:47:35 +00:00
* BUGFIX: properly parse timestamps with nanosecond precision for logs ingested via [jsonline format](https://docs.victoriametrics.com/victorialogs/data-ingestion/#json-stream-api). The bug has been introduced in [v0.20.0 release](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.20.0-victorialogs).
app/vlinsert: properly parse timestamps with nanosecond precision at /insert/jsonline HTTP endpoint This has been broken in 2b6a634ec0676c6f8bc71bcf51fab488dd4ab2e1
2024-06-17 22:23:17 +00:00
docs/VictoriaLogs/CHANGELOG.md: cut v0.20.1-victorialogs
2024-06-17 21:29:17 +00:00
## [v0.20.1](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.20.1-victorialogs)
Released at 2024-06-18
Revert "removed unneeded ref shortcodes, updated VM changelog to use relative markdown links (#6691)" This reverts commit 2e9b1efeb9422c60a345c67ebbbcaa5f8d37469d. Reason for revert: relative links in docs are much harder to maintain in consistent state comparing to absolute links: - It is non-trivial to figure out the proper relative link path when creating and editing docs. - Relative links break after moving the doc files to another paths, and it is non-trivial to figure which links are broken after that. - The updated relative links do not work properly right now in the docs. For example, the https://docs.victoriametrics.com/victorialogs/quickstart.md#building-from-source-code link at https://docs.victoriametrics.com/victorialogs/changelog/ leads to 404 page. This is documented at https://docs.victoriametrics.com/#images-in-documentation .
2024-07-25 10:47:35 +00:00
* FEATURE: allow configuring multiple receivers with distinct configs for syslog messages. See [these docs](https://docs.victoriametrics.com/victorialogs/data-ingestion/syslog/#multiple-configs).
app/vlinsert/syslog: allow accepting syslog messages with different configs at different ports
2024-06-17 21:16:34 +00:00
Revert "removed unneeded ref shortcodes, updated VM changelog to use relative markdown links (#6691)" This reverts commit 2e9b1efeb9422c60a345c67ebbbcaa5f8d37469d. Reason for revert: relative links in docs are much harder to maintain in consistent state comparing to absolute links: - It is non-trivial to figure out the proper relative link path when creating and editing docs. - Relative links break after moving the doc files to another paths, and it is non-trivial to figure which links are broken after that. - The updated relative links do not work properly right now in the docs. For example, the https://docs.victoriametrics.com/victorialogs/quickstart.md#building-from-source-code link at https://docs.victoriametrics.com/victorialogs/changelog/ leads to 404 page. This is documented at https://docs.victoriametrics.com/#images-in-documentation .
2024-07-25 10:47:35 +00:00
* BUGFIX: properly read syslog messages over TCP and TLS connections according to [RFC5425](https://datatracker.ietf.org/doc/html/rfc5425) when [data ingestion for syslog protocol](https://docs.victoriametrics.com/victorialogs/data-ingestion/syslog/) is enabled.
app/vlinsert: properly parse length-delimited syslog messages sent over TCP according to RFC5425
2024-06-17 20:28:15 +00:00
lib/logstorage: work-in-progress
2024-06-17 10:13:18 +00:00
## [v0.20.0](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.20.0-victorialogs)
Released at 2024-06-17
Revert "removed unneeded ref shortcodes, updated VM changelog to use relative markdown links (#6691)" This reverts commit 2e9b1efeb9422c60a345c67ebbbcaa5f8d37469d. Reason for revert: relative links in docs are much harder to maintain in consistent state comparing to absolute links: - It is non-trivial to figure out the proper relative link path when creating and editing docs. - Relative links break after moving the doc files to another paths, and it is non-trivial to figure which links are broken after that. - The updated relative links do not work properly right now in the docs. For example, the https://docs.victoriametrics.com/victorialogs/quickstart.md#building-from-source-code link at https://docs.victoriametrics.com/victorialogs/changelog/ leads to 404 page. This is documented at https://docs.victoriametrics.com/#images-in-documentation .
2024-07-25 10:47:35 +00:00
* FEATURE: add ability to accept logs in [Syslog format](https://en.wikipedia.org/wiki/Syslog). See [these docs](https://docs.victoriametrics.com/victorialogs/data-ingestion/syslog/).
* FEATURE: add ability to specify timezone offset when parsing [rfc3164](https://datatracker.ietf.org/doc/html/rfc3164) syslog messages with [`unpack_syslog` pipe](https://docs.victoriametrics.com/victorialogs/logsql/#unpack_syslog-pipe).
* FEATURE: add [`top` pipe](https://docs.victoriametrics.com/victorialogs/logsql/#top-pipe) for returning top N sets of the given fields with the maximum number of matching log entries.
lib/logstorage: work-in-progress
2024-06-17 10:13:18 +00:00
lib/logstorage: work-in-progress
2024-06-11 15:50:32 +00:00
## [v0.19.0](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.19.0-victorialogs)
Released at 2024-06-11
Revert "removed unneeded ref shortcodes, updated VM changelog to use relative markdown links (#6691)" This reverts commit 2e9b1efeb9422c60a345c67ebbbcaa5f8d37469d. Reason for revert: relative links in docs are much harder to maintain in consistent state comparing to absolute links: - It is non-trivial to figure out the proper relative link path when creating and editing docs. - Relative links break after moving the doc files to another paths, and it is non-trivial to figure which links are broken after that. - The updated relative links do not work properly right now in the docs. For example, the https://docs.victoriametrics.com/victorialogs/quickstart.md#building-from-source-code link at https://docs.victoriametrics.com/victorialogs/changelog/ leads to 404 page. This is documented at https://docs.victoriametrics.com/#images-in-documentation .
2024-07-25 10:47:35 +00:00
* FEATURE: do not allow starting the [filter](https://docs.victoriametrics.com/victorialogs/logsql/#filters) with [pipe names](https://docs.victoriametrics.com/victorialogs/logsql/#pipes) and [stats function names](https://docs.victoriametrics.com/victorialogs/logsql/#stats-pipe-functions). This prevents from unexpected results returned by incorrect queries, which miss mandatory [filter](https://docs.victoriametrics.com/victorialogs/logsql/#query-syntax).
* FEATURE: treat unexpected syslog message as [RFC3164](https://datatracker.ietf.org/doc/html/rfc3164) containing only the `message` field when using [`unpack_syslog` pipe](https://docs.victoriametrics.com/victorialogs/logsql/#unpack_syslog-pipe).
* FEATURE: allow using `where` prefix instead of `filter` prefix in [`filter` pipe](https://docs.victoriametrics.com/victorialogs/logsql/#filter-pipe).
* FEATURE: disallow unescaped `!` char in [LogsQL](https://docs.victoriametrics.com/victorialogs/logsql/) queries, since it permits writing incorrect query, which may look like correct one. For example, `foo!:bar` instead of `foo:!bar`.
* FEATURE: [web UI](https://docs.victoriametrics.com/victorialogs/querying/#web-ui): add markdown support to the `Group` view. See [this pull request](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6292).
vmui/logs: add markdown support (#6292) Add support for markdown format and emoji for the `_msg` field in the "Group" view. Add markdown rendering toggle. Disabled by default. Value is stored in `localStorage`.
2024-06-10 14:38:13 +00:00
lib/logstorage: work-in-progress
2024-06-11 15:50:32 +00:00
* BUGFIX: return back the improved performance for queries with `*` filters (aka `SELECT *`). This has been broken in [v0.16.0](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.16.0-victorialogs).
lib/logstorage: work-in-progress
2024-06-06 10:27:05 +00:00
## [v0.18.0](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.18.0-victorialogs)
Released at 2024-06-06
Revert "removed unneeded ref shortcodes, updated VM changelog to use relative markdown links (#6691)" This reverts commit 2e9b1efeb9422c60a345c67ebbbcaa5f8d37469d. Reason for revert: relative links in docs are much harder to maintain in consistent state comparing to absolute links: - It is non-trivial to figure out the proper relative link path when creating and editing docs. - Relative links break after moving the doc files to another paths, and it is non-trivial to figure which links are broken after that. - The updated relative links do not work properly right now in the docs. For example, the https://docs.victoriametrics.com/victorialogs/quickstart.md#building-from-source-code link at https://docs.victoriametrics.com/victorialogs/changelog/ leads to 404 page. This is documented at https://docs.victoriametrics.com/#images-in-documentation .
2024-07-25 10:47:35 +00:00
* FEATURE: [web UI](https://docs.victoriametrics.com/victorialogs/querying/#web-ui): improve displaying of logs. See [this pull request](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6419) and the following issues: [6408](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6408), [6405](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6405), [6406](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6406) and [6407](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6407).
* FEATURE: add support for [day range filter](https://docs.victoriametrics.com/victorialogs/logsql/#day-range-filter) and [week range filter](https://docs.victoriametrics.com/victorialogs/logsql/#week-range-filter). These filters allow selecting logs on a particular time range per every day or on a particular day of the week.
* FEATURE: allow using `eval` instead of `math` keyword in [`math` pipe](https://docs.victoriametrics.com/victorialogs/logsql/#math-pipe).
lib/logstorage: allow using `eval` keyword instead of `math` keyword in `math` pipe
2024-06-05 08:07:49 +00:00
lib/logstorage: work-in-progress
2024-06-05 01:18:12 +00:00
## [v0.17.0](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.17.0-victorialogs)
Released at 2024-06-05
Revert "removed unneeded ref shortcodes, updated VM changelog to use relative markdown links (#6691)" This reverts commit 2e9b1efeb9422c60a345c67ebbbcaa5f8d37469d. Reason for revert: relative links in docs are much harder to maintain in consistent state comparing to absolute links: - It is non-trivial to figure out the proper relative link path when creating and editing docs. - Relative links break after moving the doc files to another paths, and it is non-trivial to figure which links are broken after that. - The updated relative links do not work properly right now in the docs. For example, the https://docs.victoriametrics.com/victorialogs/quickstart.md#building-from-source-code link at https://docs.victoriametrics.com/victorialogs/changelog/ leads to 404 page. This is documented at https://docs.victoriametrics.com/#images-in-documentation .
2024-07-25 10:47:35 +00:00
* FEATURE: add [`pack_logfmt` pipe](https://docs.victoriametrics.com/victorialogs/logsql/#pack_logfmt-pipe) for formatting [log fields](https://docs.victoriametrics.com/victorialogs/keyconcepts/#data-model) into [logfmt](https://brandur.org/logfmt) messages.
* FEATURE: allow using IPv4 addresses in [range comparison filters](https://docs.victoriametrics.com/victorialogs/logsql/#range-comparison-filter). For example, `ip:>'12.34.56.78'` is valid filter now.
* FEATURE: add `ceil()` and `floor()` functions to [`math` pipe](https://docs.victoriametrics.com/victorialogs/logsql/#math-pipe).
* FEATURE: add support for bitwise `and`, `or` and `xor` operations at [`math` pipe](https://docs.victoriametrics.com/victorialogs/logsql/#math-pipe).
* FEATURE: add support for automatic conversion of [RFC3339 time](https://www.rfc-editor.org/rfc/rfc3339) and IPv4 addresses into numeric representation at [`math` pipe](https://docs.victoriametrics.com/victorialogs/logsql/#math-pipe).
* FEATURE: add ability to format numeric fields into string representation of time, duration and IPv4 with [`format` pipe](https://docs.victoriametrics.com/victorialogs/logsql/#format-pipe).
* FEATURE: set `format` field to `rfc3164` or `rfc5424` depending on the [Syslog format](https://en.wikipedia.org/wiki/Syslog) parsed via [`unpack_syslog` pipe](https://docs.victoriametrics.com/victorialogs/logsql/#unpack_syslog-pipe).
lib/logstorage: work-in-progress
2024-06-05 01:18:12 +00:00
Revert "removed unneeded ref shortcodes, updated VM changelog to use relative markdown links (#6691)" This reverts commit 2e9b1efeb9422c60a345c67ebbbcaa5f8d37469d. Reason for revert: relative links in docs are much harder to maintain in consistent state comparing to absolute links: - It is non-trivial to figure out the proper relative link path when creating and editing docs. - Relative links break after moving the doc files to another paths, and it is non-trivial to figure which links are broken after that. - The updated relative links do not work properly right now in the docs. For example, the https://docs.victoriametrics.com/victorialogs/quickstart.md#building-from-source-code link at https://docs.victoriametrics.com/victorialogs/changelog/ leads to 404 page. This is documented at https://docs.victoriametrics.com/#images-in-documentation .
2024-07-25 10:47:35 +00:00
* BUGFIX: always respect the limit set in [`limit` pipe](https://docs.victoriametrics.com/victorialogs/logsql/#limit-pipe). Previously the limit could be exceeded in some cases.
lib/logstorage: work-in-progress
2024-06-05 01:18:12 +00:00
lib/logstorage: work-in-progress
2024-06-03 22:59:25 +00:00
## [v0.16.0](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.16.0-victorialogs)
Released at 2024-06-04
Revert "removed unneeded ref shortcodes, updated VM changelog to use relative markdown links (#6691)" This reverts commit 2e9b1efeb9422c60a345c67ebbbcaa5f8d37469d. Reason for revert: relative links in docs are much harder to maintain in consistent state comparing to absolute links: - It is non-trivial to figure out the proper relative link path when creating and editing docs. - Relative links break after moving the doc files to another paths, and it is non-trivial to figure which links are broken after that. - The updated relative links do not work properly right now in the docs. For example, the https://docs.victoriametrics.com/victorialogs/quickstart.md#building-from-source-code link at https://docs.victoriametrics.com/victorialogs/changelog/ leads to 404 page. This is documented at https://docs.victoriametrics.com/#images-in-documentation .
2024-07-25 10:47:35 +00:00
* FEATURE: add [`unpack_syslog` pipe](https://docs.victoriametrics.com/victorialogs/logsql/#unpack_syslog-pipe) for unpacking [syslog](https://en.wikipedia.org/wiki/Syslog) messages from [log fields](https://docs.victoriametrics.com/victorialogs/keyconcepts/#data-model).
* FEATURE: parse timestamps in [`_time` filter](https://docs.victoriametrics.com/victorialogs/logsql/#time-filter) with nanosecond precision.
* FEATURE: return the last `N` matching logs from [`/select/logsql/query` HTTP API](https://docs.victoriametrics.com/victorialogs/querying/#querying-logs) with the maximum timestamps if `limit=N` query arg is passed to it. Previously a random subset of matching logs could be returned, which could complicate investigation of the returned logs.
* FEATURE: add [`drop_empty_fields` pipe](https://docs.victoriametrics.com/victorialogs/logsql/#drop_empty_fields-pipe) for dropping [log fields](https://docs.victoriametrics.com/victorialogs/keyconcepts/#data-model) with empty values.
lib/logstorage: work-in-progress
2024-06-03 22:59:25 +00:00
lib/logstorage: work-in-progress
2024-05-30 14:19:23 +00:00
## [v0.15.0](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.15.0-victorialogs)
Released at 2024-05-30
Revert "removed unneeded ref shortcodes, updated VM changelog to use relative markdown links (#6691)" This reverts commit 2e9b1efeb9422c60a345c67ebbbcaa5f8d37469d. Reason for revert: relative links in docs are much harder to maintain in consistent state comparing to absolute links: - It is non-trivial to figure out the proper relative link path when creating and editing docs. - Relative links break after moving the doc files to another paths, and it is non-trivial to figure which links are broken after that. - The updated relative links do not work properly right now in the docs. For example, the https://docs.victoriametrics.com/victorialogs/quickstart.md#building-from-source-code link at https://docs.victoriametrics.com/victorialogs/changelog/ leads to 404 page. This is documented at https://docs.victoriametrics.com/#images-in-documentation .
2024-07-25 10:47:35 +00:00
* FEATURE: add [`row_any`](https://docs.victoriametrics.com/victorialogs/logsql/#row_any-stats) function for [`stats` pipe](https://docs.victoriametrics.com/victorialogs/logsql/#stats-pipe). This function returns a sample log entry per every calculated [group of results](https://docs.victoriametrics.com/victorialogs/logsql/#stats-by-fields).
* FEATURE: add `default` operator to [`math` pipe](https://docs.victoriametrics.com/victorialogs/logsql/#math-pipe). It allows overriding `NaN` results with the given default value.
* FEATURE: add `exp()` and `ln()` functions to [`math` pipe](https://docs.victoriametrics.com/victorialogs/logsql/#math-pipe).
* FEATURE: allow omitting result name in [`math` pipe](https://docs.victoriametrics.com/victorialogs/logsql/#math-pipe) expresions. In this case the result name is automatically set to string representation of the corresponding math expression. For example, `_time:5m | math duration / 1000` is equivalent to `_time:5m | math (duration / 1000) as "duration / 1000"`.
* FEATURE: allow omitting result name in [`stats` pipe](https://docs.victoriametrics.com/victorialogs/logsql/#stats-pipe). In this case the result name is automatically set to string representation of the corresponding [stats function expression](https://docs.victoriametrics.com/victorialogs/logsql/#stats-pipe-functions). For example, `_time:5m | count(*)` is valid [LogsQL query](https://docs.victoriametrics.com/victorialogs/logsql/) now. It is equivalent to `_time:5m | stats count(*) as "count(*)"`.
lib/logstorage: work-in-progress
2024-05-30 14:19:23 +00:00
* BUGFIX: properly calculate the number of matching rows in `* | field_values x | stats count() rows` and in `* | unroll (x) | stats count() rows` queries.
lib/logstorage: work-in-progress
2024-05-28 23:52:13 +00:00
## [v0.14.0](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.14.0-victorialogs)
Released at 2024-05-29
Revert "removed unneeded ref shortcodes, updated VM changelog to use relative markdown links (#6691)" This reverts commit 2e9b1efeb9422c60a345c67ebbbcaa5f8d37469d. Reason for revert: relative links in docs are much harder to maintain in consistent state comparing to absolute links: - It is non-trivial to figure out the proper relative link path when creating and editing docs. - Relative links break after moving the doc files to another paths, and it is non-trivial to figure which links are broken after that. - The updated relative links do not work properly right now in the docs. For example, the https://docs.victoriametrics.com/victorialogs/quickstart.md#building-from-source-code link at https://docs.victoriametrics.com/victorialogs/changelog/ leads to 404 page. This is documented at https://docs.victoriametrics.com/#images-in-documentation .
2024-07-25 10:47:35 +00:00
* FEATURE: allow specifying fields, which must be packed into JSON in [`pack_json` pipe](https://docs.victoriametrics.com/victorialogs/logsql/#pack_json-pipe) via `pack_json fields (field1, ..., fieldN)` syntax.
lib/logstorage: work-in-progress
2024-05-28 23:52:13 +00:00
Revert "removed unneeded ref shortcodes, updated VM changelog to use relative markdown links (#6691)" This reverts commit 2e9b1efeb9422c60a345c67ebbbcaa5f8d37469d. Reason for revert: relative links in docs are much harder to maintain in consistent state comparing to absolute links: - It is non-trivial to figure out the proper relative link path when creating and editing docs. - Relative links break after moving the doc files to another paths, and it is non-trivial to figure which links are broken after that. - The updated relative links do not work properly right now in the docs. For example, the https://docs.victoriametrics.com/victorialogs/quickstart.md#building-from-source-code link at https://docs.victoriametrics.com/victorialogs/changelog/ leads to 404 page. This is documented at https://docs.victoriametrics.com/#images-in-documentation .
2024-07-25 10:47:35 +00:00
* BUGFIX: properly apply `if (...)` filters to calculated results in [`stats` pipe](https://docs.victoriametrics.com/victorialogs/logsql/#stats-pipe) when [grouping by fields](https://docs.victoriametrics.com/victorialogs/logsql/#stats-by-fields) is enabled. For example, `_time:5m | stats by (host) count() logs, count() if (error) errors` now properly calculates per-`host` `errors`.
lib/logstorage: work-in-progress
2024-05-28 23:52:13 +00:00
lib/logstorage: work-in-progress
2024-05-28 17:29:41 +00:00
## [v0.13.0](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.13.0-victorialogs)
Released at 2024-05-28
Revert "removed unneeded ref shortcodes, updated VM changelog to use relative markdown links (#6691)" This reverts commit 2e9b1efeb9422c60a345c67ebbbcaa5f8d37469d. Reason for revert: relative links in docs are much harder to maintain in consistent state comparing to absolute links: - It is non-trivial to figure out the proper relative link path when creating and editing docs. - Relative links break after moving the doc files to another paths, and it is non-trivial to figure which links are broken after that. - The updated relative links do not work properly right now in the docs. For example, the https://docs.victoriametrics.com/victorialogs/quickstart.md#building-from-source-code link at https://docs.victoriametrics.com/victorialogs/changelog/ leads to 404 page. This is documented at https://docs.victoriametrics.com/#images-in-documentation .
2024-07-25 10:47:35 +00:00
* FEATURE: add [`extract_regexp` pipe](https://docs.victoriametrics.com/victorialogs/logsql/#extract_regexp-pipe) for extracting arbitrary substrings from [log fields](https://docs.victoriametrics.com/victorialogs/keyconcepts/#data-model) with [RE2 regular expressions](https://github.com/google/re2/wiki/Syntax).
* FEATURE: add [`math` pipe](https://docs.victoriametrics.com/victorialogs/logsql/#math-pipe) for mathematical calculations over [log fields](https://docs.victoriametrics.com/victorialogs/keyconcepts/#data-model).
* FEATURE: add [`field_values` pipe](https://docs.victoriametrics.com/victorialogs/logsql/#field_values-pipe), which returns unique values for the given [log field](https://docs.victoriametrics.com/victorialogs/keyconcepts/#data-model).
* FEATURE: allow omitting `stats` prefix in [`stats` pipe](https://docs.victoriametrics.com/victorialogs/logsql/#stats-pipe). For example, `_time:5m | count() rows` is a valid query now. It is equivalent to `_time:5m | stats count() as rows`.
* FEATURE: allow omitting `filter` prefix in [`filter` pipe](https://docs.victoriametrics.com/victorialogs/logsql/#filter-pipe) if the filter doesn't clash with [pipe names](#https://docs.victoriametrics.com/victorialogs/logsql/#pipes). For example, `_time:5m | stats by (host) count() rows | rows:>1000` is a valid query now. It is equivalent to `_time:5m | stats by (host) count() rows | filter rows:>1000`.
* FEATURE: allow [`head` pipe](https://docs.victoriametrics.com/victorialogs/logsql/#limit-pipe) without number. For example, `error | head`. In this case 10 first values are returned as `head` Unix command does by default.
* FEATURE: allow using [comparison filters](https://docs.victoriametrics.com/victorialogs/logsql/#range-comparison-filter) with strings. For example, `some_text_field:>="foo"` matches [log entries](https://docs.victoriametrics.com/victorialogs/keyconcepts/#data-model) with `some_text_field` field values bigger or equal to `foo`.
lib/logstorage: work-in-progress
2024-05-28 17:29:41 +00:00
lib/logstorage: work-in-progress
2024-05-25 23:55:21 +00:00
## [v0.12.1](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.12.1-victorialogs)
Released at 2024-05-26
Revert "removed unneeded ref shortcodes, updated VM changelog to use relative markdown links (#6691)" This reverts commit 2e9b1efeb9422c60a345c67ebbbcaa5f8d37469d. Reason for revert: relative links in docs are much harder to maintain in consistent state comparing to absolute links: - It is non-trivial to figure out the proper relative link path when creating and editing docs. - Relative links break after moving the doc files to another paths, and it is non-trivial to figure which links are broken after that. - The updated relative links do not work properly right now in the docs. For example, the https://docs.victoriametrics.com/victorialogs/quickstart.md#building-from-source-code link at https://docs.victoriametrics.com/victorialogs/changelog/ leads to 404 page. This is documented at https://docs.victoriametrics.com/#images-in-documentation .
2024-07-25 10:47:35 +00:00
* FEATURE: add support for comments in multi-line LogsQL queries. See [these docs](https://docs.victoriametrics.com/victorialogs/logsql/#comments).
lib/logstorage: work-in-progress
2024-05-25 23:55:21 +00:00
Revert "removed unneeded ref shortcodes, updated VM changelog to use relative markdown links (#6691)" This reverts commit 2e9b1efeb9422c60a345c67ebbbcaa5f8d37469d. Reason for revert: relative links in docs are much harder to maintain in consistent state comparing to absolute links: - It is non-trivial to figure out the proper relative link path when creating and editing docs. - Relative links break after moving the doc files to another paths, and it is non-trivial to figure which links are broken after that. - The updated relative links do not work properly right now in the docs. For example, the https://docs.victoriametrics.com/victorialogs/quickstart.md#building-from-source-code link at https://docs.victoriametrics.com/victorialogs/changelog/ leads to 404 page. This is documented at https://docs.victoriametrics.com/#images-in-documentation .
2024-07-25 10:47:35 +00:00
* BUGFIX: properly apply [`in(...)` filter](https://docs.victoriametrics.com/victorialogs/logsql/#multi-exact-filter) inside `if (...)` conditions at various [pipes](https://docs.victoriametrics.com/victorialogs/logsql/#pipes). This bug has been introduced in [v0.12.0](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.12.0-victorialogs).
lib/logstorage: work-in-progress
2024-05-25 23:55:21 +00:00
docs/VictoriaLogs/CHANGELOG.md: cut v0.12.0-victorialogs
2024-05-25 19:40:10 +00:00
## [v0.12.0](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.12.0-victorialogs)
Released at 2024-05-26
Revert "removed unneeded ref shortcodes, updated VM changelog to use relative markdown links (#6691)" This reverts commit 2e9b1efeb9422c60a345c67ebbbcaa5f8d37469d. Reason for revert: relative links in docs are much harder to maintain in consistent state comparing to absolute links: - It is non-trivial to figure out the proper relative link path when creating and editing docs. - Relative links break after moving the doc files to another paths, and it is non-trivial to figure which links are broken after that. - The updated relative links do not work properly right now in the docs. For example, the https://docs.victoriametrics.com/victorialogs/quickstart.md#building-from-source-code link at https://docs.victoriametrics.com/victorialogs/changelog/ leads to 404 page. This is documented at https://docs.victoriametrics.com/#images-in-documentation .
2024-07-25 10:47:35 +00:00
* FEATURE: add [`pack_json` pipe](https://docs.victoriametrics.com/victorialogs/logsql/#pack_json-pipe), which packs all the [log fields](https://docs.victoriametrics.com/victorialogs/keyconcepts/#data-model) into a JSON object and stores it into the given field.
* FEATURE: add [`unroll` pipe](https://docs.victoriametrics.com/victorialogs/logsql/#unroll-pipe), which can be used for unrolling JSON arrays stored in [log fields](https://docs.victoriametrics.com/victorialogs/keyconcepts/#data-model).
* FEATURE: add [`replace_regexp` pipe](https://docs.victoriametrics.com/victorialogs/logsql/#replace_regexp-pipe), which allows updating [log fields](https://docs.victoriametrics.com/victorialogs/keyconcepts/#data-model) with regular expressions.
* FEATURE: improve performance for [`format`](https://docs.victoriametrics.com/victorialogs/logsql/#format-pipe) and [`extract`](https://docs.victoriametrics.com/victorialogs/logsql/#extract-pipe) pipes.
* FEATURE: improve performance for [`/select/logsql/field_names` HTTP API](https://docs.victoriametrics.com/victorialogs/querying/#querying-field-names).
lib/logstorage: work-in-progress
2024-05-25 19:36:16 +00:00
Revert "removed unneeded ref shortcodes, updated VM changelog to use relative markdown links (#6691)" This reverts commit 2e9b1efeb9422c60a345c67ebbbcaa5f8d37469d. Reason for revert: relative links in docs are much harder to maintain in consistent state comparing to absolute links: - It is non-trivial to figure out the proper relative link path when creating and editing docs. - Relative links break after moving the doc files to another paths, and it is non-trivial to figure which links are broken after that. - The updated relative links do not work properly right now in the docs. For example, the https://docs.victoriametrics.com/victorialogs/quickstart.md#building-from-source-code link at https://docs.victoriametrics.com/victorialogs/changelog/ leads to 404 page. This is documented at https://docs.victoriametrics.com/#images-in-documentation .
2024-07-25 10:47:35 +00:00
* BUGFIX: prevent from panic in [`sort` pipe](https://docs.victoriametrics.com/victorialogs/logsql/#sort-pipe) when VictoriaLogs runs on a system with one CPU core.
docs: fix docs/ and README.md spelling errors (#6362) Fixes `docs/` and `README.md` typos and errors. Signed-off-by: Arkadii Yakovets <ark@victoriametrics.com>
2024-06-03 08:04:13 +00:00
* BUGFIX: do not return referenced fields if they weren't present in the original logs. For example, `_time:5m | format if (non_existing_field:"") "abc"` could return empty `non_exiting_field`, while it shouldn't be returned because it is missing in the original logs.
Revert "removed unneeded ref shortcodes, updated VM changelog to use relative markdown links (#6691)" This reverts commit 2e9b1efeb9422c60a345c67ebbbcaa5f8d37469d. Reason for revert: relative links in docs are much harder to maintain in consistent state comparing to absolute links: - It is non-trivial to figure out the proper relative link path when creating and editing docs. - Relative links break after moving the doc files to another paths, and it is non-trivial to figure which links are broken after that. - The updated relative links do not work properly right now in the docs. For example, the https://docs.victoriametrics.com/victorialogs/quickstart.md#building-from-source-code link at https://docs.victoriametrics.com/victorialogs/changelog/ leads to 404 page. This is documented at https://docs.victoriametrics.com/#images-in-documentation .
2024-07-25 10:47:35 +00:00
* BUGFIX: properly initialize values for [`in(...)` filter](https://docs.victoriametrics.com/victorialogs/logsql/#multi-exact-filter) inside [`filter` pipe](https://docs.victoriametrics.com/victorialogs/logsql/#filter-pipe) if the `in(...)` contains other [filters](https://docs.victoriametrics.com/victorialogs/logsql/#filters). For example, `_time:5m | filter ip:in(user_type:admin | fields ip)` now works correctly.
lib/logstorage: work-in-progress
2024-05-25 19:36:16 +00:00
lib/logstorage: work-in-progress
2024-05-24 22:30:58 +00:00
## [v0.11.0](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.11.0-victorialogs)
Released at 2024-05-25
Revert "removed unneeded ref shortcodes, updated VM changelog to use relative markdown links (#6691)" This reverts commit 2e9b1efeb9422c60a345c67ebbbcaa5f8d37469d. Reason for revert: relative links in docs are much harder to maintain in consistent state comparing to absolute links: - It is non-trivial to figure out the proper relative link path when creating and editing docs. - Relative links break after moving the doc files to another paths, and it is non-trivial to figure which links are broken after that. - The updated relative links do not work properly right now in the docs. For example, the https://docs.victoriametrics.com/victorialogs/quickstart.md#building-from-source-code link at https://docs.victoriametrics.com/victorialogs/changelog/ leads to 404 page. This is documented at https://docs.victoriametrics.com/#images-in-documentation .
2024-07-25 10:47:35 +00:00
* FEATURE: add [`replace` pipe](https://docs.victoriametrics.com/victorialogs/logsql/#replace-pipe), which allows replacing substrings in [log fields](https://docs.victoriametrics.com/victorialogs/keyconcepts/#data-model).
* FEATURE: support [comparing](https://docs.victoriametrics.com/victorialogs/logsql/#range-filter) log field values with [special numeric values](https://docs.victoriametrics.com/victorialogs/logsql/#numeric-values). For example, `duration:>1.5s` and `response_size:<15KiB` are valid filters now.
* FEATURE: properly sort [durations](https://docs.victoriametrics.com/victorialogs/logsql/#duration-values) and [short numeric values](https://docs.victoriametrics.com/victorialogs/logsql/#short-numeric-values) in [`sort` pipe](https://docs.victoriametrics.com/victorialogs/logsql/#sort-pipe). For example, `10s` goes in front of `1h`, while `10KB` goes in front of `1GB`.
* FEATURE: add an ability to preserve the original non-empty field values when executing [`extract`](https://docs.victoriametrics.com/victorialogs/logsql/#extract-pipe), [`unpack_json`](https://docs.victoriametrics.com/victorialogs/logsql/#unpack_json-pipe), [`unpack_logfmt`](https://docs.victoriametrics.com/victorialogs/logsql/#unpack_logfmt-pipe) and [`format`](https://docs.victoriametrics.com/victorialogs/logsql/#format-pipe) pipes.
* FEATURE: add an ability to preserve the original field values if the corresponding unpacked values are empty when executing [`extract`](https://docs.victoriametrics.com/victorialogs/logsql/#extract-pipe), [`unpack_json`](https://docs.victoriametrics.com/victorialogs/logsql/#unpack_json-pipe), [`unpack_logfmt`](https://docs.victoriametrics.com/victorialogs/logsql/#unpack_logfmt-pipe) and [`format`](https://docs.victoriametrics.com/victorialogs/logsql/#format-pipe) pipes.
lib/logstorage: work-in-progress
2024-05-24 22:30:58 +00:00
docs/VictoriaLogs/CHANGELOG.md: cut v0.10.0-victorialogs
2024-05-24 01:08:17 +00:00
## [v0.10.0](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.10.0-victorialogs)
Released at 2024-05-24
Revert "removed unneeded ref shortcodes, updated VM changelog to use relative markdown links (#6691)" This reverts commit 2e9b1efeb9422c60a345c67ebbbcaa5f8d37469d. Reason for revert: relative links in docs are much harder to maintain in consistent state comparing to absolute links: - It is non-trivial to figure out the proper relative link path when creating and editing docs. - Relative links break after moving the doc files to another paths, and it is non-trivial to figure which links are broken after that. - The updated relative links do not work properly right now in the docs. For example, the https://docs.victoriametrics.com/victorialogs/quickstart.md#building-from-source-code link at https://docs.victoriametrics.com/victorialogs/changelog/ leads to 404 page. This is documented at https://docs.victoriametrics.com/#images-in-documentation .
2024-07-25 10:47:35 +00:00
* FEATURE: return the number of matching log entries per returned value in [HTTP API](https://docs.victoriametrics.com/victorialogs/querying/#http-api) results. This simplifies detecting [field](https://docs.victoriametrics.com/victorialogs/keyconcepts/#data-model) / [stream](https://docs.victoriametrics.com/victorialogs/keyconcepts/#stream-fields) values with the biggest number of logs for the given [LogsQL query](https://docs.victoriametrics.com/victorialogs/logsql/).
* FEATURE: improve performance for [regexp filter](https://docs.victoriametrics.com/victorialogs/logsql/#regexp-filter) in the following cases:
follow-up after 1731c0eabfe0c69fb0fa9a3dec8b39d9546db30c * updates change log * adds VL-Debug http header * updates doc * extracts only the first value of http headers for VL-Stream-Fields and VL-Ignore-Fields. It makes behaviour the same as Query string args. And allows to easily configure client applications. Since most of the client collectors don't support multi value headers. Signed-off-by: f41gh7 <nik@victoriametrics.com>
2024-09-03 17:16:10 +00:00
* If the regexp contains just a phrase without special regular expression chars. For example, `~"foo"`.
* If the regexp starts with `.*` or ends with `.*`. For example, `~".*foo.*"`.
* If the regexp contains multiple strings delimited by `|`. For example, `~"foo|bar|baz"`.
* If the regexp contains multiple [words](https://docs.victoriametrics.com/victorialogs/logsql/#word). For example, `~"foo bar baz"`.
Revert "removed unneeded ref shortcodes, updated VM changelog to use relative markdown links (#6691)" This reverts commit 2e9b1efeb9422c60a345c67ebbbcaa5f8d37469d. Reason for revert: relative links in docs are much harder to maintain in consistent state comparing to absolute links: - It is non-trivial to figure out the proper relative link path when creating and editing docs. - Relative links break after moving the doc files to another paths, and it is non-trivial to figure which links are broken after that. - The updated relative links do not work properly right now in the docs. For example, the https://docs.victoriametrics.com/victorialogs/quickstart.md#building-from-source-code link at https://docs.victoriametrics.com/victorialogs/changelog/ leads to 404 page. This is documented at https://docs.victoriametrics.com/#images-in-documentation .
2024-07-25 10:47:35 +00:00
* FEATURE: allow disabling automatic unquoting of the matched placeholders in [`extract` pipe](https://docs.victoriametrics.com/victorialogs/logsql/#extract-pipe). See [these docs](https://docs.victoriametrics.com/victorialogs/logsql/#format-for-extract-pipe-pattern).
lib/logstorage: work-in-progress
2024-05-24 01:06:55 +00:00
Revert "removed unneeded ref shortcodes, updated VM changelog to use relative markdown links (#6691)" This reverts commit 2e9b1efeb9422c60a345c67ebbbcaa5f8d37469d. Reason for revert: relative links in docs are much harder to maintain in consistent state comparing to absolute links: - It is non-trivial to figure out the proper relative link path when creating and editing docs. - Relative links break after moving the doc files to another paths, and it is non-trivial to figure which links are broken after that. - The updated relative links do not work properly right now in the docs. For example, the https://docs.victoriametrics.com/victorialogs/quickstart.md#building-from-source-code link at https://docs.victoriametrics.com/victorialogs/changelog/ leads to 404 page. This is documented at https://docs.victoriametrics.com/#images-in-documentation .
2024-07-25 10:47:35 +00:00
* BUGFIX: properly parse `!` in front of [exact filter](https://docs.victoriametrics.com/victorialogs/logsql/#exact-filter), [exact-prefix filter](https://docs.victoriametrics.com/victorialogs/logsql/#exact-prefix-filter) and [regexp filter](https://docs.victoriametrics.com/victorialogs/logsql/#regexp-filter). For example, `!~"some regexp"` is properly parsed as `not ="some regexp"`. Previously it was incorrectly parsed as `'~="some regexp"'` [phrase filter](https://docs.victoriametrics.com/victorialogs/logsql/#phrase-filter).
* BUGFIX: properly sort results by [`_time` field](https://docs.victoriametrics.com/victorialogs/keyconcepts/#time-field) when [`limit` pipe](https://docs.victoriametrics.com/victorialogs/logsql/#limit-pipe) is applied. For example, `_time:5m | sort by (_time) desc | limit 10` properly works now.
lib/logstorage: work-in-progress
2024-05-24 01:06:55 +00:00
app/vlselect: fix loading web UI
2024-05-22 21:23:38 +00:00
## [v0.9.1](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.9.1-victorialogs)
Released at 2024-05-22
Revert "removed unneeded ref shortcodes, updated VM changelog to use relative markdown links (#6691)" This reverts commit 2e9b1efeb9422c60a345c67ebbbcaa5f8d37469d. Reason for revert: relative links in docs are much harder to maintain in consistent state comparing to absolute links: - It is non-trivial to figure out the proper relative link path when creating and editing docs. - Relative links break after moving the doc files to another paths, and it is non-trivial to figure which links are broken after that. - The updated relative links do not work properly right now in the docs. For example, the https://docs.victoriametrics.com/victorialogs/quickstart.md#building-from-source-code link at https://docs.victoriametrics.com/victorialogs/changelog/ leads to 404 page. This is documented at https://docs.victoriametrics.com/#images-in-documentation .
2024-07-25 10:47:35 +00:00
* BUGFIX: [web UI](https://docs.victoriametrics.com/victorialogs/querying/#web-ui): fix loading web UI, which has been broken in [v0.9.0](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.9.0-victorialogs).
app/vlselect: fix loading web UI
2024-05-22 21:23:38 +00:00
docs/VictoriaLogs/CHANGELOG.md: cut v0.8.0-victorialogs
2024-05-22 20:07:32 +00:00
## [v0.9.0](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.9.0-victorialogs)
Released at 2024-05-22
Revert "removed unneeded ref shortcodes, updated VM changelog to use relative markdown links (#6691)" This reverts commit 2e9b1efeb9422c60a345c67ebbbcaa5f8d37469d. Reason for revert: relative links in docs are much harder to maintain in consistent state comparing to absolute links: - It is non-trivial to figure out the proper relative link path when creating and editing docs. - Relative links break after moving the doc files to another paths, and it is non-trivial to figure which links are broken after that. - The updated relative links do not work properly right now in the docs. For example, the https://docs.victoriametrics.com/victorialogs/quickstart.md#building-from-source-code link at https://docs.victoriametrics.com/victorialogs/changelog/ leads to 404 page. This is documented at https://docs.victoriametrics.com/#images-in-documentation .
2024-07-25 10:47:35 +00:00
* FEATURE: allow using `~"some_regexp"` [regexp filter](https://docs.victoriametrics.com/victorialogs/logsql/#regexp-filter) instead of `re("some_regexp")`.
* FEATURE: allow using `="some phrase"` [exact filter](https://docs.victoriametrics.com/victorialogs/logsql/#exact-filter) instead of `exact("some phrase")`.
* FEATURE: allow using `="some prefix"*` [exact prefix filter](https://docs.victoriametrics.com/victorialogs/logsql/#exact-prefix-filter) instead of `exact("some prefix"*)`.
* FEATURE: add ability to generate output fields according to the provided format string. See [these docs](https://docs.victoriametrics.com/victorialogs/logsql/#format-pipe).
* FEATURE: add ability to extract fields with [`extract` pipe](https://docs.victoriametrics.com/victorialogs/logsql/#extract-pipe) only if the given condition is met. See [these docs](https://docs.victoriametrics.com/victorialogs/logsql/#conditional-extract).
* FEATURE: add ability to unpack JSON fields with [`unpack_json` pipe](https://docs.victoriametrics.com/victorialogs/logsql/#unpack_json-pipe) only if the given condition is met. See [these docs](https://docs.victoriametrics.com/victorialogs/logsql/#conditional-unpack_json).
* FEATURE: add ability to unpack [logfmt](https://brandur.org/logfmt) fields with [`unpack_logfmt` pipe](https://docs.victoriametrics.com/victorialogs/logsql/#unpack_logfmt-pipe) only if the given condition is met. See [these docs](https://docs.victoriametrics.com/victorialogs/logsql/#conditional-unpack_logfmt).
* FEATURE: add [`row_min`](https://docs.victoriametrics.com/victorialogs/logsql/#row_min-stats) and [`row_max`](https://docs.victoriametrics.com/victorialogs/logsql/#row_max-stats) functions for [`stats` pipe](https://docs.victoriametrics.com/victorialogs/logsql/#stats-pipe), which allow returning all the [log fields](https://docs.victoriametrics.com/victorialogs/keyconcepts/#data-model) for the log entry with the minimum / maximum value at the given field.
* FEATURE: add `/select/logsql/streams` HTTP endpoint for returning [streams](https://docs.victoriametrics.com/victorialogs/keyconcepts/#stream-fields) from results of the given query. See [these docs](https://docs.victoriametrics.com/victorialogs/querying/#querying-streams) for details.
* FEATURE: add `/select/logsql/stream_field_names` HTTP endpoint for returning [stream](https://docs.victoriametrics.com/victorialogs/keyconcepts/#stream-fields) field names from results of the given query. See [these docs](https://docs.victoriametrics.com/victorialogs/querying/#querying-stream-field-names) for details.
* FEATURE: add `/select/logsql/stream_field_values` HTTP endpoint for returning [stream](https://docs.victoriametrics.com/victorialogs/keyconcepts/#stream-fields) field values for the given label from results of the given query. See [these docs](https://docs.victoriametrics.com/victorialogs/querying/#querying-stream-field-values) for details.
* FEATURE: [web UI](https://docs.victoriametrics.com/victorialogs/querying/#web-ui): change time range limitation from `_time` in the expression to `start` and `end` query args.
* BUGFIX: fix `invalid memory address or nil pointer dereference` panic when using [`extract`](https://docs.victoriametrics.com/victorialogs/logsql/#extract-pipe), [`unpack_json`](https://docs.victoriametrics.com/victorialogs/logsql/#unpack_json-pipe) or [`unpack_logfmt`](https://docs.victoriametrics.com/victorialogs/logsql/#unpack_logfmt-pipe) pipes. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6306).
* BUGFIX: [web UI](https://docs.victoriametrics.com/victorialogs/querying/#web-ui): fix an issue where logs with long `_msg` values might not display. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6281).
[vmlogs] fixed time parsing with millisecond precision time (#6293) (#6295) fix for #6293 Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2024-05-22 19:46:50 +00:00
* BUGFIX: properly handle time range boundaries with millisecond precision. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6293).
lib/logstorage: work-in-progress
2024-05-22 19:01:20 +00:00
lib/logstorage: work-in-progress
2024-05-20 02:08:30 +00:00
## [v0.8.0](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.8.0-victorialogs)
Released at 2024-05-20
Revert "removed unneeded ref shortcodes, updated VM changelog to use relative markdown links (#6691)" This reverts commit 2e9b1efeb9422c60a345c67ebbbcaa5f8d37469d. Reason for revert: relative links in docs are much harder to maintain in consistent state comparing to absolute links: - It is non-trivial to figure out the proper relative link path when creating and editing docs. - Relative links break after moving the doc files to another paths, and it is non-trivial to figure which links are broken after that. - The updated relative links do not work properly right now in the docs. For example, the https://docs.victoriametrics.com/victorialogs/quickstart.md#building-from-source-code link at https://docs.victoriametrics.com/victorialogs/changelog/ leads to 404 page. This is documented at https://docs.victoriametrics.com/#images-in-documentation .
2024-07-25 10:47:35 +00:00
* FEATURE: add ability to extract JSON fields from [log fields](https://docs.victoriametrics.com/victorialogs/keyconcepts/#data-model). See [these docs](https://docs.victoriametrics.com/victorialogs/logsql/#unpack_json-pipe).
* FEATURE: add ability to extract [logfmt](https://brandur.org/logfmt) fields from [log fields](https://docs.victoriametrics.com/victorialogs/keyconcepts/#data-model). See [these docs](https://docs.victoriametrics.com/victorialogs/logsql/#unpack_logfmt-pipe).
* FEATURE: add ability to extract arbitrary text from [log fields](https://docs.victoriametrics.com/victorialogs/keyconcepts/#data-model) into the output fields. See [these docs](https://docs.victoriametrics.com/victorialogs/logsql/#extract-pipe).
* FEATURE: add ability to put arbitrary [queries](https://docs.victoriametrics.com/victorialogs/logsql/#query-syntax) inside [`in()` filter](https://docs.victoriametrics.com/victorialogs/logsql/#multi-exact-filter).
* FEATURE: add support for post-filtering of query results with [`filter` pipe](https://docs.victoriametrics.com/victorialogs/logsql/#filter-pipe).
* FEATURE: allow applying individual [filters](https://docs.victoriametrics.com/victorialogs/logsql/#filters) per each [stats function](https://docs.victoriametrics.com/victorialogs/logsql/#stats-pipe-functions). See [these docs](https://docs.victoriametrics.com/victorialogs/logsql/#stats-with-additional-filters).
* FEATURE: allow passing string values to [`min`](https://docs.victoriametrics.com/victorialogs/logsql/#min-stats) and [`max`](https://docs.victoriametrics.com/victorialogs/logsql/#max-stats) functions. Previously only numeric values could be passed to them.
* FEATURE: speed up [`sort ... limit N` pipe](https://docs.victoriametrics.com/victorialogs/logsql/#sort-pipe) for typical cases.
* FEATURE: allow using more convenient syntax for [`range` filters](https://docs.victoriametrics.com/victorialogs/logsql/#range-filter) if upper or lower bound isn't needed. For example, it is possible to write `response_size:>=10KiB` instead of `response_size:range[10KiB, inf)`, or `temperature:<42` instead of `temperature:range(-inf, 42)`.
* FEATURE: add `/select/logsql/hits` HTTP endpoint for returning the number of matching logs per the given time bucket over the selected time range. See [these docs](https://docs.victoriametrics.com/victorialogs/querying/#querying-hits-stats) for details.
* FEATURE: add `/select/logsql/field_names` HTTP endpoint for returning [field](https://docs.victoriametrics.com/victorialogs/keyconcepts/#data-model) names from results of the given query. See [these docs](https://docs.victoriametrics.com/victorialogs/querying/#querying-field-names) for details.
* FEATURE: add `/select/logsql/field_values` HTTP endpoint for returning unique values for the given [field](https://docs.victoriametrics.com/victorialogs/keyconcepts/#data-model) obtained from results of the given query. See [these docs](https://docs.victoriametrics.com/victorialogs/querying/#querying-field-values) for details.
* BUGFIX: properly take into account `offset` at [`sort` pipe](https://docs.victoriametrics.com/victorialogs/logsql/#sort-pipe) when it already has `limit`. For example, `_time:5m | sort by (foo) offset 20 limit 10`.
lib/logstorage: work-in-progress
2024-05-20 02:08:30 +00:00
docs/VictoriaLogs/CHANGELOG.md: cut v0.7.0-victorialogs
2024-05-15 02:58:05 +00:00
## [v0.7.0](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.7.0-victorialogs)
Released at 2024-05-15
Revert "removed unneeded ref shortcodes, updated VM changelog to use relative markdown links (#6691)" This reverts commit 2e9b1efeb9422c60a345c67ebbbcaa5f8d37469d. Reason for revert: relative links in docs are much harder to maintain in consistent state comparing to absolute links: - It is non-trivial to figure out the proper relative link path when creating and editing docs. - Relative links break after moving the doc files to another paths, and it is non-trivial to figure which links are broken after that. - The updated relative links do not work properly right now in the docs. For example, the https://docs.victoriametrics.com/victorialogs/quickstart.md#building-from-source-code link at https://docs.victoriametrics.com/victorialogs/changelog/ leads to 404 page. This is documented at https://docs.victoriametrics.com/#images-in-documentation .
2024-07-25 10:47:35 +00:00
* FEATURE: add support for optional `start` and `end` query args to [HTTP querying API](https://docs.victoriametrics.com/victorialogs/querying/#http-api), which can be used for limiting the time range for [LogsQL query](https://docs.victoriametrics.com/victorialogs/logsql/).
* FEATURE: add ability to return the first `N` results from [`sort` pipe](#https://docs.victoriametrics.com/victorialogs/logsql/#sort-pipe). This is useful when `N` biggest or `N` smallest values must be returned from large amounts of logs.
* FEATURE: add [`quantile`](https://docs.victoriametrics.com/victorialogs/logsql/#quantile-stats) and [`median`](https://docs.victoriametrics.com/victorialogs/logsql/#median-stats) [stats functions](https://docs.victoriametrics.com/victorialogs/logsql/#stats-pipe).
lib/logstorage: work-in-progress
2024-05-15 02:55:44 +00:00
docs/VictoriaLogs/CHANGELOG.md: cut v0.6.1-victorialogs
2024-05-14 01:06:53 +00:00
## [v0.6.1](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.6.1-victorialogs)
Released at 2024-05-14
Revert "removed unneeded ref shortcodes, updated VM changelog to use relative markdown links (#6691)" This reverts commit 2e9b1efeb9422c60a345c67ebbbcaa5f8d37469d. Reason for revert: relative links in docs are much harder to maintain in consistent state comparing to absolute links: - It is non-trivial to figure out the proper relative link path when creating and editing docs. - Relative links break after moving the doc files to another paths, and it is non-trivial to figure which links are broken after that. - The updated relative links do not work properly right now in the docs. For example, the https://docs.victoriametrics.com/victorialogs/quickstart.md#building-from-source-code link at https://docs.victoriametrics.com/victorialogs/changelog/ leads to 404 page. This is documented at https://docs.victoriametrics.com/#images-in-documentation .
2024-07-25 10:47:35 +00:00
* FEATURE: use [natural sort order](https://en.wikipedia.org/wiki/Natural_sort_order) when sorting logs via [`sort` pipe](https://docs.victoriametrics.com/victorialogs/logsql/#sort-pipe).
lib/logstorage: work-in-progress Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6258
2024-05-13 23:49:20 +00:00
Revert "removed unneeded ref shortcodes, updated VM changelog to use relative markdown links (#6691)" This reverts commit 2e9b1efeb9422c60a345c67ebbbcaa5f8d37469d. Reason for revert: relative links in docs are much harder to maintain in consistent state comparing to absolute links: - It is non-trivial to figure out the proper relative link path when creating and editing docs. - Relative links break after moving the doc files to another paths, and it is non-trivial to figure which links are broken after that. - The updated relative links do not work properly right now in the docs. For example, the https://docs.victoriametrics.com/victorialogs/quickstart.md#building-from-source-code link at https://docs.victoriametrics.com/victorialogs/changelog/ leads to 404 page. This is documented at https://docs.victoriametrics.com/#images-in-documentation .
2024-07-25 10:47:35 +00:00
* BUGFIX: properly return matching logs in [streams](https://docs.victoriametrics.com/victorialogs/keyconcepts/#stream-fields) with small number of entries. Previously they could be skipped. The issue has been introduced in [the release v0.6.0](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.6.0-victorialogs).
* BUGFIX: fix `runtime error: index out of range` panic when using [`sort` pipe](https://docs.victoriametrics.com/victorialogs/logsql/#sort-pipe) like `_time:1h | sort by (_time)`. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6258).
lib/logstorage: work-in-progress Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6258
2024-05-13 23:49:20 +00:00
docs/VictoriaLogs/CHANGELOG.md: cut v0.6.0-victorialogs
2024-05-12 14:38:01 +00:00
## [v0.6.0](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.6.0-victorialogs)
Released at 2024-05-12
Revert "removed unneeded ref shortcodes, updated VM changelog to use relative markdown links (#6691)" This reverts commit 2e9b1efeb9422c60a345c67ebbbcaa5f8d37469d. Reason for revert: relative links in docs are much harder to maintain in consistent state comparing to absolute links: - It is non-trivial to figure out the proper relative link path when creating and editing docs. - Relative links break after moving the doc files to another paths, and it is non-trivial to figure which links are broken after that. - The updated relative links do not work properly right now in the docs. For example, the https://docs.victoriametrics.com/victorialogs/quickstart.md#building-from-source-code link at https://docs.victoriametrics.com/victorialogs/changelog/ leads to 404 page. This is documented at https://docs.victoriametrics.com/#images-in-documentation .
2024-07-25 10:47:35 +00:00
* FEATURE: return all the log fields by default in query results. Previously only [`_stream`](https://docs.victoriametrics.com/victorialogs/keyconcepts/#stream-fields), [`_time`](https://docs.victoriametrics.com/victorialogs/keyconcepts/#time-field) and [`_msg`](https://docs.victoriametrics.com/victorialogs/keyconcepts/#message-field) fields were returned by default.
* FEATURE: add support for returning only the requested log [fields](https://docs.victoriametrics.com/victorialogs/keyconcepts/#data-model). See [these docs](https://docs.victoriametrics.com/victorialogs/logsql/#fields-pipe).
* FEATURE: add support for calculating various stats over [log fields](https://docs.victoriametrics.com/victorialogs/keyconcepts/#data-model). Grouping by arbitrary set of [log fields](https://docs.victoriametrics.com/victorialogs/keyconcepts/#data-model) is supported. See [these docs](https://docs.victoriametrics.com/victorialogs/logsql/#stats-pipe) for details.
* FEATURE: add support for sorting the returned results. See [these docs](https://docs.victoriametrics.com/victorialogs/logsql/#sort-pipe).
* FEATURE: add support for returning unique results. See [these docs](https://docs.victoriametrics.com/victorialogs/logsql/#uniq-pipe).
* FEATURE: add support for limiting the number of returned results. See [these docs](https://docs.victoriametrics.com/victorialogs/logsql/#limiters).
* FEATURE: add support for copying and renaming the selected log fields. See [these](https://docs.victoriametrics.com/victorialogs/logsql/#copy-pipe) and [these](https://docs.victoriametrics.com/victorialogs/logsql/#rename-pipe) docs.
* FEATURE: allow using `_` inside numbers. For example, `score:range[1_000, 5_000_000]` for [`range` filter](https://docs.victoriametrics.com/victorialogs/logsql/#range-filter).
* FEATURE: allow numbers in hexadecimal and binary form. For example, `response_size:range[0xff, 0b10001101101]` for [`range` filter](https://docs.victoriametrics.com/victorialogs/logsql/#range-filter).
* FEATURE: allow using duration and byte size suffixes in numeric values inside LogsQL queries. See [these docs](https://docs.victoriametrics.com/victorialogs/logsql/#numeric-values).
lib/logstorage: initial implementation of pipes in LogsQL See https://docs.victoriametrics.com/victorialogs/logsql/#pipes
2024-05-12 14:33:29 +00:00
* FEATURE: improve data ingestion performance by up to 50%.
Revert "removed unneeded ref shortcodes, updated VM changelog to use relative markdown links (#6691)" This reverts commit 2e9b1efeb9422c60a345c67ebbbcaa5f8d37469d. Reason for revert: relative links in docs are much harder to maintain in consistent state comparing to absolute links: - It is non-trivial to figure out the proper relative link path when creating and editing docs. - Relative links break after moving the doc files to another paths, and it is non-trivial to figure which links are broken after that. - The updated relative links do not work properly right now in the docs. For example, the https://docs.victoriametrics.com/victorialogs/quickstart.md#building-from-source-code link at https://docs.victoriametrics.com/victorialogs/changelog/ leads to 404 page. This is documented at https://docs.victoriametrics.com/#images-in-documentation .
2024-07-25 10:47:35 +00:00
* FEATURE: optimize performance for [LogsQL query](https://docs.victoriametrics.com/victorialogs/logsql/), which contains multiple filters for [words](https://docs.victoriametrics.com/victorialogs/logsql/#word-filter) or [phrases](https://docs.victoriametrics.com/victorialogs/logsql/#phrase-filter) delimited with [`AND` operator](https://docs.victoriametrics.com/victorialogs/logsql/#logical-filter). For example, `foo AND bar` query must find [log messages](https://docs.victoriametrics.com/victorialogs/keyconcepts/#message-field) with `foo` and `bar` words at faster speed.
lib/logstorage: initial implementation of pipes in LogsQL See https://docs.victoriametrics.com/victorialogs/logsql/#pipes
2024-05-12 14:33:29 +00:00
Revert "removed unneeded ref shortcodes, updated VM changelog to use relative markdown links (#6691)" This reverts commit 2e9b1efeb9422c60a345c67ebbbcaa5f8d37469d. Reason for revert: relative links in docs are much harder to maintain in consistent state comparing to absolute links: - It is non-trivial to figure out the proper relative link path when creating and editing docs. - Relative links break after moving the doc files to another paths, and it is non-trivial to figure which links are broken after that. - The updated relative links do not work properly right now in the docs. For example, the https://docs.victoriametrics.com/victorialogs/quickstart.md#building-from-source-code link at https://docs.victoriametrics.com/victorialogs/changelog/ leads to 404 page. This is documented at https://docs.victoriametrics.com/#images-in-documentation .
2024-07-25 10:47:35 +00:00
* BUGFIX: prevent from possible corruption of short [log fields](https://docs.victoriametrics.com/victorialogs/keyconcepts/#data-model) during data ingestion.
lib/logstorage: initial implementation of pipes in LogsQL See https://docs.victoriametrics.com/victorialogs/logsql/#pipes
2024-05-12 14:33:29 +00:00
* BUGFIX: prevent from additional CPU usage for up to a few seconds after canceling the query.
Revert "removed unneeded ref shortcodes, updated VM changelog to use relative markdown links (#6691)" This reverts commit 2e9b1efeb9422c60a345c67ebbbcaa5f8d37469d. Reason for revert: relative links in docs are much harder to maintain in consistent state comparing to absolute links: - It is non-trivial to figure out the proper relative link path when creating and editing docs. - Relative links break after moving the doc files to another paths, and it is non-trivial to figure which links are broken after that. - The updated relative links do not work properly right now in the docs. For example, the https://docs.victoriametrics.com/victorialogs/quickstart.md#building-from-source-code link at https://docs.victoriametrics.com/victorialogs/changelog/ leads to 404 page. This is documented at https://docs.victoriametrics.com/#images-in-documentation .
2024-07-25 10:47:35 +00:00
* BUGFIX: prevent from returning log entries with emtpy `_stream` field in the form `"_stream":""` in [search query results](https://docs.victoriametrics.com/victorialogs/querying/). See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6042).
lib/logstorage: initial implementation of pipes in LogsQL See https://docs.victoriametrics.com/victorialogs/logsql/#pipes
2024-05-12 14:33:29 +00:00
docs/VictoriaLogs/CHANGELOG.md: cut v0.5.2-victorialogs release
2024-04-11 07:42:06 +00:00
## [v0.5.2](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.5.2-victorialogs)
Released at 2024-04-11
Revert "removed unneeded ref shortcodes, updated VM changelog to use relative markdown links (#6691)" This reverts commit 2e9b1efeb9422c60a345c67ebbbcaa5f8d37469d. Reason for revert: relative links in docs are much harder to maintain in consistent state comparing to absolute links: - It is non-trivial to figure out the proper relative link path when creating and editing docs. - Relative links break after moving the doc files to another paths, and it is non-trivial to figure which links are broken after that. - The updated relative links do not work properly right now in the docs. For example, the https://docs.victoriametrics.com/victorialogs/quickstart.md#building-from-source-code link at https://docs.victoriametrics.com/victorialogs/changelog/ leads to 404 page. This is documented at https://docs.victoriametrics.com/#images-in-documentation .
2024-07-25 10:47:35 +00:00
* BUGFIX: properly register new [log streams](https://docs.victoriametrics.com/victorialogs/keyconcepts/#stream-fields) under high data ingestion rate. The issue has been introduced in [v0.5.0](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.5.0-victorialogs).
docs/VictoriaLogs/CHANGELOG.md: cut v0.5.1-victorialogs release
2024-04-04 13:41:33 +00:00
## [v0.5.1](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.5.1-victorialogs)
Released at 2024-04-04
Revert "removed unneeded ref shortcodes, updated VM changelog to use relative markdown links (#6691)" This reverts commit 2e9b1efeb9422c60a345c67ebbbcaa5f8d37469d. Reason for revert: relative links in docs are much harder to maintain in consistent state comparing to absolute links: - It is non-trivial to figure out the proper relative link path when creating and editing docs. - Relative links break after moving the doc files to another paths, and it is non-trivial to figure which links are broken after that. - The updated relative links do not work properly right now in the docs. For example, the https://docs.victoriametrics.com/victorialogs/quickstart.md#building-from-source-code link at https://docs.victoriametrics.com/victorialogs/changelog/ leads to 404 page. This is documented at https://docs.victoriametrics.com/#images-in-documentation .
2024-07-25 10:47:35 +00:00
* BUGFIX: properly apply time range filter for queries containing [`OR` operators](https://docs.victoriametrics.com/victorialogs/logsql/#logical-filter). See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5920).
docs/VictoriaLogs/CHANGELOG.md: improve the description of the bugfix from 43b5d8bc7ab0b9d3d0b0c4b99b6a8364871fb965, so it can be googled by users
2024-03-30 04:52:18 +00:00
* BUGFIX: do not log debug lines `DEBUG: start trimLines` and `DEBUG: end trimLines`. This bug has been introduced in [v0.5.0](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.5.0-victorialogs) in [this commit](https://github.com/VictoriaMetrics/VictoriaMetrics/commit/0514091948cf8e00e42f44318c0e5e5b63b6388f).
docs/VictoriaLogs/CHANGELOG.md: cut v0.5.0-victorialogs
2024-02-29 23:35:04 +00:00
## [v0.5.0](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.5.0-victorialogs)
Released at 2024-03-01
Revert "removed unneeded ref shortcodes, updated VM changelog to use relative markdown links (#6691)" This reverts commit 2e9b1efeb9422c60a345c67ebbbcaa5f8d37469d. Reason for revert: relative links in docs are much harder to maintain in consistent state comparing to absolute links: - It is non-trivial to figure out the proper relative link path when creating and editing docs. - Relative links break after moving the doc files to another paths, and it is non-trivial to figure which links are broken after that. - The updated relative links do not work properly right now in the docs. For example, the https://docs.victoriametrics.com/victorialogs/quickstart.md#building-from-source-code link at https://docs.victoriametrics.com/victorialogs/changelog/ leads to 404 page. This is documented at https://docs.victoriametrics.com/#images-in-documentation .
2024-07-25 10:47:35 +00:00
* FEATURE: support the ability to limit the number of returned log entries from [HTTP querying API](https://docs.victoriametrics.com/victorialogs/querying/#http-api) by passing `limit` query arg. Previously all the matching log entries were returned until closing the response stream. See [this feature request](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5674). Thanks to @dmitryk-dk for [the pull request](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5778).
app/vlselect: follow-up for 451d2abf50171dbdea844b40f14e0205a173d37a - Consistently return the first `limit` log entries if the total size of found log entries doesn't exceed 1Mb. See app/vlselect/logsql/sort_writer.go . Previously random log entries could be returned with each request. - Document the change at docs/VictoriaLogs/CHANGELOG.md - Document the `limit` query arg at docs/VictoriaLogs/querying/README.md - Make the change less intrusive. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5674 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5778
2024-02-18 21:01:34 +00:00
Revert "removed unneeded ref shortcodes, updated VM changelog to use relative markdown links (#6691)" This reverts commit 2e9b1efeb9422c60a345c67ebbbcaa5f8d37469d. Reason for revert: relative links in docs are much harder to maintain in consistent state comparing to absolute links: - It is non-trivial to figure out the proper relative link path when creating and editing docs. - Relative links break after moving the doc files to another paths, and it is non-trivial to figure which links are broken after that. - The updated relative links do not work properly right now in the docs. For example, the https://docs.victoriametrics.com/victorialogs/quickstart.md#building-from-source-code link at https://docs.victoriametrics.com/victorialogs/changelog/ leads to 404 page. This is documented at https://docs.victoriametrics.com/#images-in-documentation .
2024-07-25 10:47:35 +00:00
* BUGFIX: do not panic on incorrect regular expression in [stream filter](https://docs.victoriametrics.com/victorialogs/logsql/#stream-filter). Thanks to @XLONG96 for [the bugfix](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5897).
docs/VictoriaLogs/CHANGELOG.md: document 333bda87026740f12c03d70228859565a3b550ff Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5447
2024-02-18 21:13:13 +00:00
* BUGFIX: properly determine when the assisted merge is needed. Previously the logs for determining whether the assisted merge is needed was broken. This could lead to too big number of parts under high data ingestion rate. Thanks to @lujiajing1126 for [the fix](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5447).
Revert "removed unneeded ref shortcodes, updated VM changelog to use relative markdown links (#6691)" This reverts commit 2e9b1efeb9422c60a345c67ebbbcaa5f8d37469d. Reason for revert: relative links in docs are much harder to maintain in consistent state comparing to absolute links: - It is non-trivial to figure out the proper relative link path when creating and editing docs. - Relative links break after moving the doc files to another paths, and it is non-trivial to figure which links are broken after that. - The updated relative links do not work properly right now in the docs. For example, the https://docs.victoriametrics.com/victorialogs/quickstart.md#building-from-source-code link at https://docs.victoriametrics.com/victorialogs/changelog/ leads to 404 page. This is documented at https://docs.victoriametrics.com/#images-in-documentation .
2024-07-25 10:47:35 +00:00
* BUGFIX: properly stop execution of aborted query when the query doesn't contain [`_stream` filter](https://docs.victoriametrics.com/victorialogs/logsql/#stream-filter). Previously such a query could continue consuming resources after being aborted by the client. Thanks to @z-anshun for [the fix](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5400).
docs/VictoriaLogs/CHANGELOG.md: document 333bda87026740f12c03d70228859565a3b550ff Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5447
2024-02-18 21:13:13 +00:00
docs/VictoriaLogs/README.md: cut v0.4.2-victorialogs
2023-11-15 19:43:57 +00:00
## [v0.4.2](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.4.2-victorialogs)
Released at 2023-11-15
Revert "removed unneeded ref shortcodes, updated VM changelog to use relative markdown links (#6691)" This reverts commit 2e9b1efeb9422c60a345c67ebbbcaa5f8d37469d. Reason for revert: relative links in docs are much harder to maintain in consistent state comparing to absolute links: - It is non-trivial to figure out the proper relative link path when creating and editing docs. - Relative links break after moving the doc files to another paths, and it is non-trivial to figure which links are broken after that. - The updated relative links do not work properly right now in the docs. For example, the https://docs.victoriametrics.com/victorialogs/quickstart.md#building-from-source-code link at https://docs.victoriametrics.com/victorialogs/changelog/ leads to 404 page. This is documented at https://docs.victoriametrics.com/#images-in-documentation .
2024-07-25 10:47:35 +00:00
* BUGFIX: properly locate logs for the [requested streams](https://docs.victoriametrics.com/victorialogs/logsql/#stream-filter). Previously logs for some streams may be missing in query results. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4856). Thanks to @XLONG96 for [the fix](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5295)!
* BUGFIX: [web UI](https://docs.victoriametrics.com/victorialogs/querying/#web-ui): properly sort found logs by time. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5300).
docs/VictoriaLogs/CHANGELOG.md: follow-up for 66527c598110472964171a75a539d9ec6b2a883d Document the change Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5312 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5300
2023-11-13 09:39:18 +00:00
cut v0.4.1-victorialogs release
2023-10-04 07:56:43 +00:00
## [v0.4.1](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.4.1-victorialogs)
docs/VictoriaLogs/CHANGELOG.md: add release date for v0.4.1-victorialogs
2023-10-04 15:56:37 +00:00
Released at 2023-10-04
docs/victorialogs: changelog followup for 650d72f3 (#5114) Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2023-10-03 12:26:03 +00:00
* BUGFIX: fix the free space verification process in VictoriaLogs that was erroneously shifting to read-only mode, despite there being sufficient free space available. See [this](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5112) issue.
docs/VictoriaLogs/CHANGELOG.md: cut v0.4.0-victorialogs
2023-10-02 23:46:33 +00:00
## [v0.4.0](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.4.0-victorialogs)
Released at 2023-10-03
Revert "removed unneeded ref shortcodes, updated VM changelog to use relative markdown links (#6691)" This reverts commit 2e9b1efeb9422c60a345c67ebbbcaa5f8d37469d. Reason for revert: relative links in docs are much harder to maintain in consistent state comparing to absolute links: - It is non-trivial to figure out the proper relative link path when creating and editing docs. - Relative links break after moving the doc files to another paths, and it is non-trivial to figure which links are broken after that. - The updated relative links do not work properly right now in the docs. For example, the https://docs.victoriametrics.com/victorialogs/quickstart.md#building-from-source-code link at https://docs.victoriametrics.com/victorialogs/changelog/ leads to 404 page. This is documented at https://docs.victoriametrics.com/#images-in-documentation .
2024-07-25 10:47:35 +00:00
* FEATURE: add `-elasticsearch.version` command-line flag, which can be used for specifying Elasticsearch version returned by VictoriaLogs to Filebeat at [elasticsearch bulk API](https://docs.victoriametrics.com/victorialogs/data-ingestion/#elasticsearch-bulk-api). This helps resolving [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4777).
* FEATURE: expose the following metrics at [/metrics](https://docs.victoriametrics.com/victorialogs/#monitoring) page:
* `vl_data_size_bytes{type="storage"}` - on-disk size for data excluding [log stream](https://docs.victoriametrics.com/victorialogs/keyconcepts/#stream-fields) indexes.
* `vl_data_size_bytes{type="indexdb"}` - on-disk size for [log stream](https://docs.victoriametrics.com/victorialogs/keyconcepts/#stream-fields) indexes.
app/vlinsert: add flag to limit amount of fields per line (#4976) Adding limit on ingestion allows to avoid issues like this one https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4762 Such issues are often caused by misconfigurtion on log persing/ingestion side and preventing such rows from being ingested allows to avoid performance implications created by storing such log rows. Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2023-09-15 13:18:38 +00:00
* FEATURE: add `-insert.maxFieldsPerLine` command-line flag, which can be used for limiting the number of fields per line in logs sent to VictoriaLogs via ingestion protocols. This helps to avoid issues like [this](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4762).
Revert "removed unneeded ref shortcodes, updated VM changelog to use relative markdown links (#6691)" This reverts commit 2e9b1efeb9422c60a345c67ebbbcaa5f8d37469d. Reason for revert: relative links in docs are much harder to maintain in consistent state comparing to absolute links: - It is non-trivial to figure out the proper relative link path when creating and editing docs. - Relative links break after moving the doc files to another paths, and it is non-trivial to figure which links are broken after that. - The updated relative links do not work properly right now in the docs. For example, the https://docs.victoriametrics.com/victorialogs/quickstart.md#building-from-source-code link at https://docs.victoriametrics.com/victorialogs/changelog/ leads to 404 page. This is documented at https://docs.victoriametrics.com/#images-in-documentation .
2024-07-25 10:47:35 +00:00
* FEATURE: expose `vl_http_request_duration_seconds` histogram at the [/metrics](https://docs.victoriametrics.com/victorialogs/#monitoring) page. Thanks to @crossoverJie for [this pull request](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/4934).
docs/VictoriaLogs/CHANGELOG.md: remove duplicate lines about vl_http_request_duration_seconds metric This is a follow-up after 8a23d08c210c7c2440c224debcff266de3353a64 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/4945
2023-10-01 21:31:05 +00:00
* FEATURE: add support of `-storage.minFreeDiskSpaceBytes` command-line flag to allow switching to read-only mode when running out of disk space at `-storageDataPath`. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4737).
app/vlstorage: expose vl_data_size_bytes metric at /metrics page for tracking the on-disk data size (both indexdb and the data itself)
2023-07-31 14:45:52 +00:00
docs/VictoriaLogs/CHANGELOG.md: document the fix 8d50032dd6a18b598c18da15d8d2e53a704ef8e9 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4895
2023-08-29 09:28:42 +00:00
* BUGFIX: fix possible panic when no data is written to VictoriaLogs for a long time. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4895). Thanks to @crossoverJie for filing and fixing the issue.
docs: fix docs/ and README.md spelling errors (#6362) Fixes `docs/` and `README.md` typos and errors. Signed-off-by: Arkadii Yakovets <ark@victoriametrics.com>
2024-06-03 08:04:13 +00:00
* BUGFIX: add `/insert/loki/ready` endpoint, which is used by Promtail for healthchecks. This should remove `unsupported path requested: /insert/loki/ready` warning logs. See [this comment](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4762#issuecomment-1690966722).
lib/logstorage: follow-up for 94627113dbe0b7bed23a7dc4864fc2f2903a819c - Move uniqueFields from rows to blockStreamMerger struct. This allows localizing all the references to uniqueFields inside blockStreamMerger.mustWriteBlock(), which should improve readability and maintainability of the code. - Remove logging of the event when blocks cannot be merged because they contain more than maxColumnsPerBlock, since the provided logging didn't provide the solution for the issue with too many columns. I couldn't figure out the proper solution, which could be helpful for end user, so decided to remove the logging until we find the solution. This commit also contains the following additional changes: - It truncates field names longer than 128 chars during logs ingestion. This should prevent from ingesting bogus field names. This also should prevent from too big columnsHeader blocks, which could negatively affect search query performance, since columnsHeader is read on every scan of the corresponding data block. - It limits the maximum length of const column value to 256. Longer values are stored in an ordinary columns. This helps limiting the size of columnsHeader blocks and improving search query performance by avoiding reading too long const columns on every scan of the corresponding data block. - It deduplicates columns with identical names during data ingestion and background merging. Previously it was possible to pass columns with duplicate names to block.mustInitFromRows(), and they were stored as is in the block. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4762 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/4969
2023-10-02 17:01:17 +00:00
* BUGFIX: prevent from panic during background merge when the number of columns in the resulting block exceeds the maximum allowed number of columns per block. See [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4762).
docs/VictoriaLogs/CHANGELOG.md: document the fix 8d50032dd6a18b598c18da15d8d2e53a704ef8e9 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4895
2023-08-29 09:28:42 +00:00
docs/VictoriaLogs/CHANGELOG.md: cut v0.3.0-victorialogs
2023-07-21 03:48:30 +00:00
## [v0.3.0](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.3.0-victorialogs)
Released at 2023-07-20
Revert "removed unneeded ref shortcodes, updated VM changelog to use relative markdown links (#6691)" This reverts commit 2e9b1efeb9422c60a345c67ebbbcaa5f8d37469d. Reason for revert: relative links in docs are much harder to maintain in consistent state comparing to absolute links: - It is non-trivial to figure out the proper relative link path when creating and editing docs. - Relative links break after moving the doc files to another paths, and it is non-trivial to figure which links are broken after that. - The updated relative links do not work properly right now in the docs. For example, the https://docs.victoriametrics.com/victorialogs/quickstart.md#building-from-source-code link at https://docs.victoriametrics.com/victorialogs/changelog/ leads to 404 page. This is documented at https://docs.victoriametrics.com/#images-in-documentation .
2024-07-25 10:47:35 +00:00
* FEATURE: add support for data ingestion via Promtail (aka default log shipper for Grafana Loki). See [these](https://docs.victoriametrics.com/victorialogs/data-ingestion/promtail/) and [these](https://docs.victoriametrics.com/victorialogs/data-ingestion/#loki-json-api) docs.
docs/VictoriaLogs/CHANGELOG.md: document the addition of Grafana Loki push protocol support (aka Promtail) Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/4482
2023-07-20 23:52:59 +00:00
docs/VictoriaLogs: add CHANGELOG.md
2023-07-18 06:14:05 +00:00
## [v0.2.0](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.2.0-victorialogs)
Released at 2023-07-17
Revert "removed unneeded ref shortcodes, updated VM changelog to use relative markdown links (#6691)" This reverts commit 2e9b1efeb9422c60a345c67ebbbcaa5f8d37469d. Reason for revert: relative links in docs are much harder to maintain in consistent state comparing to absolute links: - It is non-trivial to figure out the proper relative link path when creating and editing docs. - Relative links break after moving the doc files to another paths, and it is non-trivial to figure which links are broken after that. - The updated relative links do not work properly right now in the docs. For example, the https://docs.victoriametrics.com/victorialogs/quickstart.md#building-from-source-code link at https://docs.victoriametrics.com/victorialogs/changelog/ leads to 404 page. This is documented at https://docs.victoriametrics.com/#images-in-documentation .
2024-07-25 10:47:35 +00:00
* FEATURE: support short form of `_time` filters over the last X minutes/hours/days/etc. For example, `_time:5m` is a short form for `_time:(now-5m, now]`, which matches logs with [timestamps](https://docs.victoriametrics.com/victorialogs/keyconcepts/#time-field) for the last 5 minutes. See [these docs](https://docs.victoriametrics.com/victorialogs/logsql/#time-filter) for details.
* FEATURE: add ability to specify offset for the selected time range. For example, `_time:5m offset 1h` is equivalent to `_time:(now-5m-1h, now-1h]`. See [these docs](https://docs.victoriametrics.com/victorialogs/logsql/#time-filter) for details.
* FEATURE: [LogsQL](https://docs.victoriametrics.com/victorialogs/logsql/): replace `exact_prefix("...")` with `exact("..."*)`. This makes it consistent with [i()](https://docs.victoriametrics.com/victorialogs/logsql/#case-insensitive-filter) filter, which can accept phrases and prefixes, e.g. `i("phrase")` and `i("phrase"*)`. See [these docs](https://docs.victoriametrics.com/victorialogs/logsql/#exact-prefix-filter).
docs/VictoriaLogs: add CHANGELOG.md
2023-07-18 06:14:05 +00:00
## [v0.1.0](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.1.0-victorialogs)
Released at 2023-06-21
Initial release
Reference in a new issue Copy permalink