31 commits

Author	SHA1	Message	Date
Aliaksandr Valialkin	42eefebb06	app/vlselect/logsql: add missing return from ProcessLiveTailRequest() when the query cannot be live tailed ... (cherry picked from commit fe022ed795)	2024-10-11 14:27:47 +02:00
Aliaksandr Valialkin	3a556bd15a	app/vlselect/logsql: clone the query with the current timestamp when performing live tailing requests in the loop ... Previously the original timestamp was used in the copied query, so _time:duration filters were applied to the original time range: (timestamp-duration ... timestamp]. This resulted in stopped live tailing, since new logs have timestamps bigger than the original time range. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7028	2024-09-26 08:57:48 +02:00
Aliaksandr Valialkin	7e0ff1ee46	app/vlselect/logsql: call Query.Optimize() on the cloned query in order to replace * filter with filterNoop inside getLastNQueryResults() ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6785 (cherry picked from commit e86891b010)	2024-09-19 15:48:07 +02:00
Aliaksandr Valialkin	cad236003b	app/vlselect: consistently reuse the original query timestamp when executing /select/logsql/query with positive limit=N query arg ... Previously the query could return incorrect results, since the query timestamp was updated with every Query.Clone() call during iterative search for the time range with up to limit=N rows. While at it, optimize queries, which find low number of matching logs, while spend a lot of CPU time for searching across big number of logs. The optimization reduces the upper bound of the time range to search if the current time range contains zero matching rows. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6785	2024-09-08 14:34:46 +02:00
Aliaksandr Valialkin	4b49b62a58	lib/logstorage: improve error logging for incorrect queries passed to /select/logsql/stats_query and /select/logsql/stats_query_range functions	2024-09-08 12:28:33 +02:00
Aliaksandr Valialkin	c448189f69	app/vlselect: add /select/logsql/stats_query_range endpoint for building time series panels in VictoriaLogs plugin for Grafana ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6943 Updates https://github.com/VictoriaMetrics/victorialogs-datasource/issues/61	2024-09-07 00:44:34 +02:00
Aliaksandr Valialkin	01c8e12370	app/vlselect: add /select/logsql/stats_query endpoint, which is going to be used by vmalert ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6942 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6706	2024-09-06 23:00:58 +02:00
Aliaksandr Valialkin	b8a8d3d6f1	lib/logstorage: drop all the pipes from the query when calculating the number of matching logs at /select/logsql/hits API	2024-07-10 00:39:16 +02:00
Aliaksandr Valialkin	4b3477e62b	lib/logstorage: add stream_context pipe, which allows selecting surrounding logs for the matching logs	2024-06-28 19:15:19 +02:00
Aliaksandr Valialkin	bb6424aeca	app/vlselect/logsql: add optional fields_limit query arg to /select/logsql/hits HTTP endpoint ... This query arg is needed for https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6545 in order to return top N groups with the biggest number of hits.	2024-06-28 03:10:05 +02:00
Aliaksandr Valialkin	b26acec9a8	app/vlselect: properly return live tailing results	2024-06-27 15:06:15 +02:00
Aliaksandr Valialkin	dd62a2b9d6	lib/logstorage: work-in-progress	2024-06-27 14:21:03 +02:00
Aliaksandr Valialkin	9135b404d9	lib/logstorage: work-in-progress	2024-06-11 17:51:01 +02:00
Aliaksandr Valialkin	b45e466a1b	lib/logstorage: work-in-progress	2024-06-05 03:18:25 +02:00
Aliaksandr Valialkin	b7b3a9e9a3	lib/logstorage: work-in-progress	2024-06-04 01:50:55 +02:00
Aliaksandr Valialkin	03fe4c8963	lib/logstorage: work-in-progress	2024-05-25 21:36:24 +02:00
Aliaksandr Valialkin	7a2a2f173e	lib/logstorage: work-in-progress	2024-05-24 03:07:07 +02:00
Aliaksandr Valialkin	04d0dd2542	lib/logstorage: work-in-progress	2024-05-22 21:01:28 +02:00
Aliaksandr Valialkin	582e7d5439	lib/logstorage: work-in-progress	2024-05-20 04:09:15 +02:00
Aliaksandr Valialkin	28626db066	lib/logstorage: work-in-progress ... (cherry picked from commit 0aa19a2837)	2024-05-16 09:35:55 +02:00
Aliaksandr Valialkin	b1ee7bca1a	lib/logstorage: work-in-progress	2024-05-14 03:06:02 +02:00
Aliaksandr Valialkin	147704aab0	lib/logstorage: initial implementation of pipes in LogsQL ... See https://docs.victoriametrics.com/victorialogs/logsql/#pipes	2024-05-12 16:36:01 +02:00
Nikolay	60d64edd6e	app/vlselect: follow-up for 0514091948 (#6004) ... removes println lines (cherry picked from commit 43b5d8bc7a)	2024-03-25 15:42:53 +01:00
Aliaksandr Valialkin	b58c429044	app/vlselect: follow-up for 451d2abf50 ... - Consistently return the first `limit` log entries if the total size of found log entries doesn't exceed 1Mb. See app/vlselect/logsql/sort_writer.go . Previously random log entries could be returned with each request. - Document the change at docs/VictoriaLogs/CHANGELOG.md - Document the `limit` query arg at docs/VictoriaLogs/querying/README.md - Make the change less intrusive. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5674 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5778	2024-02-18 23:06:08 +02:00
Dmytro Kozlov	2d674f98d4	Enable the limit query param for the /select/logsql/query (#5778) ... * app/vlselect: add limit for logs query * app/vlselect: CHANGELOG.md * app/vlselect: stop search process if limit is reached, update logic, remove default limit * app/vlselect: fix tests * app/vlselect: fix filter tests * app/vlselect: fix tests	2024-02-18 22:59:16 +02:00
Roman Khavronenko	80768d53dd	docs: follow-up after aec4b5db81 (#4638) ... Signed-off-by: hagen1778 <roman@victoriametrics.com>	2023-07-19 14:48:17 -07:00
Aliaksandr Valialkin	352429486a	Revert "app/vlselect/logsql: use buffered writer in order to save syscalls when sending big amounts of data to clients" ... This reverts commit c19048dc13. Reason for revert: it has been appeared that the net/http.ResponseWriter is already buffered, so there in no need in double bufferring	2023-07-06 21:37:38 -07:00
Aliaksandr Valialkin	19870d42c5	app/vlselect/logsql: use buffered writer in order to save syscalls when sending big amounts of data to clients	2023-07-06 21:37:38 -07:00
Aliaksandr Valialkin	772852ff4f	app/vlselect/logsql: properly handle the error from ParseLogMessage	2023-07-06 21:33:22 -07:00
Aliaksandr Valialkin	efee71986f	app/vlselect/logsql: sort query results by _time if their summary size doesnt exceed -select.maxSortBufferSize	2023-07-06 21:25:00 -07:00
Aliaksandr Valialkin	374890294e	app/victoria-logs: initial code release	2023-07-06 17:30:05 -07:00