VictoriaMetrics/app/vlinsert/loki
Aliaksandr Valialkin 7b33a27874
lib/logstorage: follow-up for 8a23d08c21
- Compare the actual free disk space to the value provided via -storage.minFreeDiskSpaceBytes
  directly inside the Storage.IsReadOnly(). This should work fast in most cases.
  This simplifies the logic at lib/storage.

- Do not take into account -storage.minFreeDiskSpaceBytes during background merges, since
  it results in uncontrolled growth of small parts when the free disk space approaches -storage.minFreeDiskSpaceBytes.
  The background merge logic uses another mechanism for determining whether there is enough
  disk space for the merge - it reserves the needed disk space before the merge
  and releases it after the merge. This prevents from out of disk space errors during background merge.

- Properly handle corner cases for flushing in-memory data to disk when the storage
  enters read-only mode. This is better than losing the in-memory data.

- Return back Storage.MustAddRows() instead of Storage.AddRows(),
  since the only case when AddRows() can return error is when the storage is in read-only mode.
  This case must be handled by the caller by calling Storage.IsReadOnly()
  before adding rows to the storage.
  This simplifies the code a bit, since the caller of Storage.MustAddRows() shouldn't handle
  errors returned by Storage.AddRows().

- Properly store parsed logs to Storage if parts of the request contain invalid log lines.
  Previously the parsed logs could be lost in this case.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4737
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/4945
2023-10-02 16:52:23 +02:00
..
loki.go app/vlinsert: follow-up for d570763c91 2023-09-19 00:02:43 +02:00
loki_json.go lib/logstorage: follow-up for 8a23d08c21 2023-10-02 16:52:23 +02:00
loki_json_test.go lib/logstorage: follow-up for 8a23d08c21 2023-10-02 16:52:23 +02:00
loki_json_timing_test.go lib/logstorage: follow-up for 8a23d08c21 2023-10-02 16:52:23 +02:00
loki_protobuf.go lib/logstorage: follow-up for 8a23d08c21 2023-10-02 16:52:23 +02:00
loki_protobuf_test.go lib/logstorage: follow-up for 8a23d08c21 2023-10-02 16:52:23 +02:00
loki_protobuf_timing_test.go lib/logstorage: follow-up for 8a23d08c21 2023-10-02 16:52:23 +02:00
push_request.pb.go app/vlinsert/loki: manually remove bloat dependecies for generate proto file (#4686) 2023-07-22 15:36:00 -07:00
push_request.proto app/vlinsert: add support of loki push protocol (#4482) 2023-07-20 10:10:55 +02:00
timestamp.go app/vlinsert: add support of loki push protocol (#4482) 2023-07-20 10:10:55 +02:00
types.go app/vlinsert: add support of loki push protocol (#4482) 2023-07-20 10:10:55 +02:00