Commit graph

41 commits

Author SHA1 Message Date
Aliaksandr Valialkin
2b6a634ec0
lib/logstorage: work-in-progress 2024-06-17 12:13:18 +02:00
Aliaksandr Valialkin
8f5dc966f6
lib/logstorage: work-in-progress 2024-06-11 17:50:32 +02:00
Aliaksandr Valialkin
55d8379ae6
lib/logstorage: work-in-progress 2024-06-06 12:27:05 +02:00
Aliaksandr Valialkin
43cf221681
lib/logstorage: work-in-progress 2024-06-05 03:18:12 +02:00
Aliaksandr Valialkin
539fce9227
lib/logstorage: work-in-progress 2024-06-04 01:49:02 +02:00
Aliaksandr Valialkin
ceb2397381
deployment/docker: update VictoriaLogs Docker image from v0.14.0-victorialogs to v0.15.0-victorialogs
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.15.0-victorialogs
2024-05-30 16:27:26 +02:00
Aliaksandr Valialkin
1de187bcb7
lib/logstorage: work-in-progress 2024-05-29 01:52:13 +02:00
Aliaksandr Valialkin
89bd0dca0a
deployment/docker: update VictoriaLogs image from v0.12.1-victorialogs to v0.13.0-victorialogs
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.13.0-victorialogs
2024-05-28 22:14:34 +02:00
Aliaksandr Valialkin
31e23c6f6f
deployment: update VictoriaLogs image from v0.12.0-victorialogs to v0.12.1-victorialogs
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.12.1-victorialogs
2024-05-26 02:08:24 +02:00
Aliaksandr Valialkin
1882957585
docs/VictoriaLogs/CHANGELOG.md: cut v0.12.0-victorialogs 2024-05-25 21:40:10 +02:00
Aliaksandr Valialkin
e2590f0485
lib/logstorage: work-in-progress 2024-05-25 00:30:58 +02:00
Aliaksandr Valialkin
0915d6d841
deployment: update Docker image tag for VictoriaLogs from v0.8.0-victorialogs to v0.10.0-victorialogs
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.10.0-victorialogs
2024-05-24 03:15:43 +02:00
Aliaksandr Valialkin
6ad1eb9a3d
deployment: update VictoriaLogs from v0.6.0-victorialogs to v0.8.0-victorialogs
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.8.0-victorialogs
2024-05-20 04:17:37 +02:00
Aliaksandr Valialkin
de98688489
deployment: update VictoriaLogs Docker image from v0.5.2-victorialogs to v0.6.0-victorialogs
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.6.0-victorialogs
2024-05-12 23:22:50 +02:00
Denys Holius
bde4693a90
deployment/docker/victorialogs/fluentbit-docker/docker-compose.yml: update fluentbit version from v2.1.4 to v3.0.2 (#6120)
see also https://fluentbit.io/announcements/v3.0.0/
2024-04-23 01:07:45 -07:00
hagen1778
9064602d00
deployment/vector: add example for JSON stream config
Follow-up 16eeb4eb33

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-04-16 13:57:54 +02:00
Aliaksandr Valialkin
9ee51e34cc
deployment: upgrade VictoriaLogs from v0.5.1-victorialogs to v0.5.2-victorialogs
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.5.2-victorialogs
2024-04-11 09:53:24 +02:00
Aliaksandr Valialkin
7bf090525d
deployment/docker: update VictoriaLogs tag from v0.5.0-victorialogs to v0.5.1-victorialogs
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.5.1-victorialogs
2024-04-04 16:52:30 +03:00
Aliaksandr Valialkin
f45f39d80e
Revert "deployment/docs: use lower-case links to VictoriaLogs docs"
This reverts commit a0937b01c1.

Reason for revert: MixedCase links started working again.
See, for example, https://docs.victoriametrics.com/VictoriaLogs/querying/#vmui
2024-03-17 23:13:23 +02:00
hagen1778
a0937b01c1
deployment/docs: use lower-case links to VictoriaLogs docs
Links with upper-case simply don't work for unknown reason.
Once the reason is fixed on docs side, this commit can be reverted.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-03-13 11:28:06 +01:00
Aliaksandr Valialkin
a7aa119f35
deployment: update VictoriaLogs docker image from v0.4.2-victorialogs to v0.5.0-victorialogs
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.5.0-victorialogs
2024-03-01 04:18:40 +02:00
Roman Khavronenko
f0b4dd7426
deployment: create a separate env for VictoriaLogs (#5857)
* deployment: create a separate env for VictoriaLogs

The new environment consists of the following components:
* VictoriaLogs
* fluentbit for collecting logs and sending to VictoriaLogs
* VictoriaMetrics for scraping and storing metrics from fluentbit and VictoriaLogs
* Grafana with VictoriaLogs datasource for monitoring

-----------------

The motivation for creating a separate environment is to simplify existing environments
and make it easier to update or modify them in future.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-02-26 10:33:04 +01:00
Aliaksandr Valialkin
4c25ee3597
deployment: update reference to VictoriaLogs from v0.4.1-victorialogs to v0.4.2-victorialogs 2023-11-15 20:46:53 +01:00
Aliaksandr Valialkin
61935e3387
deployment/docker: update VictoriaLogs from v0.4.0-victorialogs to v0.4.1-victorialogs
See https://docs.victoriametrics.com/VictoriaLogs/CHANGELOG.html#v041
2023-10-04 14:43:42 +02:00
Aliaksandr Valialkin
3d99d4b6c0
deployment: update VictoriaLogs from v0.3.0-victorialogs to v0.4.0-victorialogs
See https://docs.victoriametrics.com/VictoriaLogs/CHANGELOG.html#v040
2023-10-03 02:33:47 +02:00
Zakhar Bessarab
4e5a68ed08
deployment/docker: add VictoriaLogs (#4929)
* deployment/docker: add VictoriaLogs configuration

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* deployment/docker/victorialogs: remove outdated comment

It was added in order to indicate that it is required to build VictoriaLogs manually before starting it at the time there was no public release available.
Currently, there is a public tag and it is not required to build it from sources.

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* deployment/docker/victorialogs/fluentbit: include log path in stream configuration

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* deployment/docker: add reference to monitoring setup for VictoriaLogs

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

---------

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2023-09-01 10:45:43 +02:00
Karan Sharma
327f63e408
fix: remove healthcheck in vector docker-compose (#4772) 2023-08-03 14:56:44 +04:00
Aliaksandr Valialkin
116623a0f0
docs/VictoriaLogs: add missing information about Promtail support for data ingestion
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4728
2023-07-28 09:47:08 -07:00
Aliaksandr Valialkin
2086c0885d
deployment: update VictoriaLogs Docker image tag from from v0.2.0-victorialogs to v0.3.0-victorialogs 2023-07-20 20:58:41 -07:00
Aliaksandr Valialkin
f548adce0b
app/vlinsert/loki: follow-up after 09df5b66fd
- Parse protobuf if Content-Type isn't set to `application/json` - this behavior is documented at https://grafana.com/docs/loki/latest/api/#push-log-entries-to-loki

- Properly handle gzip'ped JSON requests. The `gzip` header must be read from `Content-Encoding` instead of `Content-Type` header

- Properly flush all the parsed logs with the explicit call to vlstorage.MustAddRows() at the end of query handler

- Check JSON field types more strictly.

- Allow parsing Loki timestamp as floating-point number. Such a timestamp can be generated by some clients,
  which store timestamps in float64 instead of int64.

- Optimize parsing of Loki labels in Prometheus text exposition format.

- Simplify tests.

- Remove lib/slicesutil, since there are no more users for it.

- Update docs with missing info and fix various typos. For example, it should be enough to have `instance` and `job` labels
  as stream fields in most Loki setups.

- Allow empty of missing timestamps in the ingested logs.
  The current timestamp at VictoriaLogs side is then used for the ingested logs.
  This simplifies debugging and testing of the provided HTTP-based data ingestion APIs.

The remaining MAJOR issue, which needs to be addressed: victoria-logs binary size increased from 13MB to 22MB
after adding support for Loki data ingestion protocol at https://github.com/VictoriaMetrics/VictoriaMetrics/pull/4482 .
This is because of shitty protobuf dependencies. They must be replaced with another protobuf implementation
similar to the one used at lib/prompb or lib/prompbmarshal .
2023-07-20 16:48:21 -07:00
Zakhar Bessarab
09df5b66fd
app/vlinsert: add support of loki push protocol (#4482)
* app/vlinsert: add support of loki push protocol

- implemented loki push protocol for both Protobuf and JSON formats
- added examples in documentation
- added example docker-compose

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* app/vlinsert: move protobuf metric into its own file

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* deployment/docker/victorialogs/promtail: update reference to docker image

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* deployment/docker/victorialogs/promtail: make volume name unique

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* app/vlinsert/loki: add license reference

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* deployment/docker/victorialogs/promtail: fix volume name

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* docs/VictoriaLogs/data-ingestion: add stream fields for loki JSON ingestion example

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* app/vlinsert/loki: move entities to places where those are used

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* app/vlinsert/loki: refactor to use common components

- use CommonParameters from insertutils
- stop ingestion after first error similar to elasticsearch and jsonline

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* app/vlinsert/loki: address review feedback

- add missing logstorage.PutLogRows calls
- refactor tenant ID parsing to use common function
- reduce number of allocations for parsing by reusing  logfields slices
- add tests and benchmarks for requests processing funcs

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

---------

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2023-07-20 10:10:55 +02:00
Aliaksandr Valialkin
920cc26793
all: use the new release for VictoriaLogs - v0.2.0-victorialogs 2023-07-17 23:19:46 -07:00
Artem Navoiev
d1925d6a1c
fix jsonline endpoint in docs
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2023-06-22 08:48:11 +02:00
Artem Navoiev
345c7b0bc7 fix link vmui links in docker examples after the path chnage for vmui
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2023-06-21 23:21:04 -07:00
Artem Navoiev
b488399952 add information about how to access web UI
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2023-06-21 11:19:16 -07:00
Artem Navoiev
e87c38c5d3 use latest image in example for VictoriaLogs and VictoriaMetrics to simplify the flow
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2023-06-21 05:17:30 -07:00
Alexander Marshalov
cb383f546a improved vector example for victoria logs
Signed-off-by: Alexander Marshalov <_@marshalov.org>
2023-06-21 03:59:31 -07:00
Artem Navoiev
678c2cd0a1 remove grafana notion from vector readme
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2023-06-21 03:59:31 -07:00
Artem Navoiev
b43152f1ce examples add README
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2023-06-21 03:59:31 -07:00
Artem Navoiev
bc6427a44f remove docker from filebeat syslog
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2023-06-21 03:59:31 -07:00
Artem Navoiev
b024e46284 Add docker compose examples: filebeat(docker, syslog), fluentbit(docker), logstash, vector(docker)
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2023-06-21 03:59:31 -07:00