github-mirrors/VictoriaMetrics
1
0
Fork 0
mirror of https://github.com/VictoriaMetrics/VictoriaMetrics.git synced 2024-11-21 14:44:00 +00:00
Code Issues Projects Releases Packages Wiki Activity
VictoriaMetrics/docs/FAQ.md
Aziz Köksal 74a4c29729
docs: proofread FAQ.md (#2062) 
* Added missing articles.
* Changed minus characters to long dashes.
* Rephrased some words.
* Corrected grammar mistakes.
* Set correct possessive apostrophes.
* "setup" -> "set up"
* "no need in" -> "no need for"
* "comparing" -> "compared"
* "less" -> "fewer"
* "additionally" -> "in addition to"
2022-01-17 21:56:19 +02:00

30 KiB
Raw Blame History

sort
14

FAQ

What is the main purpose of VictoriaMetrics?

To provide the best monitoring solution.

Who uses VictoriaMetrics?

See case studies.

Which features does VictoriaMetrics have?

See these docs.

Are there performance comparisons with other solutions?

Yes. See these benchmarks.

How to start using VictoriaMetrics?

See these docs.

Does VictoriaMetrics support replication?

Yes. See these docs for details.

Can I use VictoriaMetrics instead of Prometheus?

Yes in most cases. VictoriaMetrics can substitute Prometheus in the following aspects:

  • Prometheus-compatible service discovery and target scraping can be done with vmagent and with single-node VictoriaMetrics. See these docs.
  • Prometheus-compatible alerting rules and recording rules can be processed with vmalert.
  • Prometheus-compatible querying in Grafana is supported by VictoriaMetrics. See these docs.

What is the difference between vmagent and Prometheus?

While both vmagent and Prometheus may scrape Prometheus targets (aka /metrics pages) according to the provided Prometheus-compatible scrape configs and send data to multiple remote storage systems, vmagent has the following additional features:

  • vmagent usually requires lower amounts of CPU, RAM and disk IO compared to Prometheus when scraping an enormous number of targets (more than 1000) or targets with a great number of exposed metrics.
  • vmagent provides independent disk-backed buffers for each configured remote storage (see -remoteWrite.url). This means that slow or temporarily unavailable storage doesn't prevent it from sending data to healthy storage in parallel. Prometheus uses a single shared buffer for all the configured remote storage systems (see remote_write->url) with a hardcoded retention of 2 hours.
  • vmagent may accept, relabel and filter data obtained via multiple data ingestion protocols in addition to data scraped from Prometheus targets. That means it supports both pull and push protocols for data ingestion. See these docs for details.
  • vmagent may be used in different use cases:

What is the difference between vmagent and Prometheus agent?

Both vmagent and Prometheus agent serve the same purpose – to efficently scrape Prometheus-compatible targets at the edge. They have the following differences:

  • vmagent usually requires lower amounts of CPU, RAM and disk IO compared to the Prometheus agent.
  • Prometheus agent supports only pull-based data collection (e.g. it can scrape Prometheus-compatible targets), while vmagent supports both pull and push data collection – it can accept data via many popular data ingestion protocols such as InfluxDB line protocol, Graphite protocol, OpenTSDB protocol, DataDog protocol, Prometheus protocol, CSV and JSON – see these docs.
  • vmagent can easily scale horizontally to multiple instances for scraping a big number of targets – see these docs.
  • vmagent supports improved relabeling.
  • vmagent can limit the number of scraped metrics per target – see these docs.
  • vmagent supports loading scrape configs from multiple files – see these docs.
  • vmagent supports data reading and data writing from/to Kafka – see these docs.
  • vmagent can read and update scrape configs from http and https URLs, while the Prometheus agent can read them only from the local file system.

Is it safe to enable remote write in Prometheus?

Yes. Prometheus continues writing data to local storage after enabling remote write, so all the existing local storage data and new data is available for querying via Prometheus as usual.

It is recommended using vmagent for scraping Prometheus targets and writing data to VictoriaMetrics.

How does VictoriaMetrics compare to other remote storage solutions for Prometheus such as M3 from Uber, Thanos, Cortex, etc.?

VictoriaMetrics is simpler, faster, more cost-effective and it provides MetricsQL query language based on PromQL. The simplicity is twofold:

  • It is simpler to configure and operate. There is no need for configuring sidecars, fighting the gossip protocol or setting up third-party systems such as Consul, Cassandra, DynamoDB or Memcached.
  • VictoriaMetrics has a simpler architecture. This means fewer bugs and more useful features in the long run compared to competing TSDBs.

See comparing Thanos to VictoriaMetrics cluster and the Remote Write Storage Wars talk from PromCon 2019.

VictoriaMetrics also uses less RAM than Thanos components.

What is the difference between VictoriaMetrics and QuestDB?

What is the difference between VictoriaMetrics and Cortex?

VictoriaMetrics is similar to Cortex in the following aspects:

The main differences between Cortex and VictoriaMetrics:

  • Cortex re-uses Prometheus source code, while VictoriaMetrics is written from scratch.
  • Cortex heavily relies on third-party services such as Consul, Memcache, DynamoDB, BigTable, Cassandra, etc. This may increase operational complexity and reduce system reliability compared to VictoriaMetrics' case, which doesn't use any external services. Compare Cortex' Architecture to VictoriaMetrics' architecture.
  • VictoriaMetrics provides production-ready single-node solution, which is much easier to set up and operate than a Cortex cluster.
  • Cortex may lose up to 12 hours of recent data on Ingestor failure – see the corresponding docs. VictoriaMetrics may lose only a few seconds of recent data, which isn't synced to persistent storage yet. See this article for details.
  • Cortex is usually slower and requires more CPU and RAM than VictoriaMetrics. See this talk from adidas at PromCon 2019 and other case studies.
  • VictoriaMetrics accepts data in multiple popular data ingestion protocols additionally to Prometheus remote_write protocol – InfluxDB, OpenTSDB, Graphite, CSV, JSON, native binary. See these docs for details.
  • VictoriaMetrics provides the MetricsQL query language, while Cortex provides the PromQL query language.
  • VictoriaMetrics can be queried via Graphite's API.

What is the difference between VictoriaMetrics and Thanos?

  • Thanos re-uses Prometheus source code, while VictoriaMetrics is written from scratch.
  • VictoriaMetrics accepts data via the standard remote_write API for Prometheus, while Thanos uses a non-standard sidecar which must run alongside each Prometheus instance.
  • The Thanos sidecar requires disabling data compaction in Prometheus, which may hurt Prometheus performance and increase RAM usage. See these docs for more details.
  • Thanos stores data in object storage (Amazon S3 or Google GCS), while VictoriaMetrics stores data in block storage (GCP persistent disks, Amazon EBS or bare metal HDD). While object storage is usually less expensive, block storage provides much lower latencies and higher throughput. VictoriaMetrics works perfectly with HDD-based block storage – there is no need for using more expensive SSD or NVMe disks in most cases.
  • Thanos may lose up to 2 hours of recent data, which wasn't uploaded yet to object storage. VictoriaMetrics may lose only a few seconds of recent data, which hasn't been synced to persistent storage yet. See this article for details.
  • VictoriaMetrics provides a production-ready single-node solution, which is much easier to set up and operate than Thanos components.
  • Thanos may be harder to set up and operate compared to VictoriaMetrics, since it has more moving parts, which can be connected with fewer reliable networks. See this article for details.
  • Thanos is usually slower and requires more CPU and RAM than VictoriaMetrics. See this talk from adidas at PromCon 2019.
  • VictoriaMetrics accepts data via multiple popular data ingestion protocols in addition to the Prometheus remote_write protocol – InfluxDB, OpenTSDB, Graphite, CSV, JSON, native binary. See these docs for details.
  • VictoriaMetrics provides the MetricsQL query language, while Thanos provides the PromQL query language.
  • VictoriaMetrics can be queried via Graphite's API.

How does VictoriaMetrics compare to InfluxDB?

  • VictoriaMetrics requires 10x less RAM and it works faster.
  • VictoriaMetrics needs lower amounts of storage space than InfluxDB for production data.
  • VictoriaMetrics provides a better query language – MetricsQL – than InfluxQL or Flux. See this tutorial for details.
  • VictoriaMetrics accepts data in multiple popular data ingestion protocols in addition to InfluxDB – Prometheus remote_write, OpenTSDB, Graphite, CSV, JSON, native binary. See these docs for details.
  • VictoriaMetrics can be queried via Graphite's API.

How does VictoriaMetrics compare to TimescaleDB?

Does VictoriaMetrics use Prometheus technologies like other clustered TSDBs built on top of Prometheus such as Thanos or Cortex?

No. VictoriaMetrics core is written in Go from scratch by fasthttp's author. The architecture is optimized for storing and querying large amounts of time series data with high cardinality. VictoriaMetrics storage uses certain ideas from ClickHouse. Special thanks to Alexey Milovidov.

What is the pricing for VictoriaMetrics?

The following versions are open source and free:

We provide commercial support for both versions. Contact us for the pricing.

The following commercial versions of VictoriaMetrics are available:

The following commercial versions of VictoriaMetrics are planned:

  • Managed VictoriaMetrics at Google Cloud.
  • Cloud monitoring solution based on VictoriaMetrics.

Contact us for more information on our plans.

Why doesn't VictoriaMetrics support the Prometheus remote read API?

The remote read API requires transferring all the raw data for all the requested metrics over the given time range. For instance, if a query covers 1000 metrics with 10K values each, then the remote read API has to return 1000*10K=10M metric values to Prometheus. This is slow and expensive. Prometheus' remote read API isn't intended for querying foreign data – aka global query view. See this issue for details.

So just query VictoriaMetrics directly via vmui, the Prometheus Querying API or via Prometheus datasource in Grafana.

Does VictoriaMetrics deduplicate data from Prometheus instances scraping the same targets (aka HA pairs)?

Yes. See these docs for details.

Where is the source code of VictoriaMetrics?

Source code for the following versions is available in the following places:

Is VictoriaMetrics a good fit for data from IoT sensors and industrial sensors?

VictoriaMetrics is able to handle data from hundreds of millions of IoT sensors and industrial sensors. It supports high cardinality data, perfectly scales up on a single node and scales horizontally to multiple nodes.

Where can I ask questions about VictoriaMetrics?

Questions about VictoriaMetrics can be asked via the following channels:

Where can I file bugs and feature requests regarding VictoriaMetrics?

File bugs and feature requests here.

Where can I find information about multi-tenancy?

See these docs. Multitenancy is supported only by the cluster version of VictoriaMetrics.

How to set a memory limit for VictoriaMetrics components?

All the VictoriaMetrics components provide command-line flags to control the size of internal buffers and caches: -memory.allowedPercent and -memory.allowedBytes (pass -help to any VictoriaMetrics component in order to see the description for these flags). These limits don't take into account additional memory, which may be needed for processing incoming queries. Hard limits may be enforced only by the OS via cgroups, Docker (see these docs) or Kubernetes (see these docs).

Memory usage for VictoriaMetrics components can be tuned according to the following docs:

How can I run VictoriaMetrics on FreeBSD?

VictoriaMetrics is included in FreeBSD ports, so just install it from there. See this link.

Does VictoriaMetrics support the Graphite query language?

Yes. See these docs.

What is an active time series?

A time series is uniquely identified by its name plus a set of its labels. For example, temperature{city="NY",country="US"} and temperature{city="SF",country="US"} are two distinct series, since they differ by the city label. A time series is considered active if it receives at least a single new sample during the last hour.

What is high churn rate?

If old time series are constantly substituted by new time series at a high rate, then such a state is called high churn rate. High churn rate has the following negative consequences:

  • Increased total number of time series stored in the database.
  • Increased size of inverted index, which is stored at <-storageDataPath>/indexdb, since the inverted index contains entries for every label of every time series with at least a single ingested sample.
  • Slow-down of queries over multiple days.

The main reason for high churn rate is a metric label with frequently changed value. Examples of such labels:

  • queryid, which changes with each query at postgres_exporter.
  • app_name or deployment_id, which changes with each new deployment in Kubernetes.
  • A label derived from the current time such as timestamp, minute or hour.
  • A hash or uuid label, which changes frequently.

The solution against high churn rate is to identify and eliminate labels with frequently changed values. The /api/v1/status/tsdb page can help determining these labels.

What is high cardinality?

High cardinality usually means a high number of active time series. High cardinality may lead to high memory usage and/or to a high percentage of slow inserts. The source of high cardinality is usually a label with a large number of unique values, which presents a big share of the ingested time series. The solution is to identify and remove the source of high cardinality with the help of /api/v1/status/tsdb.

What is a slow insert?

VictoriaMetrics maintains in-memory cache for mapping of active time series into internal series ids. The cache size depends on the available memory for VictoriaMetrics in the host system. If the information about all the active time series doesn't fit the cache, then VictoriaMetrics needs to read and unpack the information from disk on every incoming sample for time series missing in the cache. This operation is much slower than the cache lookup, so such an insert is named a slow insert. A high percentage of slow inserts on the official dashboard for VictoriaMetrics indicates a memory shortage for the current number of active time series. Such a condition usually leads to a significant slowdown for data ingestion and to significantly increased disk IO and CPU usage. The solution is to add more memory or to reduce the number of active time series. The /api/v1/status/tsdb page can be helpful for locating the source of high number of active time seriess – see these docs.

How to optimize MetricsQL query?

See this article.

Why isn't MetricsQL 100% compatible with PromQL?

MetricsQL provides better user experience than PromQL. It fixes a few annoying issues in PromQL. This prevents MetricsQL to be 100% compatible with PromQL. See this article for details.

How to migrate data from Prometheus to VictoriaMetrics?

Please see these docs.

How to migrate data from InfluxDB to VictoriaMetrics?

Please see these docs.

How to migrate data from OpenTSDB to VictoriaMetrics?

Please see these docs.

How to migrate data from Graphite to VictoriaMetrics?

Please use the whisper-to-graphite tool for reading data from Graphite and pushing them to VictoriaMetrics via Graphite's import API.

Why do the same metrics have differences in VictoriaMetrics' and Prometheus' dashboards?

There could be a slight difference in stored values for time series. Due to different compression algorithms, VM may reduce the precision for float values with more than 12 significant decimal digits. Please see this article.

The query engine may behave differently for some functions. Please see this article.