Commit graph

9516 commits

Author SHA1 Message Date
Aliaksandr Valialkin
afcc7fb167 app/vmselect/netstorage: improve error message when reading data blocks from storage
Mention the block number in the error. This should simplify troubleshooting in this code.
2019-07-28 12:12:35 +03:00
Aliaksandr Valialkin
57a57c711a package: changed the remaining /usr/local/bin to /usr/bin
This is a follo-up after 68f260d878
2019-07-28 11:08:07 +03:00
Anton Patsev
68f260d878 change /usr/local/bin to /usr/bin (#131) 2019-07-28 11:06:24 +03:00
Aliaksandr Valialkin
1eade9b358 app/vminsert: add vm_rows_per_insert summary metric
This metric should help tuning batch sizes on clients writing data to VictoriaMetrics
2019-07-27 13:21:46 +03:00
Aliaksandr Valialkin
7e8747f6ed README.md: add a section for production ARM build 2019-07-26 22:34:31 +03:00
Aliaksandr Valialkin
0168a1b658 package: various fixes
- Use `-prod` binaries instead of development binaries for both deb and rpm packages.
- Fix binary directory from /usr/sbin to /usr/local/bin as outlined in package/victoria-metrics.service
- Fix binary name from `victoriametrics` to `victoria-metrics-prod` in package/victoria-metrics.service
2019-07-26 22:31:04 +03:00
Aliaksandr Valialkin
bf6cbb762c app/vminsert: improve error messages for Influx, OpenTSDB and Graphite parsing
Include in the error message the line which failed to parse.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/127
2019-07-26 22:08:52 +03:00
Kostya Vasilyev
6aeac37fc5 pick up .service file from ./rpm (#126)
* pick up .service file from ./rpm

* feedback from @patsevanton

* remove 'start' from ExecStart command
2019-07-26 21:56:30 +03:00
Aliaksandr Valialkin
c98725db55 app/vmstorage: consistency renaming for ignored rows metrics
vm_too_big_timestamp_rows_total -> vm_rows_ignored_total{reason="big_timestamp"}
  vm_too_small_timestamp_rows_total -> vm_rows_ignored_total{reason="small_timestamp"}
2019-07-26 20:02:06 +03:00
Anton Patsev
d8043f7161 Change default value storageDataPath (#125)
Fixes #124 .
2019-07-26 14:13:55 +03:00
Aliaksandr Valialkin
f586e1f83c lib/storage: add metrics for calculating skipped rows outside the retention
The metrics are:

    - vm_too_big_timestamp_rows_total
    - vm_too_small_timestamp_rows_total
2019-07-26 14:11:01 +03:00
Kostya Vasilyev
d1132bb188 deb packaging fixes: 1) stop the service in prerm 2) reload services in postrm (#123) 2019-07-26 12:38:59 +03:00
Aliaksandr Valialkin
915fb6df79 README.md: mention that arm builds can run on Raspberry Pi 2019-07-26 12:28:40 +03:00
Kostya Vasilyev
89eb6d78a4 RPM packaging (#122) 2019-07-25 23:47:41 +03:00
Aliaksandr Valialkin
17096b5750 app/vmselect/promql: return NaN from count() over zero time series
This aligns `count` behavior with Prometheus.
2019-07-25 22:02:30 +03:00
Aliaksandr Valialkin
66efa5745f app/vmselect/promql: properly calculate incremental aggregations grouped by __name__
Previously the following query may fail on multiple distinct metric names match:

    sum(count_over_time{__name__!=''}) by (__name__)
2019-07-25 21:53:20 +03:00
Anton Patsev
106ab78a47 Add package/rpm/ (#121) 2019-07-25 11:21:55 +03:00
Aliaksandr Valialkin
8aa474d685 README.md: move how to build VictoriaMetrics section to the bottom
This streamlines `getting started` experience
2019-07-25 11:17:30 +03:00
Aliaksandr Valialkin
9e059bb330 README.md: add links to ARM build and Pure Go build in TOC 2019-07-25 11:05:35 +03:00
Aliaksandr Valialkin
2346335ea6 README.md: moved advanced topics to the bottom, so they don't clutter getting started workflow 2019-07-25 11:00:41 +03:00
Aliaksandr Valialkin
b339890dca lib/encoding/zstd: go fmt 2019-07-25 01:37:16 +03:00
Aliaksandr Valialkin
6c4ca89d75 lib/encoding/zstd: disable CRC checks in pure Go build
This should give slightly better compression and decompressions performance.
Additionally this shaves off 4 bytes per each compressed block.
2019-07-24 19:17:16 +03:00
Roman Khavronenko
f0fe7b5ad6
fix typo (#117) 2019-07-24 07:48:28 +01:00
Aliaksandr Valialkin
22ed4e7fd4 vendor: make vendor-update 2019-07-23 20:00:19 +03:00
Aliaksandr Valialkin
162f1fb1b7 all: small updates after PR #114 2019-07-23 19:54:50 +03:00
Aliaksandr Valialkin
d07f616609 lib/encoding: small fixes in tests after the PR #114 2019-07-23 19:37:51 +03:00
Roman Khavronenko
5bf4e5ffb5 all: add Pure Go build (pull request #114)
Updates #94
2019-07-23 19:26:39 +03:00
Kostya Vasilyev
8c3629a892 Debian packaging (#116)
* initial commit of deb packaging

* Incorporated feedback from @valyala:
- Put data directory under /var/lib
- More beef in systemd file
- Packaging for arm64
- Package all target which builds and packages both amd64 and arm64

* Remove PIDFile from systemd unit, useless

* per PR feedback, move debian specific files into deb subdirectory

Updates #107 .
2019-07-22 17:12:48 +03:00
Aliaksandr Valialkin
ea07cf68ba README.md: add querying Graphite data section
Mention that Graphite data may be read either via Prometheus querying API
or via go-graphite/carbonapi. See https://github.com/go-graphite/carbonapi/blob/master/cmd/carbonapi/carbonapi.example.prometheus.yaml
2019-07-21 16:10:19 +03:00
Roman Khavronenko
4ee41bab43 add versioning to dashboard description (#113) 2019-07-21 14:34:50 +03:00
Roman Khavronenko
1273f31f19 Add CPU usage panel; rename Go runtime to Resource usage (#112)
* add CPU usage panel; rename `Go runtime` to `Resource usage`

* rm irate from CPU usage panel

Updates #92 .
2019-07-20 17:24:24 +03:00
Aliaksandr Valialkin
0f2ecde0e6 lib/encoding: improve gauge series detection
- Series with negative values are always gauges
- Counters may only have increasing values with possible counter resets

This should improve compression ratio for gauge series which
were previously mistakenly detected as counters.
2019-07-20 14:05:09 +03:00
Aliaksandr Valialkin
6cd77d4847 deployment: switch builder from go1.12.6 to go1.12.7 2019-07-20 12:15:05 +03:00
Roman Khavronenko
fb14f23532 mention docker-compose as option to spin up VM (#97) 2019-07-16 00:45:21 +03:00
Aliaksandr Valialkin
daba0cdb05 lib/netutil: do not count timeouts as network errors 2019-07-15 23:05:35 +03:00
Aliaksandr Valialkin
575d2f0a91 app/vminsert: use netutil.TCPListener for collecting network-related metrics for Graphite and OpenTSDB TCP traffic 2019-07-15 22:58:00 +03:00
Aliaksandr Valialkin
ec1b439329 README.md: expand capacity planning section a bit 2019-07-12 21:19:27 +03:00
Aliaksandr Valialkin
6a943a6a58 app/vmselect/promql: remove empty time series after applying filters like q > 0
This should reduce CPU and RAM usage for queries over high number of time series.
2019-07-12 19:59:27 +03:00
Aliaksandr Valialkin
998525999c vendor: update github.com/VictoriaMetrics/metrics to v1.7.0
This version adds support for `process_*` metrics similar
to metrics exposed by https://github.com/prometheus/client_golang .

Fixes https://github.com/VictoriaMetrics/VictoriaMetrics/issues/92
2019-07-12 17:22:53 +03:00
Aliaksandr Valialkin
ab88890523 app/vmselect/promql: parallelize incremental aggregation to multiple CPU cores
This may reduce response times for aggregation over big number of time series
with small step between output data points.
2019-07-12 15:52:22 +03:00
Aliaksandr Valialkin
374d681848 README.md: clarify that Prometheus replicates data to remote storage 2019-07-12 02:51:04 +03:00
Aliaksandr Valialkin
e75d5f47c4 lib/storage: remove unused function isTooBigTimeRangeForDateMetricIDs 2019-07-12 02:28:23 +03:00
Aliaksandr Valialkin
fc90ebf43c lib/storage: do not reduce maxMetrics on time ranges exceeding maxDaysForDateMetricIDs
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/95
2019-07-12 02:20:34 +03:00
Aliaksandr Valialkin
62a7353479 app/vmselect/prometheus: set start arg in /api/v1/series to the minimum allowed time by default as Prometheus does
Fixes https://github.com/VictoriaMetrics/VictoriaMetrics/issues/91
2019-07-11 17:10:14 +03:00
Aliaksandr Valialkin
54bd21eb4a app/vmselect/prometheus: convert negative times to 0, since they arent supported by the storage 2019-07-11 17:07:20 +03:00
Aliaksandr Valialkin
2bd1a01d1a lib/storage: do not pollute inverted index with data for samples outside the retention period 2019-07-11 17:04:56 +03:00
Artem Navoiev
cd4833d3d0 integration tests 2019-07-11 15:48:08 +03:00
Aliaksandr Valialkin
101fa258e5 app/vmstorage: prepare for integration tests with multiple Init / Stop cycles 2019-07-11 15:34:50 +03:00
Aliaksandr Valialkin
d031e04023 lib/storage: use fast path for orSuffix when searching for metricIDs against plain tag value 2019-07-11 14:48:37 +03:00
Aliaksandr Valialkin
43ea4ce428 lib/storage: remember and skip individual tag filters matching too many metrics
This saves CPU time by skipping useless matching for individual tag filters.
2019-07-11 14:48:30 +03:00