VictoriaMetrics/apptest/README.md

41 lines
2 KiB
Markdown
Raw Permalink Normal View History

tests: Initial version of integration tests (#7253) ### Describe Your Changes Related issue: #7199 This is the initial version of the integration tests for cluster. See `README.md` for details. Currently cluster only, but it can also be used for vm-single if needed. The code has been added to the apptest package that resides in the root directory of the VM codebase. This is done to exclude the integration tests from regular testing build targets because: - Most of the test variants do not apply to integration testing (such as pure or race). - The integtation tests may also be slow because each test must wait for 2 seconds so vmstorage flushes pending content). It may be okay when there are a few tests but when there is a 100 of them running tests will require much more time which will affect the developer wait time and CI workflows. - Finally, the integration tests may be flaky especially short term. An alternative approach would be placing apptest under app package and exclude apptest from packages under test, but that is not trivial. The integration tests rely on retrieving some application runtime info from the application logs, namely the application's host:port. Therefore some changes to lib/httpserver/httpserver.go were necessary, such as reporting the effective host:port instead the one from the flag. ### Checklist The following checks are **mandatory**: - [x] My change adheres [VictoriaMetrics contributing guidelines](https://docs.victoriametrics.com/contributing/). --------- Signed-off-by: Artem Fetishev <rtm@victoriametrics.com> (cherry picked from commit d7b3589dbd0a3f38cecf7f8e896a3686c0dd8d4c)
2024-10-30 14:22:06 +00:00
# App Integration Tests
The `apptest` package contains the integration tests for the VictoriaMetrics
applications (such as vmstorage, vminsert, and vmselect).
An integration test aims at verifying the behavior of an application as a whole,
as apposed to a unit test that verifies the behavior of a building block of an
application.
To achieve that an integration test starts an application in a separate process
and then issues HTTP requets to it and verifies the responses, examines the
metrics the app exposes and/or files it creates, etc.
Note that an object of testing may be not just a single app, but several apps
working together. A good example is VictoriaMetrics cluster. An integration test
may reproduce an arbitrary cluster configuration and verify how the components
work together as a system.
The package provides a collection of helpers to start applications and make
queries to them:
- `app.go` - contains the generic code for staring an application and should
not be used by integration tests directly.
- `{vmstorage,vminsert,etc}.go` - build on top of `app.go` and provide the
code for staring a specific application.
- `client.go` - provides helper functions for sending HTTP requests to
applications.
The integration tests themselves reside in `*_test.go` files. Apart from having
the `_test` suffix, there are no strict rules of how to name a file, but the
name should reflect the prevailing purpose of the tests located in that file.
For example, `sharding_test.go` aims at testing data sharding.
Since integration tests start applications in a separate process, they require
the application binary files to be built and put into the `bin` directory. The
build rule used for running integration tests, `make integration-test`,
accounts for that, it builds all application binaries before running the tests.
But if you want to run the tests without `make`, i.e. by executing
`go test ./app/apptest`, you will need to build the binaries first (for example,
by executing `make all`).