Multi-level vmselect setup is not intended to use storage node IDs, so it is safe to return 0 here.
Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
It is needed to rebuild snb in order to ensure that list of storage nodes and consistent hash are in sync.
Updating just consistent hash ring is not safe because it can cause misalignment of indexes of alive nodes in snb.sns and hash slots.
Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
Exclude unhealthy storage nodes from consistent hash in case persistent storage node IDs are enabled.
This is needed in order to avoid uneven distribution of load due to default(uint64(0)) IDs assigned to storage nodes.
Remove generating fallback ID from node IP address as this will cause a re-distribution of series once storage node will become available and will change its ID.
Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
Update the handshake to use an arbitrary JSON payload to transfer metadata.
Handshake sends the metadata length first as an uint64 and then the metadata itself.
Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
It is supposed to be saved right after it was generated, there is no reason re-write it again.
Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
Generate an ID based on storage node address if storage node is not available.
This is needed in order to prevent uneven load distribution if some storage nodes are not available when vminsert is starting.
Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
This is needed in order to avoid complete data re-sharding after the upgrade to a new version.
Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
Send a node ID of vmstorage as a part of vmselect and vminsert handshakes.
Use vmstorage node ID as an identifier for consistent hashing at vminsert.
Cluster native endpoints calculate vminsert and vmselect node IDs as a hash of all underlying storage node IDs, so that it will also remain consistent in case of address changes.
Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
Generate random node ID on start if it is missing or load from disk. Save to storage on storage shutdown.
Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
This reverts commit e280d90e9a.
Reason for revert: the updated code doesn't improve the performance of table.MustAddRows for the typical case
when rows contain timestamps belonging to ptws[0].
The performance may be improved in theory for the case when all the rows belong to partiton other than ptws[0],
but this partition is automatically moved to ptws[0] by the code at lines
6aad1d43e9/lib/storage/table.go (L287-L298) ,
so the next time the typical case will work.
Also the updated code makes the code harder to follow, since it introduces an additional level of indirection
with non-trivial semantics inside table.MustAddRows - the partition.TimeRangeInPartition() function.
This function needs to be inspected and understood when reading the code at table.MustAddRows().
This function depends on minTsInRows and maxTsInRows vars, which are defined and initialized
many lines above the partition.TimeRangeInPartition() call. This complicates reading and understanding
the code even more.
The previous code was using clearer loop over rows with the clear call to partition.HasTimestamp()
for every timestamp in the row. The partition.HasTimestamp() call is used in the table.MustAddRows()
function multiple times. This makes the use of partition.HasTimestamp() call more consistent,
easier to understand and easier to maintain comparing to the mix of partition.HasTimestamp() and partition.TimeRangeInPartition()
calls.
Aslo, there is no need in documenting some hardcore software engineering refactoring at docs/CHANGLELOG.md,
since the docs/CHANGELOG.md is intended for VictoriaMetrics users, who may not know software engineering.
The docs/CHANGELOG.md must document user-visible changes, and the docs must be concise and clear for VictoriaMetrics users.
See https://docs.victoriametrics.com/contributing/#pull-request-checklist for more details.
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6629
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6677
Relative links in docs are much harder to maintain in consistent state comparing to absolute links:
- It is non-trivial to figure out the proper relative link path when creating and editing docs.
- Relative links break after moving the doc files to another paths, and it is non-trivial
to figure which links are broken after that.
See also f357ee57ef , d5809f8e12 and 8cb1822b94
- Document `make docs-debug` command at https://docs.victoriametrics.com/#documentation
- Remove unneeded ROOTDIR, REPODIR and WORKDIR env vars from docs/Makefile ,
since it is documented and expected that all the Makefile commands are run from the repository root.
- Use `docker --rm` for running Docker container with local docs server, so it is automatically
removed after pressing `Ctrl+C`. This makes the container cleanup automatic.
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6677
### Describe Your Changes
The original logic is not only highly complex but also poorly readable,
so it can be modified to increase readability and reduce time
complexity.
---------
Co-authored-by: Zhu Jiekun <jiekun@victoriametrics.com>
### Describe Your Changes
Please provide a brief description of the changes you made. Be as
specific as possible to help others understand the purpose and impact of
your modifications.
### Checklist
The following checks are **mandatory**:
- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
### Describe Your Changes
Replaced global http links in docs with relative markdown ones
### Checklist
The following checks are **mandatory**:
- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
### Describe Your Changes
Use relative markdown references, removed `{{< ref >}}` shortcodes
### Checklist
The following checks are **mandatory**:
- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
- moved files from root to VictoriaMetrics folder to be able to mount
operator docs and VictoriaMetrics docs independently
- added ability to run website locally
The following checks are **mandatory**:
- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
### Describe Your Changes
Please provide a brief description of the changes you made. Be as
specific as possible to help others understand the purpose and impact of
your modifications.
### Checklist
The following checks are **mandatory**:
- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
### Describe Your Changes
Change response code to 502 to align it with behaviour of other existing
reverse proxies. Currently, the following reverse proxies will return
502 in case an upstream is not available: nginx, traefik, caddy, apache.
Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
### Describe Your Changes
Improve documentation to help:
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6670
Currently, the documentation of Kafka Integration did not describe:
- How to switch between different remote write protocols in producer
side and consumer side.
### Describe Your Changes
Add a note to clarify usage of azure credentials when multiple
credentials are available.
The user is required to specify AZURE_CLIENT_ID as otherwise Azure API
will return an error: "Multiple user assigned identities exist, please
specify the clientId / resourceId of the identity in the token request"
### Checklist
The following checks are **mandatory**:
- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
