github-mirrors/VictoriaMetrics
1
0
Fork 0
mirror of https://github.com/VictoriaMetrics/VictoriaMetrics.git synced 2025-01-10 15:14:09 +00:00
Code Issues Projects Releases Packages Wiki Activity

app/vlinsert/elasticsearch: add a command-line flag to provide ES version (#4778)

Browse source 
* app/vlinsert/elasticsearch: add a command-line flag to provide ES version

Adds a flag which will allow to change version which will be reported by ES endpoint for compatibility checks performed by external logs shippers(such as filebeat).
See: https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4777

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* Document the -elasticsearch.version command-line flag

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4777

---------

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
This commit is contained in:
Zakhar Bessarab 2023-08-11 17:52:57 +04:00 committed by Aliaksandr Valialkin
parent 15b1810dc8
commit bae701e612
No known key found for this signature in database
GPG key ID: A72BEC6CD3D0DED1
3 changed files with 24 additions and 2 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

9
app/vlinsert/elasticsearch/elasticsearch.go
View file

@ -3,6 +3,7 @@ package elasticsearch
import (
"bufio"
"errors"
"flag"
"fmt"
"io"
"math"
@ -24,6 +25,10 @@ import (
"github.com/VictoriaMetrics/metrics"
)
var (
elasticsearchVersion = flag.String("elasticsearch.version", "8.9.0", "Elasticsearch version to report to client")
)
// RequestHandler processes Elasticsearch insert requests
func RequestHandler(path string, w http.ResponseWriter, r *http.Request) bool {
w.Header().Add("Content-Type", "application/json")
@ -60,9 +65,9 @@ func RequestHandler(path string, w http.ResponseWriter, r *http.Request) bool {
// See the latest available version for Elasticsearch at https://github.com/elastic/elasticsearch/releases
fmt.Fprintf(w, `{
"version": {
"number": "8.8.0"
"number": %q
}
}`)
}`, *elasticsearchVersion)
case http.MethodHead:
// Return empty response for Logstash ping request.
}

1
docs/VictoriaLogs/CHANGELOG.md
View file

@ -5,6 +5,7 @@ according to [these docs](https://docs.victoriametrics.com/VictoriaLogs/QuickSta
## tip
* FEATURE: add `-elasticsearch.version` command-line flag, which can be used for specifying Elasticsearch version returned by VictoriaLogs to Filebeat at [elasticsearch bulk API](https://docs.victoriametrics.com/VictoriaLogs/data-ingestion/#elasticsearch-bulk-api). This helps resolving [this issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4777).
* FEATURE: expose the following metrics at [/metrics](monitoring) page:
* `vl_data_size_bytes{type="storage"}` - on-disk size for data excluding [log stream](https://docs.victoriametrics.com/VictoriaLogs/keyConcepts.html#stream-fields) indexes.
* `vl_data_size_bytes{type="indexdb"}` - on-disk size for [log stream](https://docs.victoriametrics.com/VictoriaLogs/keyConcepts.html#stream-fields) indexes.

16
docs/VictoriaLogs/data-ingestion/Filebeat.md
View file

@ -100,6 +100,22 @@ output.elasticsearch:
_stream_fields: "host.name,log.file.path"
```
Filebeat checks a version of ElasticSearch on startup and refuses to start sending logs if the version is not compatible.
In order to bypass this check please add `allow_older_versions: true` into `output.elasticsearch` section:
```yml
output.elasticsearch:
hosts: [ "http://localhost:9428/insert/elasticsearch/" ]
parameters:
_msg_field: "message"
_time_field: "@timestamp"
_stream_fields: "host.name,log.file.path"
allow_older_versions: true
```
Alternatively, is also possible to change version which VictoriaLogs reports to Filebeat by using `-elasticsearch.version`
command-line flag.
See also:
- [Data ingestion troubleshooting](https://docs.victoriametrics.com/VictoriaLogs/data-ingestion/#troubleshooting).