Как правильно передать логи через Fluentd в Elasticsearch 8.X.X
Я пытаюсь передавать логи K8s в Elasticsearch 8.2.0 используя Fluentd. Его конфиг передаётся через configmap.yaml и выглядит так:
apiVersion: v1
kind: ConfigMap
metadata:
labels:
stack: app
name: fluentd-config
namespace: kube-system
data:
fluent.conf: |-
@include kubernetes.conf
@include elasticsearch.conf
@include pods.conf
kubernetes.conf |-
<filter kubernetes.**>
@type kubernetes_metadata
</filter>
<match fluent.**>
@type null
</match>
<match kubernetes.var.log.containers.**kube-logging**.log>
@type null
</match>
<match kubernetes.var.log.containers.**kube-system**.log>
@type null
</match>
<match kubernetes.var.log.containers.**monitoring**.log>
@type null
</match>
<match kubernetes.var.log.containers.**infra**.log>
@type null
</match>
<match kubernetes.var.log.containers.**fluentd**.log>
@type null
</match>
<match ignore.**>
@type null
</match>
<filter kubernetes.**>
@id filter_concat
@type concat
key log
multiline_end_regexp /\n$/
separator ""
</filter>
<filter kubernetes.**>
@type parser
key_name log
reserve_data true
remove_key_name_field true
<parse>
@type multi_format
<pattern>
format json
</pattern>
<pattern>
format none
</pattern>
</parse>
</filter>
elasticsearch.conf: |-
<match **>
@type elasticsearch
host 10.11.111.160
port 9100
scheme http
ssl_verify false
logstash_format false
include_timestamp true
flush_interval 5s
index_name fluentd-main
<buffer>
@type "file"
path "/var/log/fluentd/buffers/elasticsearch"
flush_interval 10s
retry_max_interval 30
chunk_limit_size 2M
queue_limit_length 8
flush_thread_count 2
</buffer>
</match>
...
Настройки fluentd и плагины установлены под 8.X.X версию Elasticsearch:
elastic-transport (8.4.1)
elasticsearch (8.19.2)
elasticsearch-api (8.19.2)
fluent-plugin-elasticsearch (5.4.4)
логи k8s собираются правильно и как мне нужно, но в elasticsearch они не передаются. Телнет до elasticsearch есть. Могу вручную создавать индексы командой curl -X PUT "10.11..." В логах fluentd я вижу сообщения:
Но при старте fluentd я получаю ошибку:
[debug]: #0 'host 10.11.111.160' is tested built-in placeholder(s) but there is no valid placeholder(s). error: Parameter 'host: 10.11.111.160' doesn't have tag placeholder
[debug]: #0 'index_name fluentd-main' is tested built-in placeholder(s) but there is no valid placeholder(s). error: Parameter 'index_name: fluentd-main' doesn't have tag placeholder
[debug]: #0 'template_name ' is tested built-in placeholder(s) but there is no valid placeholder(s). error: Parameter 'template_name: ' doesn't have tag placeholder
[debug]: #0 'logstash_prefix kubernetes' is tested built-in placeholder(s) but there is no valid placeholder(s). error: Parameter 'logstash_prefix: kubernetes' doesn't have tag placeholder
[debug]: #0 'logstash_dateformat %Y.%m.%d' is tested built-in placeholder(s) but there is no valid placeholder(s). error: Parameter 'logstash_dateformat: %Y.%m.%d' has timestamp placeholders, but chunk key 'time' is not configured
[debug]: #0 'logstash_dateformat %Y.%m.%d' is tested built-in placeholder(s) but there is no valid placeholder(s). error: Parameter 'logstash_dateformat: %Y.%m.%d' doesn't have tag placeholder
[debug]: #0 'deflector_alias ' is tested built-in placeholder(s) but there is no valid placeholder(s). error: Parameter 'deflector_alias: ' doesn't have tag placeholder
[debug]: #0 'application_name default' is tested built-in placeholder(s) but there is no valid placeholder(s). error: Parameter 'application_name: default' doesn't have tag placeholder
[debug]: #0 'ilm_policy_id logstash-policy' is tested built-in placeholder(s) but there is no valid placeholder(s). error: Parameter 'ilm_policy_id: logstash-policy' doesn't have tag placeholder
[debug]: #0 Need substitution: false
[debug]: #0 'host_placeholder 10.11.111.160' is tested built-in placeholder(s) but there is no valid placeholder(s). error: Parameter 'host_placeholder: 10.11.111.160' doesn't have tag placeholder
[debug]: #0 Detected ES 8.x: This parameter has no effect
Почему я получаю эти ошибки? Как это исправить можно?