Getting [error] [flb_msgpack_to_gelf] level is 'WARN', but should be in 0..7 or a syslog keyword

[shake76]

Bug Report

After put Labels On in kubernetes filter configuration, I'm seeing the following errors logs coming from fluen-bit pods, `

[error] [flb_msgpack_to_gelf] level is 'WARN', but should be in 0..7 or a syslog keyword

I'm currently using GELF output sending logs to Graylog and I noticed that I'm missing logs for some streams

To Reproduce
Below I can provide the log stream that I'm missing

• Example log message if applicable:

{"@timestamp":"2021-03-03T07:18:54.115-05:00","@version":1,"message":"Health Check passed in 408 ms","logger_name":"com.myhost.app.rest.UptimeCheckPlan$","thread_name":"qtp-119233","level":"INFO","level_value":20000,"nv_path":"/checkup"}
{"@timestamp":"2021-03-03T07:24:28.896-05:00","@version":1,"message":"Health Check passed in 374 ms","logger_name":"com.myhost.rest.UptimeCheckPlan$","thread_name":"qtp-119285","level":"INFO","level_value":20000,"nv_path":"/checkup"}
{"@timestamp":"2021-03-03T07:41:18.887-05:00","@version":1,"message":"Health Check passed in 123

Expected behavior
I need to get all the logs from the pods running in Kubernetes

Your Environment

• Version used: fluent-bit 1.5.7
• AWS EKS kubernetes 1.17
• Configuration:

Data
====
filter-kubernetes.conf:
----
[FILTER]
   Name                kubernetes
   Match               kube.*
   Kube_URL            https://kubernetes.default.svc.cluster.local:443
   Buffer_Size         1M
   Merge_Log           On
   K8S-Logging.Parser  On
   K8S-Logging.Exclude On
   Labels              On
   Annotations         Off

# ${HOSTNAME} returns the host name.
# But Fluentbit runs in a container. So, it is not meaningful.
# Instead, copy the host name from the Kubernetes object.
[FILTER]
   Name nest
   Match *
   Operation lift
   Nested_under kubernetes

# Rename and add fields to be valid against the GELF format.
# See http://docs.graylog.org/en/3.1/pages/gelf.html
# Keep the "time" field, as it is not a timestamp.
[FILTER]
   Name modify
   Match *
   Rename log short_message
   Rename stream _stream
   Rename @timestamp timestamp
   Rename pod_name _k8s_pod_name
   Rename namespace_name _k8s_namespace_name
   Rename pod_id _k8s_pod_id
   Rename labels _k8s_labels
   Rename container_name _k8s_container_name
   Rename docker_id _docker_id
   Add version 1.1

# Remove useless fields
[FILTER]
   Name record_modifier
   Match *
   Remove_key annotations
   #Remove_key _k8s_labels
   Remove_key kubernetes

fluent-bit.conf:
----
[SERVICE]
   Flush         1
   Log_Level     info
   Daemon        off
   Parsers_File  parsers.conf
   HTTP_Server   On
   HTTP_Listen   0.0.0.0
   HTTP_Port     2020

@INCLUDE input-kubernetes.conf
@INCLUDE filter-kubernetes.conf
@INCLUDE output-graylog.conf

input-kubernetes.conf:
----
[INPUT]
   Name               tail
   Tag                kube.*
   Path               /var/log/containers/*.log
   Docker_Mode        On
   Docker_Mode_Flush  5
   Docker_Mode_Parser read_firstline
   DB                 /var/log/flb_kube.db
   Parser             docker
   Mem_Buf_Limit      100MB
   Skip_Long_Lines    On
   Refresh_Interval   10

output-graylog.conf:
----
[OUTPUT]
   Name          gelf
   Match         *
   Host          grayloghost 
   Port          12201
   Mode          tcp
   Gelf_Short_Message_Key short_message

parsers.conf:
----
[PARSER]
   Name   json
   Format json
   Time_Key time
   Time_Format %d/%b/%Y:%H:%M:%S %z

[PARSER]
   Name        docker
   Format      json
   Time_Key    time
   Time_Format %Y-%m-%dT%H:%M:%S.%L
   Time_Keep   On
   

[PARSER]
   Name          read_firstline
   Format        regex
   Regex         (?<log>(?<="log":")\d{4}[\/-]\d{1,2}[\/-]\d{1,2}[ T]\d{2}:\d{2}:\d{2}(?!\.).*?)(?<!\\)".*(?<stream>(?<="stream":").*?)".*(?<time>\d{4}-\d{1,2}-\d{1,2}T\d{2}:\d{2}:\d{2}\.\w*).*(?=})
   Time_Key      time
   Time_Format   %Y-%m-%dT%H:%M:%S.%LZ

[github-actions]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days.

[github-actions]

This issue was closed because it has been stalled for 5 days with no activity.

[Garved]

Hi, this issue wasn't include in fix v1.7.3
I have some problem with this. Can I count on you for your help?