Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

config: throw an error if a configuration property is not applied. #7783

Merged
merged 2 commits into from
Aug 18, 2023

Conversation

pwhelan
Copy link
Contributor

@pwhelan pwhelan commented Aug 2, 2023

Summary

Add a one-liner so if for some reason a property is not properly checked or applied an error is raised nonetheless.

With this configuration:

---
pipeline:
  inputs:
    - name: dummy
      dummy:
        message: '{"message: "my custom dummy"}'
  outputs:
    - name: stdout
      match: "*"
      format: json_lines

Without this change the fact that dummy is set as a key/value map is silently ignored, even though it is set as a string in the dummy plugin config_map. This change does not check the types but at least it does raise an error when the type is ignored, ie:

./bin/fluent-bit -c bad-dummy.yml
Fluent Bit v2.1.8
* Copyright (C) 2015-2022 The Fluent Bit Authors
* Fluent Bit is a CNCF sub-project under the umbrella of Fluentd
* https://fluentbit.io

[2023/08/02 15:44:22] [error] [config] could not configure property 'dummy' on input plugin with section name 'dummy'
[2023/08/02 15:44:22] [ info] [fluent bit] version=2.1.8, commit=1d83649441, pid=766101
[2023/08/02 15:44:22] [ info] [storage] ver=1.2.0, type=memory, sync=normal, checksum=off, max_chunks_up=128
[2023/08/02 15:44:22] [ info] [cmetrics] version=0.6.3
[2023/08/02 15:44:22] [ info] [ctraces ] version=0.3.1
[2023/08/02 15:44:22] [ info] [input:dummy:dummy.0] initializing
[2023/08/02 15:44:22] [ info] [input:dummy:dummy.0] storage_strategy='memory' (memory only)
[2023/08/02 15:44:22] [ info] [sp] stream processor started
[2023/08/02 15:44:22] [ info] [output:stdout:stdout.0] worker #0 started
{"date":1691005462.537879,"message":"dummy"}
{"date":1691005463.537903,"message":"dummy"}
^C[2023/08/02 15:44:24] [engine] caught signal (SIGINT)
[2023/08/02 15:44:24] [ warn] [engine] service will shutdown in max 5 seconds
[2023/08/02 15:44:24] [ info] [input] pausing dummy.0
{"date":1691005464.537889,"message":"dummy"}
[2023/08/02 15:44:25] [ info] [engine] service has stopped (0 pending tasks)
[2023/08/02 15:44:25] [ info] [input] pausing dummy.0
[2023/08/02 15:44:25] [ info] [output:stdout:stdout.0] thread worker #0 stopping...
[2023/08/02 15:44:25] [ info] [output:stdout:stdout.0] thread worker #0 stopped

This is the error that is reported in this case:

[2023/08/02 15:44:22] [error] [config] could not configure property 'dummy' on input plugin with section name 'dummy'

Fluent-bit continues on, but this is the default behaviour for any and all invalid properties.


Enter [N/A] in the box, if an item is not applicable to your change.

Testing
Before we can approve your change; please submit the following in a comment:

  • Example configuration file for the change
  • Debug log output from testing the change
  • Attached Valgrind output that shows no leaks or memory corruption was found

If this is a change to packaging of containers or native binaries then please confirm it works for all targets.

  • Run local packaging test showing all targets (including any new ones) build.
  • Set ok-package-test label to test for all targets (requires maintainer to do).

Documentation

  • Documentation required for this feature

Backporting

  • Backport to latest stable release.

Fluent Bit is licensed under Apache 2.0, by submitting this pull request I understand that this code will be released under the terms of that license.


@pwhelan pwhelan temporarily deployed to pr August 2, 2023 19:55 — with GitHub Actions Inactive
@pwhelan pwhelan temporarily deployed to pr August 2, 2023 19:55 — with GitHub Actions Inactive
@pwhelan pwhelan temporarily deployed to pr August 2, 2023 19:55 — with GitHub Actions Inactive
@pwhelan pwhelan temporarily deployed to pr August 2, 2023 20:25 — with GitHub Actions Inactive
src/flb_config.c Outdated Show resolved Hide resolved
@pwhelan pwhelan temporarily deployed to pr August 9, 2023 14:14 — with GitHub Actions Inactive
@pwhelan pwhelan temporarily deployed to pr August 9, 2023 14:14 — with GitHub Actions Inactive
@pwhelan pwhelan temporarily deployed to pr August 9, 2023 14:14 — with GitHub Actions Inactive
@pwhelan pwhelan temporarily deployed to pr August 9, 2023 14:46 — with GitHub Actions Inactive
@pwhelan pwhelan merged commit 4fab746 into master Aug 18, 2023
@pwhelan pwhelan deleted the pwhelan-fix-bad-config-types branch August 18, 2023 21:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants