Panic in decoding when log_schema.message_key is None

[spencergilbert]

A note for the community

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Problem

It is supported to provide "" or None as a path to a key, and can be used by the other members of the log_schema configuration. It's not likely to be a useful (or often used) setting to not include the "message" but panicking on it is probably not desirable.

Especially given the log namespace feature where "message" isn't necessarily going to exist in the Vector namespace, while the Legacy namespace requires a "message" key.

Configuration

data_dir = "/var/lib/vector/"

[log_schema]
message_key = ""

[sources.source0]
format = "json"
type = "demo_logs"

[sinks.sink0]
inputs = ["source0"]
target = "stdout"
type = "console"

[sinks.sink0.encoding]
codec = "json"

Version

vector 0.31.0 (x86_64-apple-darwin 0f13b22 2023-07-06 13:52:34.591204470)

Debug Output

❯ RUST_BACKTRACE=full vector -vvv -c /tmp/vector.toml
2023-07-10T21:42:17.529832Z DEBUG vector::app: Internal log rate limit configured. internal_log_rate_secs=10
2023-07-10T21:42:17.529928Z  INFO vector::app: Log level is enabled. level="vector=trace,codec=trace,vrl=trace,file_source=trace,tower_limit=trace,rdkafka=trace,buffers=trace,lapin=trace,kube=trace"
2023-07-10T21:42:17.531093Z  INFO vector::app: Loading configs. paths=["/tmp/vector.toml"]
2023-07-10T21:42:17.534377Z DEBUG vector::config::loading: No secret placeholder found, skipping secret resolution.
thread 'main' panicked at 'valid message key: InvalidPathSyntax { path: "" }', lib/codecs/src/decoding/format/bytes.rs:40:63
stack backtrace:
   0:        0x105ada995 - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::hae51cb91d407e2ef
   1:        0x10504231b - core::fmt::write::h746bc0969202388b
   2:        0x105aa6cac - std::io::Write::write_fmt::h4098c2c7437a0bd7
   3:        0x105ae1b0a - std::sys_common::backtrace::print::h8360bf0158e89b36
   4:        0x105ae1646 - std::panicking::default_hook::{{closure}}::hedf04c568eb6e0bc
   5:        0x105ae28a7 - std::panicking::rust_panic_with_hook::h22edd02828aff274
   6:        0x105ae2255 - std::panicking::begin_panic_handler::{{closure}}::h6895f44a9b7e2caa
   7:        0x105ae21b9 - std::sys_common::backtrace::__rust_end_short_backtrace::h013519e9b77978c5
   8:        0x105ae21a2 - _rust_begin_unwind
   9:        0x108d20743 - core::panicking::panic_fmt::h2a51b60c12d11322
  10:        0x108d20ab5 - core::result::unwrap_failed::h52a6a55d409027df
  11:        0x104f80a5d - codecs::decoding::format::bytes::BytesDeserializerConfig::schema_definition::h90aad0be4c30395c
  12:        0x106d08ad0 - <vector::sources::demo_logs::DemoLogsConfig as vector::config::source::SourceConfig>::outputs::h3291a7247627c659
  13:        0x1079caa73 - vector::config::compiler::expand_globs::h0ac8f72c1499c94b
  14:        0x1079c1bf9 - vector::config::compiler::compile::hd61923fd1affc377
  15:        0x108190b9a - vector::config::loading::load_from_paths_with_provider_and_secrets::{{closure}}::haba293e1d1bf50f1
  16:        0x1081d32fb - vector::app::ApplicationConfig::from_opts::{{closure}}::h641844d683f84679
  17:        0x1081c7c7d - vector::app::Application::prepare_from_opts::h2f668f7b6f6d18a2
  18:        0x104a42b0e - vector::main::he09f040946d01171
  19:        0x104a41006 - std::sys_common::backtrace::__rust_begin_short_backtrace::h318e3c3010d9bdb0
  20:        0x104a45b22 - _main

Example Data

No response

Additional Context

No response

References

No response

[pront]

Repro steps

The config:

log_schema:
  source_type_key: ""

sources:
  stdin:
    type: stdin

sinks:
  influx:
    type: influxdb_logs
    inputs:
      - stdin
    endpoint: http://localhost:8086/
    org: org
    bucket: org-app
    database: db0
    measurement: "vector-logs"

cargo run -- -c path/to/above/config.yaml