Log level filtering not working as expected.

[whynowy]

Describe the bug

When there's logs with corresponding log level, it displays previous logs with alert.

How to reproduce

Create a simple pipeline and check the sink pod logs, play around the log levels.

• Kubernetes: [e.g. v1.18.6]
• Numaflow: [e.g. v0.5.1]

---

Message from the maintainers:

Impacted by this bug? Give it a 👍. We often sort issues this way to know what to prioritize.

For quick help and support, join our slack channel.

[veds-g]

The reason filtering does not work here is because unlike source and cat vertex, sink does not have an object in logs. The log level is extracted from the log object for source and cat, but for sink it is just a string without level information. Can we generalise the log format so its easy to apply the filters or should I handle it separately for this sink log string (provided it remains consistent).