[3007.x] Address minion log flood from zeromq

[dismari]

What does this PR do?

Checks if callback object in zeromq consume method is not None when handling logging as error.

What issues does this PR fix or reference?

Fixes:
#65898
and
#66177

Previous Behavior

Every minion will see many log errors for an:
TypeError: object NoneType can't be used in 'await' expression
from zeromq callback exception handling

New Behavior

The consume method will now only log an error if there is an exception from with callback.

Merge requirements satisfied?

[NOTICE] Bug fixes or features added to Salt require tests.

• Docs
• Changelog - https://docs.saltproject.io/en/master/topics/development/changelog.html
• Tests written/updated

Commits signed with GPG?

Yes/No

Please review Salt's Contributing Guide for best practices.

See GitHub's page on GPG signing for more information about signing commits with GPG.

[twangboy]

Looks like you have some pre-commit errors

[SwimGeek]

Hi

Is there an ETA for this fix to be available (on Debian)?

It's causing lots of our minions to disconnect from master.

[merlinz01]

As I understand it from doing some debugging, the problem is not that the callback is None, but that the callback is a non-async function that is used in an await statement. See my comment, with example. So this PR as it is does not fix the issue.

Here's my understanding of coroutines as it relates to this bug:

When you write await some_function(), that means "call some_function and then await the return value". If some_function is defined with async def, then some_function() immediately returns a coroutine object that can be awaited, which is how this is supposed to work. However, if some_function is defined with a simple def, then it returns None or whatever the function decides to return. When Python gets to the await-ing part, all it has is the return value, which in this case is None. The None object can't be awaited, so it throws the error described.

[dismari]

My 'fix' (as I described in my PR) is not really fix and rather just removing the unnecessary log which floods the salt minion's logs. As far as I can tell, the code is operational/functional and thus the logging is excessive. I would like someone to resolve the root of the issue but do not have the time/bandwidth (just trying to reduce unnecessary logs in my salt minions).

[dwoz]

Please hold off on merging this until I have a chance to look at it in more detail.

[dwoz]

superseded by #66335