faust v1.9.0 not working with sasl authentication

[jabonte]

Steps to reproduce

configure faust to use sasl

app = faust.App(
,
broker=broker,
key_serializer='json',
value_serializer='json',
broker_credentials=faust.SASLCredentials(
username=,
password=,
)
)

Expected behavior

That it works.
(It works as expected with v1.8.0)

Actual behavior

program dies with a AssertionError during startup of the worker

Full traceback

[2019-11-06 14:27:33,119] [5519] [ERROR] [^Worker]: Error: AssertionError("yield from wasn't used with future",)
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/mode/worker.py", line 267, in execute_from_commandline
self.loop.run_until_complete(self._starting_fut)
File "/usr/lib64/python3.6/asyncio/base_events.py", line 484, in run_until_complete
return future.result()
File "/usr/local/lib/python3.6/site-packages/mode/services.py", line 727, in start
await self._default_start()
File "/usr/local/lib/python3.6/site-packages/mode/services.py", line 734, in _default_start
await self._actually_start()
File "/usr/local/lib/python3.6/site-packages/mode/services.py", line 758, in _actually_start
await child.maybe_start()
File "/usr/local/lib/python3.6/site-packages/mode/services.py", line 786, in maybe_start
await self.start()
File "/usr/local/lib/python3.6/site-packages/mode/services.py", line 727, in start
await self._default_start()
File "/usr/local/lib/python3.6/site-packages/mode/services.py", line 734, in _default_start
await self._actually_start()
File "/usr/local/lib/python3.6/site-packages/mode/services.py", line 758, in _actually_start
await child.maybe_start()
File "/usr/local/lib/python3.6/site-packages/mode/services.py", line 786, in maybe_start
await self.start()
File "/usr/local/lib/python3.6/site-packages/mode/services.py", line 727, in start
await self._default_start()
File "/usr/local/lib/python3.6/site-packages/mode/services.py", line 734, in _default_start
await self._actually_start()
File "/usr/local/lib/python3.6/site-packages/mode/services.py", line 751, in _actually_start
await self.on_start()
File "/usr/local/lib/python3.6/site-packages/faust/transport/drivers/aiokafka.py", line 742, in on_start
await producer.start()
File "/usr/local/lib/python3.6/site-packages/aiokafka/producer/producer.py", line 171, in start
await self.client.bootstrap()
File "/usr/local/lib/python3.6/site-packages/aiokafka/client.py", line 203, in bootstrap
version_hint=version_hint)
File "/usr/local/lib/python3.6/site-packages/aiokafka/conn.py", line 90, in create_conn
await conn.connect()
File "/usr/local/lib/python3.6/site-packages/aiokafka/conn.py", line 214, in connect
await self._do_sasl_handshake()
File "/usr/local/lib/python3.6/site-packages/aiokafka/conn.py", line 281, in _do_sasl_handshake
payload, expect_response = res
AssertionError: yield from wasn't used with future

Versions

CPython 3.6.8 (Linux x86_64)
faust=1.9.0
aiokafka=1.1.3 aiohttp=3.6.2
linux=3.10.0-957.27.2.el7.x86_64

[gvdmarck]

Same here.
A totally internal error (we only upgraded our packages) not even being responded/acknowledged in 3 months ? What is this ?

Full traceback:

Traceback (most recent call last):
File "/opt/conda/envs/app/lib/python3.7/site-packages/mode/worker.py", line 273, in execute_from_commandline
self.loop.run_until_complete(self._starting_fut)
File "/opt/conda/envs/app/lib/python3.7/asyncio/base_events.py", line 583, in run_until_complete
return future.result()
File "/opt/conda/envs/app/lib/python3.7/site-packages/mode/services.py", line 736, in start
await self._default_start()
File "/opt/conda/envs/app/lib/python3.7/site-packages/mode/services.py", line 743, in _default_start
await self._actually_start()
File "/opt/conda/envs/app/lib/python3.7/site-packages/mode/services.py", line 767, in _actually_start
await child.maybe_start()
File "/opt/conda/envs/app/lib/python3.7/site-packages/mode/services.py", line 795, in maybe_start
await self.start()
File "/opt/conda/envs/app/lib/python3.7/site-packages/mode/services.py", line 736, in start
await self._default_start()
File "/opt/conda/envs/app/lib/python3.7/site-packages/mode/services.py", line 743, in _default_start
await self._actually_start()
File "/opt/conda/envs/app/lib/python3.7/site-packages/mode/services.py", line 767, in _actually_start
await child.maybe_start()
File "/opt/conda/envs/app/lib/python3.7/site-packages/mode/services.py", line 795, in maybe_start
await self.start()
File "/opt/conda/envs/app/lib/python3.7/site-packages/mode/services.py", line 736, in start
await self._default_start()
File "/opt/conda/envs/app/lib/python3.7/site-packages/mode/services.py", line 743, in _default_start
await self._actually_start()
File "/opt/conda/envs/app/lib/python3.7/site-packages/mode/services.py", line 760, in _actually_start
await self.on_start()
File "/opt/conda/envs/app/lib/python3.7/site-packages/faust/transport/drivers/aiokafka.py", line 760, in on_start
await producer.start()
File "/opt/conda/envs/app/lib/python3.7/site-packages/aiokafka/producer/producer.py", line 171, in start
await self.client.bootstrap()
File "/opt/conda/envs/app/lib/python3.7/site-packages/aiokafka/client.py", line 203, in bootstrap
version_hint=version_hint)
File "/opt/conda/envs/app/lib/python3.7/site-packages/aiokafka/conn.py", line 90, in create_conn
await conn.connect()
File "/opt/conda/envs/app/lib/python3.7/site-packages/aiokafka/conn.py", line 214, in connect
await self._do_sasl_handshake()
File "/opt/conda/envs/app/lib/python3.7/site-packages/aiokafka/conn.py", line 281, in _do_sasl_handshake
payload, expect_response = res
RuntimeError: await wasn't used with future

Versions

CPython 3.7.6 (Linux x86_64)
faust=1.10.1
aiokafka=1.1.3 aiohttp=3.6.2
linux=3.10.0-957.27.2.el7.x86_64

[gvdmarck]

Is robinhood still maintainng faust ? Or is this repo dead ? The fix to this issue is to update your aiokafka lib so that it is in sync with the apache one, the pull request has been there for 3 months : robinhood/aiokafka#18

[hakwerk]

I can confirm that this issue still exists in Faust 1.10.2 and that the pull request fixes it.
I applied the fixes from the PR into my virtualenv and it solves the issue.

[gvdmarck]

@ask ? Can we also have a comment on the faust follow-up status ?

[gvdmarck]

Even after emailing Robinhood directly, still no reactions what so ever.

Time to change client libraries for our applications. Does any one have returns on usage of python-kafka, pykafka or confluent-kafka ?

[deed02392]

I don't think it's a big enough of a deal to switch libraries for me. Is there a PR to fix this? Maybe just maintain your own forked package with the fix in it?

[ask]

aiokafka 1.1.6 is on PyPI with the fix

If anyone here is willing to help triage issues that would be a huge help to surface fixes for important issues sooner.

[chase2981]

I would definitely strongly consider doing my part in helping maintain this repo seems it is impressive to all who come across it; however, from what I gather this repo/package is no longer maintained. (#701 (comment) | #468 (comment)) If this is true, which shouldn't even be an uncertainty, but assuming this is true, then the least we could do is put up a "NO LONGER MAINTAINED" banner on the top of the docs and the README.md, so that way others don't go down this same path thinking this is actually a legit solution only to later discover nobody is actively facilitating the proper maintenance of it.

[chase2981]

For my company, the very fact that this repo's maintenance is questionable is a huge red flag. We are currently trying to get away from another open source kinesis streaming lib where the same thing happened, it became un-maintained a few years back and now we simply will not tolerate moving to any streaming lib whose maintenance is even somewhat questionable. (https://github.com/LeoPlatform) Our entire organization's data is on the line. It's been a huge hassle across our entire organization. That's why we were moving to kafka and why we started looking into faust in the first place.