Kafka source is disabled in the datahub-ingestion docker image for the tag v0.9.5 and later

[daha]

Describe the bug
The kafka source is disabled in the datahub-ingestion image for the tag v0.9.5.

This was probably introduced by #6783 which imports ResourceType,

datahub/metadata-ingestion/src/datahub/ingestion/source/kafka.py

Lines 11 to 15 in 0fded4d

	from confluent_kafka.admin import (
	AdminClient,
	ConfigEntry,
	ConfigResource,
	ResourceType,

, which was introduced in confluent-kafka version 1.9.0, while the docker image has confluent-kafka version 1.8.2.

To Reproduce
Just run datahub check plugins in the docker image

$ docker run --rm -it acryldata/datahub-ingestion:v0.9.5 check plugins | grep disabled
kafka          (disabled)
For details on why a plugin is disabled, rerun with '--verbose'
If a plugin is disabled, try running: pip install 'acryl-datahub[<plugin>]'

Expected behavior
kafka source should be enabled by default.

Desktop (please complete the following information):
OS: Linux

Additional context
Partial stacktrace when trying to use the kafka source with the docker image acryldata/datahub-ingestion:v0.9.5:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/datahub/ingestion/api/registry.py", line 97, in _ensure_not_lazy
    plugin_class = import_path(path)
  File "/usr/local/lib/python3.10/site-packages/datahub/ingestion/api/registry.py", line 32, in import_path
    item = importlib.import_module(module_name)
  File "/usr/local/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/usr/local/lib/python3.10/site-packages/datahub/ingestion/source/kafka.py", line 11, in <module>
    from confluent_kafka.admin import (
ImportError: cannot import name 'ResourceType' from 'confluent_kafka.admin' (/usr/local/lib/python3.10/site-packages/confluent_kafka/admin/__init__.py)

[chriscollins3456]

hey @daha ! just wanted to let you know this is on our radar and something we're looking into

[github-actions]

This issue is stale because it has been open for 30 days with no activity. If you believe this is still an issue on the latest DataHub release please leave a comment with the version that you tested it with. If this is a question/discussion please head to https://slack.datahubproject.io. For feature requests please use https://feature-requests.datahubproject.io

[github-actions]

This issue was closed because it has been inactive for 30 days since being marked as stale.