Pinning sqlalchemy conflicts with airflow 2.4.x

[jdu]

datahub/metadata-ingestion-modules/airflow-plugin/setup.py

Line 32 in 928f294

"sqlalchemy==1.3.24",

The requirement being pinned for sqlalchemy in the acryl-datahub-airflow-plugin conflicts with airflows own dependency on sqlalchemy.

NFO: pip is looking at multiple versions of sqlalchemy to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of acryl-datahub-airflow-plugin to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of python-dateutil to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of psycopg2-binary to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of fire to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of feedparser to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of chardet to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of apache-airflow-providers-slack to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of <Python from Requires-Python> to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of apache-airflow-providers-cncf-kubernetes to determine which version is compatible with other requirements. This could take a while.
ERROR: Cannot install -r /opt/setup/requirements.txt (line 8) and apache-airflow-providers-cncf-kubernetes because these package versions have conflicting dependencies.

The conflict is caused by:
    acryl-datahub-airflow-plugin 0.8.45.2 depends on sqlalchemy==1.3.24
    apache-airflow 2.4.1 depends on sqlalchemy>=1.4
    acryl-datahub-airflow-plugin 0.8.45.2 depends on sqlalchemy==1.3.24
    apache-airflow 2.4.0 depends on sqlalchemy>=1.4
    acryl-datahub-airflow-plugin 0.8.45.2 depends on sqlalchemy==1.3.24
    apache-airflow 2.3.4 depends on sqlalchemy>=1.4
    acryl-datahub-airflow-plugin 0.8.45.2 depends on sqlalchemy==1.3.24
    apache-airflow 2.3.3 depends on sqlalchemy>=1.4
    acryl-datahub-airflow-plugin 0.8.45.2 depends on sqlalchemy==1.3.24
    apache-airflow 2.3.2 depends on sqlalchemy<1.4.10 and >=1.4
    acryl-datahub-airflow-plugin 0.8.45.2 depends on sqlalchemy==1.3.24
    apache-airflow 2.3.1 depends on sqlalchemy<1.4.10 and >=1.4
    acryl-datahub-airflow-plugin 0.8.45.2 depends on sqlalchemy==1.3.24
    apache-airflow 2.3.0 depends on sqlalchemy<1.4.10 and >=1.4

To fix this you could try to:
1. loosen the range of package versions you've specified
2. remove package versions to allow pip attempt to solve the dependency conflict

ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/topics/dependency-resolution/#dealing-with-dependency-conflicts

[jdu]

I got this to install by custom compiling the plugin without the sqlalchemy version pinned, however, the plugin is no longer compatible with airflow after the latest airflow updates. See #6145

[github-actions]

This issue is stale because it has been open for 30 days with no activity. If you believe this is still an issue on the latest DataHub release please leave a comment with the version that you tested it with. If this is a question/discussion please head to https://slack.datahubproject.io. For feature requests please use https://feature-requests.datahubproject.io