Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[airflow]: extend names moved from core to provider (AIR303) #15159

Merged
merged 1 commit into from
Dec 27, 2024

Conversation

Lee-W
Copy link
Contributor

@Lee-W Lee-W commented Dec 27, 2024

Summary

Many core Airflow features have been deprecated and moved to Airflow Providers since users might need to install an additional package (e.g., apache-airflow-provider-fab==1.0.0); a separate rule (AIR303) is created for this.

  • apache-airflow-providers-common-sql >= 1.0.0
    • airflow.hooks.dbapi.ConnectorProtocolairflow.providers.common.sql.hooks.sql.ConnectorProtocol
    • airflow.hooks.dbapi.DbApiHookairflow.providers.common.sql.hooks.sql.DbApiHook
  • apache-airflow-providers-cncf-kubernetes >= 7.4.0
    • airflow.executors.kubernetes_executor_types.ALL_NAMESPACESairflow.providers.cncf.kubernetes.executors.kubernetes_executor_types.ALL_NAMESPACES
    • airflow.executors.kubernetes_executor_types.POD_EXECUTOR_DONE_KEYairflow.providers.cncf.kubernetes.executors.kubernetes_executor_types.POD_EXECUTOR_DONE_KEY
  • apache-airflow-providers-celery >= 3.3.0
    • airflow.executors.celery_executor.appairflow.providers.celery.executors.celery_executor_utils.app
    • airflow.config_templates.default_celery.DEFAULT_CELERY_CONFIGairflow.providers.celery.executors.default_celery.DEFAULT_CELERY_CONFIG
    • airflow.executors.celery_kubernetes_executor.CeleryKubernetesExecutorairflow.providers.celery.executors.celery_kubernetes_executor
    • airflow.executors.celery_executor.CeleryExecutorairflow.providers.celery.executors.celery_executor.CeleryExecutor
  • apache-airflow-providers-apache-hive >= 1.0.0
    • airflow.hooks.hive_hooks.HIVE_QUEUE_PRIORITIESairflow.providers.apache.hive.hooks.hive.HIVE_QUEUE_PRIORITIES
  • apache-airflow-providers-apache-hive >= 5.1.0
    • airflow.macros.hive.closest_ds_partitionairflow.providers.apache.hive.macros.hive.closest_ds_partition
      airflow.executors.kubernetes_executor.KubernetesExecutor → `airflow.providers.cncf.kubernetes.executors.kubernetes_executor.KubernetesE
  • apache-airflow-providers-cncf-kubernetes >= 7.4.0
    • airflow.executors.kubernetes_executor_types.ALL_NAMESPACES → `airflow.providers.cncf.kubernetes.executors.kubernetes_executor_types.ALL
    • airflow.executors.kubernetes_executor_types.POD_EXECUTOR_DONE_KEY → `airflow.providers.cncf.kubernetes.executors.kubernetes_executor_ty
    • airflow.executors.kubernetes_executor_utils.AirflowKubernetesScheduler → `airflow.providers.cncf.kubernetes.executors.kubernetes_execut
    • airflow.executors.kubernetes_executor_utils.KubernetesJobWatcher → `airflow.providers.cncf.kubernetes.executors.kubernetes_executor_uti
    • airflow.executors.kubernetes_executor_utils.ResourceVersion → `airflow.providers.cncf.kubernetes.executors.kubernetes_executor_utils.Re
    • airflow.executors.local_kubernetes_executor.LocalKubernetesExecutorairflow.providers.cncf.kubernetes.executors.LocalKubernetesExecu airflow.macros.hive.max_partitionairflow.providers.apache.hive.macros.hive.max_partition`
  • apache-airflow-providers-daskexecutor >= 1.0.0
    • airflow.executors.dask_executor.DaskExecutorairflow.providers.daskexecutor.executors.dask_executor.DaskExecutor

Test Plan

A test fixture has been included for the rule.

Copy link
Contributor

github-actions bot commented Dec 27, 2024

ruff-ecosystem results

Linter (stable)

✅ ecosystem check detected no linter changes.

Linter (preview)

✅ ecosystem check detected no linter changes.

* apache-airflow-providers-common-sql >= 1.0.0
    * `airflow.hooks.dbapi.ConnectorProtocol` → `airflow.providers.common.sql.hooks.sql.ConnectorProtocol`
    * `airflow.hooks.dbapi.DbApiHook` → `airflow.providers.common.sql.hooks.sql.DbApiHook`
* apache-airflow-providers-cncf-kubernetes >= 7.4.0
    * `airflow.executors.kubernetes_executor_types.ALL_NAMESPACES` → `airflow.providers.cncf.kubernetes.executors.kubernetes_executor_types.ALL_NAMESPACES`
    * `airflow.executors.kubernetes_executor_types.POD_EXECUTOR_DONE_KEY` → `airflow.providers.cncf.kubernetes.executors.kubernetes_executor_types.POD_EXECUTOR_DONE_KEY`
* apache-airflow-providers-celery == 3.3.0
    * `airflow.executors.celery_executor.app` → `airflow.providers.celery.executors.celery_executor_utils.app`
    * `airflow.config_templates.default_celery.DEFAULT_CELERY_CONFIG` → `airflow.providers.celery.executors.default_celery.DEFAULT_CELERY_CONFIG`
* apache-airflow-providers-apache-hive == 1.0.0
    * `airflow.hooks.hive_hooks.HIVE_QUEUE_PRIORITIES` → `airflow.providers.apache.hive.hooks.hive.HIVE_QUEUE_PRIORITIES`
* apache-airflow-providers-apache-hive == 5.1.0
    * `airflow.macros.hive.closest_ds_partition` → `airflow.providers.apache.hive.macros.hive.closest_ds_partition`
    * `airflow.macros.hive.max_partition` → `airflow.providers.apache.hive.macros.hive.max_partition`
@MichaReiser MichaReiser added rule Implementing or modifying a lint rule preview Related to preview mode features labels Dec 27, 2024
Copy link
Member

@MichaReiser MichaReiser left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice

@MichaReiser MichaReiser merged commit 2288cc7 into astral-sh:main Dec 27, 2024
21 checks passed
dcreager added a commit that referenced this pull request Dec 30, 2024
* main:
  Add all PEP-585 names to UP006 rule (#5454)
  [`flake8-simplify`] More precise inference for dictionaries (`SIM300`) (#15164)
  `@no_type_check` support (#15122)
  Visit PEP 764 inline `TypedDict`s' keys as non-type-expressions (#15073)
  [red-knot] Add diagnostic for invalid unpacking (#15086)
  [`flake8-use-pathlib`] Catch redundant joins in `PTH201` and avoid syntax errors (#15177)
  Update Rust crate glob to v0.3.2 (#15185)
  Update astral-sh/setup-uv action to v5 (#15193)
  Update dependency mdformat-mkdocs to v4.1.1 (#15192)
  Update Rust crate serde_with to v3.12.0 (#15191)
  Update NPM Development dependencies (#15190)
  Update pre-commit hook rhysd/actionlint to v1.7.5 (#15189)
  Update Rust crate syn to v2.0.93 (#15188)
  Update Rust crate serde to v1.0.217 (#15187)
  Update Rust crate quote to v1.0.38 (#15186)
  Update Rust crate compact_str to v0.8.1 (#15184)
  [`flake8-type-checking`] Disable TC006 & TC007 in stub files (#15179)
  Test explicit shadowing involving `def`s (#15174)
  Fix typo in `NameImport.qualified_name` docstring (#15170)
  [airflow]: extend names moved from core to provider (AIR303) (#15159)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
preview Related to preview mode features rule Implementing or modifying a lint rule
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants