Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[airflow] Extend names moved from core to provider (AIR303) #15216

Merged
merged 2 commits into from
Jan 2, 2025

Conversation

Lee-W
Copy link
Contributor

@Lee-W Lee-W commented Jan 2, 2025

Summary

Many core Airflow features have been deprecated and moved to Airflow Providers since users might need to install an additional package (e.g., apache-airflow-provider-fab==1.0.0); a separate rule (AIR303) is created for this.

  • airflow.kubernetes.kubernetes_helper_functions.add_pod_suffixairflow.providers.cncf.kubernetes.kubernetes_helper_functions.add_pod_suffix
  • airflow.kubernetes.kubernetes_helper_functions.annotations_for_logging_task_metadataairflow.providers.cncf.kubernetes.kubernetes_helper_functions.annotations_for_logging_task_metadata
  • airflow.kubernetes.kubernetes_helper_functions.annotations_to_keyairflow.providers.cncf.kubernetes.kubernetes_helper_functions.annotations_to_key
  • airflow.kubernetes.kubernetes_helper_functions.create_pod_idairflow.providers.cncf.kubernetes.kubernetes_helper_functions.create_pod_id
  • airflow.kubernetes.kubernetes_helper_functions.get_logs_task_metadataairflow.providers.cncf.kubernetes.kubernetes_helper_functions.get_logs_task_metadata
  • airflow.kubernetes.kubernetes_helper_functions.rand_strairflow.providers.cncf.kubernetes.kubernetes_helper_functions.rand_str
  • airflow.kubernetes.pod.Portkubernetes.client.models.V1ContainerPort
  • airflow.kubernetes.pod.Resourceskubernetes.client.models.V1ResourceRequirements
  • airflow.kubernetes.pod_launcher.PodLauncherairflow.providers.cncf.kubernetes.pod_launcher.PodLauncher
  • airflow.kubernetes.pod_launcher.PodStatusairflow.providers.cncf.kubernetes.pod_launcher.PodStatus
  • airflow.kubernetes.pod_launcher_deprecated.PodLauncherairflow.providers.cncf.kubernetes.pod_launcher_deprecated.PodLauncher
  • airflow.kubernetes.pod_launcher_deprecated.PodStatusairflow.providers.cncf.kubernetes.pod_launcher_deprecated.PodStatus
  • airflow.kubernetes.pod_launcher_deprecated.get_kube_clientairflow.providers.cncf.kubernetes.kube_client.get_kube_client
  • airflow.kubernetes.pod_launcher_deprecated.PodDefaultsairflow.providers.cncf.kubernetes.pod_generator_deprecated.PodDefaults
  • airflow.kubernetes.pod_runtime_info_env.PodRuntimeInfoEnvkubernetes.client.models.V1EnvVar
  • airflow.kubernetes.volume.Volumekubernetes.client.models.V1Volume
  • airflow.kubernetes.volume_mount.VolumeMountkubernetes.client.models.V1VolumeMount
  • airflow.kubernetes.k8s_model.K8SModelairflow.providers.cncf.kubernetes.k8s_model.K8SModel
  • airflow.kubernetes.k8s_model.append_to_podairflow.providers.cncf.kubernetes.k8s_model.append_to_pod
  • airflow.kubernetes.kube_client._disable_verify_sslairflow.kubernetes.airflow.providers.cncf.kubernetes.kube_client._disable_verify_ssl
  • airflow.kubernetes.kube_client._enable_tcp_keepaliveairflow.kubernetes.airflow.providers.cncf.kubernetes.kube_client._enable_tcp_keepalive
  • airflow.kubernetes.kube_client.get_kube_clientairflow.kubernetes.airflow.providers.cncf.kubernetes.kube_client.get_kube_client
  • airflow.kubernetes.pod_generator.datetime_to_label_safe_datestringairflow.providers.cncf.kubernetes.pod_generator.datetime_to_label_safe_datestring
  • airflow.kubernetes.pod_generator.extend_object_fieldairflow.kubernetes.airflow.providers.cncf.kubernetes.pod_generator.extend_object_field
  • airflow.kubernetes.pod_generator.label_safe_datestring_to_datetimeairflow.providers.cncf.kubernetes.pod_generator.label_safe_datestring_to_datetime
  • airflow.kubernetes.pod_generator.make_safe_label_valueairflow.providers.cncf.kubernetes.pod_generator.make_safe_label_value
  • airflow.kubernetes.pod_generator.merge_objectsairflow.providers.cncf.kubernetes.pod_generator.merge_objects
  • airflow.kubernetes.pod_generator.PodGeneratorairflow.providers.cncf.kubernetes.pod_generator.PodGenerator
  • airflow.kubernetes.pod_generator.PodGeneratorDeprecatedairflow.providers.cncf.kubernetes.pod_generator.PodGenerator
  • airflow.kubernetes.pod_generator.PodDefaultsairflow.providers.cncf.kubernetes.pod_generator_deprecated.PodDefaults
  • airflow.kubernetes.pod_generator.add_pod_suffixairflow.providers.cncf.kubernetes.kubernetes_helper_functions.add_pod_suffix
  • airflow.kubernetes.pod_generator.rand_strairflow.providers.cncf.kubernetes.kubernetes_helper_functions.rand_str
  • airflow.kubernetes.pod_generator_deprecated.make_safe_label_valueairflow.providers.cncf.kubernetes.pod_generator_deprecated.make_safe_label_value
  • airflow.kubernetes.pod_generator_deprecated.PodDefaultsairflow.providers.cncf.kubernetes.pod_generator_deprecated.PodDefaults
  • airflow.kubernetes.pod_generator_deprecated.PodGeneratorairflow.providers.cncf.kubernetes.pod_generator_deprecated.PodGenerator
  • airflow.kubernetes.secret.Secretairflow.providers.cncf.kubernetes.secret.Secret
  • airflow.kubernetes.secret.K8SModelairflow.providers.cncf.kubernetes.k8s_model.K8SModel

Test Plan

A test fixture has been included for the rule.

* `airflow.kubernetes.kubernetes_helper_functions.add_pod_suffix` → `airflow.providers.cncf.kubernetes.kubernetes_helper_functions.add_pod_suffix`
* `airflow.kubernetes.kubernetes_helper_functions.annotations_for_logging_task_metadata` → `airflow.providers.cncf.kubernetes.kubernetes_helper_functions.annotations_for_logging_task_metadata`
* `airflow.kubernetes.kubernetes_helper_functions.annotations_to_key` → `airflow.providers.cncf.kubernetes.kubernetes_helper_functions.annotations_to_key`
* `airflow.kubernetes.kubernetes_helper_functions.create_pod_id` → `airflow.providers.cncf.kubernetes.kubernetes_helper_functions.create_pod_id`
* `airflow.kubernetes.kubernetes_helper_functions.get_logs_task_metadata` → `airflow.providers.cncf.kubernetes.kubernetes_helper_functions.get_logs_task_metadata`
* `airflow.kubernetes.kubernetes_helper_functions.rand_str` → `airflow.providers.cncf.kubernetes.kubernetes_helper_functions.rand_str`
* `airflow.kubernetes.pod.Port` → `kubernetes.client.models.V1ContainerPort`
* `airflow.kubernetes.pod.Resources` → `kubernetes.client.models.V1ResourceRequirements`
* `airflow.kubernetes.pod_launcher.PodLauncher` → `airflow.providers.cncf.kubernetes.pod_launcher.PodLauncher`
* `airflow.kubernetes.pod_launcher.PodStatus` → `airflow.providers.cncf.kubernetes.pod_launcher.PodStatus`
* `airflow.kubernetes.pod_launcher_deprecated.PodLauncher` → `airflow.providers.cncf.kubernetes.pod_launcher_deprecated.PodLauncher`
* `airflow.kubernetes.pod_launcher_deprecated.PodStatus` → `airflow.providers.cncf.kubernetes.pod_launcher_deprecated.PodStatus`
* `airflow.kubernetes.pod_launcher_deprecated.get_kube_client` → `airflow.providers.cncf.kubernetes.kube_client.get_kube_client`
* `airflow.kubernetes.pod_launcher_deprecated.PodDefaults` → `airflow.providers.cncf.kubernetes.pod_generator_deprecated.PodDefaults`
* `airflow.kubernetes.pod_runtime_info_env.PodRuntimeInfoEnv` → `kubernetes.client.models.V1EnvVar`
* `airflow.kubernetes.volume.Volume` → `kubernetes.client.models.V1Volume`
* `airflow.kubernetes.volume_mount.VolumeMount` → `kubernetes.client.models.V1VolumeMount`
* `airflow.kubernetes.k8s_model.K8SModel` → `airflow.providers.cncf.kubernetes.k8s_model.K8SModel`
* `airflow.kubernetes.k8s_model.append_to_pod` → `airflow.providers.cncf.kubernetes.k8s_model.append_to_pod`
* `airflow.kubernetes.kube_client._disable_verify_ssl` → `airflow.kubernetes.airflow.providers.cncf.kubernetes.kube_client._disable_verify_ssl`
* `airflow.kubernetes.kube_client._enable_tcp_keepalive` → `airflow.kubernetes.airflow.providers.cncf.kubernetes.kube_client._enable_tcp_keepalive`
* `airflow.kubernetes.kube_client.get_kube_client` → `airflow.kubernetes.airflow.providers.cncf.kubernetes.kube_client.get_kube_client`
* `airflow.kubernetes.pod_generator.datetime_to_label_safe_datestring` → `airflow.providers.cncf.kubernetes.pod_generator.datetime_to_label_safe_datestring`
* `airflow.kubernetes.pod_generator.extend_object_field` → `airflow.kubernetes.airflow.providers.cncf.kubernetes.pod_generator.extend_object_field`
* `airflow.kubernetes.pod_generator.label_safe_datestring_to_datetime` → `airflow.providers.cncf.kubernetes.pod_generator.label_safe_datestring_to_datetime`
* `airflow.kubernetes.pod_generator.make_safe_label_value` → `airflow.providers.cncf.kubernetes.pod_generator.make_safe_label_value`
* `airflow.kubernetes.pod_generator.merge_objects` → `airflow.providers.cncf.kubernetes.pod_generator.merge_objects`
* `airflow.kubernetes.pod_generator.PodGenerator` → `airflow.providers.cncf.kubernetes.pod_generator.PodGenerator`
* `airflow.kubernetes.pod_generator.PodGeneratorDeprecated` → `airflow.providers.cncf.kubernetes.pod_generator.PodGenerator`
* `airflow.kubernetes.pod_generator.PodDefaults` → `airflow.providers.cncf.kubernetes.pod_generator_deprecated.PodDefaults`
* `airflow.kubernetes.pod_generator.add_pod_suffix` → `airflow.providers.cncf.kubernetes.kubernetes_helper_functions.add_pod_suffix`
* `airflow.kubernetes.pod_generator.rand_str` → `airflow.providers.cncf.kubernetes.kubernetes_helper_functions.rand_str`
* `airflow.kubernetes.pod_generator_deprecated.make_safe_label_value` → `airflow.providers.cncf.kubernetes.pod_generator_deprecated.make_safe_label_value`
* `airflow.kubernetes.pod_generator_deprecated.PodDefaults` → `airflow.providers.cncf.kubernetes.pod_generator_deprecated.PodDefaults`
* `airflow.kubernetes.pod_generator_deprecated.PodGenerator` → `airflow.providers.cncf.kubernetes.pod_generator_deprecated.PodGenerator`
* `airflow.kubernetes.secret.Secret` → `airflow.providers.cncf.kubernetes.secret.Secret`
* `airflow.kubernetes.secret.K8SModel` → `airflow.providers.cncf.kubernetes.k8s_model.K8SModel`
Copy link
Contributor

github-actions bot commented Jan 2, 2025

ruff-ecosystem results

Linter (stable)

✅ ecosystem check detected no linter changes.

Linter (preview)

✅ ecosystem check detected no linter changes.

@Lee-W
Copy link
Contributor Author

Lee-W commented Jan 2, 2025

I think this is the last PR that contains this many rules

Copy link
Member

@dhruvmanila dhruvmanila left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you! I think we should do a similar refactor as I did in 86bdc2e to simplify the match expression. I can do that as a follow-up, shouldn't be more than a couple of minutes.

@dhruvmanila dhruvmanila changed the title [airflow]: extend names moved from core to provider (AIR303) [airflow] Extend names moved from core to provider (AIR303) Jan 2, 2025
@dhruvmanila dhruvmanila added rule Implementing or modifying a lint rule preview Related to preview mode features labels Jan 2, 2025
@dhruvmanila dhruvmanila merged commit f8c9665 into astral-sh:main Jan 2, 2025
21 checks passed
@Lee-W
Copy link
Contributor Author

Lee-W commented Jan 2, 2025

Would love to try it! Will create a PR later today

@Lee-W
Copy link
Contributor Author

Lee-W commented Jan 2, 2025

just created #15220

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
preview Related to preview mode features rule Implementing or modifying a lint rule
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants