Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove overlooked kubernetes/pod_generator_deprecated.py file #46377

Closed
wants to merge 65 commits into from

Conversation

insomnes
Copy link
Contributor

@insomnes insomnes commented Feb 3, 2025

providers/src/airflow/providers/cncf/kubernetes/pod_generator_deprecated.py was missed in the #43689

  • Delete unused and deprecated pod generator module
  • Remove some already deleted deprecated modules from tests presence exclusion

^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in newsfragments.

@insomnes
Copy link
Contributor Author

insomnes commented Feb 3, 2025

After looking up issues, PRs, tests, and code I am pretty sure this file was missed.

Should I add significant.rst about it (delete is backward incompatible)?

@eladkal
Copy link
Contributor

eladkal commented Feb 3, 2025

I am not sure if we can remove it now.
Should it wait for next major release? cc @romsharon98 @jedcunningham WDYT?

@insomnes
Copy link
Contributor Author

insomnes commented Feb 3, 2025

Sure, I'm not in a rush.
I was digging for another issue so I made this PR to not overlook it again.

I would be happy to adhere to the established release process. If any, tell me: should I close this one, make it WIP, or set the destination to another branch?

@insomnes
Copy link
Contributor Author

insomnes commented Feb 4, 2025

@romsharon98 @jedcunningham UP with the Elad's question above. I am not in a rush but I would like to know what to do with this one and remove this PR from my "working RAM" if it's not needed.

@insomnes insomnes force-pushed the remove-deprecated-pod-gen branch from e95da45 to 56c5a55 Compare February 4, 2025 17:40
pierrejeambrun and others added 19 commits February 6, 2025 21:15
Since we always have a standalone DAG processor now, we no longer need
the special logging config to have a separate log file for the
DagProcessorManager. Using LoggingMixin instead, all of the logs end up
on stdout, just like other Airflow components.
When PROD image is build using sources, it should use source
requireents rather than pip requirements - they might be conflicting
if the main providers are different than released prividers.
* Limiting retryhttp to not use 1.3.0 to fix CI mypy checks

* making it <1.3.0
* Raise custom exception

* fix static checks

* Raise custom exception

* fix static checks

* Raise custom exception

* fix static checks

* Raise custom exception

* fix static checks

* add exception messgae

* Increment version

* change misc to fix

---------

Co-authored-by: Majoros Donat (XC-DX/EET2-Bp) <[email protected]>
Bumps [vitest](https://github.com/vitest-dev/vitest/tree/HEAD/packages/vitest) from 2.1.1 to 2.1.9.
- [Release notes](https://github.com/vitest-dev/vitest/releases)
- [Commits](https://github.com/vitest-dev/vitest/commits/v2.1.9/packages/vitest)

---
updated-dependencies:
- dependency-name: vitest
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* extra forbidden

* test

* gen_file

* fix gen_file

* rm clear_db_*

* rm testcase

* rm testcase # 2
…ab. (apache#46074)

* Add triggerer info and extra links section to task instance details tab.

* Change button variant from default solid to surface to make it lighter.

* Add map_index support.

* PR comments refactor.

* Add section name for task instance details.
* Undoing the retryhttp limitation (<1.3.0)

* adding !=1.3.0
vincbeck and others added 14 commits February 6, 2025 21:15
apache#46480)

Closes apache#46477

The chart version 1.16.0 would be released before airflow v3, so it's better to keep api server in beta so it's easier to make changes.

Signed-off-by: Andrii Korotkov <[email protected]>
…s defined (apache#46522)

This is part of the AIP-83 amendments.
* Fix task sdk client dry-run mode

The `run_after` field was missing in the fake dag run response.

* Fix trailing whitespaces
* refactor: Moved microsoft winrm provider

* refactor: fixed static checks
* Update console link to filter on multiple instance ids
* Add state filter to task page.

* Refactor task instance and dagrun states to constants util.
* Move CNCF Kubernetes to new provider structure

* Fix doc include path and k8s test

* Fix taskflow tutorial

* Fix test_project_structure

* Strip src. prefix instead of replacing all src.

Co-authored-by: Kalyan R <[email protected]>

* Merge fix get_classes_from_file apache#46454

* Fix TestCncfProviderProjectStructure
- rename PROVIDER from "cncf" to "cncf/kubernetes"
- remove MISSING_EXAMPLES_FOR_CLASSES

* Fix k8s CI requirements

* fixup! Fix k8s CI requirements

---------

Co-authored-by: Kalyan R <[email protected]>
Co-authored-by: Jarek Potiuk <[email protected]>
The big change here (other than just moving code around) is to introduce a
conceptual separation between Definition/Execution time and Scheduler time.

This means that the expansion of tasks (creating the TaskInstance rows with
different map_index values) is now done on the scheduler, and we now
deserialize to different classes. For example, when we deserialize the
`DictOfListsExpandInput` it gets turned into an instance of
SchedulerDictOfListsExpandInput. This is primarily designed so that DB access
is kept 100% out of the TaskSDK.

Some of the changes here are on the "wat" side of the scale, and this is
mostly designed to not break 100% of our tests, and we have apache#45549 to look at
that more holistically.

To support the "reduce" style task which takes as input a sequence of all the
pushed (mapped) XCom values, and to keep the previous behaviour of not loading
all values in to memory at once, we have added a new HEAD route to the Task
Execution interface that returns the number of mapped XCom values so that it
is possible to implement `__len__` on the new LazyXComSequence class.

I have deleted a tranche of tests from tests/models that were to do with
runtime behavoir and and now tested in the TaskSDK instead.
…he#46485)

* Fix bug in query invalidation and remove custom predicate logic

* Address PR feedback

* Only use key fn when we have props to pass
* Moving yandex provider to new provider structure

* fixup! Moving yandex provider to new provider structure

---------

Co-authored-by: Jarek Potiuk <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:providers provider:cncf-kubernetes Kubernetes provider related issues
Projects
None yet
Development

Successfully merging this pull request may close these issues.