Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add always (or finally) tasks that run at the end regardless of the final state — useful for cleanup tasks #6649

Closed
anna-geller opened this issue Jan 7, 2025 · 7 comments · Fixed by #6686
Closed

Add always (or finally) tasks that run at the end regardless of the final state — useful for cleanup tasks #6649

anna-geller opened this issue Jan 7, 2025 · 7 comments · Fixed by #6686
Assignees
@tchiotludo
Labels
area/backend Needs backend code changes enhancement New feature or request

Comments

@anna-geller
Copy link
Member

Feature description

Internal use case: shut down Docker containers after QA tasks are finished

Possible syntax:

id: always_example
namespace: company.team

tasks:
  - id: fail
    type: io.kestra.plugin.core.execution.Fail
    errorMessage: Test downstream tasks

errors:
  - id: send_alert
    type: io.kestra.plugin.core.log.Log
    message: alert on failure

always:
  - id: cleanup_task
    type: io.kestra.plugin.core.log.Log
    message: cleaning up resources

Later on, we can add the same concept within flowable tasks that currently support errors
@anna-geller anna-geller added area/backend Needs backend code changes enhancement New feature or request area/frontend Needs frontend code changes labels Jan 7, 2025
@github-project-automation github-project-automation bot moved this to Backlog in Issues Jan 7, 2025
@anna-geller anna-geller removed the area/frontend Needs frontend code changes label Jan 7, 2025
@tchiotludo tchiotludo self-assigned this Jan 7, 2025
@Ben8t
Copy link
Member

Ben8t commented Jan 7, 2025

That sounds a lot like the old "listeners" property or did I miss something?

@anna-geller
Copy link
Member Author

Yup, a lot of similarities there. The main reason is to enable cleanup tasks for things like QA flows. In contrast to listener which was a complex list of tasks and conditions, the always/finally would be intended as a simpler concept implying: always run those listed tasks at the end unconditionally (regardless of the state etc). In the end, a much simpler concept without much functionality overlap with Flow triggers as listener did

@Ben8t
Copy link
Member

Ben8t commented Jan 7, 2025

Alright! Good idea IMO - was in favor to keep flow listeners so I'm happy to see the concept emerge again :)

@loicmathieu
Copy link
Member

Listeners happen after the flow is terminated.

Heremy advise would be for the implementation to call the finally (I strongly prefer finally here) tasks after tasks/errors then only to set the execution final state.

@tchiotludo
Copy link
Member

The goal is to remove finally the listeners in favor of this always, you can achieve the same goal, and by this way, you could remove the whole complexity of listeners inside the code.
I don't think there is usecase any more for listeners when we will have the always

tchiotludo added a commit that referenced this issue Jan 8, 2025
@tchiotludo
feat(core): introduce an always block on flow & flowable
abc1bc1 
close #6649
@tchiotludo tchiotludo mentioned this issue Jan 8, 2025
Merged
tchiotludo added a commit that referenced this issue Jan 8, 2025
@tchiotludo
feat(core): introduce an always block on flow & flowable
4c71f15 
close #6649
@tchiotludo
Copy link
Member

@anna-geller seems that the team is pushy to have finally, ok?

@anna-geller
Copy link
Member Author

sure, no problem, go for finally then

tchiotludo added a commit that referenced this issue Jan 13, 2025
@tchiotludo
feat(core): introduce an always block on flow & flowable
bde015d 
close #6649
@github-project-automation github-project-automation bot moved this from Backlog to Done in Issues Jan 13, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@tchiotludo tchiotludo

Labels
area/backend Needs backend code changes enhancement New feature or request
Projects
Issues
Status: Done
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

feat(core): introduce an always block on flow & flowable kestra-io/kestra
4 participants
@loicmathieu @tchiotludo @Ben8t @anna-geller