Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: merge worker into main process and optimize graceful shutdown mechanism #713

Merged
merged 1 commit into from
Oct 7, 2024
Merged

feat: merge worker into main process and optimize graceful shutdown mechanism #713

merged 1 commit into from
Oct 7, 2024

Conversation

donch1989
Copy link
Member

Because

  • We don't have a proper graceful shutdown mechanism.
  • The data exchange between the API server and Temporal worker is unstable and suboptimal, introducing latency.

This commit

  • Merges the worker process into the main process, allowing the API server and Temporal worker to share memory for improved stability and performance.
  • Enhances the graceful shutdown mechanism.

Note

  • This is a temporary solution. We will refactor the run log feature to serve as the foundation for pipeline state. Once that is complete, we can separate the API server and Temporal worker again while maintaining stability.

@linear Linear
Copy link

linear bot commented Oct 7, 2024

INS-6300 implement a proper graceful shutdown for `pipeline-backend`

@donch1989 donch1989 marked this pull request as draft October 7, 2024 19:23
@donch1989 donch1989 marked this pull request as ready for review October 7, 2024 19:47
@donch1989 donch1989 merged commit 69b64a0 into main Oct 7, 2024
12 checks passed
@donch1989 donch1989 deleted the huitang/ins-6300-5 branch October 7, 2024 19:47
@droplet-bot droplet-bot mentioned this pull request Oct 7, 2024
Merged
jvallesm pushed a commit that referenced this pull request Oct 8, 2024
@droplet-bot
chore(main): release 0.42.0-beta (#700)
2c29c7f 
🤖 I have created a release *beep* *boop*
---


##
[0.42.0-beta](v0.41.0-beta...v0.42.0-beta)
(2024-10-08)


### Features

* **document:** repair pdf with libreoffice
([#683](#683))
([b6a738f](b6a738f))
* **integration:** add identity to OAuth connections
([#699](#699))
([e2e9237](e2e9237))
* merge worker into main process and optimize graceful shutdown
mechanism
([#713](#713))
([69b64a0](69b64a0))
* **universalai:** add instill credit function for universal ai
component
([#680](#680))
([9ce10b5](9ce10b5))
* use explicit parameter for target namespace when cloning pipeline
([#711](#711))
([ab2e3a6](ab2e3a6))
* **web:** fix bug and add mock users' behaviours by chromedp
([#701](#701))
([ca312fe](ca312fe))


### Bug Fixes

* **component:** use kebab-case for icon name
([#703](#703))
([cd587e2](cd587e2))
* **document:** catch the error if the bbox is out of boundary
([#718](#718))
([4fe03e6](4fe03e6))
* fix the component render error was not being returned
([#707](#707))
([931f067](931f067))
* fix the wrong target namespace when cloning pipeline
([#706](#706))
([75a757b](75a757b))
* **integration:** update GitHub scopes to read/write repository issues
([#709](#709))
([c1cf4ce](c1cf4ce))
* **text:** fix the bug about the list position
([#712](#712))
([49aed28](49aed28))

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@pinglin pinglin Awaiting requested review from pinglin

@xiaofei-du xiaofei-du Awaiting requested review from xiaofei-du

@jvallesm jvallesm Awaiting requested review from jvallesm jvallesm is a code owner

Assignees
No one assigned
Labels
instill vdp
Projects
No open projects
🔮 Instill Core
Status: No status
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@donch1989 @droplet-bot