Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(medusa): Bulk emit events #3407

Merged
merged 25 commits into from
Mar 13, 2023
Merged

feat(medusa): Bulk emit events #3407

merged 25 commits into from
Mar 13, 2023

Conversation

adrien2p
Copy link
Member

@adrien2p adrien2p commented Mar 7, 2023

What
I can see during the perf tests that inserting lots of events create a bottleneck which is resolve by the bulk emit. During the refactoring we happen a lot to have the ability to bulk emit instead of having to emit them one by one. The perf are improved a lot

FIXES CORE-1220

@changeset-bot
Copy link

changeset-bot bot commented Mar 7, 2023

🦋 Changeset detected

Latest commit: 761745e

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 3 packages
Name Type
@medusajs/inventory Patch
medusa-test-utils Patch
@medusajs/medusa Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@vercel
Copy link

vercel bot commented Mar 7, 2023

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated
medusa-docs ✅ Ready (Inspect) Visit Preview 💬 Add your feedback Mar 13, 2023 at 0:34AM (UTC)
staging ✅ Ready (Inspect) Visit Preview 💬 Add your feedback Mar 13, 2023 at 0:34AM (UTC)

Copy link
Contributor

@olivermrbl olivermrbl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like this! Would be great to add some tests to the PR :)

packages/medusa/src/services/event-bus.ts Outdated Show resolved Hide resolved
packages/medusa/src/services/event-bus.ts Outdated Show resolved Hide resolved
packages/medusa/src/repositories/staged-job.ts Outdated Show resolved Hide resolved
packages/medusa/src/services/claim.ts Outdated Show resolved Hide resolved
packages/medusa/src/services/event-bus.ts Outdated Show resolved Hide resolved
packages/medusa/src/services/order.ts Outdated Show resolved Hide resolved
packages/medusa/src/services/event-bus.ts Outdated Show resolved Hide resolved
packages/medusa/src/services/event-bus.ts Outdated Show resolved Hide resolved
packages/medusa/src/services/event-bus.ts Outdated Show resolved Hide resolved
@olivermrbl olivermrbl force-pushed the feat/event-bus-bulk-emit branch from 1c80382 to 4eb96b7 Compare March 10, 2023 12:53
@olivermrbl olivermrbl force-pushed the feat/event-bus-bulk-emit branch from 4eb96b7 to 3401e4a Compare March 10, 2023 12:56
@olivermrbl
Copy link
Contributor

@adrien2p Will you investigate the failing check? 🙏

@adrien2p
Copy link
Member Author

adrien2p commented Mar 13, 2023

@adrien2p Will you investigate the failing check? 🙏

Sure 👌
what db is used in the CI? I ll look at it. But it seams that returning * is only supported by postgres, maria and MSSql, ill find another solution

@olivermrbl
Copy link
Contributor

But it seams that returning * is only supported by postgres, maria and MSSql, ill find another solution

Got it - that was what we were dreading.

We are testing against a SQLite DB in the pipeline as well, so that's why it's failing.

@adrien2p
Copy link
Member Author

But it seams that returning * is only supported by postgres, maria and MSSql, ill find another solution

Got it - that was what we were dreading.

We are testing against a SQLite DB in the pipeline as well, so that's why it's failing.

It seams that it should be available in sqlite from v3.35 https://www.sqlite.org/lang_returning.html i wonder which version are we using

@adrien2p
Copy link
Member Author

@olivermrbl I have opened an issue here in the mean time I ll do a fix

Copy link
Contributor

@olivermrbl olivermrbl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it worth spending some time looking into how/if we could use the addBulk in the enqueuer as well? Otherwise, most emit calls will not benefit from these optimizations, as they are part of a transaction i.e. the staged job flow.

@adrien2p
Copy link
Member Author

Is it worth spending some time looking into how/if we could use the addBulk in the enqueuer as well? Otherwise, most emit calls will not benefit from these optimizations, as they are part of a transaction i.e. the staged job flow.

Yes I will do that it is a very good point, though, the ones in the transaction benefit from the bulk insert of the staged jobs

@adrien2p
Copy link
Member Author

@olivermrbl I have updated the enqueuer, let me know wdyt

@olivermrbl
Copy link
Contributor

Merge as you please. Afterward (or now), I'll publish a canary, and we can run tests in a multi-process env before we release. Okay?

@adrien2p
Copy link
Member Author

/snapshot-this

@github-actions
Copy link
Contributor

🚀 A snapshot release has been made for this PR

Test the snapshots by updating your package.json with the newly published versions:

yarn add @medusajs/[email protected]
yarn add @medusajs/[email protected]

Latest commit: 601d20e

@olivermrbl olivermrbl merged commit f0a1355 into master Mar 13, 2023
@olivermrbl olivermrbl deleted the feat/event-bus-bulk-emit branch March 13, 2023 14:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants