post: Introducing Jaiminho, an implementation of the Transactional Outbox Pattern for Django #71
Conversation
✅ Deploy Preview for loadsmart-engineering ready!
To edit notification comments on pull requests, go to your Netlify site settings. |
|
|
||
| The Event Relaying Command will query the database for all events that were not sent in a given timebox and will try to re-send them using the original method, message and kwargs. Note that if the event relay runs after a deploy that changed or removed the original method, it will not be able to re-send the failed events and they will have to be dealt with manually. Also, the `DELETE_AFTER_SEND` configuration also applies to this command. | ||
|
|
||
| Tip: Schedule this command to be regularly executed and Jaiminho will ensure all failed events are re-sent. |
There was a problem hiding this comment.
Would be nice to include the consumer mode when the relayer runs forever (no scheduling)
There was a problem hiding this comment.
Done! Can you check the new text to see if it's well explained?
|
|
||
| # How Jaiminho works | ||
|
|
||
| With the default configuration, when you add the `@save_to_outbox` decorator to a method, whenever it is called and raises an exception, Jaiminho will create a database entry storing: |
There was a problem hiding this comment.
If we use the keep order option, we save it always and store it for sending it in order. Right @antunesleo ?
There was a problem hiding this comment.
Yeah! With the keep order option, the persist all events is mandatory. It is explained on the Persist all events subsection on the additional settings section.
| --- | ||
| title: Introducing Jaiminho, an implementation of the Transactional Outbox Pattern for Django | ||
| author: Rafael Zanetti | ||
| twitter: LoadsmartEng |
There was a problem hiding this comment.
Do you have a Twitter account @rfzanetti ?
There was a problem hiding this comment.
I do, but it has too much dumb stuff and not to much tech stuff haha I could also create an account just to link it here, what do you suggest?
rfzanetti
requested review from
barbosa,
antunesleo,
cristianowa,
Yanhenning and
gustahrodrigues
| } | ||
| ``` | ||
|
|
||
| 4) Add the `@save_to_outbox` decorator to any method responsible for communicating with external systems (brokers, external APIs, etc): |
There was a problem hiding this comment.
suggestion: I would give more colors around save_to_outbox_stream as well.
There was a problem hiding this comment.
I think the post is already a bit crowded with a lot of the configuration options, so maybe might be worth leaving it just on the documentation. What do you think?
| comments: true | ||
| --- | ||
|
|
||
| At Loadsmart, many of our services are organized within an event-driven architecture and face some usual challenges that come with such an architecture. One of them is **dual writes**, which happens when an application has to change data on two different places. A common example of dual writes is an application that needs to store data on a database and notify that change by emitting an event which may be consumed by different services. The inherent issue with this scenario is that if one of these two operations fails, the producer and consumer applications will have data inconsistencies. If we choose to write the change to the database first and then notify the event, in the case of a notification failure the database would have the updated data and the consumer wouldn’t. If we reverse it (i.e. notifying first and then storing), a failure when committing the transaction in the database would cause the consumer receiving data that isn’t on the application database. |
There was a problem hiding this comment.
What about linking some material about dual writes ?
There was a problem hiding this comment.
I feel it's a bit of overkill...I think that the main audience would be people that already are experienced with this issue (and hopefully are considering adding jaiminho to their projects? 😄), but for other people who don't know what that is, the explanations on the post are enough.
What do you think?
|
|
||
| At Loadsmart, many of our services are organized within an event-driven architecture and face some usual challenges that come with such an architecture. One of them is **dual writes**, which happens when an application has to change data on two different places. A common example of dual writes is an application that needs to store data on a database and notify that change by emitting an event which may be consumed by different services. The inherent issue with this scenario is that if one of these two operations fails, the producer and consumer applications will have data inconsistencies. If we choose to write the change to the database first and then notify the event, in the case of a notification failure the database would have the updated data and the consumer wouldn’t. If we reverse it (i.e. notifying first and then storing), a failure when committing the transaction in the database would cause the consumer receiving data that isn’t on the application database. | ||
|
|
||
| There are well-known methods to deal with such issues, and we chose to apply the transactional outbox pattern. Since we couldn’t find any related solution on [Django community](https://djangopackages.org/search/?q=outbox), we decided to implement an open-source library called [Jaiminho](https://github.com/loadsmart/jaiminho), which is a broker agnostic implementation of the outbox pattern. |
There was a problem hiding this comment.
Same, linking something digging deeper into Transaction Outbox Pattern
There was a problem hiding this comment.
same thing as above
| comments: true | ||
| --- | ||
|
|
||
| At Loadsmart, many of our services are organized within an event-driven architecture and face some usual challenges that come with such an architecture. One of them is **dual writes**, which happens when an application has to change data on two different places. A common example of dual writes is an application that needs to store data on a database and notify that change by emitting an event which may be consumed by different services. The inherent issue with this scenario is that if one of these two operations fails, the producer and consumer applications will have data inconsistencies. If we choose to write the change to the database first and then notify the event, in the case of a notification failure the database would have the updated data and the consumer wouldn’t. If we reverse it (i.e. notifying first and then storing), a failure when committing the transaction in the database would cause the consumer receiving data that isn’t on the application database. |
There was a problem hiding this comment.
I would change when an application has to change to when an application has to write
No description provided.