Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for middleware when consuming messages #19

Open
ghost opened this issue Feb 10, 2016 · 2 comments
Open

Add support for middleware when consuming messages #19

ghost opened this issue Feb 10, 2016 · 2 comments

Comments

@ghost
Copy link

ghost commented Feb 10, 2016

For each message I receive, I want to perform the following

  • Make a logging statement
  • Send a notification to monitoring tools like Newrelic
  • Set some ThreadLocal context information
  • Deserialize
  • Validate input satisfies some schema
  • Perform my actual computation
  • Validate computation results, putting certain error types back onto the queue for retry, while sending others straight to the DLQ.
  • Log how long this process took .
  • Clear my ThreadLocal context.

Clojure Ring middleware provides great hooks for reducing the boilerplate required in order to follow this kind of workflow. It would be nice to be able to provide similar middleware for SQS message handling.

@ghost ghost added the enhancement label Feb 10, 2016
@mtkp
Copy link
Contributor

mtkp commented Feb 10, 2016

+1

we should have hooks for both enqueue (similar to clj-http) and dequeue (similar to ring) middlewares

enqueue middleware could, for example, automatically inject some data into SQS message attributes taken from the current threadlocal context.

@ghost
Copy link
Author

ghost commented Feb 29, 2016

So I've been working with basically implementing this on the application side to test out one approach, and I think we need the following to add this to the library.

  1. Better document exactly what the Message object coming out of SQS looks like. Right now, it can be found at https://github.com/TheClimateCorporation/squeedo/blob/master/src/com/climate/squeedo/sqs.clj#L162, but it would be nice to not have to dive deep into a private function to find that information, and it can be a pain when you have to do it to properly implement each one of your middlewares.
  2. Add some default middleware as illustrations of how to create your own middleware. Deserialization and logging are obvious choices.
  3. Doing it similar to the way ring works will work well whenever we're consuming from many different queues, as you could define a set of middleware in a single place that will work the same for all queues that you're listening to. However, it is simple enough to wrap your message handler in middleware client side if you're only listening to a single queue. We should add examples of both to the readme.

mtkp added a commit that referenced this issue Jun 26, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant