Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adds support for additional kwargs in middlewares (Solves issue #82) #83

Closed
wants to merge 3 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
55 changes: 55 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -233,6 +233,61 @@ delete_old_tasks.send(max_task_age=86400)
</dd>
</dl>

#### Providing custom kwargs to middleware classes

Some middleware classes require extra arguments (ie: rate limiter backend in `dramatiq.middleware.GroupCallbacks`).

Given the above scenario, you need to setup your middleware class in your django `settings.py`:

```python
DRAMATIQ_BROKER = {
...
"MIDDLEWARE": [
...
"dramatiq.middleware.GroupCallbacks",
...
]
...
}
```

Now we need to extend the default `DjangoDramatiqConfig` class to specify the custom args for
`dramatiq.middleware.GroupCallbacks`.


You can create your own django app or add this class to some `app.py` module:

```python
from django_dramatiq.apps import DjangoDramatiqConfig


class CustomDjangoDramatiqConfig(DjangoDramatiqConfig):

@classmethod
def middleware_groupcallbacks_kwargs(cls):
return {"rate_limiter_backend": cls.get_rate_limiter_backend()}


CustomDjangoDramatiqConfig.initialize()
```

Notice the naming convention, if you need to provide arguments to `dramatiq.middleware.GroupCallbacks`
you will need to add a `@classmethod` with the name `middleware_<middleware_name>_kwargs`,
where `<middleware_name>` is the class name of the middleware in lowercase.

In this class method, you need to return a `dict` of kwargs for your middleware.

The final step is setup this new custom *dramatiq* app config instead of the default one in your `settings.py`:


```python
INSTALLED_APPS = [
...
"yourapp.apps.CustomDjangoDramatiqConfig",
...
]
```


### Usage with [django-configurations]

Expand Down
33 changes: 20 additions & 13 deletions django_dramatiq/apps.py
Original file line number Diff line number Diff line change
Expand Up @@ -29,16 +29,15 @@
]
}

RATE_LIMITER_BACKEND = None


class DjangoDramatiqConfig(AppConfig):
name = "django_dramatiq"
verbose_name = "Django Dramatiq"

RATE_LIMITER_BACKEND = None

@classmethod
def initialize(cls):
global RATE_LIMITER_BACKEND

dramatiq.set_encoder(cls.select_encoder())

Expand All @@ -62,27 +61,38 @@ def initialize(cls):
)
rate_limiter_backend_class = import_string(rate_limiter_backend_path)
rate_limiter_backend_options = rate_limiter_backend_settings.get("BACKEND_OPTIONS", {})
RATE_LIMITER_BACKEND = rate_limiter_backend_class(**rate_limiter_backend_options)
cls.RATE_LIMITER_BACKEND = rate_limiter_backend_class(**rate_limiter_backend_options)

broker_settings = cls.broker_settings()
broker_path = broker_settings["BROKER"]
broker_class = import_string(broker_path)
broker_options = broker_settings.get("OPTIONS", {})
middleware = [load_middleware(path) for path in broker_settings.get("MIDDLEWARE", [])]
middleware = [
load_middleware(path, **cls.get_middleware_kwargs(path))
for path in broker_settings.get("MIDDLEWARE", [])
]

if result_backend is not None:
middleware.append(results_middleware)

broker = broker_class(middleware=middleware, **broker_options)
dramatiq.set_broker(broker)

@property
def rate_limiter_backend(self):
global RATE_LIMITER_BACKEND
if RATE_LIMITER_BACKEND is None:
@classmethod
def get_rate_limiter_backend(cls):
if cls.RATE_LIMITER_BACKEND is None:
raise RuntimeError("The rate limiter backend has not been configured.")

return RATE_LIMITER_BACKEND
return cls.RATE_LIMITER_BACKEND

@classmethod
def get_middleware_kwargs(cls, path):
if isinstance(path, str):
middleware_path = path.rsplit('.', 1)[1].lower()
middleware_kwargs_method = "middleware_{}_kwargs".format(middleware_path)
if hasattr(cls, middleware_kwargs_method):
return getattr(cls, middleware_kwargs_method)()
return {}

@classmethod
def broker_settings(cls):
Expand All @@ -104,6 +114,3 @@ def tasks_database(cls):
def select_encoder(cls):
encoder = getattr(settings, "DRAMATIQ_ENCODER", DEFAULT_ENCODER)
return import_string(encoder)()


DjangoDramatiqConfig.initialize()
4 changes: 2 additions & 2 deletions django_dramatiq/utils.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
from django.utils.module_loading import import_string


def load_middleware(path_or_obj):
def load_middleware(path_or_obj, **kwargs):
if isinstance(path_or_obj, str):
return import_string(path_or_obj)()
return import_string(path_or_obj)(**kwargs)
return path_or_obj