Research if Django instrumentation can auto-detect DJANGO_SETTINGS_MODULE when auto-instrumentation is used

[owais]

Currently users are required to set the DJANGO_SETTINGS_MODULE env var when auto-instrumenting Django. It would be convenient for the auto-instrumentation to be able to figure this out automatically.

hint: it might be possible to fix this by lazily import django.conf.settings instead of here: https://github.com/open-telemetry/opentelemetry-python-contrib/blob/main/instrumentation/opentelemetry-instrumentation-django/src/opentelemetry/instrumentation/django/__init__.py#L80

Currenly the instrumentation fails if the env var is not setup:

2021-09-10 19:05:16,525 ERROR [/Users/olone/playground/django-otel/venv/lib/python3.9/site-packages/opentelemetry/instrumentation/auto_instrumentation/sitecustomize.py] [sitecustomize.py:84] [trace_id=0 span_id=0 resource.service.name=my-django-service] - Instrumenting of django failed
Traceback (most recent call last):
  File "/Users/olone/playground/django-otel/venv/lib/python3.9/site-packages/opentelemetry/instrumentation/auto_instrumentation/sitecustomize.py", line 81, in _load_instrumentors
    distro.load_instrumentor(entry_point, skip_dep_check=True)
  File "/Users/olone/playground/django-otel/venv/lib/python3.9/site-packages/opentelemetry/instrumentation/distro.py", line 63, in load_instrumentor
    instrumentor().instrument(**kwargs)
  File "/Users/olone/playground/django-otel/venv/lib/python3.9/site-packages/opentelemetry/instrumentation/instrumentor.py", line 109, in instrument
    result = self._instrument(  # pylint: disable=assignment-from-no-return
  File "/Users/olone/playground/django-otel/venv/lib/python3.9/site-packages/opentelemetry/instrumentation/django/__init__.py", line 132, in _instrument
    settings_middleware = getattr(settings, "MIDDLEWARE", [])
  File "/Users/olone/playground/django-otel/venv/lib/python3.9/site-packages/django/conf/__init__.py", line 82, in __getattr__
    self._setup(name)
  File "/Users/olone/playground/django-otel/venv/lib/python3.9/site-packages/django/conf/__init__.py", line 63, in _setup
    raise ImproperlyConfigured(
django.core.exceptions.ImproperlyConfigured: Requested setting MIDDLEWARE, but settings are not configured. You must either define the environment variable DJANGO_SETTINGS_MODULE or call settings.configure() before accessing settings.
2021-09-10 19:05:16,528 ERROR [/Users/olone/playground/django-otel/venv/lib/python3.9/site-packages/opentelemetry/instrumentation/auto_instrumentation/sitecustomize.py] [sitecustomize.py:113] [trace_id=0 span_id=0 resource.service.name=my-django-service] - Failed to auto initialize opentelemetry
Traceback (most recent call last):
  File "/Users/olone/playground/django-otel/venv/lib/python3.9/site-packages/opentelemetry/instrumentation/auto_instrumentation/sitecustomize.py", line 111, in initialize
    _load_instrumentors(distro)
  File "/Users/olone/playground/django-otel/venv/lib/python3.9/site-packages/opentelemetry/instrumentation/auto_instrumentation/sitecustomize.py", line 85, in _load_instrumentors
    raise exc
  File "/Users/olone/playground/django-otel/venv/lib/python3.9/site-packages/opentelemetry/instrumentation/auto_instrumentation/sitecustomize.py", line 81, in _load_instrumentors
    distro.load_instrumentor(entry_point, skip_dep_check=True)
  File "/Users/olone/playground/django-otel/venv/lib/python3.9/site-packages/opentelemetry/instrumentation/distro.py", line 63, in load_instrumentor
    instrumentor().instrument(**kwargs)
  File "/Users/olone/playground/django-otel/venv/lib/python3.9/site-packages/opentelemetry/instrumentation/instrumentor.py", line 109, in instrument
    result = self._instrument(  # pylint: disable=assignment-from-no-return
  File "/Users/olone/playground/django-otel/venv/lib/python3.9/site-packages/opentelemetry/instrumentation/django/__init__.py", line 132, in _instrument
    settings_middleware = getattr(settings, "MIDDLEWARE", [])
  File "/Users/olone/playground/django-otel/venv/lib/python3.9/site-packages/django/conf/__init__.py", line 82, in __getattr__
    self._setup(name)
  File "/Users/olone/playground/django-otel/venv/lib/python3.9/site-packages/django/conf/__init__.py", line 63, in _setup
    raise ImproperlyConfigured(
django.core.exceptions.ImproperlyConfigured: Requested setting MIDDLEWARE, but settings are not configured. You must either define the environment variable DJANGO_SETTINGS_MODULE or call settings.configure() before accessing settings.
Watching for file changes with StatReloader
Performing system checks...

System check identified no issues (0 silenced).

You have 18 unapplied migration(s). Your project may not work properly until you apply the migrations for app(s): admin, auth, contenttypes, sessions.
Run 'python manage.py migrate' to apply them.
September 10, 2021 - 13:35:17
Django version 3.2.7, using settings 'mysite.settings'
Starting development server at http://127.0.0.1:8000/
Quit the server with CONTROL-C.

[ocelotl]

This should not be required and auto-instrumentation should be able to figure this out automatically.

But how? 🤔 I may be missing something but if Django itself needs this to be defined directly by the user in an environment variable (because it can't figure it out by itself), I don't see how our instrumentation can do it.

[owais]

I don't know how :) Part of the issue is to research and figure out the answer and the answer may very well be that is is not possible to do.

[ocelotl]

Ah, ok. I was a bit confused 😆

I have edited this issue to reflect the actual intention. 👍

[jeremydvoss]

I see this issue is old but still open. I used to have this issue, but it seems to have stopped. Was there an intentional fix? from django.conf import settings has been there awhile (https://github.com/open-telemetry/opentelemetry-python-contrib/blame/9a2285a42b059e7e88912ad410c1d585f2622545/instrumentation/opentelemetry-instrumentation-django/src/opentelemetry/instrumentation/django/__init__.py#L197)

[jeremydvoss]

Testing this out, here's what I found:

1. Auto-instrumentation does utilize the DJANGO_SETTINGS_MODULE env var.
2. If the django instrumentation is installed, but django itself is not, then using auto-instrumentation works correctly.
   • sitesutomize catches a dependency conflict because django is not installed and therefore skips the django instrumentation.
3. If Django is installed but the DJANGO_SETTINGS_MODULE is not set, auto-instrumentation breaks -- even when auto-instrumented non-django apps.
   • Since django is installed, sitecustomize attempts to call load_instrumentor but fails when it attempts to pull the MIDDLEWARE setting.

Instrumenting of django failed
Traceback (most recent call last):
  File "c:\users\jeremyvoss\workplace\opentelemetry-python-contrib\opentelemetry-instrumentation\src\opentelemetry\instrumentation\auto_instrumentation\sitecustomize.py", line 89, in _load_instrumentors
    distro.load_instrumentor(entry_point, skip_dep_check=True)
  File "C:\Users\jeremyvoss\workplace\auto_instrumentation\lib\site-packages\opentelemetry\instrumentation\distro.py", line 63, in load_instrumentor
    instrumentor().instrument(**kwargs)
  File "C:\Users\jeremyvoss\workplace\auto_instrumentation\lib\site-packages\opentelemetry\instrumentation\instrumentor.py", line 109, in instrument
    raise exc
  File "c:\users\jeremyvoss\workplace\opentelemetry-python-contrib\opentelemetry-instrumentation\src\opentelemetry\instrumentation\auto_instrumentation\sitecustomize.py", line 89, in _load_instrumentors
    distro.load_instrumentor(entry_point, skip_dep_check=True)
  File "C:\Users\jeremyvoss\workplace\auto_instrumentation\lib\site-packages\opentelemetry\instrumentation\distro.py", line 63, in load_instrumentor
    instrumentor().instrument(**kwargs)
  File "C:\Users\jeremyvoss\workplace\auto_instrumentation\lib\site-packages\opentelemetry\instrumentation\instrumentor.py", line 109, in instrument
    result = self._instrument(  # pylint: disable=assignment-from-no-return
  File "C:\Users\jeremyvoss\workplace\opentelemetry-python-contrib\instrumentation\opentelemetry-instrumentation-django\src\opentelemetry\instrumentation\django\__init__.py", line 280, in _instrument
    settings_middleware = getattr(settings, _middleware_setting, [])
  File "C:\Users\jeremyvoss\workplace\auto_instrumentation\lib\site-packages\django\conf\__init__.py", line 92, in __getattr__
    self._setup(name)
  File "C:\Users\jeremyvoss\workplace\auto_instrumentation\lib\site-packages\django\conf\__init__.py", line 72, in _setup
    raise ImproperlyConfigured(
django.core.exceptions.ImproperlyConfigured: Requested setting MIDDLEWARE, but settings are not configured. You must either define the environment variable DJANGO_SETTINGS_MODULE or call settings.configure() before accessing settings.

Since this would cause issues for non-django apps, and it is unreasonable to ask that users always uninstall django on their machines when running non-django apps or to have a DJANGO_SETTINGS_MODULE env var configured when running non-django apps, I see 2 possible fixes:

1. If django settings are not configured, call settings.configure(). This leaves the settings empty, but configured. Which means we can still add ['opentelemetry.instrumentation.django.middleware.otel_middleware._DjangoMiddleware'] to it. The reason why we would actually set the settings to nothing instead of just the settings_middleware variable is that the remaining code uses getattr again.

if not settings.configured:
            settings.configure()
settings_middleware = getattr(settings, _middleware_setting, [])

2. We exit the django instrumentation peacefully but allow sitecustomize to continue other instrumentations. This would likely require changes to sitecustomize, since it is correctly set up to stop instrumenting when an exception occurs.