issues initiating uvicorn with multiprocessing logqueue in python version 3.12.4

[dtrai2]

When running the logprep acceptance test with python 3.12.4 the following two tests of the http input connector fail:

FAILED tests/acceptance/test_http_input.py::test_http_input_accepts_message_for_single_pipeline - TimeoutError: Timer expired
FAILED tests/acceptance/test_http_input.py::test_http_input_accepts_message_for_multiple_pipelines - TimeoutError: Timer expired

When running the acceptance tests with debugging information (adding print statements to the tests/acceptance/util.py:wait_for_output method), we get the following stacktraces:

2024-06-20 09:17:21 2620   Manager    INFO    : Created new pipeline
2024-06-20 09:17:21 2620   py.warnings WARNING : /opt/hostedtoolcache/Python/3.12.4/x64/lib/python3.12/multiprocessing/popen_fork.py:66: DeprecationWarning: This process (pid=2620) is multi-threaded, use of fork() may lead to deadlocks in the child.
  self.pid = os.fork()

2024-06-20 09:17:21 2620   Manager    WARNING : Restarting failed pipeline on index 1 with exit code: 1
Process Process-14:
Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.12.4/x64/lib/python3.12/logging/config.py", line 581, in configure
    handler = self.configure_handler(handlers[name])
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.12.4/x64/lib/python3.12/logging/config.py", line 792, in configure_handler
    proxy_queue = MM().Queue()
                  ^^^^
  File "/opt/hostedtoolcache/Python/3.12.4/x64/lib/python3.12/multiprocessing/context.py", line 57, in Manager
    m.start()
  File "/opt/hostedtoolcache/Python/3.12.4/x64/lib/python3.12/multiprocessing/managers.py", line 562, in start
    self._process.start()
  File "/opt/hostedtoolcache/Python/3.12.4/x64/lib/python3.12/multiprocessing/process.py", line 118, in start
    assert not _current_process._config.get('daemon'), \
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: daemonic processes are not allowed to have children

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.12.4/x64/lib/python3.12/multiprocessing/process.py", line 314, in _bootstrap
    self.run()
  File "/opt/hostedtoolcache/Python/3.12.4/x64/lib/python3.12/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/home/runner/work/Logprep/Logprep/logprep/framework/pipeline.py", line 210, in run
    self._setup()
  File "/home/runner/work/Logprep/Logprep/logprep/framework/pipeline.py", line 50, in _inner
    return func(self)
           ^^^^^^^^^^
  File "/home/runner/work/Logprep/Logprep/logprep/framework/pipeline.py", line 186, in _setup
    self._input.setup()
  File "/home/runner/work/Logprep/Logprep/logprep/connector/http/input.py", line 466, in setup
    self.http_server = http.ThreadingHTTPServer(
                       ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/runner/work/Logprep/Logprep/logprep/util/http.py", line 71, in __init__
    uvicorn_config = uvicorn.Config(**uvicorn_config, app=app, log_config=logprep_log_config)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.12.4/x64/lib/python3.12/site-packages/uvicorn/config.py", line 273, in __init__
    self.configure_logging()
  File "/opt/hostedtoolcache/Python/3.12.4/x64/lib/python3.12/site-packages/uvicorn/config.py", line 365, in configure_logging
    logging.config.dictConfig(self.log_config)
  File "/opt/hostedtoolcache/Python/3.12.4/x64/lib/python3.12/logging/config.py", line 920, in dictConfig
    dictConfigClass(config).configure()
  File "/opt/hostedtoolcache/Python/3.12.4/x64/lib/python3.12/logging/config.py", line 588, in configure
    raise ValueError('Unable to configure handler '
ValueError: Unable to configure handler 'queue'

This indicates that there is an issue when configuring the logqueue for the uvcorn.Config. In the logprep version 3.12.4 the logging configuration initiates a multiprocessing queue to check for the configured type. Initiating this queue inside a daemonic thread probably issues the AssertionError: daemonic processes are not allowed to have children exception.

Python 3.12.3: https://github.com/python/cpython/blob/v3.12.3/Lib/logging/config.py#L789
Python 3.12.4: https://github.com/python/cpython/blob/v3.12.4/Lib/logging/config.py#L792 (containing the MM().Queue() which causes the error).

The change was probably introduced by this pull request: https://github.com/python/cpython/pull/120030/files

When the logqueue is given to a uvicorn server that runs in the main process this issue does not arise. As reference following scratch python file can be used for validating this:

import multiprocessing as mp

import falcon.asgi
import uvicorn

logqueue = mp.Queue(-1)

log_config = {
    "version": 1,
    "formatters": {
        "logprep": {
            "class": "logprep.util.logging.LogprepFormatter",
        }
    },
    "handlers": {
        "console": {
            "class": "logging.StreamHandler",
            "formatter": "logprep",
            "stream": "ext://sys.stdout",
        },
        "queue": {
            "class": "logging.handlers.QueueHandler",
            "queue": logqueue,
        },
    },
    "loggers": {
        "root": {"level": "INFO", "handlers": ["queue"]},
    },
}

app = falcon.asgi.App()
uvicorn.run(app=app, log_config=log_config)

[dtrai2]

is fixed in a higher python version, so from 3.12.5 everything should work again