Starlette app hangs on Azure Functions when middleware based on BaseHTTPMiddleware is used

[robintw]

Describe the bug

I am running a Starlette app through Azure Functions, using the ASGI support in Azure Functions to run the app. When I set up the app with no middlewares, it works fine. When I add the CORSMiddleware it also works fine, but when I add any middleware which is a subclass of BaseHTTPMiddleware, then all requests to the app hang and eventually timeout.

To reproduce

A simple reproducible example is available here. This defines a simple Starlette app as follows:

from starlette.applications import Starlette
from starlette.middleware import Middleware
from starlette.responses import JSONResponse
from starlette.routing import Route
from starlette.middleware.base import BaseHTTPMiddleware
from starlette.middleware.cors import CORSMiddleware

async def homepage(request):
    return JSONResponse({"hello": "world"})


class CustomHeaderMiddleware(BaseHTTPMiddleware):
    async def dispatch(self, request, call_next):
        response = await call_next(request)
        response.headers["Custom"] = "Example"
        return response


app = Starlette(
    debug=True,
    routes=[
        Route("/", homepage),
    ],
    # Works with no middlewares defined
    #
    # Hangs with this line:
    # middleware=[Middleware(CustomHeaderMiddleware)]
    #
    # Works with this line:
    middleware = [Middleware(CORSMiddleware)]
)

This app is then deployed on Azure Functions, using their AsgiMiddleware class, which interfaces between Azure Functions triggers and ASGI requests. The code is all available in the repo, but the key part is __init__.py, which contains:

import azure.functions as func
from azure.functions import AsgiMiddleware

from .simple_app import app

def main(req: func.HttpRequest, context: func.Context) -> func.HttpResponse:
    return AsgiMiddleware(app).handle(req, context) 

Expected behavior

App works with all types of middleware

Actual behavior

As you can see from the comments at the bottom of the app, this works with no middlewares, or just the CORSMiddleware, but hangs with the CustomHeaderMiddleware which is derived from BaseHTTPMiddleware.

Debugging material

As the app just hangs, there is no traceback to share.

Environment

• OS: Linux, on Azure Functions
• Python version: 3.9
• Starlette version: 0.16.0 (latest release at time of writing)

Additional context

This was first discovered in a FastAPI app called titiler - see developmentseed/titiler#388. The author of that app realised it was only failing when using middleware which were a subclass of BaseHTTPMiddleware, and suggested I try with raw Starlette and post an issue here.

[Kludex]

After some hours debugging this, I've come to the conclusion that the issue is on Azure Functions.

The problem lies on the fact that they are patching the receive() method and returning always the same message. That becomes an issue when you see our cancellation logic on the StreamingResponse, and then you confirm that listen_for_disconnect gets into an infinite loop (blocking the event loop for streaming_response).

If we add a simple await anyio.sleep(0) on the listen_for_disconnect loop, we are able to pass the block and finish our cycle, because streaming_response will finish and then wrap will cancel the task group. But I think the ideal solution would be for Azure Functions to do something similar as Mangum does.

/cc @tonybaloney Maybe you're interested in this? I was not able to debug the changes on the package while using the VSCode extension, otherwise I could have tried to push a fix for it there. 😞

[robintw]

Thanks so much for this investigation @Kludex. I've linked it on the Azure Functions repo too, so hopefully someone from Microsoft will pick it up.

[Kludex]

If someone is willing to teach me how to "live edit" the Azure Functions package while on the VSCode Azure plugin, I can try to submit a PR there. 😗

[vincentsarago]

@Kludex you might have already found this, but here are the vscode tutorial https://docs.microsoft.com/en-us/azure/azure-functions/create-first-function-vs-code-python#run-the-function-locally 🤷

[tonybaloney]

@Kludex @robintw I'll pick this up on the Microsoft/Azure Functions side and debug

[tonybaloney]

Looks like the easiest way to solve this is for receive() to return a disconnect if it has already been called once with the body. Functions doesn't support streaming bodies, so it would never check for more body.