Middleware causing hanging when running on Azure Functions

[robintw]

Problem description

I've been working on getting titiler working on Azure Functions (the Azure equivalent of AWS Lambda). The good news is that I've got it working - using their AsgiMiddleware that wraps an ASGI app. The less good news is that I had loads of problems with the function basically hanging (the HTTP request just not completing). I did a load of commenting out of parts of the app definition (in application\main.py) to see what was causing the problem, and after a load of trial and error, I found that a couple of the bits of middleware that are added to the app seemed to be causing the hang.

Specifically, if I commented out these lines, then it all works - but if I uncomment any of them then it hangs:

app.add_middleware(
    CacheControlMiddleware,
    cachecontrol=api_settings.cachecontrol,
    exclude_path={r"/healthz"},
)

if api_settings.debug:
    app.add_middleware(LoggerMiddleware, headers=True, querystrings=True)
    app.add_middleware(TotalTimeMiddleware)

(these are from application\main.py.

I've had a look at the definitions of the middleware, and I really can't see why they would make it fail. Does anyone have any idea what might be going on here? I thought I'd ask here first, but I can take it to the Azure people and ask if no-one here has any ideas.

Does anyone have

Expected Output

Works with the original code, without having to comment anything out.

Environment Information

Running on Azure Functions, using Python 3.9, and the standard Azure Functions runners. This is not through Docker - for various reasons - but instead using a standard requirements.txt file and installing the dependencies through that.

[vincentsarago]

@robintw are you saying that the CompressionMiddleware works but not the other ones?

titiler/src/titiler/application/titiler/application/main.py

Lines 64 to 74 in 04308ef

	app.add_middleware(
	CompressionMiddleware,
	minimum_size=0,
	exclude_mediatype={
	"image/jpeg",
	"image/jpg",
	"image/png",
	"image/jp2",
	"image/webp",
	},
	)

[robintw]

Yes, the CORSMiddleware and CompressionMiddleware seem to be fine, but any of the others seem to make it hang - I've no idea why!

[vincentsarago]

No clue 🤷‍♂️

The only thing I can tell is that the custom middleware which are blocking are built on top of BaseHTTPMiddleware https://github.com/encode/starlette/blob/3282d2364119c205adcd13b4e1db8a53af2202c7/starlette/middleware/base.py#L15

The other are not 🤷

Could you try removing the default middleware in titiler application and add a really simple one:

from titiler.core.factory import TilerFactory

from fastapi import FastAPI
from starlette.middleware.base import BaseHTTPMiddleware

app = FastAPI()
cog = TilerFactory()
app.include_router(cog.router)

class CustomHeaderMiddleware(BaseHTTPMiddleware):
    async def dispatch(self, request, call_next):
        response = await call_next(request)
        response.headers['Custom'] = 'Example'
        return response

app.add_middleware(CustomHeaderMiddleware)

[robintw]

Sorry for the delay in getting back to you.

I've tried that example code, and that hangs too. But if I remove the app.add_middleware call then it works fine.

Very strange...obviously something in the BaseHTTPMiddleware is doing something that Azure Functions doesn't like, but I have no idea what.

Do you have any suggestions? I sounds like it isn't really a titiler issue, so I'll try raising it as an issue on one of the Azure Functions libraries repos.

[vincentsarago]

I sounds like it isn't really a titiler issue, so I'll try raising it as an issue on one of the Azure Functions libraries repos.

yeah either on Azure Functions libraries repos or starlette 🤷

[robintw]

I've tested this with a simple Starlette app (see https://github.com/robintw/starlette-test-azure) and it still hangs only when a middleware derived from BaseHTTPMiddleware is used. I've raised issues on the Starlette repo here and the Azure Functions Python repo here.

[vincentsarago]

Excellent
thanks for debugging @robintw

let's closes this here for now then