Streaming Promises on a +page.server file misbehave in a built environment

[Sheco]

Describe the bug

Streamed Promises work correctly in the development environment (npm run dev), but block the page load in a built environment (For example npm run build && npm run preview)

I'm using the latest version of sveltekit

Reproduction

• Install a new sveltekit app npm create svelte@latest my-app
• Install dependencies npm install
• Create a src/routes/+page.svelte file with the example from the SvelteKit docs

<script>
  /** @type {import('./$types').PageData} */  
  export let data;
</script>

<p>
  one: {data.one}
</p>
<p>
  two: {data.two}
</p>
<p>
  three:
  {#await data.streamed.three}
    Loading...
  {:then value}
    {value}
  {:catch error}
    {error.message}
  {/await}
</p>

• Create a src/routes/+page.server.js with the example in the SvelteKit docs

/** @type {import('./$types').PageServerLoad} */
export function load() {
  return {
    one: Promise.resolve(1),
    two: Promise.resolve(2),
    streamed: {
      three: new Promise((fulfil) => {
        setTimeout(() => {
          fulfil(3)
        }, 1000);
      })
    }
  };
}

• Run the dev environment npm run dev and load the dev page at http://localhost:5174/ (adjust the port if needed)
  Enjoy the "loading..." message for a second and then enjoy the final "3"
• Now build the app and preview it, npm run build && npm run preview and load the page at http://localhost:4173/ (adjust the port if needed)
  The page blocks for a second, shows "loading..." for an instant and then shows the final "3"

Logs

No response

System Info

System:
    OS: Linux 6.1 Fedora Linux 37 (Workstation Edition)
    CPU: (8) x64 11th Gen Intel(R) Core(TM) i7-1165G7 @ 2.80GHz
    Memory: 4.74 GB / 15.35 GB
    Container: Yes
    Shell: 5.2.15 - /bin/bash
  Binaries:
    Node: 18.15.0 - /usr/bin/node
    npm: 9.5.0 - /usr/bin/npm
  Browsers:
    Firefox: 111.0
  npmPackages:
    @sveltejs/adapter-auto: ^2.0.0 => 2.0.0 
    @sveltejs/kit: ^1.5.0 => 1.14.0 
    svelte: ^3.54.0 => 3.57.0 
    vite: ^4.2.0 => 4.2.1

Severity

annoyance

Additional Information

No response

[dummdidumm]

The problem is somewhere within the node stream or the browser itself. We're calling res.write(chunk) but that does not flush out the chunk immediately. Instead it seems to wait for more data and flush everything once res.end() is called. When doing curl localhost:4173 the first data chunk shows up immediately, which makes me think it's the browser. I'm not sure how to get it to flush out the data eagerly, other than appending junk data to the chunk which pushes it over the threshold.

[danieldiekmeier]

I have also encountered this problem and would like to help find a fix. Maybe this helps:

Command	Chrome/Firefox	curl 7.87.0
vite dev	✅	✅
vite preview	❌	✅
node build locally	✅	✅
node build in prod, behind nginx	❌	✅ but chunks in a weird place inside the SvelteKit <script> tag – BEFORE the </html>

It feels like there might be several factors at play here? I don't understand why npm run preview does not work, but node build DOES, except that behind nginx, it doesn't. Can I try anything else to help get to the bottom of this?

[danieldiekmeier]

(I now realize that my nginx problems are not what this issue was originally about, but I still wanted to follow up)

I looked into this a little more and I came to the conclusion that vite preview not working is probably unrelated (and might even be considered "correct" if you're using adapters/platforms that don't support streaming.) vite preview seems to use a different setup from what you're getting out of adapter-node.

I was still unsure why streaming didn't work for me on production, behind nginx. This seems to be a feature of nginx: By default, nginx buffers the response from the proxied server, so the streaming does not reach the client. (Except if the buffer fills up.)

This gets amplified when nginx also gzips the responses, because then we have another buffer. (this also explains why __data.json seemed to work better for me: I didn't gzip the content type text/sveltekit-data.)

If I turn the proxy_buffering setting off, streaming responses immediately work:

# nginx.conf

location / {
	proxy_buffering off;
    # ...
}

But I think this may be a bad idea, so I'm not sure what I'm going to do.

[rodshtein]

@danieldiekmeier you can disable NGINX proxy buffer by set X-Accel-Buffering header on specific route:

export async function load({ setHeaders }) {
  setHeaders({
    'X-Accel-Buffering': 'no'
  });
  ...
}

NGINX Docs: nginx.org/en/docs/http/ngx_http_proxy_module

[ddxv]

(I now realize that my nginx problems are not what this issue was originally about, but I still wanted to follow up)

I looked into this a little more and I came to the conclusion that vite preview not working is probably unrelated (and might even be considered "correct" if you're using adapters/platforms that don't support streaming.) vite preview seems to use a different setup from what you're getting out of adapter-node.

I was still unsure why streaming didn't work for me on production, behind nginx. This seems to be a feature of nginx: By default, nginx buffers the response from the proxied server, so the streaming does not reach the client. (Except if the buffer fills up.)

This gets amplified when nginx also gzips the responses, because then we have another buffer. (this also explains why __data.json seemed to work better for me: I didn't gzip the content type text/sveltekit-data.)

If I turn the proxy_buffering setting off, streaming responses immediately work:

# nginx.conf

location / {
	proxy_buffering off;
    # ...
}

But I think this may be a bad idea, so I'm not sure what I'm going to do.

In the end I was only able to get streaming working with "proxy_buffering off;" Also, it still doesn't work locally when running:

npm run build then npm run preview

[isaacharrisholt]

I'm also experiencing this issue. It runs fine on the dev server, but not once built, including when deployed to Vercel. I'm using:

• @sveltejs/kit: v2.5.8
• @sveltejs/adapter-vercel: v5.3.0

There are workarounds, but they're not at all ideal.

[khromov]

As for why npm run preview is not working with streaming promises, it seems to be down the browser sending the Accept-encoding: gzip header, and the Vite preview server then responds with content-encoding: gzip and a gzipped payload. Importantly this gzipped response does not stream and is delivered as a single chunk. Here's a comparison of gzipped and non gzipped responses, the non-gzipped one is streamed in correctly:

You can reproduce by running this curl:

curl -N -H "Accept: text/html" -H "Accept-Encoding: gzip" http://127.0.0.1:4173/streaming -vvv --output -

I don't know what has to be tweaked but my guess is something inside Vite for this to work properly.

[eltigerchino]

This issue seems to be specific to vite preview and does not happen when running the node adapter directly even with the gzip header.

Maybe related to #11377 . @benmccann do you know if it's safe to remove any of the other base vite middlewares? There also doesn't seem to be a middleware named viteBaseMiddleware anymore. However, there is an anonymous function that performs the gzip. Streaming works if I remove it but should it be removed?

Here's the body of the anonymous function base Vite middleware

(req, res, next = NOOP) => {
                const accept = req.headers['accept-encoding'] + '';
                const encoding = ((brotli && accept.match(/\bbr\b/)) || (gzip && accept.match(/\bgzip\b/)) || [])[0];

                // skip if no response body or no supported encoding:
                if (req.method === 'HEAD' || !encoding) return next();

                /** @type {zlib.Gzip | zlib.BrotliCompress} */
                let compress;
                /** @type {Array<[string, function]>?} */
                let pendingListeners = [];
                let pendingStatus = 0;
                let started = false;
                let size = 0;

                function start() {
                        started = true;
                        // @ts-ignore
                        size = res.getHeader('Content-Length') | 0 || size;
                        const compressible = mimes.test(
                                String(res.getHeader('Content-Type') || 'text/plain')
                        );
                        const cleartext = !res.getHeader('Content-Encoding');
                        const listeners = pendingListeners || [];

                        if (compressible && cleartext && size >= threshold) {
                                res.setHeader('Content-Encoding', encoding);
                                res.removeHeader('Content-Length');
                                if (encoding === 'br') {
                                        compress = zlib$1.createBrotliCompress({
                                                params: Object.assign({
                                                        [zlib$1.constants.BROTLI_PARAM_QUALITY]: level,
                                                        [zlib$1.constants.BROTLI_PARAM_SIZE_HINT]: size,
                                                }, brotliOpts)
                                        });
                                } else {
                                        compress = zlib$1.createGzip(
                                                Object.assign({ level }, gzipOpts)
                                        );
                                }
                                // backpressure
                                compress.on('data', chunk => write.call(res, chunk) || compress.pause());
                                on.call(res, 'drain', () => compress.resume());
                                compress.on('end', () => end.call(res));
                                listeners.forEach(p => compress.on.apply(compress, p));
                        } else {
                                pendingListeners = null;
                                listeners.forEach(p => on.apply(res, p));
                        }

                        writeHead.call(res, pendingStatus || res.statusCode);
                }

                const { end, write, on, writeHead } = res;

                res.writeHead = function (status, reason, headers) {
                        if (typeof reason !== 'string') [headers, reason] = [reason, headers];
                        if (headers) for (let k in headers) res.setHeader(k, headers[k]);
                        pendingStatus = status;
                        return this;
                };

                res.write = function (chunk, enc) {
                        size += getChunkSize(chunk, enc);
                        if (!started) start();
                        if (!compress) return write.apply(this, arguments);
                        return compress.write.apply(compress, arguments);
                };

                res.end = function (chunk, enc) {
                        if (arguments.length > 0 && typeof chunk !== 'function') {
                                size += getChunkSize(chunk, enc);
                        }
                        if (!started) start();
                        if (!compress) return end.apply(this, arguments);
                        return compress.end.apply(compress, arguments);
                };

                res.on = function (type, listener) {
                        if (!pendingListeners) on.call(this, type, listener);
                        else if (compress) compress.on(type, listener);
                        else pendingListeners.push([type, listener]);
                        return this;
                };

                next();
        }