fix(server/diffdownload): fix the bug where client connection close events were not propogated to the database stream connection

[iainsproat]

Description & motivation

This is a more complete fix which #3889 aimed to resolve, but which introduced additional bugs on some models.

This PR fixes a long existing bug with prematurely closed streaming connections. Prior to this PR, for the /api/getobjects/* REST endpoint, if the client prematurely closed a connection to the server it was not propogated to the database connection. This resulted in the database connection never being closed, and inevitably saturation for the server and the inability to serve further traffic (reffectively a Denial of Service until the server is restarted).

The root cause of the bug is an issue with Node.js stream.pipeline implementation, described in the documentation as so:

stream.pipeline() leaves dangling event listeners on the streams after the callback has been invoked. In the case of reuse of streams after failure, this can cause event listener leaks and swallowed errors.

Previously we had an improper implementation of the workaround for the above. This PR fixes that implementation.

The changes which are relevant are best reviewed by diffing against the changes prior to the previous 'fix':

git diff fc634ebfe5deec336f7d6c84e584ae77f24a65c1 -- packages/server/modules/core/rest/diffDownload.ts

Changes:

• Listen to close events on the Response, not the Request.

• .bind(dbStream) did not work.

• Adds a further test for scenarios which caused the previous solution (fix(server/objects): solves database connection resource leak when client connections are prematurely closed #3889) to fail. This is intended to prevent regression.

• The CSV file is a partial data dump of the objects for the Speckle onboarding model, reduced to the minimum failing example.

• Most of this PR removes the previous, failing, 'fix'.

To-do before merge:

• test should have minimally failing example

Screenshots:

Validation of changes:

Checklist:

• My pull request follows the guidelines in the Contributing guide?
• My pull request does not duplicate any other open Pull Requests for the same update/change?
• My commits are related to the pull request and do not amend unrelated code or documentation.
• My code follows a similar style to existing code.
• I have added appropriate tests.
• I have updated or added relevant documentation.

References

[github-actions]

📸 Preview service has generated an image.

[github-actions]

📸 Preview service has generated an image.

[github-actions]

📸 Preview service has generated an image.

[iainsproat]

This was the fix, so simple in the end.
Listen to events on the response, not the request. And .bind wasn't required.