Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(check uploads content): Check that a client upload's extension matches the file contents #371

Merged
merged 3 commits into from
Feb 14, 2024

Conversation

mdial89f
Copy link
Contributor

@mdial89f mdial89f commented Feb 8, 2024

Purpose

This remediates a finding where a user could upload a file that has an extension that doesn't match its contents; the example was a jpg containing xml with embedded javascript

Linked Issues to Close

Closes https://qmacbis.atlassian.net/browse/OY2-26965

Approach

Please access the ticket above for more information. This PR will deliberately refrain from listing all contents of the ticket in this PR.

This finding calls for a way to verify a file's extension against its contents. We decided the best, and most secure, way to do that is to check it server side. Here's how it works:

  • A user drags and drops a file into the application form
  • The file's extension is checked against an allowed extension list. If it's a disallowed extension, the UI rejects the file and shows an error message.
  • If the file's extension is allowed, the file is accepted.
  • On submit, the file is uploaded to S3.
  • When the file arrives in S3, an event triggers our file scanning lambda. The role of this lambda is to verify the file is safe, and tag it CLEAN if it is. Until the file is tagged CLEAN, the file is entirely inaccessible to all entities except for the scanner itself.
  • The lambda gets the declared extension of the file, or what it says it is. (new)
  • The lambda uses the npm file-type package to check the contents of the file (new)
  • If the declared extension and the detected content type do not match, the scanner tags the file as EXTMISMATCH and exits; in this scenario, the file continues to be inaccessible (new)
  • If we haven't errored out yet, clamav scans the file and tags it clean or not, as it does today.

How to test

We were provided the file that was used in the finding. It is named as a jpg but contains xml embedded with javascript.
As part of testing:

  • the specific jpg/xml mentioned above was uploaded, and was flagged EXTMISMATCH as expected. It remained inaccessible
  • a .pdf file was renamed to be a .png file. That file was also flagged EXTMISMATCH and remained inaccessible
  • a wide array of correctly extensioned files were uploaded, and were marked CLEAN

Assorted Notes/Considerations/Learning

None

@mdial89f mdial89f self-assigned this Feb 8, 2024
@mdial89f mdial89f added type: FIX Submit bug fixes status: READY PR is ready for review labels Feb 8, 2024
@mdial89f mdial89f requested review from pkim-gswell and benjaminpaige and removed request for pkim-gswell February 8, 2024 15:20
Copy link
Collaborator

@benjaminpaige benjaminpaige left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah this looks pretty sweet

Copy link
Contributor

@pkim-gswell pkim-gswell left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very cool 😎

@@ -39,6 +39,10 @@ export const STATUS_ERROR_PROCESSING_FILE: string =
process.env.STATUS_ERROR_PROCESSING_FILE || "ERROR";
export const STATUS_SKIPPED_FILE: string =
process.env.STATUS_SKIPPED_FILE || "SKIPPED";
export const STATUS_EXTENSION_MISMATCH_FILE: string =
process.env.STATUS_EXTENSION_MISMATCH_FILE || "EXTMISMATCH"
export const STATUS_UNKNOWN_EXTENSION: string =
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nitpicking here. For naming consistency:

STATUS_EXTENSION_MISMATCH_FILE
STATUS_EXTENSION_UNKNOWN

But that'll also mean changing the env var name too. Completely up to you.
Aside from that, everything looks great

@mdial89f mdial89f merged commit 4d157db into master Feb 14, 2024
16 checks passed
Copy link
Contributor

🎉 This PR is included in version 1.5.0-val.1 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
released on @val status: READY PR is ready for review type: FIX Submit bug fixes
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants