Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

s3: fix handling of NULL blob.contentType values #1353

Merged
merged 5 commits into from
Jan 13, 2025

Conversation

alxndrsn
Copy link
Contributor

Ref #1351

This aligns handling of NULL values for blob."contentType" between postgres-stored and s3-stored blobs.

test/e2e/util/api.js Outdated Show resolved Hide resolved
This example was taken from https://en.wikipedia.org/wiki/GeoJSON; licence may require attribution, which is a bit tricky in a JSON file.
Copy link
Member

@lognaturel lognaturel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems like a reasonable fix for now and we can take some time to consider what an improvement to both cases would look like.

@ktuite ktuite requested review from ktuite and removed request for ktuite January 10, 2025 21:39
@ktuite
Copy link
Member

ktuite commented Jan 10, 2025

I was able to locally reproduce the error TypeError: response header response-content-type should be of type "string" (with a dev server connected to s3) and then verify that this fixes it. Checking out the related PRs now.

@ktuite
Copy link
Member

ktuite commented Jan 11, 2025

The only other thing I was wondering about was if it was possible/worth checking this in the integration tests

    it.only('should get back null content-type when blob is on s3', testService(async (service, { Blobs, all }) => {
      global.s3.enableMock();
      const asAlice = await service.login('alice');

      await asAlice.post('/v1/projects/1/forms?publish=true')
        .set('Content-Type', 'application/xml')
        .send(testData.forms.binaryType)
        .expect(200);

      await asAlice.post('/v1/projects/1/forms/binaryType/submissions')
        .send(testData.instances.binaryType.both)
        .set('Content-Type', 'text/xml')
        .expect(200);

      await asAlice.post('/v1/projects/1/forms/binaryType/submissions/both/attachments/my_file1.mp4')
        .send('testvideo')
        .set('Content-Type', '') // set empty content type
        .expect(200);

      const blobs = await all(sql`select id, "contentType", md5 from blobs`);
      console.log(blobs);

      await Blobs.s3UploadPending();

      await asAlice.get('/v1/projects/1/forms/binaryType/submissions/both/attachments/my_file1.mp4')
        .expect(307)
        .then(({ headers, body }) => {
          console.log("headers", headers);
          console.log("body", body);
        });
    }));

but the bug fix is in lib/external/s3.js getRespHeaders which is just a utility of the s3 external utility and isn't part of s3 the mock.

If we go down a longer adventure of setting content types more appropriately, we would probably want to test it more here, but for now, this patch is good.

@alxndrsn
Copy link
Contributor Author

The only other thing I was wondering about was if it was possible/worth checking this in the integration tests

    it.only('should get back null content-type when blob is on s3', testService(async (service, { Blobs, all }) => {
      global.s3.enableMock();
      const asAlice = await service.login('alice');

      await asAlice.post('/v1/projects/1/forms?publish=true')
        .set('Content-Type', 'application/xml')
        .send(testData.forms.binaryType)
        .expect(200);

      await asAlice.post('/v1/projects/1/forms/binaryType/submissions')
        .send(testData.instances.binaryType.both)
        .set('Content-Type', 'text/xml')
        .expect(200);

      await asAlice.post('/v1/projects/1/forms/binaryType/submissions/both/attachments/my_file1.mp4')
        .send('testvideo')
        .set('Content-Type', '') // set empty content type
        .expect(200);

      const blobs = await all(sql`select id, "contentType", md5 from blobs`);
      console.log(blobs);

      await Blobs.s3UploadPending();

      await asAlice.get('/v1/projects/1/forms/binaryType/submissions/both/attachments/my_file1.mp4')
        .expect(307)
        .then(({ headers, body }) => {
          console.log("headers", headers);
          console.log("body", body);
        });
    }));

but the bug fix is in lib/external/s3.js getRespHeaders which is just a utility of the s3 external utility and isn't part of s3 the mock.

If we go down a longer adventure of setting content types more appropriately, we would probably want to test it more here, but for now, this patch is good.

Can you clarify what you're suggesting testing here? Is it the Content-Type header of the 307 redirect?

@alxndrsn alxndrsn merged commit 7574030 into getodk:master Jan 13, 2025
6 checks passed
@alxndrsn alxndrsn deleted the s3-null-mimetype-handling branch January 13, 2025 05:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants