Customize upload ACL when uploading in object storage

[dougluce]

I'm using S3 object storage with a bucket that is not publicly readable. Instead, it feeds a CloudFront distribution. This works smashingly well and I'm very grateful that the ability to store videos to S3 was added to the code base.

After upgrading to 4.1.0, attempts to upload to the bucket result in an error:

2022-03-11 09:55:59.076 error: Cannot move video https://candidates-viz.ligo.org/videos/watch/9b938c28-9fe1-4726-8a0b-d84785d40cb3 to object storage. {
  "err": {
    "stack": "AccessDenied: Access Denied\n    at deserializeAws_restXmlPutObjectCommandError (/var/www/peertube/versions/peertube-v4.1.0/node_modules/@aws-sdk/client-s3/dist-cjs/protocols/Aws_restXml.js:8266:41)\n    at processTicksAndRejections (node:internal/process/task_queues:96:5)\n    at async /var/www/peertube/versions/peertube-v4.1.0/node_modules/@aws-sdk/middleware-serde/dist-cjs/deserializerMiddleware.js:7:24\n    at async /var/www/peertube/versions/peertube-v4.1.0/node_modules/@aws-sdk/middleware-signing/dist-cjs/middleware.js:11:20\n    at async StandardRetryStrategy.retry (/var/www/peertube/versions/peertube-v4.1.0/node_modules/@aws-sdk/middleware-retry/dist-cjs/StandardRetryStrategy.js:51:46)\n    at async /var/www/peertube/versions/peertube-v4.1.0/node_modules/@aws-sdk/middleware-logger/dist-cjs/loggerMiddleware.js:6:22",
    "message": "Access Denied",
    "Code": "AccessDenied",
    ...

This is due to ACL being set to public-read in the PutObjectCommand in server/lib/object-storage/shared/object-storage-helpers.ts. If I remove that option in code the upload works just fine.

What was the motivation for adding that to the S3 call? Would an addition to the config of an explicit ACL be the best way to mitigate this issue?

[aldy120]

I got errors AccessDenied or AccessControlListNotSupported and finally I solve it with S3 configuration.

Uploading with ACL: 'public-read' caused unexpected 403 error if the S3 bucket is created with default settings in AWS console.

To mitigate this issue in Amazon S3 side without modifying peertube source code, here are the steps we can do in S3 permission tab:

• Disable Block Public Access
• Enable ACLs

[gsugambit]

I'm having this same issue with Google Cloud Storage

[dougluce]

@aldy120 That is not an appropriate solution as uploaded objects are then publicly readable from the S3 bucket.

[dougluce]

@gsugambit I believe the problem you've described in #4855 is not the same as this one.

[gsugambit]

@dougluce i had to make all objects in my storage public because of the ACL defined public-read in the source code. It's same exact issue. There are just other things that came up after making the bucket public

[Chocobozzz]

Hello,

See the discussion: #4290 (comment)

I'm not sure to understand this issue: video files uploaded in the bucket should be publicly readable by anyone so why should we remove this ACL?

[gsugambit]

@Chocobozzz If the bucket is public, even the files marked private in Peertube can be browsed if address is known.

[dougluce]

@Chocobozzz I'm using CloudFront for the actual video distribution and an AWS S3 bucket as the store for CloudFront. The bucket doesn't need to be public for CloudFront to access it. While it's not a given that public access is transitive in this scenario, even in cases where it is open S3 access can show up as a red flag on over-enthusiastic AWS audits (whereas CloudFront is commonly assumed to be public).

From that linked discussion it appears that the public ACL was added as a convenience for people who have the alternative to properly configure their bucket. Unfortunately it leaves no alternative for some of us who have properly configured our buckets.

How does dougluce@3301e0c look?

[Chocobozzz]

If the bucket is public, even the files marked private in Peertube can be browsed if address is known.

See #4417

I'm using CloudFront for the actual video distribution and an AWS S3 bucket as the store for CloudFront.

Thanks for the explanations, I understand now.

How does dougluce@3301e0c look?

It seems good, you can do a PR!