Merge how-to section on S3 buckets into S3 store docs (#7972)

Rather than having a misc tutorial page in the grab-bag "package management" section, this information should just be part of the S3 store docs. --------- Co-authored-by: John Ericson <[email protected]>
NixOS · Oct 23, 2023 · cd680bd · cd680bd
1 parent c9528d2
commit cd680bd
Show file tree

Hide file tree

Showing 4 changed files with 100 additions and 121 deletions.
diff --git a/doc/manual/src/SUMMARY.md.in b/doc/manual/src/SUMMARY.md.in
@@ -24,7 +24,6 @@
     - [Serving a Nix store via HTTP](package-management/binary-cache-substituter.md)
     - [Copying Closures via SSH](package-management/copy-closure.md)
     - [Serving a Nix store via SSH](package-management/ssh-substituter.md)
-    - [Serving a Nix store via S3](package-management/s3-substituter.md)
 - [Nix Language](language/index.md)
   - [Data Types](language/values.md)
   - [Language Constructs](language/constructs.md)

diff --git a/doc/manual/src/advanced-topics/post-build-hook.md b/doc/manual/src/advanced-topics/post-build-hook.md
@@ -17,9 +17,8 @@ the build loop.
 
 # Prerequisites
 
-This tutorial assumes you have [configured an S3-compatible binary
-cache](../package-management/s3-substituter.md), and that the `root`
-user's default AWS profile can upload to the bucket.
+This tutorial assumes you have configured an [S3-compatible binary cache](@docroot@/command-ref/new-cli/nix3-help-stores.md#s3-binary-cache-store) as a [substituter](../command-ref/conf-file.md#conf-substituters),
+and that the `root` user's default AWS profile can upload to the bucket.
 
 # Set up a Signing Key
 

diff --git a/doc/manual/src/package-management/s3-substituter.md b/doc/manual/src/package-management/s3-substituter.md
diff --git a/src/libstore/s3-binary-cache-store.md b/src/libstore/s3-binary-cache-store.md
@@ -2,7 +2,103 @@ R"(
 
 **Store URL format**: `s3://`*bucket-name*
 
-This store allows reading and writing a binary cache stored in an AWS
-S3 bucket.
+This store allows reading and writing a binary cache stored in an AWS S3 (or S3-compatible service) bucket.
+This store shares many idioms with the [HTTP Binary Cache Store](#http-binary-cache-store).
+
+For AWS S3, the binary cache URL for a bucket named `example-nix-cache` will be exactly <s3://example-nix-cache>.
+For S3 compatible binary caches, consult that cache's documentation.
+
+### Anonymous reads to your S3-compatible binary cache
+
+> If your binary cache is publicly accessible and does not require authentication,
+> it is simplest to use the [HTTP Binary Cache Store] rather than S3 Binary Cache Store with
+> <https://example-nix-cache.s3.amazonaws.com> instead of <s3://example-nix-cache>.
+
+Your bucket will need a
+[bucket policy](https://docs.aws.amazon.com/AmazonS3/v1/userguide/bucket-policies.html)
+like the following to be accessible:
+
+```json
+{
+    "Id": "DirectReads",
+    "Version": "2012-10-17",
+    "Statement": [
+        {
+            "Sid": "AllowDirectReads",
+            "Action": [
+                "s3:GetObject",
+                "s3:GetBucketLocation"
+            ],
+            "Effect": "Allow",
+            "Resource": [
+                "arn:aws:s3:::example-nix-cache",
+                "arn:aws:s3:::example-nix-cache/*"
+            ],
+            "Principal": "*"
+        }
+    ]
+}
+```
+
+### Authentication
+
+Nix will use the
+[default credential provider chain](https://docs.aws.amazon.com/sdk-for-cpp/v1/developer-guide/credentials.html)
+for authenticating requests to Amazon S3.
+
+Note that this means Nix will read environment variables and files with different idioms than with Nix's own settings, as implemented by the AWS SDK.
+Consult the documentation linked above for further details.
+
+### Authenticated reads to your S3 binary cache
+
+Your bucket will need a bucket policy allowing the desired users to perform the `s3:GetObject` and `s3:GetBucketLocation` action on all objects in the bucket.
+The [anonymous policy given above](#anonymous-reads-to-your-s3-compatible-binary-cache) can be updated to have a restricted `Principal` to support this.
+
+### Authenticated writes to your S3-compatible binary cache
+
+Your account will need an IAM policy to support uploading to the bucket:
+
+```json
+{
+  "Version": "2012-10-17",
+  "Statement": [
+    {
+      "Sid": "UploadToCache",
+      "Effect": "Allow",
+      "Action": [
+        "s3:AbortMultipartUpload",
+        "s3:GetBucketLocation",
+        "s3:GetObject",
+        "s3:ListBucket",
+        "s3:ListBucketMultipartUploads",
+        "s3:ListMultipartUploadParts",
+        "s3:PutObject"
+      ],
+      "Resource": [
+        "arn:aws:s3:::example-nix-cache",
+        "arn:aws:s3:::example-nix-cache/*"
+      ]
+    }
+  ]
+}
+```
+
+### Examples
+
+With bucket policies and authentication set up as described above, uploading works via [`nix copy`](@docroot@/command-ref/new-cli/nix3-copy.md) (experimental).
+
+- To upload with a specific credential profile for Amazon S3:
+
+  ```console
+  $ nix copy nixpkgs.hello \
+    --to 's3://example-nix-cache?profile=cache-upload&region=eu-west-2'
+  ```
+
+- To upload to an S3-compatible binary cache:
+
+  ```console
+  $ nix copy nixpkgs.hello --to \
+    's3://example-nix-cache?profile=cache-upload&scheme=https&endpoint=minio.example.com'
+  ```
 
 )"