Skip to content

Commit

Permalink
docs(core): add azure and gcs powerpack plugins (#28668)
Browse files Browse the repository at this point in the history
- Adds powerpack-gcs-cache plugin docs
- Adds powerpack-azure-cache-plugin docs
- Removes a duplicate rspack entry on the /nx-api page

---------

Co-authored-by: Jonathan Cammisuli <[email protected]>
  • Loading branch information
isaacplmann and Cammisuli authored Nov 5, 2024
1 parent e2f5eaa commit 5ddcb56
Show file tree
Hide file tree
Showing 23 changed files with 663 additions and 43 deletions.
3 changes: 0 additions & 3 deletions docs/blog/2024-09-25-introducing-nx-powerpack.md
Original file line number Diff line number Diff line change
Expand Up @@ -156,9 +156,6 @@ permissions:
id-token: write
...

env:
NX_DB_CACHE: true

jobs:
main:
runs-on: ubuntu-latest
Expand Down
67 changes: 67 additions & 0 deletions docs/external-generated/packages-metadata.json
Original file line number Diff line number Diff line change
@@ -1,4 +1,27 @@
[
{
"description": "A Nx Powerpack plugin which provides a Nx cache which can be self hosted on Azure Blob Storage.",
"documents": [
{
"id": "overview",
"name": "Overview",
"description": "A Nx Powerpack plugin which provides a Nx cache which can be self hosted on Azure Blob Storage.",
"file": "external-generated/packages/powerpack-azure-cache/documents/overview",
"itemList": [],
"isExternal": false,
"path": "powerpack-azure-cache/documents/overview",
"tags": [],
"originalFilePath": "shared/packages/powerpack-azure-cache/powerpack-azure-cache-plugin"
}
],
"executors": [],
"generators": [],
"githubRoot": "https://github.com/nrwl/nx/blob/master",
"name": "powerpack-azure-cache",
"packageName": "@nx/powerpack-azure-cache",
"root": "/libs/nx-packages/powerpack-azure-cache",
"source": "/libs/nx-packages/powerpack-azure-cache/src"
},
{
"description": "A Nx Powerpack plugin which allows users to write and apply rules for your entire workspace that help with consistency, maintainability, reliability and security.",
"documents": [
Expand Down Expand Up @@ -32,6 +55,50 @@
"root": "/libs/nx-packages/powerpack-conformance",
"source": "/libs/nx-packages/powerpack-conformance/src"
},
{
"description": "A Nx Powerpack plugin which is specific to Nx Enterprise Cloud workspaces.",
"documents": [],
"executors": [],
"generators": [
{
"description": "Initialize Nx Powerpack Enterprise Cloud config",
"file": "external-generated/packages/powerpack-enterprise-cloud/generators/init.json",
"hidden": false,
"name": "init",
"originalFilePath": "/libs/nx-packages/powerpack-enterprise-cloud/src/generators/init/schema.json",
"path": "powerpack-enterprise-cloud/generators/init",
"type": "generator"
}
],
"githubRoot": "https://github.com/nrwl/nx/blob/master",
"name": "powerpack-enterprise-cloud",
"packageName": "@nx/powerpack-enterprise-cloud",
"root": "/libs/nx-packages/powerpack-enterprise-cloud",
"source": "/libs/nx-packages/powerpack-enterprise-cloud/src"
},
{
"description": "A Nx Powerpack plugin which provides a Nx cache which can be self hosted on Google Cloud Storage.",
"documents": [
{
"id": "overview",
"name": "Overview",
"description": "A Nx Powerpack plugin which provides a Nx cache which can be self hosted on Google Cloud Storage.",
"file": "external-generated/packages/powerpack-gcs-cache/documents/overview",
"itemList": [],
"isExternal": false,
"path": "powerpack-gcs-cache/documents/overview",
"tags": [],
"originalFilePath": "shared/packages/powerpack-gcs-cache/powerpack-gcs-cache-plugin"
}
],
"executors": [],
"generators": [],
"githubRoot": "https://github.com/nrwl/nx/blob/master",
"name": "powerpack-gcs-cache",
"packageName": "@nx/powerpack-gcs-cache",
"root": "/libs/nx-packages/powerpack-gcs-cache",
"source": "/libs/nx-packages/powerpack-gcs-cache/src"
},
{
"description": "Package to provide the ability to activate and read licenses for Nx Powerpack.",
"documents": [],
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,83 @@
---
title: Overview of the Nx powerpack-azure-cache Plugin
description: The powerpack-azure-cache Nx plugin enables you to use Azure Storage to host your remote cache instead of Nx Cloud
---

The `@nx/powerpack-azure-cache` plugin enables you to use [Azure Storage](https://azure.microsoft.com/en-us/products/storage/blobs) instead of Nx Cloud to host your remote cache.

This plugin will enable the remote cache for your Nx workspace, but does not provide any of the other features of Nx Cloud. If you want to leverage [distributed task execution](/ci/features/distribute-task-execution), [re-running flaky tasks](/ci/features/flaky-tasks) or [automatically splitting tasks](/ci/features/split-e2e-tasks), you'll need to [connect to Nx Cloud](/ci/intro/connect-to-nx-cloud) and use [Nx Replay](/ci/features/remote-cache) instead.

{% callout type="warning" title="Potential Cache Poisoning" %}
Using your own Azure Storage account to host the remote cache opens you up to the possibility of [cache poisoning](/troubleshooting/unknown-local-cache). To avoid this, use [Nx Replay](/ci/features/remote-cache).
{% /callout %}

{% callout title="This plugin requires an active Nx Powerpack license" %}
In order to use `@nx/powerpack-azure-cache`, you need to have an active Powerpack license. If you don't have a license or it has expired, your cache will no longer be shared and each machine will use its local cache.
{% /callout %}

## Set Up @nx/powerpack-azure-cache

### 1. Install the Package

1. [Activate Powerpack](/nx-enterprise/activate-powerpack) if you haven't already
2. Install the package

```shell
nx add @nx/powerpack-azure-cache
```

### 2. Authenticate with Azure

There are several ways to [authenticate with Azure Storage](https://github.com/Azure/login#login-with-openid-connect-oidc-recommended), but the method recommended by Azure is to use OpenID Connect, like this:

```yaml {% fileName=".github/workflows/ci.yml" %}
name: CI
...
permissions:
id-token: write
...

jobs:
main:
env:
NX_POWERPACK_LICENSE: ${{ secrets.NX_POWERPACK_LICENSE }}
runs-on: ubuntu-latest
steps:
...

- name: Azure login
uses: azure/login@v2
with:
client-id: ${{ secrets.AZURE_CLIENT_ID }}
tenant-id: ${{ secrets.AZURE_TENANT_ID }}
subscription-id: ${{ secrets.AZURE_SUBSCRIPTION_ID }}
...

- run: pnpm exec nx affected -t lint test build
```
You need to set the `AZURE_CLIENT_ID`, `AZURE_TENANT_ID` and `AZURE_SUBSCRIPTION_ID` secrets as defined in the [Azure documentation](https://github.com/Azure/login#login-with-openid-connect-oidc-recommended).

Note: Any authentication method that [sets up the `DefaultAzureCredentials`](https://learn.microsoft.com/en-us/azure/developer/javascript/sdk/credential-chains#use-defaultazurecredential-for-flexibility) will enable the plugin to work.

{% callout type="note" title="Custom Azure Endpoint" %}
If you are using a custom Azure endpoint, you will need to authenticate by [setting the `AZURE_STORAGE_CONNECTION_STRING` environment variable](https://learn.microsoft.com/en-us/azure/storage/common/storage-configure-connection-string). The `@nx/powerpack-azure-cache` plugin will detect the environment variable and automatically use it to connect to Azure.
{% /callout %}

### 3. Configure the Nx Cache to Use Azure Storage

Finally, you need to configure your Nx cache in the `nx.json` file. The `container` that you specify needs to already exist - Nx doesn't create it for you.

```jsonc {% fileName="nx.json" %}
{
"azure": {
"container": "mycontainer",
"accountName": "myaccount"
}
}
```

| **Property** | **Description** |
| --------------- | -------------------------------- |
| **container** | The name of the container to use |
| **accountName** | The name of blob storage account |
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
{
"name": "init",
"factory": "./src/generators/init/init",
"schema": {
"$schema": "http://json-schema.org/schema",
"id": "NxPowerpackEnterpriseCloudInit",
"title": "Add Powerpack Enterprise Cloud plugin to the workspace",
"type": "object",
"cli": "nx",
"properties": {},
"additionalProperties": false,
"required": [],
"presets": []
},
"description": "Initialize Nx Powerpack Enterprise Cloud config",
"implementation": "/libs/nx-packages/powerpack-enterprise-cloud/src/generators/init/init.ts",
"aliases": [],
"hidden": false,
"path": "/libs/nx-packages/powerpack-enterprise-cloud/src/generators/init/schema.json",
"type": "generator"
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,82 @@
---
title: Overview of the Nx powerpack-gcs-cache Plugin
description: The powerpack-gcs-cache Nx plugin enables you to use Google Cloud Storage to host your remote cache instead of Nx Cloud
---

The `@nx/powerpack-gcs-cache` plugin enables you to use [Google Cloud Storage](https://cloud.google.com/storage) instead of Nx Cloud to host your remote cache.

This plugin will enable the remote cache for your Nx workspace, but does not provide any of the other features of Nx Cloud. If you want to leverage [distributed task execution](/ci/features/distribute-task-execution), [re-running flaky tasks](/ci/features/flaky-tasks) or [automatically splitting tasks](/ci/features/split-e2e-tasks), you'll need to [connect to Nx Cloud](/ci/intro/connect-to-nx-cloud) and use [Nx Replay](/ci/features/remote-cache) instead.

{% callout type="warning" title="Potential Cache Poisoning" %}
Using your own Google Cloud Storage account to host the remote cache opens you up to the possibility of [cache poisoning](/troubleshooting/unknown-local-cache). To avoid this, use [Nx Replay](/ci/features/remote-cache).
{% /callout %}

{% callout title="This plugin requires an active Nx Powerpack license" %}
In order to use `@nx/powerpack-gcs-cache`, you need to have an active Powerpack license. If you don't have a license or it has expired, your cache will no longer be shared and each machine will use its local cache.
{% /callout %}

## Set Up @nx/powerpack-gcs-cache

### 1. Install the Package

1. [Activate Powerpack](/nx-enterprise/activate-powerpack) if you haven't already
2. Install the package

```shell
nx add @nx/powerpack-gcs-cache
```

### 2. Authenticate with Google Cloud

There are several ways to [authenticate with Google Cloud Storage](https://github.com/google-github-actions/setup-gcloud#authorization), but the method recommended by Google is to use Workload Identity Federation, like this:

```yaml {% fileName=".github/workflows/ci.yml" %}
name: CI
...
permissions:
id-token: write
...

jobs:
main:
env:
NX_POWERPACK_LICENSE: ${{ secrets.NX_POWERPACK_LICENSE }}
runs-on: ubuntu-latest
steps:
...

- id: 'auth'
name: 'Authenticate to Google Cloud'
uses: 'google-github-actions/auth@v2'
with:
token_format: 'access_token'
workload_identity_provider: 'projects/123456789/locations/global/workloadIdentityPools/my-pool/providers/my-provider'
service_account: '[email protected]'

- name: 'Set up Cloud SDK'
uses: 'google-github-actions/setup-gcloud@v2'
with:
version: '>= 363.0.0'

...

- run: pnpm exec nx affected -t lint test build
```
Note: Any authentication method that [sets up the Application Default Credentials](https://cloud.google.com/docs/authentication/application-default-credentials) will enable the plugin to work.
### 3. Configure the Nx Cache to Use Google Cloud Storage
Finally, you need to configure your Nx cache in the `nx.json` file. The `bucket` that you specify needs to already exist - Nx doesn't create it for you.

```jsonc {% fileName="nx.json" %}
{
"gcs": {
"bucket": "my-bucket"
}
}
```

| **Property** | **Description** |
| ------------ | ----------------------------- |
| **bucket** | The name of the bucket to use |
Original file line number Diff line number Diff line change
Expand Up @@ -57,11 +57,10 @@ permissions:
id-token: write
...

env:
NX_DB_CACHE: true

jobs:
main:
env:
NX_POWERPACK_LICENSE: ${{ secrets.NX_POWERPACK_LICENSE }}
runs-on: ubuntu-latest
steps:
...
Expand Down
78 changes: 78 additions & 0 deletions docs/generated/manifests/menus.json
Original file line number Diff line number Diff line change
Expand Up @@ -10501,6 +10501,32 @@
"isExternal": false,
"disableCollapsible": false
},
{
"id": "powerpack-azure-cache",
"path": "/nx-api/powerpack-azure-cache",
"name": "powerpack-azure-cache",
"children": [
{
"id": "documents",
"path": "/nx-api/powerpack-azure-cache/documents",
"name": "documents",
"children": [
{
"name": "Overview",
"path": "/nx-api/powerpack-azure-cache/documents/overview",
"id": "overview",
"isExternal": false,
"children": [],
"disableCollapsible": false
}
],
"isExternal": false,
"disableCollapsible": false
}
],
"isExternal": false,
"disableCollapsible": false
},
{
"id": "powerpack-conformance",
"path": "/nx-api/powerpack-conformance",
Expand Down Expand Up @@ -10544,6 +10570,58 @@
"isExternal": false,
"disableCollapsible": false
},
{
"id": "powerpack-enterprise-cloud",
"path": "/nx-api/powerpack-enterprise-cloud",
"name": "powerpack-enterprise-cloud",
"children": [
{
"id": "generators",
"path": "/nx-api/powerpack-enterprise-cloud/generators",
"name": "generators",
"children": [
{
"id": "init",
"path": "/nx-api/powerpack-enterprise-cloud/generators/init",
"name": "init",
"children": [],
"isExternal": false,
"disableCollapsible": false
}
],
"isExternal": false,
"disableCollapsible": false
}
],
"isExternal": false,
"disableCollapsible": false
},
{
"id": "powerpack-gcs-cache",
"path": "/nx-api/powerpack-gcs-cache",
"name": "powerpack-gcs-cache",
"children": [
{
"id": "documents",
"path": "/nx-api/powerpack-gcs-cache/documents",
"name": "documents",
"children": [
{
"name": "Overview",
"path": "/nx-api/powerpack-gcs-cache/documents/overview",
"id": "overview",
"isExternal": false,
"children": [],
"disableCollapsible": false
}
],
"isExternal": false,
"disableCollapsible": false
}
],
"isExternal": false,
"disableCollapsible": false
},
{
"id": "powerpack-license",
"path": "/nx-api/powerpack-license",
Expand Down
Loading

0 comments on commit 5ddcb56

Please sign in to comment.