Rego template endpoint #96

samteb · 2024-02-06T17:08:37Z

No description provided.

apps/authz/src/app/http/rest/dto/policy-criterion-builder.dto.ts

wcalderipe · 2024-02-07T08:43:19Z

apps/authz/src/app/core/admin.service.ts

@@ -84,4 +87,10 @@ export class AdminService {

    return payload.request.data
  }
+
+  async setPolicyRules(payload: SetPolicyRulesRequest): Promise<PolicyCriterionBuilder[]> {
+    await this.opaService.generateRegoFile(payload.request.data)


Do you plan to save these policies in the database as well, no?

yes but not on the engine part. see this discussion:
https://narvalxyz.slack.com/archives/C067Q1TDA9G/p1707310563319479

wcalderipe · 2024-02-07T08:56:00Z

apps/authz/src/app/opa/opa.service.ts

+  generateRegoFile(policies: PolicyCriterionBuilder[]): void {
+    Handlebars.registerHelper('criterion', function (item) {
+      const criterion: Criterion = item.criterion
+      const args = item.args
+
+      if (args === null) {
+        return `${criterion}`
+      }
+
+      if (!isEmpty(args)) {
+        if (Array.isArray(args)) {
+          if (typeof args[0] === 'string') {
+            return `${criterion}({${args.map((el) => `"${el}"`).join(', ')}})`
+          }
+
+          if (criterion === Criterion.CHECK_APPROVALS) {
+            return `approvals = ${criterion}([${args.map((el) => JSON.stringify(el)).join(', ')}])`
+          }
+
+          return `${criterion}([${args.map((el) => JSON.stringify(el)).join(', ')}])`
+        }
+
+        return `${criterion}(${JSON.stringify(args)})`
+      }
+    })
+
+    Handlebars.registerHelper('reason', function (item) {
+      if (item.then === Then.PERMIT) {
+        const reason = [
+          `"type": "${item.then}"`,
+          `"policyId": "${item.name}"`,
+          '"approvalsSatisfied": approvals.approvalsSatisfied',
+          '"approvalsMissing": approvals.approvalsMissing'
+        ]
+        return `reason = {${reason.join(', ')}}`
+      }
+
+      if (item.then === Then.FORBID) {
+        const reason = {
+          type: item.then,
+          policyId: item.name,
+          approvalsSatisfied: [],
+          approvalsMissing: []
+        }
+        return `reason = ${JSON.stringify(reason)}`
+      }
+    })
+
+    const templateSource = readFileSync('./apps/authz/src/opa/template/template.hbs', 'utf-8')
+
+    const template = Handlebars.compile(templateSource)
+
+    const regoContent = template({ policies })
+
+    writeFileSync(`./apps/authz/src/opa/rego/generated/${uuidv4()}.rego`, regoContent, 'utf-8')
+
+    console.log('Policy .rego file generated successfully.')
+  }


Here's a suggestion on how to unit test this logic:

Separate side effects from policy generation logic. If you move the template file reading and Rego file writing elsewhere, the logic essentially becomes: given a policy schema and a template string, you produce a policy string on the other side (the Rego logic).

Unit test each criterion defined in policy-build.type.ts. For a given template and criterion A, specify what you expect as a policy. Do this without merging criteria into a single policy. This approach will provide comprehensive documentation of what you expect for smaller system components.

Unit test a few criteria together to generate a policy and check if the outcome matches your expectations. This will provide us confidence on criteria composition.

These types of logic to map from standard A to B are often a major source of bugs. Let's get that covered so we can move faster without breaking things 😉

apps/authz/src/app/opa/opa.service.ts

packages/authz-shared/src/lib/type/policy-builder.type.ts

apps/authz/src/app/__test__/e2e/admin.spec.ts

apps/authz/src/app/http/rest/controller/admin.controller.ts

packages/authz-shared/src/index.ts

packages/authz-shared/src/lib/decorators/is-account-id.decorator.ts

packages/authz-shared/src/lib/decorators/is-asset-id.decorator.ts

packages/authz-shared/src/lib/type/policy-builder.type.ts

packages/authz-shared/src/lib/util/caip.util.ts

* Migration, data is persisted per-connection (#88) * schema migration to scope all data per connection * data migration script * fix provider type per connection * unique keys order * fixed failing tests * added connectionId to mock data in transfer and sync tests * handle known destination connections to connectionId * abstract away single tie to connection change from known destinations * filter on correct row in wallet repository * updated migration script to duplicate provider resources * updates existing transfers * Pass through known destination (#90) * Add CSR public key format for FB connection (#92) * Add CSR public key format for FB connection * Add optional CSR format in the public key DTO * Add describe block * Pin node-forge version * Remove wallet connections list (#95) * Scope operations by connection (#96) * Scope operations by connection * Scope sync operations by connection * Fix supertest wrapper to work with query method * Use signed request util in the wallet E2E tests * Fix flaky tests * Add back connection list endpoint * Fix address E2E tests * Bump SDK version * Add network and asset endpoints in the SDK * Adding BitGo connection * ScopedSync by RawAccounts (#84) * duplicated sync, poc working * fixed rebase conflicts and format * cleanup rebase merge, correctly insert connectionId on scope sync * fireblocks scoped-sync updates * use vault label * added updates to anchorage's scoped sync service * add tests on anchorage scoped sync * cleaned up anchorage scope sync tests * map fireblocks assetWallets to network specific accounts * fixed not found test expectation * test fireblocks scoped sync * fixed rebase conflicts * changed externalId standard * Adding Raw Accounts endpoint & Assed/Network bootstrap * Adding includeAddress query param so fireblocks does not always fetch addresses * Cache assets & refactor network cache (#98) * Cache asset Refactor network to use cache manager * Remove draft config * Pin cache-manager version * Lower case cache keys * Fix findAll method * Endpoints to fetch scoped-syncs (#101) * scoped sync persists stringified raw accounts * add status fetching endpoints for scoped syncs * use prismaservice methods to json stringify and parse * Minor fixes before release connection-specific (#102) * Add back the nested wallet and account endpoints in the connection * Minor fix on scope sync logs Connection ID is mandatory in the Vault Client * connect a new connection triggers full sync on anchorage (#103) * connect a new connection triggers full sync on anchorage * fix test excpetation for empty raw accounts on anchorage * Add listProviderRawAccounts method in the Vault SDK * removed references to 'sync' and dropped table * removed syncService from connection test * deleted sync e2e test * re-generated sdk without scoped sync methods * Revert "re-generated sdk without scoped sync methods" This reverts commit 66dccdfa2bd909d2f6c015b153e87c3728004dd2. * Revert "removed references to 'sync' and dropped table" This reverts commit 9c2cd022b95439ce9bc4a45a44d7c67da0cdc535. * Revert "deleted sync e2e test" This reverts commit 2f4a252650fb871662cbe92ba149aa9e2dc5d0c6. * Revert "removed syncService from connection test" This reverts commit 450ece6a34ec5ef04bf170c176f56de4e71b97c9. * removed sync functionality except list * removed deprecated imports --------- Co-authored-by: Ptroger <[email protected]> Co-authored-by: Pierre Troger <[email protected]> * Feature/nar 2059 make scoped sync quicker (#104) * fireblocks query uniquely scoped accounts, handle duplicated rawAccounts * optimized queries per wallet * anchorage scoped syncs swallow not found errors * batch requests to control rate limiting * removed connection-sync event handler * manage error handling in promiseAllSettled * use httpstatus * expect precisely one account * Asset fallback (#105) * Asset fallback for Anchorage and Fireblocks * Add TransferAssetService * Remove findTransferAsset from AssetService * Remove fallback type * Refactor TransferAssetService * Fix integration test types * Fix E2E tests * Fix provider --------- Co-authored-by: Ptroger <[email protected]> Co-authored-by: Matt Schoch <[email protected]> Co-authored-by: Pierre Troger <[email protected]>

* Migration, data is persisted per-connection (#88) * schema migration to scope all data per connection * data migration script * fix provider type per connection * unique keys order * fixed failing tests * added connectionId to mock data in transfer and sync tests * handle known destination connections to connectionId * abstract away single tie to connection change from known destinations * filter on correct row in wallet repository * updated migration script to duplicate provider resources * updates existing transfers * Pass through known destination (#90) * Add CSR public key format for FB connection (#92) * Add CSR public key format for FB connection * Add optional CSR format in the public key DTO * Add describe block * Pin node-forge version * Remove wallet connections list (#95) * Scope operations by connection (#96) * Scope operations by connection * Scope sync operations by connection * Fix supertest wrapper to work with query method * Use signed request util in the wallet E2E tests * Fix flaky tests * Add back connection list endpoint * Fix address E2E tests * Bump SDK version * Add network and asset endpoints in the SDK * Adding BitGo connection * ScopedSync by RawAccounts (#84) * duplicated sync, poc working * fixed rebase conflicts and format * cleanup rebase merge, correctly insert connectionId on scope sync * fireblocks scoped-sync updates * use vault label * added updates to anchorage's scoped sync service * add tests on anchorage scoped sync * cleaned up anchorage scope sync tests * map fireblocks assetWallets to network specific accounts * fixed not found test expectation * test fireblocks scoped sync * fixed rebase conflicts * changed externalId standard * Adding Raw Accounts endpoint & Assed/Network bootstrap * Adding includeAddress query param so fireblocks does not always fetch addresses * Cache assets & refactor network cache (#98) * Cache asset Refactor network to use cache manager * Remove draft config * Pin cache-manager version * Lower case cache keys * Fix findAll method * Endpoints to fetch scoped-syncs (#101) * scoped sync persists stringified raw accounts * add status fetching endpoints for scoped syncs * use prismaservice methods to json stringify and parse * Minor fixes before release connection-specific (#102) * Add back the nested wallet and account endpoints in the connection * Minor fix on scope sync logs Connection ID is mandatory in the Vault Client * connect a new connection triggers full sync on anchorage (#103) * connect a new connection triggers full sync on anchorage * fix test excpetation for empty raw accounts on anchorage * Add listProviderRawAccounts method in the Vault SDK * removed references to 'sync' and dropped table * removed syncService from connection test * deleted sync e2e test * re-generated sdk without scoped sync methods * Revert "re-generated sdk without scoped sync methods" This reverts commit 66dccdfa2bd909d2f6c015b153e87c3728004dd2. * Revert "removed references to 'sync' and dropped table" This reverts commit 9c2cd022b95439ce9bc4a45a44d7c67da0cdc535. * Revert "deleted sync e2e test" This reverts commit 2f4a252650fb871662cbe92ba149aa9e2dc5d0c6. * Revert "removed syncService from connection test" This reverts commit 450ece6a34ec5ef04bf170c176f56de4e71b97c9. * removed sync functionality except list * removed deprecated imports --------- Co-authored-by: Ptroger <[email protected]> Co-authored-by: Pierre Troger <[email protected]> * Feature/nar 2059 make scoped sync quicker (#104) * fireblocks query uniquely scoped accounts, handle duplicated rawAccounts * optimized queries per wallet * anchorage scoped syncs swallow not found errors * batch requests to control rate limiting * removed connection-sync event handler * manage error handling in promiseAllSettled * use httpstatus * expect precisely one account * Asset fallback (#105) * Asset fallback for Anchorage and Fireblocks * Add TransferAssetService * Remove findTransferAsset from AssetService * Remove fallback type * Refactor TransferAssetService * Fix integration test types * Fix E2E tests * Fix provider * Update Unified API examples * Remove pagination * Update README * Update quickstart README * Quickstart, Select Accounts section * Adding `.` before quickstart readme so it orders first in gist * Update SDK version * Ignore config.*.yaml files in git --------- Co-authored-by: Ptroger <[email protected]> Co-authored-by: Matt Schoch <[email protected]> Co-authored-by: Pierre Troger <[email protected]>

samuel added 6 commits February 6, 2024 17:38

wip

c46781f

Merge remote-tracking branch 'origin/main' into rego-template-endpoint

d9d1532

fix

9070f69

fix

8f577cf

fix

f1666d5

fix

2ab7f70

wcalderipe approved these changes Feb 7, 2024

View reviewed changes

samuel and others added 21 commits February 7, 2024 12:44

Merge remote-tracking branch 'origin/main' into rego-template-endpoint

744ea93

fix

ae4d1b0

CR

d34c0be

wip

ed11311

Work in progress with Sam

8cb4862

Working example

c64b1c2

Work example of union type array validation

23c93b8

wip

db0166c

Merge remote-tracking branch 'origin/main' into rego-template-endpoint

29b067d

fix

2a780e5

fix

9b3b72d

delete

cc079f2

delete

04a5f9d

done

ecd1912

fix

bde44d3

wip

21eecfe

Merge remote-tracking branch 'origin/main' into rego-template-endpoint

26fdc7a

fix

8ff00bd

fix test

9b0c30b

fix

df18e77

fix

af43302

wcalderipe reviewed Feb 9, 2024

View reviewed changes

fix

c32ab06

samuel added 5 commits February 9, 2024 14:56

CR

e5852d1

FIx circular dep

616f4b9

fix

b9b22ca

Merge remote-tracking branch 'origin/main' into rego-template-endpoint

40441f9

fix

e82a83c

samteb merged commit 45008d0 into main Feb 12, 2024
3 checks passed

samteb deleted the rego-template-endpoint branch February 12, 2024 07:34

mattschoch pushed a commit that referenced this pull request Jun 19, 2024

Rego template endpoint with DTOs (#96)

532c28b

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Rego template endpoint #96

Rego template endpoint #96

samteb commented Feb 6, 2024

wcalderipe Feb 7, 2024

samteb Feb 9, 2024

wcalderipe Feb 7, 2024 •

edited

Loading

wcalderipe Feb 7, 2024

Rego template endpoint #96

Rego template endpoint #96

Conversation

samteb commented Feb 6, 2024

wcalderipe Feb 7, 2024

Choose a reason for hiding this comment

samteb Feb 9, 2024

Choose a reason for hiding this comment

wcalderipe Feb 7, 2024 • edited Loading

Choose a reason for hiding this comment

wcalderipe Feb 7, 2024

Choose a reason for hiding this comment

wcalderipe Feb 7, 2024 •

edited

Loading