Allow `identity` mount to be tunable #14723

ianferguson · 2022-03-25T13:54:56Z

resolves #14696

I didn't expect this to be so simple, but it seems like removing identity from the "non tunable" mounts list just works, at least for the audit hmac tuning I've tested.

There may be other impacts I'm not aware of due to this, if y'all can think of any you want me to test for specifically please let me know, I'm happy to do so

Testing

Setup

I built my branch locally and started a Vault test server

> bin/vault server -dev -dev-root-token-id=test

Execution

In another shell I ran the follow commands to enable audit logs, write some groups, tune identity to allow the name field to be printed in plaintext in audit logs, and then write some groups again:

# turn on audit log
>  vault audit enable file file_path=/tmp/vault.log
Success! Enabled the file audit device at: file/

# make read/write requests to show that by default audit logs still hmac the identity fields
> vault write identity/group name=a-group
Key     Value
---     -----
id      5fdab8ca-5127-97c6-38ce-0467373f91ad
name    a-group

> vault read identity/group/name/a-group
Key                  Value
---                  -----
alias                map[]
creation_time        2022-03-25T13:48:42.909741Z
id                   5fdab8ca-5127-97c6-38ce-0467373f91ad
last_update_time     2022-03-25T13:48:42.909741Z
member_entity_ids    []
member_group_ids     <nil>
metadata             <nil>
modify_index         1
name                 a-group
namespace_id         root
parent_group_ids     <nil>
policies             <nil>
type                 internal

# tune the identity mount to allow `name` to print
> vault secrets tune -audit-non-hmac-request-keys=name -audit-non-hmac-response-keys=name identity
Success! Tuned the secrets engine at: identity/

# make the same read/write requests for another group to show that now the audit logs print the `name` in response/request audit events 

> vault write identity/group name=b-group
Key     Value
---     -----
id      ff5011a0-d904-d098-63b2-501d9f497c86
name    b-group

> vault read identity/group/name/b-group
Key                  Value
---                  -----
alias                map[]
creation_time        2022-03-25T13:49:06.218824Z
id                   ff5011a0-d904-d098-63b2-501d9f497c86
last_update_time     2022-03-25T13:49:06.218824Z
member_entity_ids    []
member_group_ids     <nil>
metadata             <nil>
modify_index         1
name                 b-group
namespace_id         root
parent_group_ids     <nil>
policies             <nil>
type                 internal

Results: How that looked in the audit logs

Before tuning (`name` in request/response is hmac'd)

{
  "request": {
    "id": "a880573c-e072-af6c-64cc-dc63b127aeef",
    "operation": "read",
    "mount_type": "identity",
    "client_token": "hmac-sha256:8b4822c1345b3e43dba5d0aa12080981f593e478ae3dfcc232036cce52e2cd0e",
    "client_token_accessor": "hmac-sha256:7c584491e9f651d53424048d122fe9ec93297ac37900e2077891bb189df4454b",
    "namespace": {
      "id": "root"
    },
    "path": "identity/group/name/a-group",
    "remote_address": "127.0.0.1",
    "remote_port": 62417
  },
  "response": {
    "mount_type": "identity",
    "data": {
      "alias": {},
      "creation_time": "hmac-sha256:bf9f710dc5a8acd12925d6ece25a52cc3c3168bab8f7dfbafc7929a8c6a8fe2c",
      "id": "hmac-sha256:bcea8f768a75e7c291f27761ccc25f6b988fde6d61e322f077f4592ed8764e49",
      "last_update_time": "hmac-sha256:bf9f710dc5a8acd12925d6ece25a52cc3c3168bab8f7dfbafc7929a8c6a8fe2c",
      "member_entity_ids": [],
      "member_group_ids": null,
      "metadata": null,
      "modify_index": 1,
      "name": "hmac-sha256:c145de2c8cef9005a0677f858fbb89f988addc7f868cb06dad63a4cfad54ae0d",
      "namespace_id": "hmac-sha256:d2d2bee396f2bb4526845aa0736bc241d857de704d4a9d460031248c9bad2a2d",
      "parent_group_ids": null,
      "policies": null,
      "type": "hmac-sha256:793110e4ab7e6151e89f1f26aad53d4470774ec189380538abc0d0090faf65be"
    }
  }
}
{
  "time": "2022-03-25T13:48:57.720818Z",
  "type": "request",
  "auth": {
    "client_token": "hmac-sha256:8b4822c1345b3e43dba5d0aa12080981f593e478ae3dfcc232036cce52e2cd0e",
    "accessor": "hmac-sha256:7c584491e9f651d53424048d122fe9ec93297ac37900e2077891bb189df4454b",
    "display_name": "token",
    "policies": [
      "root"
    ],
    "token_policies": [
      "root"
    ],
    "token_type": "service",
    "token_issue_time": "2022-03-25T09:48:26-04:00"
  },
  "request": {
    "id": "b09197a2-57e8-82f2-1540-60db5621a03d",
    "client_id": "0DHqvq2D77kL2/JTPSZkTMJbkFVmUu0TzMi0jiXcFy8=",
    "operation": "update",
    "mount_type": "system",
    "client_token": "hmac-sha256:8b4822c1345b3e43dba5d0aa12080981f593e478ae3dfcc232036cce52e2cd0e",
    "client_token_accessor": "hmac-sha256:7c584491e9f651d53424048d122fe9ec93297ac37900e2077891bb189df4454b",
    "namespace": {
      "id": "root"
    },
    "path": "sys/mounts/identity/tune",
    "data": {
      "audit_non_hmac_request_keys": [
        "hmac-sha256:5e65419a2601b1d743429f0f63c13e54250141211ce7c7c4be0aa95bb931a3af"
      ],
      "audit_non_hmac_response_keys": [
        "hmac-sha256:5e65419a2601b1d743429f0f63c13e54250141211ce7c7c4be0aa95bb931a3af"
      ],
      "default_lease_ttl": "hmac-sha256:db1aae827d2f648e42d41235c83a3efebe28825349fb68d690662dc7fa15040f",
      "force_no_cache": false,
      "max_lease_ttl": "hmac-sha256:db1aae827d2f648e42d41235c83a3efebe28825349fb68d690662dc7fa15040f",
      "options": null
    },
    "remote_address": "127.0.0.1",
    "remote_port": 62418
  }
}

After tuning (`name` is printed in plaintext`):

{
  "time": "2022-03-25T13:49:06.218756Z",
  "type": "request",
  "auth": {
    "client_token": "hmac-sha256:8b4822c1345b3e43dba5d0aa12080981f593e478ae3dfcc232036cce52e2cd0e",
    "accessor": "hmac-sha256:7c584491e9f651d53424048d122fe9ec93297ac37900e2077891bb189df4454b",
    "display_name": "token",
    "policies": [
      "root"
    ],
    "token_policies": [
      "root"
    ],
    "token_type": "service",
    "token_issue_time": "2022-03-25T09:48:26-04:00"
  },
  "request": {
    "id": "7b19034c-3cc4-c859-b314-98d4f0b46db7",
    "client_id": "0DHqvq2D77kL2/JTPSZkTMJbkFVmUu0TzMi0jiXcFy8=",
    "operation": "update",
    "mount_type": "identity",
    "client_token": "hmac-sha256:8b4822c1345b3e43dba5d0aa12080981f593e478ae3dfcc232036cce52e2cd0e",
    "client_token_accessor": "hmac-sha256:7c584491e9f651d53424048d122fe9ec93297ac37900e2077891bb189df4454b",
    "namespace": {
      "id": "root"
    },
    "path": "identity/group",
    "data": {
      "name": "b-group"
    },
    "remote_address": "127.0.0.1",
    "remote_port": 62419
  }
}
{
  "time": "2022-03-25T13:49:06.218989Z",
  "type": "response",
  "auth": {
    "client_token": "hmac-sha256:8b4822c1345b3e43dba5d0aa12080981f593e478ae3dfcc232036cce52e2cd0e",
    "accessor": "hmac-sha256:7c584491e9f651d53424048d122fe9ec93297ac37900e2077891bb189df4454b",
    "display_name": "token",
    "policies": [
      "root"
    ],
    "token_policies": [
      "root"
    ],
    "token_type": "service",
    "token_issue_time": "2022-03-25T09:48:26-04:00"
  },
  "request": {
    "id": "7b19034c-3cc4-c859-b314-98d4f0b46db7",
    "operation": "update",
    "mount_type": "identity",
    "client_token": "hmac-sha256:8b4822c1345b3e43dba5d0aa12080981f593e478ae3dfcc232036cce52e2cd0e",
    "client_token_accessor": "hmac-sha256:7c584491e9f651d53424048d122fe9ec93297ac37900e2077891bb189df4454b",
    "namespace": {
      "id": "root"
    },
    "path": "identity/group",
    "data": {
      "name": "b-group"
    },
    "remote_address": "127.0.0.1",
    "remote_port": 62419
  },
  "response": {
    "mount_type": "identity",
    "data": {
      "id": "hmac-sha256:aa7c5e0a8c96bb3a17b6f90e6c6f12e5c8c362e611c9014ef5426603f06d3c9b",
      "name": "b-group"
    }
  }
}
{
  "time": "2022-03-25T13:49:09.954194Z",
  "type": "request",
  "auth": {
    "client_token": "hmac-sha256:8b4822c1345b3e43dba5d0aa12080981f593e478ae3dfcc232036cce52e2cd0e",
    "accessor": "hmac-sha256:7c584491e9f651d53424048d122fe9ec93297ac37900e2077891bb189df4454b",
    "display_name": "token",
    "policies": [
      "root"
    ],
    "token_policies": [
      "root"
    ],
    "token_type": "service",
    "token_issue_time": "2022-03-25T09:48:26-04:00"
  },
  "request": {
    "id": "41ebb69e-828e-b8fe-48a2-c73d22d58d78",
    "client_id": "0DHqvq2D77kL2/JTPSZkTMJbkFVmUu0TzMi0jiXcFy8=",
    "operation": "read",
    "mount_type": "identity",
    "client_token": "hmac-sha256:8b4822c1345b3e43dba5d0aa12080981f593e478ae3dfcc232036cce52e2cd0e",
    "client_token_accessor": "hmac-sha256:7c584491e9f651d53424048d122fe9ec93297ac37900e2077891bb189df4454b",
    "namespace": {
      "id": "root"
    },
    "path": "identity/group/name/b-group",
    "remote_address": "127.0.0.1",
    "remote_port": 62420
  }
}
{
  "time": "2022-03-25T13:49:09.954345Z",
  "type": "response",
  "auth": {
    "client_token": "hmac-sha256:8b4822c1345b3e43dba5d0aa12080981f593e478ae3dfcc232036cce52e2cd0e",
    "accessor": "hmac-sha256:7c584491e9f651d53424048d122fe9ec93297ac37900e2077891bb189df4454b",
    "display_name": "token",
    "policies": [
      "root"
    ],
    "token_policies": [
      "root"
    ],
    "token_type": "service",
    "token_issue_time": "2022-03-25T09:48:26-04:00"
  },
  "request": {
    "id": "41ebb69e-828e-b8fe-48a2-c73d22d58d78",
    "operation": "read",
    "mount_type": "identity",
    "client_token": "hmac-sha256:8b4822c1345b3e43dba5d0aa12080981f593e478ae3dfcc232036cce52e2cd0e",
    "client_token_accessor": "hmac-sha256:7c584491e9f651d53424048d122fe9ec93297ac37900e2077891bb189df4454b",
    "namespace": {
      "id": "root"
    },
    "path": "identity/group/name/b-group",
    "remote_address": "127.0.0.1",
    "remote_port": 62420
  },
  "response": {
    "mount_type": "identity",
    "data": {
      "alias": {},
      "creation_time": "hmac-sha256:3cda9fccd26e9736444df512fc1838084bcc8e1a9df68c68a4aa0ff69a97c993",
      "id": "hmac-sha256:aa7c5e0a8c96bb3a17b6f90e6c6f12e5c8c362e611c9014ef5426603f06d3c9b",
      "last_update_time": "hmac-sha256:3cda9fccd26e9736444df512fc1838084bcc8e1a9df68c68a4aa0ff69a97c993",
      "member_entity_ids": [],
      "member_group_ids": null,
      "metadata": null,
      "modify_index": 1,
      "name": "b-group",
      "namespace_id": "hmac-sha256:d2d2bee396f2bb4526845aa0736bc241d857de704d4a9d460031248c9bad2a2d",
      "parent_group_ids": null,
      "policies": null,
      "type": "hmac-sha256:793110e4ab7e6151e89f1f26aad53d4470774ec189380538abc0d0090faf65be"
    }
  }
}

heatherezell · 2022-04-04T19:18:30Z

Hi there! Thanks for this contribution. I'll get some eyes on it, but please don't forget to add a changelog entry. Thanks! :)

ianferguson · 2022-04-06T17:46:30Z

@hsimon-hashicorp thanks! changelog added in e9114b5a9

maxb · 2022-05-08T10:44:54Z

I happened to notice this, and wondered - is there any reason to keep the concept of an "untunable mount" at all?

vault/vault/mount.go

Lines 86 to 91 in 254d8da

    
           untunableMounts = []string{ 
        
           	cubbyholeMountPath, 
        
           	systemMountPath, 
        
           	"audit/", 
        
           	identityMountPath, 
        
           }

Wanting to tune fields displayed in the audit logs for sys/ seems quite possible. For cubbyhole/ it's less likely but technically possibly a useful debugging technique. And audit/ isn't even a path used by Vault at all, AFAIK - I'm guessing someone just stuck it in here whilst contemplating making audit backends more like secrets engines and auth methods, but the change was never implemented?

Just something to think about whilst people's thoughts are in this area.

HridoyRoy · 2022-05-10T19:25:28Z

I happened to notice this, and wondered - is there any reason to keep the concept of an "untunable mount" at all?

vault/vault/mount.go

Lines 86 to 91 in 254d8da

untunableMounts = []string{

cubbyholeMountPath,

systemMountPath,

"audit/",

identityMountPath,

}

Wanting to tune fields displayed in the audit logs for sys/ seems quite possible. For cubbyhole/ it's less likely but technically possibly a useful debugging technique. And audit/ isn't even a path used by Vault at all, AFAIK - I'm guessing someone just stuck it in here whilst contemplating making audit backends more like secrets engines and auth methods, but the change was never implemented?

Just something to think about whilst people's thoughts are in this area.

Hi @maxb , some mounts such as the identity mount are more sensitive than others. As such, we'd like to make these mounts as standardized as possible from environment to environment to reduce the chances of complicated bugs arising due to, say, tuning.

HridoyRoy

Hi @ianferguson, thanks for submitting this PR!

Regarding the this particular change/request, I do agree that users should have the ability to change audit settings for identity if they so choose. However, I'm unclear about the repercussions of tuning the other parameters for this mount. Would you please update this PR in such a way that only a subset of the tunable parameters can be submitted for certain mounts (like the identity mount)?

ianferguson · 2022-07-28T20:33:13Z

@HridoyRoy just getting back to this now

Reading the mount tuning table on Vault v1.11.1 several of the tuning values are already set by default:

> vault read sys/mounts/identity/tune
Key                            Value
---                            -----
default_lease_ttl              768h
description                    identity store
force_no_cache                 false
max_lease_ttl                  768h
passthrough_request_headers    [Authorization]

looking at the secrets mount tuning parameters they all seem like they'd have non-catastrophic and reasonable behaviors when set

I tested tuning all possible values on a build of this branch:

$ vault secrets tune \
  -allowed-response-headers="Authorization" \
  -audit-non-hmac-request-keys="id" \ 
  -audit-non-hmac-response-keys="id" \
  -default-lease-ttl="769h" \
  -description="Custom identity" \
  -listing-visibility="hidden" \
  -max-lease-ttl="780h" \
  -passthrough-request-headers "Authorization" \
  -allowed-managed-keys "something" \ 
  identity/
Success! Tuned the secrets engine at: identity/

$ vault read sys/mounts/identity/tune
Key                             Value
---                             -----
allowed_managed_keys            [something]
allowed_response_headers        [Authorization]
audit_non_hmac_request_keys     [id]
audit_non_hmac_response_keys    [id]
default_lease_ttl               769h
description                     Custom identity
force_no_cache                  false
listing_visibility              hidden
max_lease_ttl                   780h
passthrough_request_headers     [Authorization]

and at least a few entity create/lookup type commands all worked fine before and after those tuning settings.

I think it should be safe for the identity mount to be tuned the same as a "normal" secrets mount

ianferguson · 2022-09-15T21:40:22Z

@HridoyRoy would you or someone else be able to review my last response? #14723 (comment)

VioletHynes · 2024-06-04T14:55:43Z

Hey there @ianferguson ! I'm sorry this languished a little without eyes on it. I've been exploring the code/area and I think this makes sense to merge. I appreciate your comment illustrating all of the options, too!

Before we can merge it, you'll need to pull main into your branch, and there'll be a conflict I think, but an easy one (the line you're removing got renamed to mountPathIdentity). Once that gets updated, I'll be happy to approve and merge this in if all the tests pass etc.

I really appreciate your patience, and I'm sorry this waited for so long. I'm eager to see this one over the finish line to the end, though, so rest assured that your efforts won't be in vain.

VioletHynes

Approved! Thanks for being so patient, and for responding to the feedback. I'm sorry this one took so long and slipped through the cracks a little, but I'm super eager to get this merged and I'm really glad we could get this in finally.

ianferguson · 2024-06-12T18:58:29Z

@VioletHynes thanks for merging so quickly, I was waiting for tests to pass before pinging you!

heatherezell added the core/identity label Apr 4, 2022

heatherezell requested a review from hghaf099 April 6, 2022 19:22

heatherezell requested a review from HridoyRoy May 9, 2022 18:19

HridoyRoy suggested changes May 10, 2022

View reviewed changes

VioletHynes added the waiting-for-response label Jun 7, 2024

ianferguson added 2 commits June 12, 2024 11:30

Allow identity mount to be tunable

dca724b

Add changelog

42d55e0

ianferguson force-pushed the ianferguson/tunable_identity branch from e9114b5 to 42d55e0 Compare June 12, 2024 15:30

VioletHynes approved these changes Jun 12, 2024

View reviewed changes

VioletHynes merged commit 663c9a3 into hashicorp:main Jun 12, 2024
66 of 67 checks passed

github-actions bot removed the waiting-for-response label Jun 12, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Allow `identity` mount to be tunable #14723

Allow `identity` mount to be tunable #14723

ianferguson commented Mar 25, 2022 •

edited

Loading

heatherezell commented Apr 4, 2022

ianferguson commented Apr 6, 2022

maxb commented May 8, 2022

HridoyRoy commented May 10, 2022

HridoyRoy left a comment

ianferguson commented Jul 28, 2022

ianferguson commented Sep 15, 2022

VioletHynes commented Jun 4, 2024

VioletHynes left a comment

ianferguson commented Jun 12, 2024

Allow identity mount to be tunable #14723

Allow identity mount to be tunable #14723

Conversation

ianferguson commented Mar 25, 2022 • edited Loading

Testing

Setup

Execution

Results: How that looked in the audit logs

Before tuning (name in request/response is hmac'd)

After tuning (name is printed in plaintext`):

heatherezell commented Apr 4, 2022

ianferguson commented Apr 6, 2022

maxb commented May 8, 2022

HridoyRoy commented May 10, 2022

HridoyRoy left a comment

Choose a reason for hiding this comment

ianferguson commented Jul 28, 2022

ianferguson commented Sep 15, 2022

VioletHynes commented Jun 4, 2024

VioletHynes left a comment

Choose a reason for hiding this comment

ianferguson commented Jun 12, 2024

Allow `identity` mount to be tunable #14723

Allow `identity` mount to be tunable #14723

ianferguson commented Mar 25, 2022 •

edited

Loading

Before tuning (`name` in request/response is hmac'd)

After tuning (`name` is printed in plaintext`):