Orphaned Group Aliases Created when When Re-Using Group Canonical ID

[arusso]

Describe the bug

When creating group aliases, if one is to use a canonical_id already associated with another group alias an orphaned group alias is created. This orphaned alias is not updatable (to change canonical_id) nor deletable.

To Reproduce

Steps to reproduce the behavior

note: while userpass doesnt support groups, it appears to exhibit the same behavior that we observed in our ldap/okta setup, and so was used for simplicity.

1. Setup our auth backends (2x userpass) and record their accessors:

vault auth enable -path=up1/ userpass
vault auth enable -path=up2/ userpass
UP1_MNT_ACC=$(vault auth list -format=json  | jq -r '.["up1/"].accessor')
UP2_MNT_ACC=$(vault auth list -format=json  | jq -r '.["up2/"].accessor')

2. Create our group and 2x group aliases for that group

GRP_ID=$(vault write -field id identity/group name=fubar type=external)
UP1_GAID=$(vault write -field id identity/group-alias name=grp1 mount_accessor=$UP1_MNT_ACC canonical_id=$GRP_ID)
UP2_GAID=$(vault write -field id identity/group-alias name=grp2 mount_accessor=$UP2_MNT_ACC canonical_id=$GRP_ID)

3. Observe both group alias objects were successfully created using the same canonical_id despite violating the 1:1 relationship between group/group-alias.

vault read identity/group-alias/id/$UP1_GAID
vault read identity/group-alias/id/$UP2_GAID 

4. Observer group considers the 2nd group alias ($UP2_GAID) as it's alias:

vault read -field alias identity/group/id/$GRP_ID

At this point, we have a few ways to make things fail:

1. Attempt to create a new group and fix the canonical_id on the first alias, observing that we are unable to do so:

GRP2_ID=$(vault write -field id identity/group name=fubar2 type=external)
vault write identity/group-alias/id/$UP1_GAID \
    name=grp1 \
    canonical_id=$GRP2_ID \
    mount_accessor=$UP1_MNT_ACC

2. Attempt to delete the alias and observe it does not delete despite the success message:

vault delete identity/group-alias/id/$UP1_GAID
vault read identity/group-alias/id/$UP1_GAID

3. Attempt a 2nd delete and observe an EOF error along with a stack trace on the vault server:

vault delete identity/group-alias/id/$UP1_GAID

Example stack trace output from vault server:

2018-09-17T16:36:54.676-0700 [INFO ] http: panic serving 127.0.0.1:48690: runtime error: invalid memory address or nil pointer dereference
goroutine 350 [running]:
net/http.(*conn).serve.func1(0xc42020ee60)
        /goroot/src/net/http/server.go:1726 +0xd0
panic(0x21fed60, 0x3bc4e30)
        /goroot/src/runtime/panic.go:502 +0x229
github.com/hashicorp/vault/vault.(*IdentityStore).pathGroupAliasIDDelete.func1(0x28fa8a0, 0xc420577240, 0xc420631440, 0xc4202554a0, 0x0, 0x0, 0x0)
        /gopath/src/github.com/hashicorp/vault/vault/identity_store_group_aliases.go:282 +0x309
github.com/hashicorp/vault/logical/framework.(*Backend).HandleRequest(0xc420902f70, 0x28fa8a0, 0xc420577240, 0xc420631440, 0x0, 0x0, 0x0)
        /gopath/src/github.com/hashicorp/vault/logical/framework/backend.go:224 +0x475
github.com/hashicorp/vault/vault.(*Router).routeCommon(0xc42023e840, 0x28fa8a0, 0xc420577240, 0xc420631440, 0x0, 0x0, 0x0, 0x0, 0x0)
        /gopath/src/github.com/hashicorp/vault/vault/router.go:523 +0x7c2
github.com/hashicorp/vault/vault.(*Router).Route(0xc42023e840, 0x28fa8a0, 0xc420577240, 0xc420631440, 0xc4206a22a0, 0x0, 0x0)
        /gopath/src/github.com/hashicorp/vault/vault/router.go:378 +0x4e
github.com/hashicorp/vault/vault.(*Core).handleRequest(0xc4205fc900, 0x28fa8a0, 0xc420577240, 0xc420631440, 0x0, 0x0, 0x0, 0x0)
        /gopath/src/github.com/hashicorp/vault/vault/request_handling.go:523 +0xcfb
github.com/hashicorp/vault/vault.(*Core).HandleRequest(0xc4205fc900, 0x28fa960, 0xc420570ed0, 0xc420631440, 0x0, 0x0, 0x0)
        /gopath/src/github.com/hashicorp/vault/vault/request_handling.go:320 +0xefe
github.com/hashicorp/vault/http.request(0xc4205fc900, 0x28f8da0, 0xc420abc460, 0xc420272c00, 0xc420631440, 0x0, 0x0)
        /gopath/src/github.com/hashicorp/vault/http/handler.go:439 +0x62
github.com/hashicorp/vault/http.handleLogical.func1(0x28f8da0, 0xc420abc460, 0xc420272c00)
        /gopath/src/github.com/hashicorp/vault/http/logical.go:152 +0xed
net/http.HandlerFunc.ServeHTTP(0xc42069fe20, 0x28f8da0, 0xc420abc460, 0xc420272c00)
        /goroot/src/net/http/server.go:1947 +0x44
github.com/hashicorp/vault/http.handleRequestForwarding.func1(0x28f8da0, 0xc420abc460, 0xc420272c00)
        /gopath/src/github.com/hashicorp/vault/http/handler.go:391 +0x1ca
net/http.HandlerFunc.ServeHTTP(0xc42069fe40, 0x28f8da0, 0xc420abc460, 0xc420272c00)
        /goroot/src/net/http/server.go:1947 +0x44
net/http.(*ServeMux).ServeHTTP(0xc4204c0de0, 0x28f8da0, 0xc420abc460, 0xc420272c00)
        /goroot/src/net/http/server.go:2337 +0x130
github.com/hashicorp/vault/http.wrapHelpHandler.func1(0x28f8da0, 0xc420abc460, 0xc420272c00)
        /gopath/src/github.com/hashicorp/vault/http/help.go:22 +0x158
net/http.HandlerFunc.ServeHTTP(0xc42069fea0, 0x28f8da0, 0xc420abc460, 0xc420272c00)
        /goroot/src/net/http/server.go:1947 +0x44
github.com/hashicorp/vault/http.wrapCORSHandler.func1(0x28f8da0, 0xc420abc460, 0xc420272c00)
        /gopath/src/github.com/hashicorp/vault/http/cors.go:29 +0x545
net/http.HandlerFunc.ServeHTTP(0xc42069fec0, 0x28f8da0, 0xc420abc460, 0xc420272c00)
        /goroot/src/net/http/server.go:1947 +0x44
github.com/hashicorp/vault/http.wrapGenericHandler.func1(0x28f8da0, 0xc420abc460, 0xc420272b00)
        /gopath/src/github.com/hashicorp/vault/http/handler.go:147 +0x143
net/http.HandlerFunc.ServeHTTP(0xc4204c0f00, 0x28f8da0, 0xc420abc460, 0xc420272b00)
        /goroot/src/net/http/server.go:1947 +0x44
github.com/hashicorp/vault/vendor/github.com/hashicorp/go-cleanhttp.PrintablePathCheckHandler.func1(0x28f8da0, 0xc420abc460, 0xc420272b00)
        /gopath/src/github.com/hashicorp/vault/vendor/github.com/hashicorp/go-cleanhttp/handlers.go:40 +0xcf
net/http.HandlerFunc.ServeHTTP(0xc42069fee0, 0x28f8da0, 0xc420abc460, 0xc420272b00)
        /goroot/src/net/http/server.go:1947 +0x44
net/http.serverHandler.ServeHTTP(0xc42069d930, 0x28f8da0, 0xc420abc460, 0xc420272b00)
        /goroot/src/net/http/server.go:2694 +0xbc
net/http.(*conn).serve(0xc42020ee60, 0x28fa8a0, 0xc420577040)
        /goroot/src/net/http/server.go:1830 +0x651
created by net/http.(*Server).Serve
        /goroot/src/net/http/server.go:2795 +0x27b

4. Attempt to delete the group and observe the grp2 alias is deleted but grp1 still exists:

vault delete identity/group/id/$GRP_ID
vault read identity/group-alias/id/$UP1_GAID
vault read identity/group-alias/id/$UP2_GAID

Expected behavior

• An error should be thrown when a second group alias is created with the same canonical id.
• (optional) A flag should be provided to allow skipping the current canonical_id against an existing group checks when updating or deleting a group-alias.

Environment:

note: this was experienced in our production environment (first vault status) and reproduced on a dev instance:

CLI:

$ vault version
Vault v0.11.0 ('87492f9258e0227f3717e3883c6a8be5716bf564')

Production:

$ VAULT_ADDR=https://vault.example.com:8200 vault status
Key             Value
---             -----
Seal Type       shamir
Sealed          false
Total Shares    6
Threshold       2
Version         0.11.0
Cluster Name    company-name
Cluster ID      2e09c8dd-feed-dead-beef-73843effa59c
HA Enabled      true
HA Cluster      n/a
HA Mode         active

Local Dev Instance:

$ vault status
Key             Value
---             -----
Seal Type       shamir
Sealed          false
Total Shares    1
Threshold       1
Version         0.11.0
Cluster Name    vault-cluster-992ee882
Cluster ID      215cb217-486c-6bdf-da5b-3edef71c61a3
HA Enabled      false

All systems are running RHEL 7.x x86_64

Vault server configuration file(s):

storage "consul" {
  address = "127.0.0.1:8500"
  path    = "secrets/"
}

ui = true
cluster_name = "company"
pid_file = "/var/run/vault.pid"
disable_clustering = true

listener "tcp" {
  address     = "hostip:8200"
  tls_cert_file = "/etc/pki/tls/certs/vault.crt"
  tls_key_file = "/etc/pki/tls/private/vault.key"
  tls_min_version = "tls12"
  proxy_protocol_behavior = "allow_authorized"
  proxy_protocol_authorized_addrs = "ip1,ip2,ip3"
}

api_addr = "https://hostname.example.com:8200"

Additional context

All reproduction was done with a root token.

In cases where the group is deleted (and possibly even when it is not), the orphaned group-alias will be unable to resolve the group and will prevent users within that group from authenticating completely.

In our production environment, in place of userpass auth backends we had ldap and okta backends which prevented users from the impacted groups from logging in via ldap (our primary means) since it was the first alias created and therefore orphaned.