[vault] unable to create provider

[phac008]

On kind Deployment only - following error occurs when configuring oidc provider via cmdline:
“unable to create provider: oidc: issuer did not match the issuer returned by provider” with keycloak idp

oidc_discovery_url=https://keycloak-127-0-0-1.nip.io/realms/sx-cnp-oss
2024-08-06T21:22:55.065Z [ERROR] auth.oidc.auth_oidc_28b6b006: error checking oidc discovery URL: error="error creating provider with given values: NewProvider: unable to create provider: Get "https://keycloak-127-0-0-1.nip.io/realms/sx-cnp-oss/.well-known/openid-configuration\": dial tcp 127.0.0.1:443: connect: connection refused"

oidc_discovery_url=http://keycloak-service.keycloak.svc.cluster.local:8080/realms/sx-cnp-oss
2024-08-06T21:36:42.019Z [ERROR] auth.oidc.auth_oidc_28b6b006: error checking oidc discovery URL: error="error creating provider with given values: NewProvider: unable to create provider: oidc: issuer did not match the issuer returned by provider, expected "http://keycloak-service.keycloak.svc.cluster.local:8080/realms/sx-cnp-oss\" got "https://keycloak-127-0-0-1.nip.io/realms/sx-cnp-oss\""

this is because of frontend_url defined in keycloak realm (which is needed for backstage oidc plugin)

Issues:
hashicorp/vault/issues/25024
https://discuss.hashicorp.com/t/unable-to-create-provider-oidc-issuer-did-not-match-the-issuer-returned-by-provider-with-keycloak-idp/61851

currently no Workaround after deployment via argo:
remove frontend url from Keycloak realm
add oidc config:

vault write auth/oidc/config oidc_discovery_url="https://keycloak-127-0-0-1.nip.io/realms/sx-cnp-oss" oidc_client_id="vault" oidc_client_secret="" default_role="default"

when re-add frontend url to keycloak

Missing auth_url. Please check that allowed_redirect_uris for the role include this mount path.

....
so for development backstage oidc login will not work so far...

[phac008]

rewrite rule as alternative to solve this issue?
spring-projects/spring-security#14633 (comment)

[phac008]

rewrite rule not successfully either...

[phac008]

hostname v2 and dynamic backend as next step
https://www.keycloak.org/server/hostname

[Aladex]

I encountered an issue where Vault needed to resolve the Keycloak domain (keycloak.example.com) to an internal IP, but the domain points to an external IP. Here's how I solved it:

1. Using hostAliases:
   I added a hostAliases entry on one of the nodes. Since my NGINX ingress is running as a DaemonSet with hostNetwork: true, this allowed Vault to resolve keycloak.example.com to the internal IP.

2. Alternative solution with a Service:
   Another approach is to create a Service that either points directly to Keycloak or to the NGINX ingress. By assigning a static IP to this Service, you can reference it from Vault and avoid exposing NGINX externally, while still ensuring that traffic is routed to Keycloak internally.

[phac008]

@Aladex
nice one - thank you guiding me back to right direction.
setup with hostAlias and additional service is working now. still some automation work to do, but for demo this should work

[phac008]

several different types of solution tested, 2 options left. Both of them come with additional changes needed. Still working on it

[phac008]

short description:
Variant with HostAlias and Service worked, but not that "k8s like". switched back to rewrite in coredns - more reliable regarding restarting/redeployment of single services. #778

A new issue occurred for necessary CA Cert for Vault OIDC connection - upbound/provider-vault#45, also there is another open issue, when oidc creation fails: upbound/provider-vault#25

so, currently oidc creation was disabled from crossplane and implemented in vault-initializer container.