initial workload identity addition

[milesmatthias]

No description provided.

[morgante]

Code LGTM, just need to:

• Fix CI tests
• Add the new test fixture to the Cloud Build tests

[milesmatthias]

I thought I did add the workload identity tests to CI by adding it to the kitchen.yml: https://github.com/terraform-google-modules/terraform-google-kubernetes-engine/pull/307/files#diff-d3fbd21bc6be7ce4531c27a82b2619de ?

Nevermind, I see where I need to add this.

[milesmatthias]

And CI is failing because of a different test, not mine, so I'll look at it, but it may be most efficient to review with whoever wrote those tests.

[bharathkkb]

I had to add identity_namespace = "${module.example-project.project_id}.svc.id.goog" to have workload identity enabled with //modules/beta-public-cluster/.
Otherwise workload identity fails with Error 400: Identity namespace does not exist (proj-id.svc.id.goog)

[bharathkkb]

I think we need the kubernetes provider also initialized here like:

provider "kubernetes" {
  version                = "~> 1.10"
  host                   = module.gke.endpoint
  cluster_ca_certificate = base64decode(module.gke.ca_certificate)
}

Without this kubernetes provider will try to use the credentials from kubectl.

[milesmatthias]

@morgante I'm having trouble finding where the CI tools set which IAM credentials it's using to run tests. I'm running into these errors when running in cloudbuild:

Error: Error creating Network: googleapi: Error 403: Required 'compute.networks.create' permission for 'projects/ci-gke-dcf1/global/networks/cft-gke-test-9u5a', forbidden
       
on network.tf line 27, in resource "google_compute_network" "main":
27: resource "google_compute_network" "main" {
       
Error: Error creating service account: googleapi: Error 403: Permission iam.serviceAccounts.create is required to perform this operation on project projects/ci-gke-dcf1., forbidden
       
on ../../../sa.tf line 37, in resource "google_service_account" "
37: resource "google_service_account" "cluster_service_account" {

Any advice would be appreciated.

[bharathkkb]

@milesmatthias I received 403s on my PR yesterday as well. This is due to terraform-google-modules/terraform-google-project-factory#333
Prepare stage for GKE seems to be using ~> 2.18 bumping it to 2.20.1 seems to fix prepare for me.
https://github.com/bharathkkb/terraform-google-kubernetes-engine/blob/8d568de156660d8fc7d049d9645c7b25a0311d64/test/setup/versions.tf#L22

[milesmatthias]

I may need to bring in some of @bharathkkb's work from #381 to get this working... It seems IAM permissions in this CI test environment are misconfigured for this repo.

[milesmatthias]

@morgante would love any thoughts you have here to help unblock me. Thanks!

[morgante]

@milesmatthias Try rebasing on master. The key changes have been merged already.

[milesmatthias]

Doing those tests here: #391

[milesmatthias]

@morgante with updating from master, the integration service account doesn't have the right perms, even though we're giving it SA admin role in test/setup:

       Error: serviceaccounts is forbidden: User "system:anonymous" cannot create resource "serviceaccounts" in API group "" in the namespace "default"
       
         on ../../../modules/workload-identity/main.tf line 31, in resource "kubernetes_service_account" "main":
         31: resource "kubernetes_service_account" "main" {

Why is cloudbuild running as system:anonymous?

[morgante]

I think it might be because we authenticate the Kubernetes provider using the CA certificate?

Could you try creating a separate Kubernetes provider config specific to the workload identity test?

[bharathkkb]

@milesmatthias can I help in anyway to push this out? I have been planning to use this submodule in a project

[morgante]

@bharathkkb Since @milesmatthias isn't working with Google anymore, it might be a good idea for you to take this over and (a) fix merge conflicts + (b) get tests passing.

[bharathkkb]

@morgante sg, I will do that

[bharathkkb]

closed via #417