Add readOnlyRootFilesystem to security context #2771
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
@mr-miles Could you tell us how you are using GateKeeper and why a policy in gatekeeper is requiring this? Thanks for the PR. |
|
Sorry @david-yu, you caught this before I'd made a ticket with more explanation in! Our network team use gatekeeper to enforce their security policy over our k8s cluster and previously consul was exempted from all the checks. There were a couple of updates to the helm chart recently to allow consul + sidecar to run in a restricted/unprivileged environment so we attempted to re-enabled it. On doing so, our "readonlyfilesystem" validation was flagging, and preventing sidecars from starting (I think our restricted environment is a little more restricted than just non-root user accounts). The change to readonly containers was small - but useful security hardening (plus it has caught some bugs for us in other areas in the past) - hence submitting a PR. Do you think this is a useful addition? |
|
Sorry did you mean you used the latest changes from #2755 to run in an un-privileged mode, and these were extra settings needed to allow your sidecars to run? In this case gateways are writing to |
|
Yes that's it - they are extra settings that I needed to make everything work. Though it was the changes in #2702 and #2572 that I had been looking at. Thinking about it, it doesn't need to be medium=memory since the bootstrap file doesn't have anything sensitive in. Any mount (disk or memory) of /tmp will avoid the "container filesystem is readonly" error message. |
pglass
approved these changes
Aug 17, 2023
pglass
added
pr/no-changelog
|
@pglass - thanks for looking at this - done! |
2 tasks
|
What version of the Helm chart are you? @mr-miles trying to understand what releases to backport this into. We can at least do consul k8s 1.2.x. |
|
1.2.x is fine for me - we're on 1.2.1 |
david-yu
pushed a commit
that referenced
this pull request
Aug 23, 2023
* Add readOnlyRootFilesystem to security context (#2771) --------- Co-authored-by: mr-miles <[email protected]> Co-authored-by: Paul Glass <[email protected]>
2 tasks
david-yu
pushed a commit
that referenced
this pull request
Aug 23, 2023
* Add readOnlyRootFilesystem to security context (#2771) --------- Co-authored-by: mr-miles <[email protected]> Co-authored-by: Paul Glass <[email protected]>
2 tasks
david-yu
pushed a commit
that referenced
this pull request
Aug 23, 2023
…2.x (#2831) Add readOnlyRootFilesystem to security context (#2771) (#2789) --------- Co-authored-by: mr-miles <[email protected]> Co-authored-by: Paul Glass <[email protected]>
david-yu
pushed a commit
that referenced
this pull request
Aug 25, 2023
hc-github-team-consul-core
added a commit
that referenced
this pull request
Aug 30, 2023
…2771)" into release/1.2.x (#2849) backport of commit ef6088c Co-authored-by: David Yu <[email protected]>
absolutelightning
added a commit
that referenced
this pull request
Sep 12, 2023
* test image form consul-enterprise * Revert "test image form consul-enterprise" This reverts commit 2fb794450c8d64a502ebdb296f6836de7be06d59. * Convert acceptance to use github actions (#2046) * Terraform: increase node sizes * update GKE to use already created subnets * Dispatch: dispatch to consul-k8s-workflows * Remove CircleCI (#2050) * Update status on PRs (#2054) * Update status on PRs * Split pr and push into 2 different files so that context can be passed through * Update backport assistant to support -gh-automerge (#2047) * Add a cleanup cron job (#2059) * Add a cleanup cron job * add sameness group CRD (#2048) * draft of adding sameness group CRD * move sameness group tests to ent test file * update tests * fix lint issues * generate yaml and update helm charts * update field descriptions and validation and its test * remove unwanted files, add license comments back * rename samenessgroups to samenessgroup * fix resource names * update failing unit test * Supply chain updates (#2072) * Fix Sync Catalog ACL Token Environment Var Name (#2068) * Fix Sync Catalog ACL Token Environment Var Name * Update ACL variable name in tests * Add changelog for NET 2422 (#2080) * add sameness group to exported services (#2075) * add sameness group to exported services * update CRDs * update deep copy * re add license line * check if sameness group is wildcard * remove experimental tag on peering fields * update error message case * update error message case in webhook test * Adjust API gateway controller deployment appropriately when Vault configured as secrets backend (#2083) * Adjust mount based on whether Vault is enabled as secrets backend * Add changelog entry * Improve wording of changelog entry * Use Vault serverca for CONSUL_CACERT when secrets backend enabled * Add comment to Helm template explaining logic * Add unit test for CONSUL_CACERT with Vault secret path * Add unit tests for removing mounts when Vault is secrets backend * Result of tsccr-helper -pin-all-workflows . (#2089) Co-authored-by: hashicorp-tsccr[bot] <hashicorp-tsccr[bot]@users.noreply.github.com> * set consul server locality from k8s node labels (#2093) * add sameness group to service resolver, update manifests (#2086) * add sameness group to service resolver, update manifests * get the latest api and update acceptance tests * get the latest api in acceptanc tests * update validation code, remove dynamic validations, update tests * check nil pointer * go get latest api * revert acceptance changes * add sameness group to source intention (#2097) * add sameness group to source intention * add more test coverage * add comment on metaValueMaxLength variable * fix comment lint issue * security: update Go version to 1.20.4 (#2102) * Spatel/net 1646 add max ejection percent and base ejection time (#2064) * Add MaxEjectionPercent and BaseEjectionTime to servicedefaults * test with sister branch in consul repo * missed one * fix tag names * fix json tags and duration type * update test * generate yaml files and fix imports --------- Co-authored-by: Semir Patel <[email protected]> * chore(ci): fix changelog action for non-main base branches (#2105) * chore(ci): fix backport assistant not finding new branches (#2113) * Customizing Vault Version for WanFed Test (#2043) * Customizing Vault Version for WanFed Test * Modified * Changed according to the review comments * Removed the commented line * Vault server version type changed to String * changed back to VaultServerVersion type * Changing "VaultServerVersion" to type "String" * add config read command (#2078) * add config read command * add tests * lint * update docs * add changelog * fix linting errors * PR feedback * Update CRDs for Permissive mTLS (#2100) * Add mutualTLSMode to service-defaults and proxy-defaults * Add allowEnablingPermisiveMutualTLS to mesh config entry * helm: add HOST_IP to mesh-gateway (#1808) * add HOST_IP to mesh-gateway * chore(ci): fix typo in changelog checker (#2127) * Add support for syncing Ingress hostname to the Consul Catalog (#2098) * Add support for syncing Ingress hostname to the Consul Catalog * fix changelog-checker syntax error * Add telemetry collector deployment to consul-k8s (#2134) * Create values.yaml section for telemetry-collector * Initial telemetry-collector validation and bats test * Add nodeSelector * Add connect-init initContainer * Add consul-dataplane container * Conditionally add ca-cert volume * Include vault annotations * Prune tests to pertinent test cases * Move consul server env vars * Check ca mount for dataplane container * Check correct env var * Set default resources * Set initContainer and tolerations * Support priorityClassName * Support setting initContainer resources * Fix replicas unit test * Turn off tproxy and remove unneeded security context * Set -tls-disabled if global.tls.enabled=false * Set -ca-certs correct if tls is enabled * Set external server args * Set partition flag tests * Label bats tests, remove duplicate flags * Bats tests for service, add metricsserver port * Support annotations and imagePullSecret on serviceAccount * Create configmap for custom configuration * Add configmap to deployment * Fix test names * Remove unneeded cloud validation. fixup comment * Comment values.yaml changes * Switch from sidecar auth method to component auth method * changelog * Add PodSecurityPolicy for consul-telemetry-collector * Rename init container + add comment * Remove logLevel bats tests as it is unsupported right now * Remove auth-method special cases * Replace LOGIN_DATACENTER login with LOGIN_NAMESPACE * Remove unneeded LOGIN_DATACENTER test * NET-2619 - save ClusterIPs to manual vips table (#2124) * Get the consul version from values.yaml (#2146) * [COMPLIANCE] Add Copyright and License Headers (#2079) Co-authored-by: hashicorp-copywrite[bot] <110428419+hashicorp-copywrite[bot]@users.noreply.github.com> * Update go-discover (#2157) * update go-discove so we're not pulling in a version of tencent cloud that no longer exists * Update go discover to latest * add helm chart values to configure global server side rate limiting (#2170) * add helm chart values to configure global server side rate limiting * add changelog. * update server checksum for configmap * fix the other 2 checksums * Disable DNS redirection when tproxy is disabled (#2176) * Disable DNS redirection when tproxy is disabled DNS redirection and the various settings that make that possible (like the dataplane binding to a port for DNS) is only useful if tproxy is enabled. Most of the code checked if tproxy was enabled but there was one location where we didn't check. This resulted in a bug with our multiport support where even though tproxy is disabled, we tried to setup the dataplane to proxy DNS. This meant each dataplane tried to bind to 8600 but because there are >1 dataplanes with multiport, there was a port conflict. This PR fixes the location where we didn't check if tproxy was enabled and as a result fixes the multiport issue. * Fix tests (#2181) * [API Gateway] Add stub acceptance test (#2185) * Update consul image so that acceptance tests run (#2189) * API Gateways for Consul on Kubernetes `BETA` (#2152) * Add API Gateway subcommand to Control Plane. Co-authored-by: Thomas Eckert <[email protected]> * Add GatewayClassConfig CRD (#2036) * Update dependencies so that CRDs can be added * Generate CRD for GatewayClassConfig * Return empty logger instead of nil due to dependency update * Update sidecar webhook to use ProbeHandler instead of Handler * Update controller sub resources to use sub resource update options * Re-add copyright header that got removed on generation * Use NewTestLogger and ProbeHandler in tests * Add api_gateway_types_test * Remove boilerplate from ctrl-generate as it is no longer required * Add app-copyright-header to Makefile * Clarify GatewayClassConfig description * Remove unneeded fields from GatewayClassConfig * Fix lint issues * Fix TestLogger in enterprise tests * Add Changelog * Fix TestLogger in enterprise test in one more place * Remove the helpers * Remove unused consts * Adds API Gateway Class Config controller * Add Hack for Generating CRDs from external sources (#2060) * Add generate-external-crds to Makefile * Add contributing docs * Add comment about Helm ignoring kustomization.yaml * Update Makefile Co-authored-by: Luke Kysow <[email protected]> * Update CONTRIBUTING.md Co-authored-by: Nathan Coleman <[email protected]> --------- Co-authored-by: Luke Kysow <[email protected]> Co-authored-by: Nathan Coleman <[email protected]> * Remove the api-gateway subcommand we decided not to use (#2062) * APIGW Resource Translation (#2070) * WIP: api-gateway resource conversion * convert meta for apigw from k8s * Added tests and updated config entry translation for APIGW * Fix linting issue, move translation code to correct location * Updates from PR comments * Update config entry translation to use k8s type NamedNamespace, updated tests * switch to standard import rename for consul api * Add GatewayClass Controller (#2055) * Add permissions to connect-inject clusterrole * Add gateway api crd deps * Stub out the gatewayclass controller * Add finalizer functions * Use finalizer functions * Add tests for GatewayClass Controller * Change the controller name * Only register gwv1beta1 * Run tests in parallel * Remove RBAC comments * Remove perms from resources not yet implemented * shouldUpdate -> expectedDidUpdate * Don't requeue if in use * Address PR feedback * Apply suggestions from code review Co-authored-by: Andrew Stucki <[email protected]> * Make gatewayClassFinalizer private * Separate out indexers * Move validation of parametersRef to a helper func * Add reason to ensureStatus * Rename GatewayClassReconciler -> GatewayClassController * Add perms to list gateways * Clean up status conditions * Clean up indexes * Set conditions properly and test them * Test incorrect parametersRef * Fix comments on indexer funcs * Fix lint issues * Set conditions without unnecessary updates * Set ObservedGeneration from parent object * Fix infinite loop issue with invalid config * Fix update issue * Return error if the GatewayClass cannot be reached --------- Co-authored-by: Andrew Stucki <[email protected]> * Updates GatewayClassConfig Controller to use common finalizer methods * APIGW4CONK8S: HTTP Route/TCPRoute/Secrets Translation (#2088) * Add http route translation * Added copywrite headers * Add namespace translation for service * handle potential nil pointer on section name, check if parent ref if an api gateway, fix comment from PR Review * Added TCPRoute Translation * Fix potential nil pointer deref in tcp service namespace, update tcproute tests * Add inline certs translation, clean up some potential nil pointer derefs * Clean up comments * Linting * Switch out env var usage for field on translator * rename api-gateway/consul package to api-gateway/translation * Adds stub for Gateway Controller * Use the non-deprecated logr test (#2125) * APIGW4CONK8s: Add Consul Cache (#2118) * Added basic cache functionality with most tests, todo: add get method for cache and expand tests * Updated tests for Cache.Run function, removed tests of unexported methods called by Run function * Moved translation function def to translation package, added translate apigw config entry * Add translation for consul config entries to k8s namespaced name meta * Added Get method to cache * Add watch for contoller and setup in inject command * Updated comments, renamed TranslateConsulInlineSecret method to TranslateConsulInlineCertificate * Updates from PR review * Parallelize tests * Bump consul api version * Set api timeout for cache calls * Revert "Bump consul api version" This reverts commit c074b0f749d891f78ddff86b3a7eb62ba1e52a17. * Linting fun * Add Gatekeeper for managing gateway deployment resources (#2117) * Stub out the gatewayclass controller * Change the controller name * Only register gwv1beta1 * Address PR feedback * Adds stub of Gateway Controller * cannot understand why the indexes are not working * some updates, want to do cleanup * rebase and cleanup * Start adding deployer * Flesh out tests * Refactor into a "gatekeeper" * Integrate the gatekeeper into the gateway controller * Simplify the api * Remove the creation of helm config until later * Remove use and rename package to gatekeeper * Add labels to apigateway * Manage ServiceAccount * Manage Deployment * Add more to deployment * Update Helm Values * WIP fleshing out the gateway deployment upsert behavior * Update role and service * Fix merge conflicts * Round out tests * Add test for respecting replicas * Change the Gatekeeper New API and add comments for Upsert and Delete * implement joinResources * accept suggestions from @jm96441n * Use pointer receivers * Separate out mutator * Update deployment correctly * Update Role and ServiceAccount * Fix that silly linting error * Comments on HelmConfig * Add Image to deployment * Merge api-gateway into branch --------- Co-authored-by: Melisa Griffin <[email protected]> * Net 3490/reference grants (#2122) * Adds reference grant validation * Adds all necessary methods and tests * lint * some cleanup, fix copypasta test errors * lint * more linting * PR updates, fix capitalization * Add a bunch of TODOs for teamwork * Split out cleanup func and clear up todos * APIGW4CONK8S: Serialize the GatewayClassConfig onto the Gateway for easier retrieval (#2126) * Add serialization of gateway class config * Parallelize tests * Remove prints, fix cache tests * Add outer managed check to ensure we don't fetch config if we don't need to * Stub out where the openshift role info will go (#2145) * APIGW4CONK8S: Function to get all refs for a gateway (#2139) * Added function to get all refs for a gateway * Use k8s objects for references rather than consul objects * Fix comment * [API Gateway] API Gateway Binding Logic (#2142) * initial commit * Add additional TODO * Add some basic lifecycle unit tests * split up implementation * Add more tests and fix some bugs * remove one parallel call in a loop * Fix binding * Add resolvedRefs statuses for routes * Fix issue with empty parent ref that k8s doesn't like * Fix up updates/status ordering * Add basic gateway status setting * Finish up first pass on gateway statuses * Re-organize and begin adding comments * More comments * More comments * More comments * More comments * More comments * Add file that wasn't saved * Add utils unit tests * Add more tests * Final tests * Fix tests * Fix up gateway annotation with binding logic * Update doc comments for linter * Add forgotten file * Fix block in tests due to buffered channel size and better handle context cancelation * Add basic acceptance tests for route binding behavior (#2161) * Configure Gateway Controller with Helm values (#2158) * Stub out the gatewayclass controller * Change the controller name * Only register gwv1beta1 * Address PR feedback * Adds stub of Gateway Controller * cannot understand why the indexes are not working * some updates, want to do cleanup * rebase and cleanup * Start adding deployer * Flesh out tests * Refactor into a "gatekeeper" * Integrate the gatekeeper into the gateway controller * Simplify the api * Remove the creation of helm config until later * Remove use and rename package to gatekeeper * Add labels to apigateway * Manage ServiceAccount * Manage Deployment * Add more to deployment * Update Helm Values * WIP fleshing out the gateway deployment upsert behavior * Update role and service * Fix merge conflicts * Round out tests * Add test for respecting replicas * Change the Gatekeeper New API and add comments for Upsert and Delete * implement joinResources * accept suggestions from @jm96441n * Use pointer receivers * Separate out mutator * Update deployment correctly * Update Role and ServiceAccount * Fix that silly linting error * Comments on HelmConfig * Add Image to deployment * Add Gateway flags to inject-connect * Pass through env vars * Add environment variables to the deployment template * Add conditional injection of environment variables * Add env vars back in * Fix up issues from merge * Test default env vars * Test all of the env vars * Fix up more issues from merge * Pass in values to HelmConfig then to Controller * Just pass config in as a struct * Add gateway-gatewayclass * Add gateway-gatewayclassconfig * Add DeploymentSpec to GatewayClassConfig * Remove deployment configuration settings from HelmConfig * Remove BATs on deployment configuration * Expand gatewayclassconfig * Set deployment replicas in test * Place GatewayClassConfig in the crds/ dir * Update control-plane/api-gateway/gatekeeper/gatekeeper_test.go Co-authored-by: Andrew Stucki <[email protected]> --------- Co-authored-by: Melisa Griffin <[email protected]> Co-authored-by: Andrew Stucki <[email protected]> * Net 4124/handle syncing consul lifecycle events (#2173) * with type switch * latest changes * remove debugging panic * Updated error in test * Fix bug with capacity v length in the cache list and type that is being subscribed to * Fix linting issues/naming from PR review * Added tests for delete function * Plumbing for gatekeeper with snapshot * [API Gateway] Hooking up API Gateways End-to-End (#2175) * updated gatekeeper, added update call, still needs work * still has some print statements, seeing issues with updates * some linting * run ctrl-manifests and generate * get the whole gamut finally working in a minimum configuration * Fix up tests * Add some tests * Move cache package * Fix up tests after other fixes * Fix up test lifecycle * Fix up linter issues * Remove unnecessary test that panics * Add MeshService CRD * fix bats tests * bats bats bats * baaaatttss * Fix up acceptance test cleanup by introducing uninstall hook to cleanup managed GatewayClass and GatewayClassConfig resources * Add test for deletion failures due to finalizers * reorder commands --------- Co-authored-by: Melisa Griffin <[email protected]> * Fix crd loading (#2179) * Fix CRD loading for CLI * Adds crds directory to install with consul-k8s cli * fix tests * testing * fix bats tests --------- Co-authored-by: Thomas Eckert <[email protected]> Co-authored-by: Andrew Stucki <[email protected]> * Add Changelog * Fix up issues after merge back * Fix wildcard usage on enterprise * Don't subscribe to peerings when not enabled * Remove additional changelog entries since we're only going to use 1 --------- Co-authored-by: Melisa Griffin <[email protected]> Co-authored-by: Luke Kysow <[email protected]> Co-authored-by: Nathan Coleman <[email protected]> Co-authored-by: John Maguire <[email protected]> Co-authored-by: Andrew Stucki <[email protected]> Co-authored-by: Melisa Griffin <[email protected]> * Update consul image on prepare-dev and prepare-release (#2180) Update consul image on prepare-dev and prepare-release * Fix dev mode on main (#2193) * Fix CVEs by updating controller-runtime (#2183) * Bump version of controller runtime * Use SubResourceUpdateOption * Fix test loggr * Fix ProbeHandler * Set runtime to 0.14.6 * Add Changelog * Fix up a few more breaking change issues * Adding support for idleTimeout in Service Router spec (#2156) * Adding support for idleTimeout in Service Router spec * Changelog: add support for idleTimeout in Service Router config (#2200) * add changelog * build(deps): update controller UBI base to 9.2 (#2204) * inject envoy_telemetry_bind_socket_dir proxy config when telemetry collector is enabled (#2143) * inject envoy_telemetry_bind_socket_dir proxy config when telemetry collector is enabled * use metrics.enableTelemetryCollector value to gate controller logic * add changelog entry and unit test * update cloud preset to enable telemetry collector (#2205) * Consul Telemetry acceptance test (#2195) * Fix bug on service intention CRDs causing source partitions and namespaces not to be compared. (#2194) This bug means that swapping partitions and namespaces on sources wouldn't get reflected in Consul. * Add CRD for jwt-provider config entry (#2209) * Add CRD for jwt-provider config entry * Pin consul/api to versions containing the jwt-provider config entry * Update Makefile to use v0.10.0 of sigs.k8s.io/controller-tools/cmd/controller-gen * API Gateway tenancy tests + fixes (#2201) * Initial scaffolding * Fix up some infinite reconciliation issues and initial other bugs * overhaul * get basic e2e working again * Add resource ref validation * Fix up namespace/reference grants * fix binding * clean up logging * cleanup * Get some binder unit tests working again * log guard * Fix unit test * Fix up more binder tests * get more binder tests working * finish binder tests * fix setter test * light touches and un-bak passing tests * Remove controller test as the wiring of deployments is predominantly tests via acceptance tests * Update reference grant tests * fix linter issues * fix acceptance test linters * Fix validation tests * Fix up consul cache tests * fixing up a few more tests * Finish up translation test work * Fix last bit of tests * Update ServiceIntentions CRD for JWT auth (#2213) * Fix setting args for the telemetry-collector (#2224) * Fix setting args for the telemetry-collector Either the docker container or the execution method for the telemetry-collector is making the args not get included on the process. Switch to putting it directly in the command so we can ensure this works as expected * Fix bats test * Fix telemetry collector issue and fix for bat test (#2223) * Get consul-dataplane image from helm chart (#2232) * Add acceptance test cleanup for API Gateway resources (#2237) * improve code readability and fix flaky tests re acl token generation (#2210) * Increase timeout and backoff for retry on flaky test (#2242) * Add fake demo/crds to get around that expectation in chart install (#2245) * NET-4285 add check for pointer (#2246) * Persist virtual-ips for intentions / service-defaults. (#2222) * Allow API Gateways to bind to privileged ports (#2253) * API Gateway lifecycle acceptance tests (#2248) * initial test * More lifecycle work * functional lifecycle tests * accepance: extend api gateway lifecycle test retryCheck timeouts (#2256) To reduce the likelihood of flakes. * api-gateway: create RoleBinding attaching Role to ServiceAccount (#2252) * Create RoleBinding attaching Role to ServiceAccount * Update ClusterRole for controller to allow management of RoleBindings * Separate logic for RoleBinding management from logic for Role * Use pointer receiver for all functions on Gatekeeper struct * Use more descriptive name for NamespacedName arg on delete * Clean up missed code in cherrypick * Remove out-of-scope TODO * Make Upsert docstring more robust, explaining dependency ordering * Add RoleBindings to unit tests for Gatekeeper * Add missing resources to kustomization.yaml (#2255) * Add missing JWT provider resource to kustomization.yaml - Add missing assertions for JWT provider too. * Add OSS tests for exported-services * Fix Gateway trigger for when secret is modified (#2261) * Fix Gateway trigger for when secret is modified * Add some simple unit tests * up some testing timeouts for acceptance tests * Add CRD for ControlPlane RequestLimits (#2166) * Update casing of json tag for ServiceDefault field (#2266) * Add the endpoint ignoring logic for triggering gateway reconciliation (#2227) * [COMPLIANCE] Add Copyright and License Headers (#2271) Co-authored-by: hashicorp-copywrite[bot] <110428419+hashicorp-copywrite[bot]@users.noreply.github.com> * Add additional helm hook for resource management (#2259) * Add additional helm hook for resource management * Move GatewayClassConfig CRD to templates * Add CRDs to templates * Add value to values.yaml * Remove GatewayClass and GatewayClassConfig bats * Fix CRD ExportedServices * Change -release to -release-name on gateway-resources subcommand * switch to pointer to avoid lock copy for linter * Move forcible test cleanup to before helm delete since it will now drop CRDs * adjust cleanup logic since it looks like the testing framework sometimes uninstalls the helm chart early * Fix cli unit test and drop CRD reading data since it's no longer embedded in the CLI * Add BATs for Gateway CRDs * Add BATs for Gateway Resources * Update Contributing --------- Co-authored-by: Thomas Eckert <[email protected]> * Add missing entries to main CHANGELOG (#2275) * Fixing changelog for 2195 (#2277) * [API Gateway] Add external consul servers test (#2270) * [API Gateway] Add external consul servers test * Fix up releaseName usage on CLI-based tests to mirror helm-based tests * Add check for timeout error (#2280) * Add Consul status to routes and gateways (#2281) * Update alpine to 3.18 to fix CVE-2023-2650 (#2284) * Update alpine to 3.18 * Remove check for reference grant for route to gateway (#2283) * Remove check for reference grant for route to gateway * Fix tenancy tests * Final cleaning up of acceptance test * [API Gateway] Add partition test (#2278) * Add partition test * drop superfluous sprintf * fix linter issue on acceptance test * Add predicated watch for pods * Update memory defaults for connect inject controller (#2249) * Update memory defaults for connect inject controllers * Add changelog entry * Bump up Consul server statefulset memory defaults too * Mw/fix pipeline 1 1 6 (#2282) * update eks and aks to use latest kubernetes version * updated the terraform provider as some fields were deprecated * Add bug to changelog so that go-changelog works (#2276) * Fix retry loops that use `t` (#2311) * Add FIPS builds (#2165) * Add FIPS builds for linux amd64 * add version check * fix CI labels and add local dev commands * fix ci version tagging * switch to ubuntu 20.04 * add CLI version tag * add gcompat for alpine glibc cgo compatibility * remove FIPS version check from connect-init * address comments * activated weekly acceptance tests for 1-2-x (#2315) - making this trigger nightly until after 1.2.0 GA - leaving 0.49.x active until after 1.2.0 GA * Net 4230/add tcp to basic acceptance test (#2297) * first run through, needs help * still need to make secure pass * left something uncommented * it works and also cleanup * fix acceptance tests * [API Gateway] Add acceptance test for cluster peering (#2306) * [API Gateway] Add acceptance test for cluster peering * Fix linter * Fix random unrelated linter errors to get CI to run: revert later? * one more linter fix to later probably revert * more linter fixes * Revert "more linter fixes" This reverts commit 6210dff0e51bbcf2f754f6d666c08292ba958aaa. * Revert "one more linter fix to later probably revert" This reverts commit 030c563bbe0b0a9ef73b33cbea32464416156d8f. * Revert "Fix random unrelated linter errors to get CI to run: revert later?" This reverts commit fdeccabb2f6c4418168cad9be5b2459435b7e30b. * Mw/net 3598 update kind for consul k8s acceptance tests with latest version of kind and k8s 1.27 (#2304) * update cloud tests to use 1.24, 1.25 and 1.26 version of kubernetes for more coverage * updated readme for supported kubernetes versions * added changelog * [API Gateway] WAN Federation test and fixes (#2295) * [API Gateway] WAN Federation test and fixes * Fix unit tests * [API Gateway] fix dangling service registrations (#2321) * Fix when gateways are deleted before we get services populated into cache * a bit of cleanup * api-gateway: add unit tests verifying scaling parameters on GatewayClassConfig are obeyed (#2272) * Add unit tests verifying that scaling parameters on GatewayClassConfig are obeyed * Add test case for scaling w/ no min or max configured * Rename GatewayClassController to prevent name collision (#2317) * Rename GatewayClassController to prevent name collision * Use gateway instead of gatewayclass in name * Use the constant in ownership checks * Change GatewayClass name to "consul" * Change GatewayClass name in cases * Change ApiGatewayClass back * [API Gateway] Conformance Test Fixes (#2326) * Fix SupportedKinds array to be what Conformance test expects * Fix cert validation status condition for listeners * Add programmed condition for listeners * Fix unit test --------- Co-authored-by: Nathan Coleman <[email protected]> * pin for 1.2.x-rc latest Consul submodules (#2327) * Ensure Reconciliation Stops (#2305) * first pass at halting: got httproute and api-gateway done * clean up test * Handle all set for infinite reconcile check * Add table tests for minimal setup * Added some odd field names to test normalization is handled correctly * Use funky casing http routes * Add CRT docker changes for release workflow (#2333) * Update var check with appropriate quotes (#2330) * Revert "Ensure Reconciliation Stops (#2305)" (#2341) This reverts commit 7f6e1cb5c4c2d8797944c1a3e0dcd12943f75138. * Improvement- [NET-189] Added helm inputs for managing audit logs (#2265) * Added helm inputs for managing audit logs * Remove unwanted changes from values * Set Consul service instance localities from K8s node labels (#2346) * fix: use correct flag when translating namespaces (#2353) * fix: use correct flag when translating namespaces * Use non-normalized namespace when deregistering services * Guard against namespace queries when namespaces not enabled in cache * added imagePullPolicy for images in values.yaml (#2310) * added imagePullPolicy for images in values.yaml * fix: renamed pullPolicy key according to image * fixed dafault always in tmpl * changed structure of image in yaml * revert changes * added global imagePullPolicy * fixed typo * added changelog file * [chore]: Pin github action workflows (#2356) * ci: update backport assistant to 0.3.4 (#2365) This brings consul-k8s in line with consul. Most importantly, the backport assistant was updated to automatically assign created PRs to the author of the PR that is being backported. * update changelog based on changes made to 1.2.x (#2348) * update changelog based on changes made to 1.2.x * fixed test cases - enterprise cases were in the OSS test cases * api-gateway: nightly conformance test action (#2257) * trigger conformance tests nightly, squash * remove extra line * Update nightly-api-gateway-conformance.yml * add crds for prioritize by locality (#2357) * set everything to correct version (#2342) making scripts more robust and removing changing helm chart * api-gateway: fix cache and service deletion issue (#2377) * Fix cache and service deletion issue * Add comments * add in acceptance test * Fix indentation * Fix unit test for deleting gateway w/ consul services * Remove redundant service deregistration code * Exit loop early once registration is found for service * Fix import blocking * Set status on pods added to test * Apply suggestions from code review * Reduce count of test gateways to 10 from 100 --------- Co-authored-by: Nathan Coleman <[email protected]> Co-authored-by: Sarah Alsmiller <[email protected]> * Adding support for weighted k8s service (#2293) * Adding support for weighted k8s service * Adding changelog * if per-app weight is 0 then pull the weight to 1 * Addressing review comments * Addressing review comments * Addressing review comments * Comment update * Comment update * Parameterized table test * Parameterized table test * fixing linting issue * fixing linting issue --------- Co-authored-by: srahul3 <[email protected]> * Bumping go-discover to the lastest version (#2390) * Bumping go-discover to the lastest version * Pin Kind versions on release branches (#2384) * pinned kind configuration for CI tests - created a yaml file with the desired pinned versions - created a script to read the yaml - added a make target which can be used in CI to get the desired kind inputs/config --------- Co-authored-by: Curt Bushko <[email protected]> * [COMPLIANCE] Add Copyright and License Headers (#2400) Co-authored-by: hashicorp-copywrite[bot] <110428419+hashicorp-copywrite[bot]@users.noreply.github.com> * update consul-dataplane on main to use 1.2-dev (#2325) * Acceptance test for permissive mTLS (#2378) * Revert "added imagePullPolicy for images in values.yaml (#2310)" (#2415) This reverts commit 285096241e0d5c5b6d53dd8a37889ab3ea5a8af2. * update with new make targets (#2411) - allow configuration of acceptance testing matrices * feat(helm): add configurable server-acl-init and cleanup resource limits (#2416) * feat(helm): add configurable server-acl-init and cleanup resource limits * Apply suggestions from code review Co-authored-by: Ashwin Venkatesh <[email protected]> * bugfix yaml path * fix bats test --------- Co-authored-by: Ashwin Venkatesh <[email protected]> * update redhat registry id (#2337) * Fix auditlog config (#2434) * Add acceptance test to test sync + ingress (#2421) * [COMPLIANCE] Add Copyright and License Headers (#2456) Co-authored-by: hashicorp-copywrite[bot] <110428419+hashicorp-copywrite[bot]@users.noreply.github.com> * Fix GatewayClassConfig Test Timing Issue (#2409) * Add retryCheckWithWait func * Fix retry timing on GatewayClassConfig test * remove redundant scale, make scale up number max + 1 * NET-4627, fix acceptance tests flake --------- Co-authored-by: Sarah Alsmiller <[email protected]> * always update acl policy if it exists (#2392) * always update acl policy if it exists * added changelog * added unit test * fix typo * added some additional assertions to test * refactored create_or_update unit test * Proxy Lifecycle helm, connect-inject and acceptance tests (#2233) Proxy Lifecycle helm, connect-inject and acceptance tests (#2233) Co-authored-by: Nitya Dhanushkodi <[email protected]> * PR breaking change release note change (#2469) * Add breaking change to release notes * Adds back gateway controller halting integration test (#2412) Co-authored-by: John Maguire <[email protected]> * api-gateway: Fix nil pointer exception panic (#2487) * fix nil pointer exception * add unit test * added changelog * delete changelog * Use correct length for certificate RSA key for tests (#2490) * Use correct length for certificate RSA key * api-gateway: Fix nil pointer exception panic (#2487) * fix nil pointer exception * add unit test * added changelog * delete changelog * Remove skip for fixed test --------- Co-authored-by: sarahalsmiller <[email protected]> * APIGW: Validate length of RSA Keys (#2478) * Validate length of RSA key for inline certs * Bring key length check functions over from consul * move validation of key length from certificate parsing into validation of cert * Update to use sentinel errors * Add changelog * Addressing PR comments: fixing text in changelog, fixing import blocks, slight refactor of cert validation for readability * Ensure cert is removed from consul if an invalid one is presented * Fix linting issues, added tests for validating keys * add changelog for 1.2.0 dataplane and consul 1.16.0 (#2496) * add changelog for Consul 1.16.0 * add changelog for dataplane 1.2.0 * Adds chanelog values for 0.49.7 (#2501) * ci: fix eks terraform quota error by cleaning up oidc providers (#2470) cleans up oidc providers older than 8 hours. * build: update versions to 1.3.0-dev (#2511) * [COMPLIANCE] Add Copyright and License Headers (#2507) Co-authored-by: hashicorp-copywrite[bot] <110428419+hashicorp-copywrite[bot]@users.noreply.github.com> * values.yaml - replace connect with service mesh for some instances (#2516) * fix connect/service mesh * Update values.yaml * docs: self service changelog instructions (#2526) * feat: adding security context and annotations to tls and acl init/cleanup jobs (#2525) * feat: adding security context and annotations to tls and acl init/cleanup jobs * changelog --------- Co-authored-by: Chinikins <[email protected]> * NET-4813: Fix issue where virtual IP saving had insufficient ACLs. (#2520) Fix issue where virtual IP saving had insufficient ACLs. * reactivate proxy-lifecycle tests (#2532) * Fix test flakes. (#2483) * Update chart to use OSS image (#2528) * Remove todo.txt (#2548) * makes gateway controllers less chatty (#2524) * HCP Observability acceptance test (#2254) * HCP bootstrap preset to always downcase datacenter (#2551) * Lowercase datacenter name from HCP bootstrap response * Add test cases to cloud bootstrap * api-gateway: when multiple listeners have the same port, only add to K8s Service once (#2413) * Modify unit tests to include multiple listeners w/ same port Running the tests on this commit will demonstrate the bug * When multiple listeners have the same port, only add to K8s Service once * Add changelog entry * NET-4482: set route condition appropriately when parent ref includes non-existent section (#2420) * Set route accepted condition appropriately when no listener with section name matching parent * Adjust error message for bind errors that aren't specific to one listener * Include section name in message for NoMatchingParent when available * Add unit test coverage for conditions derived from binding results * Add changelog entry * test: update nightly tests to consul 1.17-dev (#2556) * Update Release Scripts (#2558) * update environment variables with CONSUL_K8s prefix - This will let us check that we have all the environment variables set more easily with `printenv | grep "CONSUL_K8S"` * update imageConsulDataplane without quotes - this makes it consistent with the other images - allows scripting to work similarly to other images * updated utils script - handle replace case where consul-enterprise is in the values.yaml file and charts.yaml file - handle adding pre-release tag in changelog - handle updating consul-dataplane * added missing changelogs (#2565) * added missing changelogs * Update CHANGELOG.md for 0.49.8 --------- Co-authored-by: Curt Bushko <[email protected]> * Refactor test framework to allow for more than two kube contexts (#2534) * updated contributing example with new configuration lists add new make target "kind" to makefile * This lets us setup our standard kind environment for testing refactor framework to take config list flags * removed primary/secondary kube flags as this limited us to only two clusters * added flags for kube configs, contexts and namespaces. This way we can support n clusters where n is the length of the longest list. The flags are then combined into a list of objects for use in testing added tests for new helper methods refactored tests * now TestMain for multicluster check that the test arguments contain the expected number of clusters * use helper method `env.GetSecondaryContextKey(t)` which grabs the second context in the list instead of using the defunct environment.SecondaryContextName refactored flag test to use new config lists refactored cli cluster to use get primary helper added multicluster check for vault acceptance * vault tests are multi-cluster but we weren't performing the necessary checks * [COMPLIANCE] Add Copyright and License Headers (#2577) Add copyright and license headers * Consume gateway-api v0.7.1 for acceptance testing (#2578) Changes proposed in this PR: - Consume the same version of gateway-api for acceptance testing that we're consuming in the control plane: https://github.com/hashicorp/consul-k8s/blob/29b6ed36923498afc8f377455d4275653960230f/control-plane/go.mod#L42 How I've tested this PR: - 👀 - 🤖 tests pass How I expect reviewers to test this PR: - See above Checklist: - [ ] Tests added - [ ] [CHANGELOG entry added](https://github.com/hashicorp/consul-k8s/blob/main/CONTRIBUTING.md#adding-a-changelog-entry) * Update to handle validation endpoints (#2580) Changes proposed in this PR: - add in new validation call in endpoint How I've tested this PR: Ran it locally and tested the changes How I expect reviewers to test this PR: Read the code and run the command themselves to verify: ``` ./consul-k8s/acceptance/tests/cloud && go test -run TestBasicCloud -v -p 1 -timeout 20m \ -use-kind \ -kubecontext="kind-dc1" \ -consul-image hashicorppreview/consul-enterprise:1.17-dev -consul-k8s-image hashicorppreview/consul-k8s-control-plane:1.3.0-dev -consul-collector-image hashicorp/consul-telemetry-collector:0.0.1 \ -enable-enterprise ``` Checklist: - [X] Tests added - [n/a] [CHANGELOG entry added](https://github.com/hashicorp/consul-k8s/blob/main/CONTRIBUTING.md#adding-a-changelog-entry) * test(eks): fix deprecated CSI driver terraform (#2584) Changes proposed in this PR: - Replacing the deprecated [`resolve_conflicts`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/eks_addon#resolve_conflicts) with the new attributes. I don't know if we really need this setting since it is optional and the addon has no user-defined config, but I'm keeping this to keep the behavior consistent. How I've tested this PR: I did not. How I expect reviewers to test this PR: 👀 Checklist: - [ ] ~Tests added~ - [ ] ~[CHANGELOG entry added](https://github.com/hashicorp/consul-k8s/blob/main/CONTRIBUTING.md#adding-a-changelog-entry)~ * Add a check to prevent a nil-pointer dereference on Ingress LB (#2592) * test: remove unused workflow inputs (#2589) Changes proposed in this PR: - Removed unused workflow inputs. * chore: Update actions for security (#2601) Changes proposed in this PR: - Update actions that are out of date How I've tested this PR: 👀 How I expect reviewers to test this PR: 👀 Checklist: - [ ] Tests added - [ ] [CHANGELOG entry added](https://github.com/hashicorp/consul-k8s/blob/main/CONTRIBUTING.md#adding-a-changelog-entry) * [NET-4122] Doc guidance for federation with externalServers (#2583) Add guidance for proper configuration when joining to a secondary cluster using WAN fed with external servers also enabled. Also clarify federation requirements and fix formatting for an unrelated value. Changes proposed in this PR: - Update base content for generating Helm chart docs to clarify the use case encountered in https://github.com/hashicorp/consul-k8s/issues/2138 - Minor additional fixes - _Follow-up: propagate generated doc changes to `consul` and additionally update https://developer.hashicorp.com/consul/docs/k8s/deployment-configurations/servers-outside-kubernetes there_ How I've tested this PR: N/A (docs only) How I expect reviewers to test this PR: 👀 Checklist: - [ ] Tests added - [ ] [CHANGELOG entry added](https://github.com/hashicorp/consul-k8s/blob/main/CONTRIBUTING.md#adding-a-changelog-entry) * Handle errors properly when services are de-registered from the catalog (#2571) - In the past, kubernetes nodes were used as the source of truth to determine the list of services that should exist in Consul. - In most cases this was ok but becomes a problem when nodes are quickly deleted from kubernetes such as the case when using spot instances. - Instead, use consul synthetic-nodes to get the list of services and deregister the services that do not have endpoint addresses. --------- Co-authored-by: mr-miles <[email protected]> * Adding support for Enterprise and other improvement on the Customizing Vault Version for WanFed Test (#2481) * Adding support for Enterprise and other improvement on the Customizing Vault Version for WanFed Test This is the extension of the PR - https://github.com/hashicorp/consul-k8s/pull/2043 In this PR, the followings were addressed - 1. Now the vault enterprise version can be provided in the cli command. The previous PR only addressed Vault OSS. 2. Two flags “-no-cleanup-wan-fed” and “test-duration” were introduced to not to cleanup the test environment after successful setup to give it time to do manual testing for features/to reproduce customer issues. Default is 1 hour. 3. This was tested in Kind environment and it works fine. The following was taken out to use the “use-kind” option for WanFed test. //if cfg.UseKind { // t.Skipf("Skipping this test because it's currently flaky on kind") //} * Fix indentation * Fix unit test for deleting gateway w/ consul services * Remove redundant service deregistration code * Exit loop early once registration is found for service * Fix import blocking * Set status on pods added to test * Apply suggestions from code review * Reduce count of test gateways to 10 from 100 --------- Co-authored-by: Nathan Coleman <[email protected]> Co-authored-by: Sarah Alsmiller <[email protected]> Changes proposed in this PR: - - How I've tested this PR: How I expect reviewers to test this PR: Checklist: - [ ] Tests added - [ ] CHANGELOG entry added > HashiCorp engineers only, community PRs should not add a changelog entry. > Entries should use present tense (e.g. Add support for...) * Removing the changes in vault_namespaces_test.go * Introducing new flag no-cleanup * Removed "go 1.20" from go.work file * cfg.USEKind check is added back * Removed previousy added "Test Duration" flag * Some changes * Some changes * Differentiate FIPS linux package names (#2599) * added make target for checking for hashicorppreview (#2603) * added make target for checking for hashicorppreview * added check to prepare-release make target * Increase golangci-lint timeout to 10m (#2621) This is meant to solve for recurrent timeouts in several steps, particularly `golangci-lint-control-plane` and `golang-ci-lint-cli`. An accompanying change in `consul-k8s-workflows` should disable caching until the (unclear) root of the issue can be resolved, or we can disable or clear cache in a more targeted way that solves for these cases. * Fix TestAPIGateway_GatewayClassConfig (#2631) * Fix TestAPIGateway_GatewayClassConfig * Remove stray files from bad merge * Support running with restricted PSA enforcement enabled (part 1) (#2572) Support restricted PSA enforcement in a basic setup. This is enough to get a basic setup with ACLs and TLS working and an acceptance test passing (but does not update every component). On OpenShift, we have the option to set the security context or not. If the security context is unset, then it is set automatically by OpenShift SCCs. However, we prefer to set the security context to avoid useless warnings on OpenShift and to reduce the config difference between OpenShift and plain Kube. By default, OpenShift namespaces have the audit and warn PSA labels set to restricted, so we receive pod security warnings when deploying Consul to OpenShift even though the pods will be able to run. Helm chart changes: * Add a helper to the helm chart to define a "restricted" container security context (when pod security policies are not enabled) * Update the following container securityContexts to use the "restricted" settings (not exhaustive) - gateway-cleanup-job.yaml - gateway-resources-job.yaml - gossip-encryption-autogenerate-job.yaml - server-acl-init-cleanup-job.yaml - only if `.Values.server.containerSecurityContext.server.acl-init` is unset - server-acl-init-job.yaml - only if `.Values.server.containerSecurityContext.server.acl-init` is unset - server-statefulset.yaml: - the locality-init container receives the restricted context - the consul container receives the restricted context only if `.Values.server.containerSecurityContext.server` is unset - tls-init-cleanup-job.yaml - only if `.Values.server.containerSecurityContext.server.tls-init` is unset - tls-init-job.yaml - only if `.Values.server.containerSecurityContext.server.tls-init` is unset - webhook-cert-manager-deployment.yaml Acceptance test changes: * When `-enable-openshift` and `-enable-cni` are set, configure the CNI settings correctly for OpenShift. * Add the `-enable-restricted-psa-enforcement` test flag. When this is set, the tests assume the Consul namespace has restricted PSA enforcement enabled. The tests will deploy the CNI (if enabled) into the `kube-system` namespace. Compatible test cases will deploy applications outside of the Consul namespace. * Update the ConnectHelper to configure the NetworkAttachmentDefinition required to be compatible with the CNI on OpenShift. * Add fixtures for static-client and static-server for OpenShift. This is necessary because the deployment configs must reference the network attachment definition when using the CNI on OpenShift. * Update tests in the `acceptance/tests/connect` directory to either run or skip based on -enable-cni and -enable-openshift * change fips delimiter to + (#2480) (#2591) * [NET-4865] security: Upgrade Go and net/http CVE-2023-29406 (#2642) security: Upgrade Go and net/http Upgrade to Go 1.20.6 and `net/http` 1.12.0 to resolve CVE-2023-29406. * Consul client always logs into the local datacenter (#2652) The consul client always logs into the local datacenter * Add support for requestTimeout in Service Resolver spec (#2641) * Add support for requestTimeout in Service Resolver spec * preserve serviceresolvers.yaml Preserving yaml from main, only adding requesttimeout property. * update generated.deepcopy.go * Use latest controller-gen to generate CRDs --------- Co-authored-by: Ashwin Venkatesh <[email protected]> * Increase timeout for acl replication to 60 seconds and poll every 500 ms (#2656) increase timeout for acl replication to 60 seconds and poll every 500 ms * Update changelog to address cloud auto-join change in 1.0.0 (#2667) * NET-4967: Fix helm install when setting copyAnnotations or nodeSelector for apiGateway (#2597) * Support multiline nodeSelector arg * Support multiline service annotations arg * Update test assertions * Add changelog entry * Fix ordering of licence in templates (#2675) * Mw/net 4260 phase 2 automate the k8s sameness tests (#2579) * add kustomize files - These reflect the different test cases - sameness.yaml defines the ordered list of failovers - static-server responds with a unique name so we can track failover order - static-client includes both DNS and CURL in the image used so we can exec in for testing * add sameness tests - We do a bunch of infra setup for peering and partitions, but after the initial setup only partitions are tested - We test service failover, dns failover and PQ failover scenarios * add 4 kind clusters to make target - The sameness tests require 4 kind clusters, so the make target will now spin up 4 kind clusters - not all tests need 4 kind clusters, but the entire suite of tests can be run with 4 * increase kubectl timeout to 90s - add variable for configuring timeout - timeout was triggering locally on intel mac machine, so this timeout should cover our devs lowest performing machines * add sameness test to test packages * Fix comments on partition connect test * Added logLevel field for components (#2302) * Added logLevel field for components * Add changelog * Fix tests * Rename 2298.txt to 2302.txt * Address comments * Fix tests * Fix helm tests * Address comments * Add client and server loglevels * Fix bats * Update changelog * Fix bats tests * Add missing tsccr entries (#2682) * Use controller-gen 0.8.0 for CRDs (#2684) - Add missing license headers. * Fix ingress (#2687) * [NET-4865] Bump golang.org/x/net to 0.12.0 in cni (#2668) * Bump golang.org/x/net to 0.12.0 in cni This was missed in 5b57e6340dff44157cb7a984ac7220e47849dfb9 as part of a general upgrade of that dependency. * Bump server-connection-manager to v0.1.3 Tidying up following CVE dependency bumps, leading to a new release of this library. * Fix default Ent image tag in acceptance tests (#2683) * Fix default Ent image tag in acceptance tests Rather than hard-coding the Docker repository and parsing the non-Ent image tag for a version, simply replace the image name and retain other coordinates. This is consistent with our tagging scheme introduced in https://github.com/hashicorp/consul/pull/13541 and will allow for using `hashicorppreview` images seamlessly regardless of whether OSS or Ent is being tested. * Add make target for loading images in kind Complement other multi-cluster make targets by supporting image loading across kind clusters. * [NET-5146] security: Upgrade Go and `x/net` (#2710) security: Upgrade Go and x/net Upgrade to Go 1.20.7 and `x/net` 1.13.0 to resolve [CVE-2023-29409](https://nvd.nist.gov/vuln/detail/CVE-2023-29409) and [CVE-2023-3978](https://nvd.nist.gov/vuln/detail/CVE-2023-3978). * Increase timeout while waiting for vault server to be ready (#2709) increase timeout while waiting for server to be ready and fix require.Equal check * Acceptance tests: increase api-gateway retries (#2716) * Increase the retries and add config entry retries * NET-3908: allow configuration of SecurityContextConstraints when running on OpenShift (#2184) Co-authored-by: Melisa Griffin <[email protected]> * Gateway privileged port mapping (#2707) * Adds port mapping to Gateway Class Config to avoid running container on privileged ports Co-authored-by: Nathan Coleman <[email protected]> * Support restricted PSA enforcement part 2 (#2702) * NET-4413 Implement translation + validation of TLS options (#2711) * Implement validation of TLS options * Use constants for annotation keys * Add changelog entry * Implement TLS options translation * Update changelog entry * Add unit test coverage for TLS option validation * Code review feedback * NET-4993 JWT auth basic acceptance test (#2706) * JWT auth basic acceptance test * Update to run only in enterprise mode, update comment to be correct * Remove usage of `testing.t` in retry block * Fixed last `t` in retry block in tests * Update acceptance/tests/api-gateway/api_gateway_test.go Co-authored-by: Nathan Coleman <[email protected]> * Update acceptance/tests/api-gateway/api_gateway_test.go Co-authored-by: Nathan Coleman <[email protected]> * Updating filenames for gw jwt cases and adding message about why this test is skipped --------- Co-authored-by: Nathan Coleman <[email protected]> * [NET-5217] Apply K8s node locality to services and sidecars (#2748) Apply K8s node locality to services and sidecars Locality-aware routing is based on proxy locality rather than the proxied service. Ensure we propagate locality to both when registering services. * Adds changelog for release of 1.1.4 (#2754) * Set privileged to false unless on OpenShift without CNI (#2755) * Set privileged to false unless on OpenShift without CNI * Update consul-enterprise-version script to add -ent (#2756) * Automate the k8s sameness tests add peering (#2725) * added fixtures * removed fixtures - intentions only gets added now if acls are enabled - payment-service-resolver is only for locality aware which isn't in scope for this PR * updated sameness tests to include peering - refactored with some helper functions for members (now TestClusters) - made names more uniform, tend more towards the cluster-01-a/cluster-02-a/etc. nomenclature * added 4 clusters to cni make target * disable proxy lifecycle * Updates changelog to include 1.0.9 (#2758) * Adds changelog for 1.2.1, reorders 1.1.4 and 1.0.9 (#2768) * Mw/net 4260 add tproxy coverage (#2776) * add additional tproxy static-client - this doesn't specify an upstream so that tproxy will be able to handle routing * add tproxy coverage - add control-flow to handle using the virtual host name when tproxy is enabled * [NET-2880] Add `PrioritizeByLocality` to `ProxyDefaults` CRD (#2784) Add `PrioritizeByLocality` to `ProxyDefaults` CRD In addition to service resolver, add this field to the CRD for proxy defaults for parity with Consul config options. * AKS 1.24 is deprecated, update to latest 1.25 patch (#2792) * Net 4889 implement retry feature on the api gateway (#2735) * squash, add support for retry loops and timeouts to api-gateway NET-4889, NET-4890 * Update .changelog/2735.txt Co-authored-by: Andrew Stucki <[email protected]> * clean up extra files * delete custom struct, just use client.Object * delete * revert kustomization * lint cleanups * fix merge reversion, last bit of cleanup --------- Co-authored-by: Andrew Stucki <[email protected]> * Update Kustomize to use `patches` instead of `patchesStrategicMerge` (#2786) * Fix Kustomization for cases * Fix patches in config * Update `Contributing` * [NET-4498] Test locality propagation to services from k8s (#2791) Test locality propagation to services from k8s Verify that we propagate locality (region and zone) from standard k8s annotations to services registered by consul-k8s. This will later be expanded to exercise multi-cluster locality-based failover. * Use Kubernetes 1.25 on AKS (#2801) * Point mod to main to fix build errors (#2805) point mod to main to fix build errors * Fix peer test flakes. (#2812) This commit fixes an issue where the peering tests would flake due to the fact that we were concurrently modifying a global map. It also adds in retry logic so that the consul servers have sufficient time to initialize before attempting to generate peering tokens. * NET-4806: Fix ACL tokens for pods don't have pod name set (#2808) Fix issue where tokens had missing pod name. Prior to this commit, tokens descriptions would have a missing pod name and would have the form: {pod: "default/"} This poses issues for the endpoints controller, which will try to parse the metadata and use it to clean up the token. Without the pod name, consul-k8s will continually leak tokens. * net-1776, add job lifecycle test and changes to connhelper (#2669) * changes to connhelper, add job lifecycle test * yaml fixes * move around job yaml files, update grace period times * yaml change * timer change * wait for job to start when deploying * fix file paths * Skip Lifecycle Test on t-proxy --------- Co-authored-by: Thomas Eckert <[email protected]> * Net 1784 inject sidecar first (#2743) * change container creation order. Change order of container creation so that envoy container is created before app container. * change tests to fit proxy container added first * add sidecar first iff lifecycle enabled * update tests to include/exclude lifecycle * container ordering in multiport + lifecycle, test case * create changelog * change exec calls to specify container specify containers when exec'ing * Update 2743.txt * small fixes to appending sidecar * Add readOnlyRootFilesystem to security context (#2771) (#2789) * Add readOnlyRootFilesystem to security context (#2771) --------- Co-authored-by: mr-miles <[email protected]> Co-authored-by: Paul Glass <[email protected]> * feat: func to create V2 resource client (#2823) * feat: add helm value for consul resource-apis experiment (#2800) * feat: add helm value for consul resource-apis experiment * Apply suggestions from code review Co-authored-by: John Murret <[email protected]> * PR feedback part 2 --------- Co-authored-by: John Murret <[email protected]> * add sameness testing performance enhancement (#2822) * NET-5186 Add NET_BIND_SERVICE capability to Consul's restricted securityContext (#2787) * Add NET_BIND_SERVICE capability to Consul's restricted securityContext * Add changelog entry * Update related bats tests * Change type of release note * Added tests for partition dns/pq (#2816) * Added tests for partition dns/pq - did some light refactoring * Mw/net 4888 add namespace tests failover wan fed (#2797) * added fixtures * modified connHelper Create Intention - Function can now take optional intention ops. For now just supports overriding the source/destination namespaces * added WAN Federation test - split out into own test because TestWANFederation also does some PSA related tests. Didn't want to change this test too much, and my test requires consul-k8s mirroring - added new test TestWANFederationFailover which tests some failover scenarios, including to different namespaces and datacenters * refactored connHelper to use opts * fix: lifecycle enabled iptables mismatch (#2842) * refactor: make space for v2 controllers (#2832) refator: make space for v2 controllers * build: update SDK version to use commit from (#2846) * Revert "Add readOnlyRootFilesystem to security context (#2771)" (#2847) Revert "Add readOnlyRootFilesystem to security context (#2771) (#2789)" This reverts commit b75d8034b96ae1e21c0cca66ad5ee9a63af20505. * Fix issue where CLI install test was running Tproxy manually (#2843) * Configure Gateway Deployment Resources (#2723) * Update comments on Deployment * Move resources into managedGatewayClass * Add resource configuration to GatewayClassConfig * Regenerate CRDs * Pass resource configuration into the gateway-resources-job * Pull in resources from GatewayClassConfig * Add flag for resources in `gateway-resources` subcommand * Clean up some comments in existing code * Add gateway-resources configmap * Load configmap into gateway-resources job * Load resources from json * Update CRDs * Read resources in from the configmap * Add BATs for Gateway Resources Configmap * Add Changelog * Fix unquoted value in BATs * Fix how resources.json is read * Fix BATs errors for real * Fix seg fault bug * Fix reading of resources file * Quote "$actual" * Fix zsh/sh differences in BATs * Update control-plane/api-gateway/common/helm_config.go Co-authored-by: Nathan Coleman <[email protected]> * Move resources into DeploymentSpec * Remove extra split in crds --------- Co-authored-by: Nathan Coleman <[email protected]> * correct prometheus port and scheme annotations if tls is enabled (#2782) * correct prometheus port and scheme annotations if tls is enabled * Adds missing fields for PassiveHealthCheck on IngressGateway and ServiceDefault CRDs (#2796…
skpratt
added a commit
that referenced
this pull request
Oct 26, 2023
2 tasks
sarahalsmiller
pushed a commit
that referenced
this pull request
Apr 24, 2024
* readOnlyRootFilesystem * Add mount for /tmp * Add /tmp mountpoint * Update ingress-gateways-deployment.yaml * Update terminating-gateways-deployment.yaml * Update helm unit tests
sarahalsmiller
pushed a commit
that referenced
this pull request
May 8, 2024
* readOnlyRootFilesystem * Add mount for /tmp * Add /tmp mountpoint * Update ingress-gateways-deployment.yaml * Update terminating-gateways-deployment.yaml * Update helm unit tests
sarahalsmiller
added a commit
that referenced
this pull request
May 8, 2024
* Add readOnlyRootFilesystem to security context (#2771) * readOnlyRootFilesystem * Add mount for /tmp * Add /tmp mountpoint * Update ingress-gateways-deployment.yaml * Update terminating-gateways-deployment.yaml * Update helm unit tests * Create 2781.txt * rename changelog file * rename changelog file * Mount /tmp to volume for snapshots * rename changelog * changelog --------- Co-authored-by: mr-miles <[email protected]> Co-authored-by: Paul Glass <[email protected]> Co-authored-by: Sarah Alsmiller <[email protected]>
hc-github-team-consul-core
added a commit
that referenced
this pull request
May 21, 2024
…ycleShutdown… into release/1.4.x (#4007) * Fix meshgw tests (#3532) * Fix meshgw tests * change protocol on mesh gw tests to tcp from mesh * add nightly for rc branch (#3533) * [NET-7243] Stub APIGateway Controller for v2 (#3507) * stub api-gateway-controller * Add setup to v2 controller * Net 7376 Status struct on api gateway with required info from kubesig (#3530) * add status structs * update status * updated script to point at RC version correctly (#3541) * updated script to point at RC version correctly * Mw/prepare main for 1.5 dev (#3535) * bump versions to next version * updated script to handle new Consul-k8s images * [COMPLIANCE] Add Copyright and License Headers (#3499) Co-authored-by: hashicorp-copywrite[bot] <110428419+hashicorp-copywrite[bot]@users.noreply.github.com> * Net 7279 consul k8s write failing acceptance test for tcp route (#3540) * add status structs * update status * fixtures for v2 * checkpoint * add hook to only run test when flag is enabled * clean up reversions, delte extra files * remove http listeners * delete extra file * revert accidental IDE changes * clean up lint issues * Add json tags to api-gateway types (#3550) * reconcile consul-k8s with changes made in Consul (#3543) * [NET-7656] Add GatewayClassConfig watch for MeshGateway controller (#3537) * Add GatewayClass[Config] watches for MeshGateway controller * Update merge logic for deployment + service * Add test coverage for MergeDeployment * Add test coverage for MergeService * Copy over owner references to new Service + Deployment * Ensure signals are passed to commands (#3548) * Ensure signals are passed to commands Change `/bin/sh -ec "<command>"` to `/bin/sh -ec "exec <command>"`. Adding `exec` ensures that `<command>` is not executed as a child process but replaces the `/bin/sh` process. This ensure that `<command>` receives any signals. Specifically this is an issue when attempting to trap SIGTERMs as part of graceful pod shutdown. Without this change, we weren't receiving any signals because they aren't passed down by `/bin/sh -c`. * Fix broken bats tests and add changelog Signed-off-by: Ashwin Venkatesh <[email protected]> --------- Signed-off-by: Ashwin Venkatesh <[email protected]> Co-authored-by: Ashwin Venkatesh <[email protected]> * [NET-7158] CRUD hooks for api gateway v2 (#3519) * Add hooks for CRUD side effects for apigateway controller * Added tests for controller * [NET-6465] Respect connectInject.initContainer.resources for v1 API gateways (#3531) * Respect connectInject.initContainer.resources for v1 API gateways * Add changelog entry * Add test coverage for init container resources on API gateway Pods * Add NET_BIND_SERVICE to the security context in the deployment of Mesh Gateway (NET-6463) (#3549) * Add NET_BIND_SERVICE to the security context in the deployment of Mesh Gateway * [NET-7657,NET-6934] Define v2 GatewayClass + GatewayClassConfig locally (#3559) * Define GatewayClass's spec model locally instead of consuming proto from Consul * Update gateway resources job to use new types, constants * Make description optional, regenerate CRD definitions * Remove GatewayClass columns related to syncing into Consul * [NET-7156] Gateways Controllers Reusability (#3574) * make controller setup for gateway controllers generic and reusable, add indices onto gateway resources in k8s for more efficient lookups * cleanup from PR review * Update control-plane/controllers/resources/gateway_controller_setup.go Co-authored-by: Nathan Coleman <[email protected]> * Update control-plane/controllers/resources/gateway_indices.go Co-authored-by: Nathan Coleman <[email protected]> * Update control-plane/controllers/resources/gateway_controller_setup.go Co-authored-by: Nathan Coleman <[email protected]> * Update control-plane/controllers/resources/gateway_controller_setup.go Co-authored-by: Nathan Coleman <[email protected]> * clean up from PR review --------- Co-authored-by: Nathan Coleman <[email protected]> * [NET-6465] Consider init container resources when determining if existing + desired deployments are equal (#3575) * Consider init container resources when determining if existing + desired deployments are equal * Add test coverage for compareDeployments * Update control-plane/api-gateway/gatekeeper/deployment_test.go * [NET-7657] Consume version of proto-public with GatewayClass[Config] removed (#3581) [NET-7657] Consume version of proto-public with GatewayClass + GatewayClassConfig removed * Update multicluster v2beta1 to v2 (#3560) Co-authored-by: skpratt <[email protected]> * [NET-7156] Generalize MeshGatewayBuilder to just GatewayBuilder (#3538) * update gateway builder to be generic * Add api gateway to gateway builder * Updated service test for gateway listeners/ports * update test names * update listener functions * remove check for listener name * fix tests * release: Update 10-util.sh to adjust formatting (#3588) Update 10-util.sh * use go 1.21.7 (#3591) * add make target script (#3596) add new make target for go mod tidy check * v2tenancy: namespace mirroring acceptance tests (#3590) * add linting back (#3603) added linting back * [COMPLIANCE] Add Copyright and License Headers (#3610) Co-authored-by: hashicorp-copywrite[bot] <110428419+hashicorp-copywrite[bot]@users.noreply.github.com> * Datadog Integration (#3407) * datadog-integration: updated consul-server agent telemetry-config.json with dd specific items as well as additional missing VM based options, unit tests, dd unix socket integration, dd agent acl token generation, deployment override failsafes * datadog-integration: updated consul-server agent telemetry-config.json with dd specific items as well as additional missing VM based options, unit tests, dd unix socket integration, dd agent acl token generation | final initial-push * changelog entry update * datadog-integration: updated consul-server agent server.config (enable_debug) and telemetry.config update | enable_debug to server.config * curt pr review changes (minus extraConfig templating verification changes) * global.metrics.AgentMetrics -> global.metrics.enableAgentMetrics * dogstatsd and otlp mutually exclusive verification checks * breaking changes now incorporated into consul.validateExtraConfig helper template function as precheck * extraConfig hash updates post merge conflict update * fix helpers.tpl consul.extraConfig from merge --> /consul/tmp/extra-config/extra-from-values.json | add labels to rolebinding for datadog secrets * update changelog .txt to match new PR number * updated server-statefulset.yaml to correct ad.datadoghq.com/consul.logs annotation to valid single quote string * fix helpers.tpl consul.extraConfig from merge --> /consul/tmp/extra-config/extra-from-values.json | add labels to rolebinding for datadog secrets * fix helpers.tpl consul.extraConfig from merge --> /consul/tmp/extra-config/extra-from-values.json | add labels to rolebinding for datadog secrets * update UDP dogstatsdPort behavior to exclude including a port value if using a kube service address (as determined by user overrides) * update _helpers.tpl consul.ValidateDatadogConfiguration func to account for using 'https' as protocol => should fail * update server-statefulset.yaml to exclude prometheus.io annotations if enabling datadog openmetrics method for consul server metrics scrape. conflict present with http vs https that breaks openemtrics scrape on consul * update server-statefulset.yaml to exclude prometheus.io annotations if enabling datadog openmetrics method for consul server metrics scrape. conflict present with http vs https that breaks openemtrics scrape on consul * correct otlp protocol helpers.tpl check to lower-case the protocol to match the open-telemetry-deployment.yaml behavior * fix server-acl-init command_test.go for datadog token policy - datacenter should have been dc1 * add in server-statefulset bats test for extraConfig validation testing * Net 7238 - consul k8s modify gateway resources job to create apigw gatewayclass and gatewayclassconfig (#3564) * configmap update * udpate chart to respect api-gateway-config * fix typo * added unit tests, added some stuff missed in initial pass * added thorough unit tests for gateway-resources-configmap.yaml * remove unneeded extra line * additional debugging * test * test * remove extra escapes * final test * test again * one more test * this should work * fix spacing issue * Fix logic on apigateway that ignores current annotations on services (#3597) * [NET-7449] Generalize CRUD hooks for Gateways (#3576) Generalize the crud hooks for gateways * [NET-5932] chore: remove comment from closed ticket (#3636) chore: remove comment from closed ticket * [NET-2420] security: Upgrade helm containerd and several other dependencies (#3625) * security: upgrade helm/v3 to 3.13.3 Addresses multiple CVEs: - CVE-2023-25165 - CVE-2022-23524 - CVE-2022-23526 - CVE-2022-23525 * chore: upgrade k8s dependencies to match controller-runtime * security: upgrade containerd to latest Addresses GHSA-7ww5-4wqc-m92c (GO-2023-2412) * security: upgrade docker/docker to latest Addresses GHSA-jq35-85cj-fj4p * security: upgrade docker/distribution to latest Addresses CVE-2023-2253 * security: upgrade filepath-securejoin to latest patch Addresses GHSA-6xv5-86q9-7xr8 (GO-2023-2048) * chore: upgrade oras-go to fix docker incompatibility * Add changelog * build: Create arm64 packages as well (#3428) During the CRT on-boarding, packaging for other Linux architectures (arm64) was not enabled. This change adds packaging support for those architectures. I've specifically opted not to include 32-bit. See #1132. Related to hashicorp/releng-support#178. Other related updates: - To make future support a bit easier, I've enabled the build workflow from releng prefixed branches. - Using qemu emulation for testing package installs on other architectures, thus allowing us to validate the binaries work as intended - Minor alteration to the package install tests to use yum instead of rpm Co-authored-by: David Yu <[email protected]> * [NET-2420] security: re-enable security scan release block (#3628) * security: upgrade helm/v3 to 3.13.3 Addresses multiple CVEs: - CVE-2023-25165 - CVE-2022-23524 - CVE-2022-23526 - CVE-2022-23525 * chore: upgrade k8s dependencies to match controller-runtime * security: upgrade containerd to latest Addresses GHSA-7ww5-4wqc-m92c (GO-2023-2412) * security: upgrade docker/docker to latest Addresses GHSA-jq35-85cj-fj4p * security: upgrade docker/distribution to latest Addresses CVE-2023-2253 * security: upgrade filepath-securejoin to latest patch Addresses GHSA-6xv5-86q9-7xr8 (GO-2023-2048) * chore: upgrade oras-go to fix docker incompatibility * Add changelog * security: re-enable security scan release block This was previously disabled due to an unresolved false-positive CVE. Re-enabling both secrets and OSV + Go Modules scanning, which per our current scan results should not be a blocker to future releases. Also add security scans on PR and merge to protected branches to allow proactive triage going forward. See hashicorp/consul#19978 for similar change in that repo, adapted here. * [NET-8174] security: add scan triage for CVE-2024-25620 (helm/v3) (#3657) security: add scan triage for CVE-2024-25620 (helm/v3) Triage this scan result as `consul-k8s` should not be directly impacted and it is medium severity. Follow-up ticket filed for remediation. Also improve formatting of scan config since this change will be backported. * Update main changelog for 1.1.10, 1.2.6 and 1.3.3 (#3662) * Update main changelog for 1.1.10, 1.2.6 and 1.3.3 * include previous missed releases * [COMPLIANCE] Add Copyright and License Headers (#3654) Co-authored-by: hashicorp-copywrite[bot] <110428419+hashicorp-copywrite[bot]@users.noreply.github.com> * [NET-7450] setup crud hooks for APIGateway v2 (#3580) * setup crud hooks for APIGateway v2 * update CRDS and reorganize code in api gateway type * pass in gateway kind for annotations * Fix tests * Fix tests * register all types needed for test * values.yaml - tlsServerName docs (#3656) * Update values.yaml Co-authored-by: Jeff Boruszak <[email protected]> --------- Co-authored-by: Jeff Boruszak <[email protected]> * [NET-6741] make: Add target for updating dependencies across all modules (#3669) make: Add target for updating dependencies across all modules To enable more consistent and error-proof dependency management, add a Make target that will set a dependency version across all submodules that require it. Also runs `go mod tidy`. This first ensures the dependency addition is reverted if the module in question does not require it; it also ensures that any additional cleanup needed in `go.mod`/`go.sum` is applied. * build.yml: Add ECR images back (#3668) * Update build.yml * Create 3668.txt * build.yml: typo on tags (#3681) * bump kind to v0.22.0 and update k8s support (#3675) * bump kind to v0.22.0 and update k8s support * Create 3675.txt * Update README.md * [NET-8174] security: add scan triage for CVE-2024-26147 (helm/v3) (#3688) security: add scan triage for CVE-2024-26147 (helm/v3) * chore: upgrade Consul dependencies to latest (#3695) * chore: upgrade Consul dependencies to latest * chore: upgrade control-plane submodule dependencies to latest * fix: update GatewayClass finalizer reference * release: add \n to end of NOTE for releases (#3700) * Update 10-util.sh * Update control-plane/build-support/functions/10-util.sh Co-authored-by: Michael Zalimeni <[email protected]> --------- Co-authored-by: Michael Zalimeni <[email protected]> * chore: upgrade `consul/api` to latest (#3702) chore: upgrade consul/api to latest v1.28.0 was retracted due to double-publish. * [NET-8174] security: add triage alias for GO-2024-2554 (#3705) security: add triage alias for GO-2024-2554 This vulnerability was already triaged via its GHSA alias, but the scanner is flagging it under this name, so adding an explicit entry. * docs: update `CHANGELOG` for K8s 1.4.0 release (#3710) docs: update CHANGELOG for K8s 1.4.0 release * docs: update 1.4.0 Helm docs per Docs team feedback (#3714) * [NET-8367] security: upgrade google.golang.org/protobuf to 1.33.0 (#3719) * update protobuf lib * add changelog * NET-6878: Fix Flake API Gateway Acceptance (#3717) * test upgraded library * remove toolchain reference * add toolchain * NET-8391: fix cleanup script (#3725) * NET-8391: fix cleanup script * cleanup testing comments * NET-8391: fix cleanup script - remove network interface(s) (#3730) * cleanup network interfaces * clean up test * updates k8s version (#3731) * fix(control-plane): acl tokens deleted while pods in graceful shutdown (#3736) * NET-6878: Remove finalizers from CRDs during test resource cleanup (#3739) * remove finalizers from crds * add comments * Upgrade to go 1.21.8 (#3741) * Upgrade to use Go `1.21.8`. This resolves CVEs [CVE-2024-24783](https://nvd.nist.gov/vuln/detail/CVE-2024-24783) (`crypto/x509`). [CVE-2023-45290](https://nvd.nist.gov/vuln/detail/CVE-2023-45290) (`net/http`). [CVE-2023-45289](https://nvd.nist.gov/vuln/detail/CVE-2023-45289) (`net/http`, `net/http/cookiejar`). [CVE-2024-24785](https://nvd.nist.gov/vuln/detail/CVE-2024-24785) (`html/template`). [CVE-2024-24784](https://nvd.nist.gov/vuln/detail/CVE-2024-24784) (`net/mail`). Update the Consul Build Go base image to `alpine3.19`. This resolves CVEs [CVE-2023-52425](https://nvd.nist.gov/vuln/detail/CVE-2023-52425) [CVE-2023-52426](https://nvd.nist.gov/vuln/detail/CVE-2023-52426) * Add changelog * Fix typo in values file for sync catalog test (#3760) * upgraded helm v3 to address GHSA-jw44-4f3j-q396 (#3768) * disable scan for "GHSA-jw44-4f3j-q396" until patch fix in helm v3 * addressed comments * Net 6821 - Regenerate Terminating Gateway CRD with new field (#3737) * initial updates * regen crds * Add fixes for flaky-cni and failing cloud-nightly tests (#3764) Add fixes for flaky-cni * Catalog: Use EndpointSlice and propagate Kubernetes Topology information to synced consul service (#3693) * Use EndpointSlice and propagate zone metadata to consul service * Fix tests * Add test for zone metadata * Cleanup and changelog entry * Fix clusterrole permissions and type on Informer * Include region info for NodePort services * Include topology region for all service types * Update release note * Fix tests * fix sync-catalog-clusterrole and tests * fix stash conflict * adding endpoints permission back to sync catalog since it still uses it. * Fix endpointslice map * Fix topology region * Remove region lookups, remove endpoints permissions, use pointers for endpointslice map * Drop region test --------- Co-authored-by: John Murret <[email protected]> * Increase timeout for running commands in acceptance test (#3784) increase timeout for running commands * Bugfix: Don't recreate servicemap for catalog sync (#3785) * test: fix TestConnectInject_ProxyLifecycleShutdown (#3774) * Removes Legacy API Gateway Stanza that was deprecated in Consul 1.16 (#3718) * Removes Legacy API Gateway Stanza that was deprecated in Consul 1.16 * remove unit test for previously removed `consul-cni` validation (#3794) In #1527, we added support for OpenShift and Multus, which meant that the `consul-cni` plugin was no longer necessarily the final CNI plugin run. While working on a patch to allow compatibility with Nomad transparent proxy, I discovered we'd never removed a now-failing unit test of the plugin for the validation step. It looks like the remaining unit tests still cover the remaining validation, so we can safely remove this test. Ref: #1527 Ref: hashicorp/nomad#10628 * [NET-8412] Fix order of APIGW ACL policy/role creation (#3779) * Reorder gateway policy and role creation to avoid error messages in consul when policy/role already exists * refactor for readability * fix spacing * Added changelog * improve reliability of acceptance tests (#3800) * improve reliability of acceptance tests * remove update to timeout * add output to error * [net-8411] bug: fix premature token and service instance deletion due to pod fetch errors (#3758) * API gateway metrics (#3811) * First metrics pass * Fix up build * move to non-deprecated chart options * Fix up charts and defaults * Add changelog * Fix bad merge * Fix test * fix linter error * Fix extra yaml block from bad merge * Switch == true check to use ParseBool * Add support for Nomad transparent proxy (#3795) Nomad will implement support for Connect transparent proxy. Unlike in K8s, the CNI plugin can't contact the Nomad API to read allocation metadata (pod labels) to get the iptables configuration, and doesn't use the rest of the Consul-K8s control plane to inject that metadata. Instead, Nomad will pass the iptables configuration JSON-serialized in the CNI arguments. This changeset implements the behavior switch by detecting the `CONSUL_IPTABLES_CONFIG` argument in the CNI arguments. This hypothetically allows for non-Nomad workflows to use the same code path, if desired. Ref: hashicorp/nomad#10628 * fix version output for `consul-cni` (#3829) The `consul-cni` plugin emits "version unknown" because the CNI library's `PluginMain` uses a global variable that isn't being set as part of our build process. Import the `control-plane/version` package so that we have an identical version in builds across both binaries. * [NET-8601] Upgrade `vault/api` and `docker/docker` to resolve open CVEs (#3837) * security: upgrade vault/api to remove go-jose.v2 * security: upgrade docker/docker to v25.0.5 * add changelog * Remove anyuid SCC requirement for OpenShift (#3813) Remove SCC requirement for anyuid for OpenShift * Cleanup formatting to follow consul-k8s standard (#3852) * Datadog Unix Socket Path Custom Path fix (#3635) * Update dogstatsd hostPath rendering for Unix domain sockets -- override customizable and volumeMount/volume should align * changelog update * changelog: reviewer update to include datadog specific context * readd dev image tags for fips ubi (#3881) * readd dev image tags for fips ubi * fix up bad copy paste * [net-7710] don't overwrite prometheus path annotation if it's already been specified (#3846) don't overwrite prometheus path annotation if it's already been specified * feat: Add startup-grace-period-seconds and graceful-startup-path (#3878) * feat: Add startup-grace-period-seconds and graceful-startup-path * Add changelog --------- Co-authored-by: Michael Zalimeni <[email protected]> * NET-8594: Disable TestSyncCatalog (#3815) * [NET-8946 NET-8947 NET-8948] security: bump go, x/net and envoy versions (#3893) security: bump go and x/net * NET-8594: Disable TestSyncCatalogIngress (#3904) * Helm: support sync-lb-services-endpoints for sync catalog (#3905) * Helm: support sync-lb-services-endpoints for sync catalog * add test * fix template tag order --------- Co-authored-by: jukie <[email protected]> * Datadog Integration Acceptance Tests / Bug fixes (#3685) * datadog: acceptance tests - initial commit (not fully working yet) * server-statefulset: update logic for prometheus annotations (only enabled if using dogstatsd, otherwise disabled) * datadog: acceptance test working with dd-client api and operator deployment frameword * datadog-acceptance: main branch rebase merge conflict cherry-pick * datadog: acceptance testing update to metric name matching using regex * datadog: acceptance testing helper update for backoff retry * datadog: acceptance testing working timeseries query verification udp + uds * datadog: update helpers for /v1/query * server-statefulset.yaml: update to correct release name prepend to consul-server URL * datadog: acceptance testing consul integration checks working * server-statefulset: yaml and bats updates for datadog openmetrics and consul integration check URLs to use consul.fullname-server * PR3685: changelog update * datadog: openmetrics acceptance test update * datadog: added OTEL_EXPORTER_OTLP_ENDPOINT to consul telemetry collector deployment for dd-agent ingestion (passes tag info to DD) * otlp: datadog otlp acceptance test updates for telemetry-collector (grpc => http prefix) | staged otlp acceptance test * datadog-acceptance: fake-intake fixture addition * datadog-acceptance: update _helpers.tpl for consul version sanitization (truncate to <64) * datadog-acceptance: update base fixture for fake-intake * datadog-acceptance: add DogstatsD stats enablement (required for curling agent local endpoint) * datadog-acceptance: add DogstatsD stats enablement (required for curling agent local endpoint) * datadog-acceptance: first-round fake-intake testing - works but is innaccurate * datadog-acceptance: datadog framework - remove dd client agent requirement (fake-intake) * datadog-acceptance: update flags to not require API and APP key (fake-intake) * datadog-acceptance: go mod updates for uuid downgrade * acceptance-test: remove otlp acceptance test -- no fake-intake or agent endpoint to verify * datadog-acceptance: acceptance test lint fixes * acceptance-test: update control-plane/cni/main.go l:272 comment with period for lint testing. * acceptance-test: retry lint fixes * acceptance-test: correct telemetry collector URL from grpc:// to http:// * [NET-8412] Fix APIGW policy creation ordering for upgrade path (#3918) * fix policy creation for upgrading * Added changelog * Add post-release changelogs (#3867) Add changelogs * GH-3406 - Only error for config entries from different datacenters when the config entries are different (#3873) * GH-3406 - Only error for config entries from different datacenters when the config entries are different * add changelog * fixing tests and logic * refactoring code to make tests pass and also use a switch statement for readability and also get rid of intermediate state flag of requireMigration in a long iterative section of code. * add missing license file (#3921) * add missing license file * missed copying the license file to workdir * make up missing value and remove redundant directory creation * [COMPLIANCE] Add Copyright and License Headers (#3936) Co-authored-by: hashicorp-copywrite[bot] <110428419+hashicorp-copywrite[bot]@users.noreply.github.com> * Net 9069/xw add license file to all bin (#3942) * debug: missing LICENSE * use abs path * [NET-6466] Remove secrets from termgw role (#3928) * remove unnecessary permissions for terminating gateways * add changelog * Net 9069/fix local brokerage (#3948) * make copy of license file into control plane * remove redundant copy in gh workflow * use env instead of arg * [NET-8091] Use file-system-certificate in Consul instead of inline-certificate (#3767) * Use file-system-certificate in Consul instead of inline-certificate * Actually update correctly from merges * Adds changelog * Updates go.mod in acceptance tests with latest consul api, updates the acceptance gateway lifecycle test * Small updates * Update comment --------- Co-authored-by: Melisa Griffin <[email protected]> * chore: remove workstream from JIRA sync (#3960) * NET-9154: Update Kubernetes version (#3958) Update Kubernetes version * chore: fix JIRA workflow (#3965) * [NET-9097, NET-8174] Upgrade controller-runtime (#3935) * Consume controller-runtime v0.16.3 This is the version required by gateway-api v1.0.0, which will be consumed in a future PR * Reconcile breaking changes in controller-runtime * Fix linter errors * gofmt * Update controller tests to handle new fake client requirements * Update test assertion to handle changes in controller-runtime * Restore incorrectly-removed flags * Use a proper delete on the fake client since DeletionTimestamp is immutable * Update enterprise tests to specify status subresources * Update controller-runtime dependency for acceptance tests * Explicitly inject decoder into webhooks * Appease the linter * Use SetupWithManager pattern from controllers for webhook setup * Consume consistent version of k8s.io/client-go everywhere * Upgrade related dependencies for CLI, including helm/v3 * Consume latest release of helm/v3 * changelog * Inline function calls for testing * Consume controller-runtime v0.16.5 --------- Co-authored-by: Ronald Ekambi <[email protected]> * Fix a panic in connect-inject when the provided upstreams list is malformed (#3956) * Check if an upstream is malformed, if so ignore it. * support multiple upstreams separator (<space>, <comma>) add tests * add /n as a separator * add changelog * added log when upstream is skipped * [NET-9152] CRD for service registeration (#3943) * service is registering * add all the fields * health checks working * handle finalizers to clean up * Add status to registration CRD * Added initial unit test for reconcile * success paths for registration and deregistration * added failure tests, moved finalizer removal logic so it occurs after service is successfully deregistered * first test for to catalog registration type * maximal registration to catalog test * test all the things * deregistration tests * update some comments and fields, re-run generators * Added changelog * linting all the things * fixing test setup for new controller runtime * Handle errors for parsing duration * Add ReadOnlyRootFilesystem to Security Context (#2909) * Add readOnlyRootFilesystem to security context (#2771) * readOnlyRootFilesystem * Add mount for /tmp * Add /tmp mountpoint * Update ingress-gateways-deployment.yaml * Update terminating-gateways-deployment.yaml * Update helm unit tests * Create 2781.txt * rename changelog file * rename changelog file * Mount /tmp to volume for snapshots * rename changelog * changelog --------- Co-authored-by: mr-miles <[email protected]> Co-authored-by: Paul Glass <[email protected]> Co-authored-by: Sarah Alsmiller <[email protected]> * activate tproxy mode even when a cluster IP is not assigned to pod (#3974) * activate tproxy mode even when a cluster IP is not assigned to pod. * add changelog * fix failing tests * security: Upgrade Go to 1.21.10 (#3980) * NET-9178-Consul-api-gateway-not-starting-after-restart (#3978) * don't error if role already exists on restart * changelog * lint * [NET-9153] Handle Terminating Gateway ACL Setup (#3975) * first pass at creating write policy for service and updating term gw acl role * handle deregistering, update tests for registering with acls * existing deregister tests passing * failures with term gw role not existing * clean up * reorg code * Move to own package * watch for terminating gateways * move files back, handle multiple terminating gateways * handle errors and ensure finalizer is set * Add tests for finalizers * remove unused file * fix import naming * linting * fix comment, extract constant * [NET-9201] Validating webhook for registrations (#3990) * Add validating webhook for registrations * cleaned up registration webhook setup * fix setup for webhook, updated docs * fix typo, remove debugging log, rename variables for readability * Updating GitHub action versions to the latest TSCCR approved version (#3979) * test: fix PeeringGateway acceptance (#3992) * Adds ability to set the imagePullPolicy for all Consul images (consul… (#3991) * Adds ability to set the imagePullPolicy for all Consul images (consul, consul-dataplane, consul-k8s, consul-telemetry-collector) * [NET-9155] Cache resources for Registrations (#3993) * Add set for adding and removing services * remove service add * first pass at populating cache * cache is working, need to fix how statuses are handled * move to new directory, fix up the status conditions (still todos on this), handle results * updated tests * unexport methods that don't need to be exported * handle consul deregistrations * clean up before code review * show ACLUpdate as false if consul deregistered service * fix issue with updating acl status on consul deregistration * fix linting errors * FLAKEY_TEST: Add retry to outbound request for ProxyLifecycleShutdownTest * increase retry count for TestAPIGateway_GatewayClassConfig test * backport of commit b7ecab4 * backport of commit 2fcccd2 --------- Signed-off-by: Ashwin Venkatesh <[email protected]> Co-authored-by: John Maguire <[email protected]> Co-authored-by: Michael Wilkerson <[email protected]> Co-authored-by: sarahalsmiller <[email protected]> Co-authored-by: hashicorp-copywrite[bot] <110428419+hashicorp-copywrite[bot]@users.noreply.github.com> Co-authored-by: Anita Akaeze <[email protected]> Co-authored-by: Nathan Coleman <[email protected]> Co-authored-by: Luke Kysow <[email protected]> Co-authored-by: Ashwin Venkatesh <[email protected]> Co-authored-by: Melisa Griffin <[email protected]> Co-authored-by: Chris S. Kim <[email protected]> Co-authored-by: skpratt <[email protected]> Co-authored-by: David Yu <[email protected]> Co-authored-by: Semir Patel <[email protected]> Co-authored-by: natemollica-dev <[email protected]> Co-authored-by: Michael Zalimeni <[email protected]> Co-authored-by: Daniel Kimsey <[email protected]> Co-authored-by: Curt Bushko <[email protected]> Co-authored-by: Jeff Boruszak <[email protected]> Co-authored-by: NicoletaPopoviciu <[email protected]> Co-authored-by: Dan Stough <[email protected]> Co-authored-by: Ashwin Venkatesh <[email protected]> Co-authored-by: Isaac Wilson <[email protected]> Co-authored-by: John Murret <[email protected]> Co-authored-by: Tim Gross <[email protected]> Co-authored-by: Nitya Dhanushkodi <[email protected]> Co-authored-by: Andrew Stucki <[email protected]> Co-authored-by: Alvin Huang <[email protected]> Co-authored-by: Andrea Scarpino <[email protected]> Co-authored-by: Deniz Onur Duzgun <[email protected]> Co-authored-by: wangxinyi7 <[email protected]> Co-authored-by: Melisa Griffin <[email protected]> Co-authored-by: Ronald Ekambi <[email protected]> Co-authored-by: Dhia Ayachi <[email protected]> Co-authored-by: mr-miles <[email protected]> Co-authored-by: Paul Glass <[email protected]> Co-authored-by: Sarah Alsmiller <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Changes proposed in this PR:
How I've tested this PR:
I was attempting to run consul in a restricted namespace and our gatekeeper policy tripper on this. I found I could not add the context by manipulating the helm values and so a PR was required,
I have tested it by deploying my updated package and confirming gatekeeper was not tripped
How I expect reviewers to test this PR:
Confirm it deploys fine in a standard test environment just for sanity: as the components don't write this labelling doesn't affect functionality and so is a no-op for anyone not already blocked by security context.
Checklist: