Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Backport of Vault CA bugfixes into release/1.16.x #19308

Conversation

hc-github-team-consul-core
Copy link
Collaborator

Backport

This PR is auto-generated from #19285 to be assessed for backporting due to the inclusion of the label backport/1.16.

The below text is copied from the body of the original PR.


Description

Fixing edge case bugs uncovered by #19051

Testing & Reproduction steps

Added tests for goroutine leak detection.

The retry functionality is a revert of #16143 and was tested manually.


Overview of commits

ishustava and others added 30 commits September 8, 2023 16:18
#18458)

This commit adds support for transparent proxy to the sidecar proxy controller. As we do not yet support inferring destinations from intentions, this assumes that all services in the cluster are destinations.
* mesh-controller: handle L4 protocols for a proxy without upstreams

* sidecar-controller: Support explicit destinations for L4 protocols and single ports.

* This controller generates and saves ProxyStateTemplate for sidecar proxies.
* It currently supports single-port L4 ports only.
* It keeps a cache of all destinations to make it easier to compute and retrieve destinations.
* It will update the status of the pbmesh.Upstreams resource if anything is invalid.

* endpoints-controller: add workload identity to the service endpoints resource

* small fixes

* review comments

* Address PR comments

* sidecar-proxy controller: Add support for transparent proxy

This currently does not support inferring destinations from intentions.

* PR review comments

* mesh-controller: handle L4 protocols for a proxy without upstreams

* sidecar-controller: Support explicit destinations for L4 protocols and single ports.

* This controller generates and saves ProxyStateTemplate for sidecar proxies.
* It currently supports single-port L4 ports only.
* It keeps a cache of all destinations to make it easier to compute and retrieve destinations.
* It will update the status of the pbmesh.Upstreams resource if anything is invalid.

* endpoints-controller: add workload identity to the service endpoints resource

* small fixes

* review comments

* Make sure endpoint refs route to mesh port instead of an app port

* Address PR comments

* fixing copyright

* tidy imports

* sidecar-proxy controller: Add support for transparent proxy

This currently does not support inferring destinations from intentions.

* tidy imports

* add copyright headers

* Prefix sidecar proxy test files with source and destination.

* Update controller_test.go

---------

Co-authored-by: Iryna Shustava <[email protected]>
Co-authored-by: R.B. Boyer <[email protected]>
Co-authored-by: github-team-consul-core <[email protected]>
fix windows integrations tests machine size
* feat: implement list command

* refactor: apply command file parsing
* added logs

* added echo

* removed pull request
)

* execute copyright headers after performing deep-copy generation.

* fix copyright install

* Apply suggestions from code review

Co-authored-by: Semir Patel <[email protected]>

* Apply suggestions from code review

Co-authored-by: Semir Patel <[email protected]>

* rename steps to match codegen naming

* remove copywrite install category

---------

Co-authored-by: Semir Patel <[email protected]>
…mode (#18606)

* mesh-controller: handle L4 protocols for a proxy without upstreams

* sidecar-controller: Support explicit destinations for L4 protocols and single ports.

* This controller generates and saves ProxyStateTemplate for sidecar proxies.
* It currently supports single-port L4 ports only.
* It keeps a cache of all destinations to make it easier to compute and retrieve destinations.
* It will update the status of the pbmesh.Upstreams resource if anything is invalid.

* endpoints-controller: add workload identity to the service endpoints resource

* small fixes

* review comments

* Address PR comments

* sidecar-proxy controller: Add support for transparent proxy

This currently does not support inferring destinations from intentions.

* PR review comments

* mesh-controller: handle L4 protocols for a proxy without upstreams

* sidecar-controller: Support explicit destinations for L4 protocols and single ports.

* This controller generates and saves ProxyStateTemplate for sidecar proxies.
* It currently supports single-port L4 ports only.
* It keeps a cache of all destinations to make it easier to compute and retrieve destinations.
* It will update the status of the pbmesh.Upstreams resource if anything is invalid.

* endpoints-controller: add workload identity to the service endpoints resource

* small fixes

* review comments

* Make sure endpoint refs route to mesh port instead of an app port

* Address PR comments

* fixing copyright

* tidy imports

* sidecar-proxy controller: Add support for transparent proxy

This currently does not support inferring destinations from intentions.

* tidy imports

* add copyright headers

* Prefix sidecar proxy test files with source and destination.

* Update controller_test.go

* NET-5132 - Configure multiport routing for connect proxies in TProxy mode

* formatting golden files

* reverting golden files and adding changes in manually.  build implicit destinations still has some issues.

* fixing files that were incorrectly repeating the outbound listener

* PR comments

* extract AlpnProtocol naming convention to getAlpnProtocolFromPortName(portName)

* removing address level filtering.

* adding license to resources_test.go

---------

Co-authored-by: Iryna Shustava <[email protected]>
Co-authored-by: R.B. Boyer <[email protected]>
Co-authored-by: github-team-consul-core <[email protected]>
Update Go version to 1.20.8

This resolves several CVEs (see changelog entry).
…omputedRoutes resource (#18460)

This new controller produces an intermediate output (ComputedRoutes) that is meant to summarize all relevant xRoutes and related mesh configuration in an easier-to-use format for downstream use to construct the ProxyStateTemplate.

It also applies status updates to the xRoute resource types to indicate that they are themselves semantically valid inputs.
… ProxyStateTemplate, and wire up leaf cert manager dependency (#18756)

* Refactors the leafcert package to not have a dependency on agent/consul and agent/cache to avoid import cycles. This way the xds controller can just import the leafcert package to use the leafcert manager.

The leaf cert logic in the controller:
* Sets up watches for leaf certs that are referenced in the ProxyStateTemplate (which generates the leaf certs too).
* Gets the leaf cert from the leaf cert cache
* Stores the leaf cert in the ProxyState that's pushed to xds
* For the cert watches, this PR also uses a bimapper + a thin wrapper to map leaf cert events to related ProxyStateTemplates

Since bimapper uses a resource.Reference or resource.ID to map between two resource types, I've created an internal type for a leaf certificate to use for the resource.Reference, since it's not a v2 resource.
The wrapper allows mapping events to resources (as opposed to mapping resources to resources)

The controller tests:
Unit: Ensure that we resolve leaf cert references
Lifecycle: Ensure that when the CA is updated, the leaf cert is as well

Also adds a new spiffe id type, and adds workload identity and workload identity URI to leaf certs. This is so certs are generated with the new workload identity based SPIFFE id.

* Pulls out some leaf cert test helpers into a helpers file so it
can be used in the xds controller tests.
* Wires up leaf cert manager dependency
* Support getting token from proxytracker
* Add workload identity spiffe id type to the authorize and sign functions



---------

Co-authored-by: John Murret <[email protected]>
NET-5592 - update Nomad integration testing
* some changes to debug

* revert machines

* increased timeout

* added sleep 10 seconds before test start

* chagne envoy version

* removed sleep

* revert timeout

* replace position

* removed date

* Revert "[NET-5217] [OSS] Derive sidecar proxy locality from parent service (#18437)"

This reverts commit 05604ee.

* fix build

* Revert "replace position"

This reverts commit 48e6af4.

* Revert "Revert "[NET-5217] [OSS] Derive sidecar proxy locality from parent service (#18437)""

This reverts commit d7c568e.

* comment out api gateway http hostnames test

* fix import

* revert integ test run on PR
Add support for TCP traffic permissions
…te (#18765)

When one resource contains an inner field that is of type *pbresource.Reference we want the
Tenancy to be reasonably defaulted by the following rules:

1. The final values will be limited by the scope of the referenced type.
2. Values will be inferred from the parent's tenancy, and if that is insufficient then using
   the default tenancy for the type's scope.
3. Namespace will only be used from a parent if the reference and the parent share a
   partition, otherwise the default namespace will be used.

Until we tackle peering, this hard codes an assumption of peer name being local. The
logic for defaulting may need adjustment when that is addressed.
There's currently a bug that causes CI to be skipped on all non-PR
changes. Until that's fixed and we can be certain the check will fail CI
or default to running tests in the case of errors, disabling this check.
* delete command for resource management
The renaming of files from oss -> ce caused incorrect snapshots
to be created due to ce writes now happening prior to ent writes.
When this happens various entities will attempt to be restored
from the snapshot prior to a partition existing and will cause a
panic to occur.
* add a simple fuzz test for the resourcehcl package and fix some panics uncovered by the test

* fix default for null values
…ntainer port-mapping and Openshift Security Context Constraints

Co-authored-by: trujillo-adam <[email protected]>
@hc-github-team-consul-core hc-github-team-consul-core requested a review from a team October 20, 2023 15:04
@hc-github-team-consul-core hc-github-team-consul-core requested review from modrake and emilymianeil and removed request for a team October 20, 2023 15:04
@hc-github-team-consul-core hc-github-team-consul-core force-pushed the backport/kisunji/vault-ca-fixes/fully-electric-phoenix branch 2 times, most recently from 14f7640 to 6a24944 Compare October 20, 2023 15:04
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Auto approved Consul Bot automated PR

@github-actions github-actions bot added type/docs Documentation needs to be created/updated/clarified theme/api Relating to the HTTP API interface theme/health-checks Health Check functionality theme/acls ACL and token generation theme/cli Flags and documentation for the CLI interface theme/config Relating to Consul Agent configuration, including reloading theme/ui Anything related to the UI theme/connect Anything related to Consul Connect, Service Mesh, Side Car Proxies theme/telemetry Anything related to telemetry or observability type/ci Relating to continuous integration (CI) tooling for testing or releases pr/dependencies PR specifically updates dependencies of project theme/envoy/xds Related to Envoy support theme/contributing Additions and enhancements to community contributing materials theme/internals Serf, Raft, SWIM, Lifeguard, Anti-Entropy, locking topics theme/consul-terraform-sync Relating to Consul Terraform Sync and Network Infrastructure Automation labels Oct 20, 2023
@kisunji kisunji closed this Oct 20, 2023
auto-merge was automatically disabled October 20, 2023 15:07

Pull request was closed

@vercel vercel bot temporarily deployed to Preview – consul October 20, 2023 15:13 Inactive
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
pr/dependencies PR specifically updates dependencies of project theme/acls ACL and token generation theme/api Relating to the HTTP API interface theme/cli Flags and documentation for the CLI interface theme/config Relating to Consul Agent configuration, including reloading theme/connect Anything related to Consul Connect, Service Mesh, Side Car Proxies theme/consul-terraform-sync Relating to Consul Terraform Sync and Network Infrastructure Automation theme/contributing Additions and enhancements to community contributing materials theme/envoy/xds Related to Envoy support theme/health-checks Health Check functionality theme/internals Serf, Raft, SWIM, Lifeguard, Anti-Entropy, locking topics theme/telemetry Anything related to telemetry or observability theme/ui Anything related to the UI type/ci Relating to continuous integration (CI) tooling for testing or releases type/docs Documentation needs to be created/updated/clarified
Projects
None yet
Development

Successfully merging this pull request may close these issues.