Merge branch 'main' of ssh://github.com/hashicorp/consul into NET-4135

.changelog/13023.txt

@@ -0,0 +1,3 @@
+```release-note:improvement
+ui: the topology view now properly displays services with mixed connect and non-connect instances.
+```

.changelog/18068.txt

@@ -0,0 +1,3 @@
+```release-note:bug
+xds: Prevent partial application of non-Required Envoy extensions in the case of failure.
+```

.changelog/18168.txt

@@ -0,0 +1,3 @@
+```release-note:improvement
+hcp: Add dynamic configuration support for the export of server metrics to HCP.
+```

.changelog/18302.txt

@@ -0,0 +1,4 @@
+```release-note:bug
+snapshot: fix access denied and handle is invalid when we call snapshot save on windows - skip sync() for folders in windows in
+https://github.com/rboyer/safeio/pull/3
+```

.changelog/18319.txt

@@ -0,0 +1,6 @@
+```release-note:improvement
+acl: added builtin ACL policy that provides global read-only access (builtin/global-read-only)
+```
+```release-note:improvement
+acl: allow for a single slash character in policy names
+```

.changelog/18324.txt

@@ -0,0 +1,3 @@
+```release-note:feature
+api-gateway: add retry and timeout filters
+```

.changelog/18325.txt

@@ -0,0 +1,3 @@
+```release-note:bug
+mesh: **(Enterprise Only)** Require that `jwt-provider` config entries are created in the `default` namespace.
+```

.changelog/18336.txt

@@ -0,0 +1,7 @@
+```release-note:feature
+xds: Add a built-in Envoy extension that appends OpenTelemetry Access Logging (otel-access-logging) to the HTTP Connection Manager filter.
+```
+
+```release-note:feature
+xds: Add support for patching outbound listeners to the built-in Envoy External Authorization extension.
+```

.changelog/18358.txt

@@ -0,0 +1,7 @@
+```release-note:security
+Upgrade to use Go 1.20.7.
+This resolves vulnerability [CVE-2023-29409](https://nvd.nist.gov/vuln/detail/CVE-2023-29409)(`crypto/tls`).
+```
+```release-note:security
+Update `golang.org/x/net` to v0.13.0 to address [CVE-2023-3978](https://nvd.nist.gov/vuln/detail/CVE-2023-3978).
+```

.changelog/18367.txt

@@ -0,0 +1,3 @@
+```release-note:bug
+dev-mode: Fix dev mode has new line in responses. Now new line is added only when url has pretty query parameter.
+```

.changelog/18381.txt

@@ -0,0 +1,6 @@
+```release-note:improvement
+checks: It is now possible to configure agent TCP checks to use TLS with
+optional server SNI and mutual authentication. To use TLS with a TCP check, the
+check must enable the `tcp_use_tls` boolean. By default the agent will use the
+TLS configuration in the `tls.default` stanza. 
+```

.changelog/18437.txt

@@ -0,0 +1,3 @@
+```release-note:bug
+Inherit locality from services when registering sidecar proxies.
+```

.changelog/18439.txt

@@ -0,0 +1,3 @@
+```release-note:feature
+Support custom watches on the Consul Controller framework.
+```

.changelog/18464.txt

@@ -0,0 +1,3 @@
+```release-note:bug
+UI : Nodes list view was breaking for synthetic-nodes. Fix handles non existence of consul-version meta for node.
+```

.changelog/18504.txt

@@ -0,0 +1,3 @@
+```release-note:feature
+dataplane: Allow getting bootstrap parameters when using V2 APIs
+```

.changelog/18558.txt

@@ -0,0 +1,3 @@
+```release-note:bug
+check: prevent go routine leakage when existing Defercheck of same check id is not nil
+```

.changelog/18560.txt

@@ -0,0 +1,3 @@
+```release-note:improvement
+ui: Use Community verbiage
+```

.changelog/18583.txt

@@ -0,0 +1,3 @@
+```release-note:feature
+mesh: **(Enterprise only)** Adds rate limiting config to service-defaults
+```

.changelog/18584.txt

@@ -0,0 +1,3 @@
+```release-note:improvement
+Reduce the frequency of metric exports from Consul to HCP from every 10s to every 1m
+```

.changelog/18617.txt

@@ -0,0 +1,4 @@
+```release-note:improvement
+log: Currently consul logs files like this consul-{timestamp}.log. This change makes sure that there is always
+consul.log file with the latest logs in it.
+```

.changelog/18625.txt

@@ -0,0 +1,5 @@
+```release-note:improvement
+Adds flag -append-filename (which works on values version, dc, node and status) to consul snapshot save command.
+Adding the flag -append-filename version,dc,node,status will add consul version, consul datacenter, node name and leader/follower
+(status) in the file name given in the snapshot save command before the file extension.
+```

.changelog/18636.txt

@@ -0,0 +1,3 @@
+```release-note:bug
+connect: Fix issue where Envoy endpoints would not populate correctly after a snapshot restore.
+```

.changelog/18646.txt

@@ -0,0 +1,3 @@
+```release-note:feature
+api-gateway: Add support for response header modifiers on http-route configuration entry
+```

.changelog/18667.txt

@@ -0,0 +1,3 @@
+```release-note:improvement
+api: Add support for listing ACL tokens by service name.
+```

.changelog/18668.txt

@@ -0,0 +1,3 @@
+```release-note:breaking-change
+audit-logging: **(Enterprise only)** allowing timestamp based filename only on rotation. initially the filename will be just file.json
+```

.changelog/18681.txt

@@ -0,0 +1,3 @@
+```release-note:bug
+api: Fix `/v1/agent/self` not returning latest configuration 
+```

.changelog/18708.txt

@@ -0,0 +1,7 @@
+```release-note:feature
+acl: Added ACL Templated policies to simplify getting the right ACL token.
+```
+
+```release-note:improvement
+cli: Added `-templated-policy`, `-templated-policy-file`, `-replace-templated-policy`, `-append-templated-policy`, `-replace-templated-policy-file`, `-append-templated-policy-file` and `-var` flags for creating or updating tokens/roles.
+```

.changelog/18719.txt

@@ -0,0 +1,7 @@
+```release-note:feature
+acl: Add BindRule support for templated policies. Add new BindType: templated-policy and BindVar field for templated policy variables.
+```
+
+```release-note:feature
+cli: Add `bind-var` flag to `consul acl binding-rule` for templated policy variables.
+```

.changelog/18724.txt

@@ -0,0 +1,3 @@
+```release-note:bug
+telemetry: emit consul version metric on a regular interval.
+```

.changelog/18742.txt

@@ -0,0 +1,8 @@
+```release-note:security
+Upgrade to use Go 1.20.8. This resolves CVEs
+[CVE-2023-39320](https://github.com/advisories/GHSA-rxv8-v965-v333) (`cmd/go`),
+[CVE-2023-39318](https://github.com/advisories/GHSA-vq7j-gx56-rxjh) (`html/template`),
+[CVE-2023-39319](https://github.com/advisories/GHSA-vv9m-32rr-3g55) (`html/template`),
+[CVE-2023-39321](https://github.com/advisories/GHSA-9v7r-x7cv-v437) (`crypto/tls`), and
+[CVE-2023-39322](https://github.com/advisories/GHSA-892h-r6cr-53g4) (`crypto/tls`)
+```

.changelog/18769.txt

@@ -0,0 +1,3 @@
+```release-note:feature
+acl: Adds a new ACL rule for workload identities
+```

.changelog/18773.txt

@@ -0,0 +1,3 @@
+```release-note:bug
+ca: Vault provider now cleans up the previous Vault issuer and key when generating a new leaf signing certificate [[GH-18779](https://github.com/hashicorp/consul/issues/18779)]
+```

.changelog/18813.txt

@@ -0,0 +1,3 @@
+```release-note:improvement
+acl: Use templated policy to generate synthetic policies for tokens/roles with node and/or service identities
+```

.changelog/18816.txt

@@ -0,0 +1,3 @@
+```release-note:feature
+cli: Add `consul acl templated-policy` commands to read, list and preview templated policies. 
+```

.changelog/_18366.txt

@@ -0,0 +1,3 @@
+```release-note:feature
+config-entry(api-gateway): (Enterprise only) Add GatewayPolicy to APIGateway Config Entry listeners
+```

.changelog/_18422.txt

@@ -0,0 +1,3 @@
+```release-note:feature
+config-entry(api-gateway): (Enterprise only) Add JWTFilter to HTTPRoute Filters
+```

.changelog/_6870.txt

@@ -0,0 +1,3 @@
+```release-note:feature
+gateway: **(Enterprise only)** Add JWT authentication and authorization to APIGateway Listeners and HTTPRoutes.
+```

.copywrite.hcl

@@ -1,8 +1,8 @@
 schema_version = 1
 
 project {
-  license        = "MPL-2.0"
-  copyright_year = 2013
+  license        = "BUSL-1.1"
+  copyright_year = 2023
 
   # (OPTIONAL) A list of globs that should not have copyright/license headers.
   # Supports doublestar glob patterns for more flexibility in defining which
@@ -19,11 +19,18 @@ project {
 
     # ignore specific test data files
     "agent/uiserver/testdata/**",
+    "internal/resourcehcl/testdata/**",
 
     # generated files 
     "agent/structs/structs.deepcopy.go",
     "agent/proxycfg/proxycfg.deepcopy.go",
     "agent/grpc-middleware/rate_limit_mappings.gen.go",
     "agent/uiserver/dist/**",
+
+    # licensed under MPL - ignoring for now until the copywrite tool can support
+    # multiple licenses per repo.
+    "sdk/**",
+    "api/**",
+    "proto-public/**",
   ]
 }

.github/CODEOWNERS

@@ -8,3 +8,34 @@
 # release configuration
 /.release/                              @hashicorp/release-engineering @hashicorp/github-consul-core
 /.github/workflows/build.yml            @hashicorp/release-engineering @hashicorp/github-consul-core
+
+
+# Staff Engineer Review (protocol buffer definitions)
+/proto-public/   @hashicorp/consul-core-staff
+/proto/          @hashicorp/consul-core-staff
+
+# Staff Engineer Review (v1 architecture shared components)
+/agent/cache/                  @hashicorp/consul-core-staff
+/agent/consul/fsm/             @hashicorp/consul-core-staff
+/agent/consul/leader*.go       @hashicorp/consul-core-staff
+/agent/consul/server*.go       @hashicorp/consul-core-staff
+/agent/consul/state/           @hashicorp/consul-core-staff
+/agent/consul/stream/          @hashicorp/consul-core-staff
+/agent/submatview/             @hashicorp/consul-core-staff
+/agent/blockingquery/          @hashicorp/consul-core-staff
+
+# Staff Engineer Review (raft/autopilot)
+/agent/consul/autopilotevents/  @hashicorp/consul-core-staff
+/agent/consul/autopilot*.go     @hashicorp/consul-core-staff
+
+# Staff Engineer Review (v2 architecture shared components)
+/internal/controller/                   @hashicorp/consul-core-staff
+/internal/resource/                     @hashicorp/consul-core-staff
+/internal/storage/                      @hashicorp/consul-core-staff
+/agent/consul/controller/               @hashicorp/consul-core-staff
+/agent/grpc-external/services/resource/ @hashicorp/consul-core-staff
+
+# Staff Engineer Review (v1 security)
+/acl/                   @hashicorp/consul-core-staff
+/agent/xds/rbac*.go     @hashicorp/consul-core-staff
+/agent/xds/jwt*.go      @hashicorp/consul-core-staff

.github/ISSUE_TEMPLATE/config.yml

@@ -1,5 +1,5 @@
 # Copyright (c) HashiCorp, Inc.
-# SPDX-License-Identifier: MPL-2.0
+# SPDX-License-Identifier: BUSL-1.1
 
 blank_issues_enabled: false
 contact_links:

.github/dependabot.yml

@@ -1,5 +1,5 @@
 # Copyright (c) HashiCorp, Inc.
-# SPDX-License-Identifier: MPL-2.0
+# SPDX-License-Identifier: BUSL-1.1
 
 version: 2
 updates:

.github/pr-labeler.yml

@@ -1,5 +1,5 @@
 # Copyright (c) HashiCorp, Inc.
-# SPDX-License-Identifier: MPL-2.0
+# SPDX-License-Identifier: BUSL-1.1
 
 pr/dependencies:
     - vendor/**/*

.github/scripts/changelog_checker.sh

@@ -1,6 +1,6 @@
 #!/bin/bash
 # Copyright (c) HashiCorp, Inc.
-# SPDX-License-Identifier: MPL-2.0
+# SPDX-License-Identifier: BUSL-1.1
 
 
 set -euo pipefail

.github/scripts/filter_changed_files_go_test.sh

@@ -0,0 +1,37 @@
+#!/bin/bash
+# Copyright (c) HashiCorp, Inc.
+# SPDX-License-Identifier: BUSL-1.1
+
+
+# Get the list of changed files
+files_to_check=$(git diff --name-only origin/$GITHUB_BASE_REF)
+
+# Define the directories to check
+skipped_directories=("docs/" "ui/" "website/" "grafana/")
+
+# Initialize a variable to track directories outside the skipped ones
+other_directories=""
+trigger_ci=true
+
+# # Loop through the changed files and find directories/files outside the skipped ones
+# for file_to_check in $files_to_check; do
+# 	file_is_skipped=false
+# 	for dir in "${skipped_directories[@]}"; do
+# 		if [[ "$file_to_check" == "$dir"* ]] || [[ "$file_to_check" == *.md && "$dir" == *"/" ]]; then
+# 			file_is_skipped=true
+# 			break
+# 		fi
+# 	done
+# 	if [ "$file_is_skipped" = "false" ]; then
+# 		other_directories+="$(dirname "$file_to_check")\n"
+# 		trigger_ci=true
+# 		echo "Non doc file(s) changed - triggered ci: $trigger_ci"
+# 		echo -e $other_directories
+# 		echo "trigger-ci=$trigger_ci" >>"$GITHUB_OUTPUT"
+# 		exit 0 ## if file is outside of the skipped_directory exit script
+# 	fi
+# done
+
+# echo "Only doc file(s) changed - triggered ci: $trigger_ci"
+echo "Doc file(s) change detection is currently disabled - triggering ci"
+echo "trigger-ci=$trigger_ci" >>"$GITHUB_OUTPUT"

.github/scripts/get_runner_classes.sh

@@ -1,26 +1,26 @@
 #!/usr/bin/env bash
 # Copyright (c) HashiCorp, Inc.
-# SPDX-License-Identifier: MPL-2.0
+# SPDX-License-Identifier: BUSL-1.1
 
 #
 # This script generates tag-sets that can be used as runs-on: values to select runners.
 
 set -euo pipefail
 
 case "$GITHUB_REPOSITORY" in
-    *-enterprise)
-        # shellcheck disable=SC2129
-        echo "compute-small=['self-hosted', 'linux', 'small']" >> "$GITHUB_OUTPUT"
-        echo "compute-medium=['self-hosted', 'linux', 'medium']" >> "$GITHUB_OUTPUT"
-        echo "compute-large=['self-hosted', 'linux', 'large']" >> "$GITHUB_OUTPUT"
-        # m5d.8xlarge is equivalent to our xl custom runner in OSS
-        echo "compute-xl=['self-hosted', 'ondemand', 'linux', 'type=m5d.8xlarge']" >> "$GITHUB_OUTPUT"
-        ;;
-    *)
-        # shellcheck disable=SC2129
-        echo "compute-small=['custom-linux-s-consul-latest']" >> "$GITHUB_OUTPUT"
-        echo "compute-medium=['custom-linux-m-consul-latest']" >> "$GITHUB_OUTPUT"
-        echo "compute-large=['custom-linux-l-consul-latest']" >> "$GITHUB_OUTPUT"
-        echo "compute-xl=['custom-linux-xl-consul-latest']" >> "$GITHUB_OUTPUT"
-        ;;
+*-enterprise)
+	# shellcheck disable=SC2129
+	echo "compute-small=['self-hosted', 'linux', 'small']" >>"$GITHUB_OUTPUT"
+	echo "compute-medium=['self-hosted', 'linux', 'medium']" >>"$GITHUB_OUTPUT"
+	echo "compute-large=['self-hosted', 'linux', 'large']" >>"$GITHUB_OUTPUT"
+	# m5d.8xlarge is equivalent to our xl custom runner in CE
+	echo "compute-xl=['self-hosted', 'ondemand', 'linux', 'type=m6a.2xlarge']" >>"$GITHUB_OUTPUT"
+	;;
+*)
+	# shellcheck disable=SC2129
+	echo "compute-small=['custom-linux-s-consul-latest']" >>"$GITHUB_OUTPUT"
+	echo "compute-medium=['custom-linux-m-consul-latest']" >>"$GITHUB_OUTPUT"
+	echo "compute-large=['custom-linux-l-consul-latest']" >>"$GITHUB_OUTPUT"
+	echo "compute-xl=['custom-linux-xl-consul-latest']" >>"$GITHUB_OUTPUT"
+	;;
 esac