Skip automatically preserved request headers when rewriting

In elastic#79412 we fixed a bug that request headers got dropped when the
request is sent across to a node of different version. The fix is to
restore all existing request headers during the threadContext rewriting.
However, there are headers that are always automatically preserved by
the ThreadContext infrastructure, e.g. x-opaque-id. This causes failures
when the code tries to re-add the x-opaque-id header since it already
exists. An example of this issue is for CCS where the remote cluster is
often on a different version compared to the local cluster.

Resolves: elastic#79412

x-pack/plugin/core/src/main/java/org/elasticsearch/xpack/core/security/SecurityContext.java

@@ -175,7 +175,7 @@ public void executeAfterRewritingAuthentication(Consumer<StoredContext> consumer
                 )
             );
             existingRequestHeaders.forEach((k, v) -> {
-                if (false == AuthenticationField.AUTHENTICATION_KEY.equals(k)) {
+                if (threadContext.getHeader(k) == null) {
                     threadContext.putHeader(k, v);
                 }
             });

x-pack/plugin/security/src/test/java/org/elasticsearch/xpack/security/SecurityContextTests.java

@@ -12,6 +12,7 @@
 import org.elasticsearch.common.settings.Settings;
 import org.elasticsearch.common.util.concurrent.ThreadContext;
 import org.elasticsearch.common.util.concurrent.ThreadContext.StoredContext;
+import org.elasticsearch.tasks.Task;
 import org.elasticsearch.test.ESTestCase;
 import org.elasticsearch.test.VersionUtils;
 import org.elasticsearch.xpack.core.security.SecurityContext;
@@ -124,7 +125,11 @@ public void testExecuteAfterRewritingAuthentication() throws IOException {
             AuthenticationField.PRIVILEGE_CATEGORY_KEY,
             randomAlphaOfLengthBetween(3, 10),
             randomAlphaOfLengthBetween(3, 8),
-            randomAlphaOfLengthBetween(3, 8)
+            randomAlphaOfLengthBetween(3, 8),
+            Task.X_OPAQUE_ID,
+            randomAlphaOfLength(10),
+            Task.TRACE_ID,
+            randomAlphaOfLength(20)
         );
         threadContext.putHeader(requestHeaders);