[fix][broker] TokenAuthenticationState: authenticate token only once

[michaeljmarshall]

PIP: #12105

Builds on #19282 and similar to #19295.

Motivation

Pulsar always calls the AuthenticationState#authenticate method to authenticate authData. We can remove an unnecessary call to authenticate, which saves some resources.

Modifications

• Remove the logic to authenticate authData in the TokenAuthenticationState constructor.
• Update the AuthenticationProviderList#newAuthState implementation so that it authenticates AuthData that is passed to it. Given the testing, this design might have been the original one. It's also possible that the original creators meant to put all states into the list of potential auth states.
• Replace testNewHttpAuthState with testAuthenticateHttpRequest. This change is an extension of [feat][broker] Update AuthenticationProvider to simplify HTTP Authn #19197 where we deprecated newHttpAuthState in favor of authenticateHttpRequest.
• Update AuthenticationProviderList#newAuthState and AuthenticationProviderList#newHttpAuthState to add all providers that do not throw an exception on the respective newAuthState and newHttpAuthState method calls. This logic started to fail after I updated TokenAuthenticationState#newAuthState to not authenticate the token because the list logic considered the first one to not fail on initialization to be the one to put in the list. As such, the list never had more than one state. Given the logic in the AuthenticationListState class, this seems like an unintentional design. The side effect of this change is that authentication will be slightly more expensive because the list provider will use the "good" and the "bad" token provider in the auth state.
• Update AuthenticationService to only wrap AuthenticationProvider with AuthenticationProviderList when there are multiple authentication providers. This change could be controversial because I ran into an issue where ProxySaslAuthenticationTest failed because of a change I made in the AuthenticationProviderList logic, and that test failure improved this PR. However, it seems completely unnecessary to wrap all AuthenticationProviders with the AuthenticationProviderList when it is rarely necessary.
• Update AuthenticationProvider#authenticateHttpRequest default implementation to match the AuthenticationProviderList#authenticateHttpRequest handling, which caught all exceptions and threw them as AuthenticationException.

Verifying this change

New tests are added and old tests also verify correctness.

Does this pull request potentially affect one of the following parts:

In a sense, this breaks an implicit contract that the class had. However, because the getAuthRole() method will throw an exception if called incorrectly, it is likely that misuse of this class will result in a fail fast behavior.

Documentation

• doc-not-needed

This change affects third party broker plugins. We do not document these features yet, so the best we can do is add a comment in the release notes.

Matching PR in forked repository

PR in forked repository: michaeljmarshall#19

[michaeljmarshall]

Tests failures are legitimate. I'll look into those later tonight or tomorrow morning.

[michaeljmarshall]

I expect the tests to pass now.

[lhotari]

/pulsarbot rerun-failure-checks

[lhotari]

There are some tests failures, for example https://github.com/apache/pulsar/actions/runs/4055811136/jobs/6979681080#step:11:943

Error:  Tests run: 6, Failures: 1, Errors: 0, Skipped: 3, Time elapsed: 0.615 s <<< FAILURE! - in org.apache.pulsar.broker.auth.AuthenticationServiceTest
  Error:  testAuthenticationHttpRequestResponseWithAnonymousRole(org.apache.pulsar.broker.auth.AuthenticationServiceTest)  Time elapsed: 0.039 s  <<< FAILURE!
  java.util.concurrent.ExecutionException: javax.naming.AuthenticationException: I failed
  	at java.base/java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:396)
  	at java.base/java.util.concurrent.CompletableFuture.get(CompletableFuture.java:2073)
  	at org.apache.pulsar.broker.authentication.AuthenticationProvider.authenticateHttpRequest(AuthenticationProvider.java:159)
  	at org.apache.pulsar.broker.authentication.AuthenticationService.authenticateHttpRequest(AuthenticationService.java:133)
  	at org.apache.pulsar.broker.auth.AuthenticationServiceTest.testAuthenticationHttpRequestResponseWithAnonymousRole(AuthenticationServiceTest.java:162)
  	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
  	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
  	at org.testng.internal.invokers.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:139)
  	at org.testng.internal.invokers.InvokeMethodRunnable.runOne(InvokeMethodRunnable.java:47)
  	at org.testng.internal.invokers.InvokeMethodRunnable.call(InvokeMethodRunnable.java:76)
  	at org.testng.internal.invokers.InvokeMethodRunnable.call(InvokeMethodRunnable.java:11)
  	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
  	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
  	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
  	at java.base/java.lang.Thread.run(Thread.java:833)
  Caused by: javax.naming.AuthenticationException: I failed
  	at org.apache.pulsar.broker.auth.AuthenticationServiceTest$MockAuthenticationProviderAlwaysFail.authenticate(AuthenticationServiceTest.java:228)
  	at org.apache.pulsar.broker.authentication.AuthenticationProvider.authenticateAsync(AuthenticationProvider.java:72)
  	... 15 more