SR: 404 for compatibility request for non-confluent SR impl

[NinaLee96]

Issue submitter TODO list

• I've looked up my issue in FAQ
• I've searched for an already existing issues here
• I've tried running main-labeled docker image and the issue still persists there
• I'm running a supported version of the application which is listed here

Describe the bug (actual behavior)

When I navigate to the Schema Registry tab, in the Browser Developer Tool, I see that the request to retrieve the schema registry returns a 200 with a response body of my schemas.

However, after that it calls/schemas/compatibility, the response body returns a 404 not found. I noticed the API appended an extra / at the end resulting in /config/ in the response body

Based on the confluent documentation: https://docs.confluent.io/platform/7.8/schema-registry/develop/api.html#config

It looks like the official confluent api endpoint is using /config for the PUT and the GET request. Is it possible to get the API changed to reflect that as well?

I see that the swagger API has /config/ listed:
https://github.com/kafbat/kafka-ui/blob/main/contract/src/main/resources/swagger/kafka-sr-api.yaml#L168

Prior, I was using ghcr.io/kafbat/kafka-ui:v1.0.0 and the above issue didn't seem to be a problem when it came to rendering the schemas on the UI.

Expected behavior

I expected the Schema Registry to list my schemas on the UI

Your installation details

app version: 4cf17a0 v1.1.0

Image using: ghcr.io/kafbat/kafka-ui:v1.1.0.

I've deployed the application via helm chart that I've created.

Steps to reproduce

1. Connect kafbat to a cloud provider's schema registry
2. Navigate to the Schema Registry Tab on the UI

Screenshots

Logs

GET request

GET https://REDACTED/api/clusters/event-streams/schemas/compatibility
Status 500
Version HTTP/2
Transferred 8.33 kB (7.63 kB size)
Referrer Policy no-referrer
DNS Resolution System

Response header:

{
  "code": 5000,
  "message": "404 Not Found from GET <REDACTED>/confluent/config/",
  "timestamp": 1738096873080,
  "requestId": "b1ea1820-91",
  "fieldsErrors": [],
  "stackTrace": "org.springframework.web.reactive.function.client.WebClientResponseException$NotFound: 404 Not Found from GET <REDACTED>/confluent/config/\n\tat org.springframework.web.reactive.function.client.WebClientResponseException.create(WebClientResponseException.java:324)\n\tSuppressed: The stacktrace has been enhanced by Reactor, refer to additional information below: \nError has been observed at the following site(s):\n\t*__checkpoint ⇢ 404 NOT_FOUND from GET <REDACTED>/confluent/config/ [DefaultWebClient]\n\t*__checkpoint ⇢ Handler io.kafbat.ui.controller.SchemasController#getGlobalSchemaCompatibilityLevel(String, ServerWebExchange) [DispatcherHandler]\n\t*__checkpoint ⇢ io.kafbat.ui.config.CorsGlobalConfiguration$$Lambda/0x00007f0bac73b1a0 [DefaultWebFilterChain]\n\t*__checkpoint ⇢ io.kafbat.ui.config.ReadOnlyModeFilter [DefaultWebFilterChain]\n\t*__checkpoint ⇢ io.kafbat.ui.config.CustomWebFilter [DefaultWebFilterChain]\n\t*__checkpoint ⇢ AuthorizationWebFilter [DefaultWebFilterChain]\n\t*__checkpoint ⇢ ExceptionTranslationWebFilter [DefaultWebFilterChain]\n\t*__checkpoint ⇢ LogoutWebFilter [DefaultWebFilterChain]\n\t*__checkpoint ⇢ ServerRequestCacheWebFilter [DefaultWebFilterChain]\n\t*__checkpoint ⇢ SecurityContextServerWebExchangeWebFilter [DefaultWebFilterChain]\n\t*__checkpoint ⇢ LogoutPageGeneratingWebFilter [DefaultWebFilterChain]\n\t*__checkpoint ⇢ LoginPageGeneratingWebFilter [DefaultWebFilterChain]\n\t*__checkpoint ⇢ StaticFileWebFilter [DefaultWebFilterChain]\n\t*__checkpoint ⇢ DefaultResourcesWebFilter [DefaultWebFilterChain]\n\t*__checkpoint ⇢ OAuth2LoginAuthenticationWebFilter [DefaultWebFilterChain]\n\t*__checkpoint ⇢ OAuth2AuthorizationRequestRedirectWebFilter [DefaultWebFilterChain]\n\t*__checkpoint ⇢ ReactorContextWebFilter [DefaultWebFilterChain]\n\t*__checkpoint ⇢ HttpHeaderWriterWebFilter [DefaultWebFilterChain]\n\t*__checkpoint ⇢ ServerWebExchangeReactorContextWebFilter [DefaultWebFilterChain]\n\t*__checkpoint ⇢ org.springframework.security.web.server.WebFilterChainProxy [DefaultWebFilterChain]\n\t*__checkpoint ⇢ HTTP GET \"/provectus/api/clusters/event-streams-enterprise/schemas/compatibility\" [ExceptionHandlingWebHandler]\nOriginal Stack Trace:\n\t\tat org.springframework.web.reactive.function.client.WebClientResponseException.create(WebClientResponseException.java:324)\n\t\tat org.springframework.web.reactive.function.client.DefaultClientResponse.lambda$createException$1(DefaultClientResponse.java:214)\n\t\tat reactor.core.publisher.FluxMap$MapSubscriber.onNext(FluxMap.java:106)\n\t\tat reactor.core.publisher.FluxOnErrorReturn$ReturnSubscriber.onNext(FluxOnErrorReturn.java:162)\n\t\tat reactor.core.publisher.FluxDefaultIfEmpty$DefaultIfEmptySubscriber.onNext(FluxDefaultIfEmpty.java:122)\n\t\tat reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onNext(FluxMapFuseable.java:129)\n\t\tat reactor.core.publisher.FluxContextWrite$ContextWriteSubscriber.onNext(FluxContextWrite.java:107)\n\t\tat reactor.core.publisher.FluxMapFuseable$MapFuseableConditionalSubscriber.onNext(FluxMapFuseable.java:299)\n\t\tat reactor.core.publisher.FluxFilterFuseable$FilterFuseableConditionalSubscriber.onNext(FluxFilterFuseable.java:337)\n\t\tat reactor.core.publisher.Operators$BaseFluxToMonoOperator.completePossiblyEmpty(Operators.java:2097)\n\t\tat reactor.core.publisher.MonoCollect$CollectSubscriber.onComplete(MonoCollect.java:145)\n\t\tat reactor.core.publisher.FluxMap$MapSubscriber.onComplete(FluxMap.java:144)\n\t\tat reactor.core.publisher.FluxPeek$PeekSubscriber.onComplete(FluxPeek.java:260)\n\t\tat reactor.core.publisher.FluxMap$MapSubscriber.onComplete(FluxMap.java:144)\n\t\tat reactor.netty.channel.FluxReceive.onInboundComplete(FluxReceive.java:413)\n\t\tat reactor.netty.channel.ChannelOperations.onInboundComplete(ChannelOperations.java:455)\n\t\tat reactor.netty.channel.ChannelOperations.terminate(ChannelOperations.java:509)\n\t\tat reactor.netty.http.client.HttpClientOperations.onInboundNext(HttpClientOperations.java:817)\n\t\tat reactor.netty.channel.ChannelOperationsHandler.channelRead(ChannelOperationsHandler.java:115)\n\t\tat io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444)\n\t\tat io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)\n\t\tat io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)\n\t\tat io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:436)\n\t\tat io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346)\n\t\tat io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318)\n\t\tat io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:251)\n\t\tat io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442)\n\t\tat io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)\n\t\tat io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)\n\t\tat io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1503)\n\t\tat io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1366)\n\t\tat io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1415)\n\t\tat io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:530)\n\t\tat io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:469)\n\t\tat io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:290)\n\t\tat io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444)\n\t\tat io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)\n\t\tat io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)\n\t\tat io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1357)\n\t\tat io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440)\n\t\tat io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)\n\t\tat io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:868)\n\t\tat io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:799)\n\t\tat io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:501)\n\t\tat io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:399)\n\t\tat io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)\n\t\tat io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)\n\t\tat io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)\n\t\tat java.base/java.lang.Thread.run(Thread.java:1583)\n"
}

Additional context

I'm not too sure of the functionality of the project, but I tried the changes locally and it seems to work.

I'm unsure of the side effects/regression it could cause by changing the API.

Steps:

1. Pulled repo with latest changes on main
2. Changed the /config/ to /config in kafka-sr-api.yaml
3. Rebuilt the project using ./mvnw clean install -Dmaven.test.skip=true -Pprod
4. Pushed image to a private registry
5. Deploy

[Haarolean]

Hi, thanks for reporting this.
A few observations:

1. We haven't changed the SR API since July 2023 according to git history of the file. So there shouldn't be any difference between 1.0.0 and 1.1.0.
2. Which SR implementation do you use? <REDACTED>/confluent/config/ implies it uses some confluent-compatible API, so my guess is it's not the confluent's SR, am I right? In this case, the implementation of SR might not treat these URLs the same way (with or without slash) as confluent does.

Can you call the same URL manually but remove the slash, does it work?

[NinaLee96]

Hi!

The schema registry uses Confluent's APIs but only the HTTP REST endpoints with the following paths are supported:

• compatibility
• config
• schemas
• subjects

Docs:

• https://cloud.ibm.com/docs/EventStreams?topic=EventStreams-ES_schema_registry#using_schema_registry_confluent

Testing locally

If I try to call the endpoint locally, I get a response similar to what I would get on the UI with/without the slash

curl https://token:<apikey>@REDACTED_HOST/confluent/config/

404 page not found

curl https://token:<apikey>@REDACTED_HOST/confluent/config

{"compatibilityLevel":"NONE"}

[Haarolean]

Ah, I see. That's what I've been thinking of. This is not confluent's SR per se, it's a third-party SR provider emulating confluent's API, which doesn't do that properly (the presence of a slash at the end affecting the result).

I'll adjust the path in our SR API reference, but you should report this to IBM as well, this is a sane presumption that it shouldn't affect the output.

[Haarolean]

Should be available under main-tagged image shortly