IllegalStateException while reading VirtualServices #3709

rbaeurle · 2022-01-03T14:25:59Z

Describe the bug

I'm the author of bug snowdrop/istio-java-api#137 and finally was directed to #3613. As you can see from this ticket I was able to successfully test the fix on my system. Now I'm using the 'official' kubernetes-client 5.11.1 and I get the following IllegalStateException while reading/creating a VirtualService:

io.fabric8.kubernetes.client.KubernetesClientException: Error executing: GET at: https://172.21.0.1/apis/networking.istio.io/v1beta1/namespaces/statusmanager-pr-291/virtualservices/core--2-1-3--fegip. Cause: Do not know how to construct standard type id resolver for idType: DEDUCTION

Fabric8 Kubernetes Client version

other (please specify in additional context)

Steps to reproduce

get an IstioClient instance:

  @Bean
  public IstioClient istioClient() {
    var config = new ConfigBuilder().withNamespace(configurationService.getNamespace())
                                    .build();

    return new DefaultIstioClient(config);
  }

reading existing VirtualServices:

istioClient.v1beta1()
                   .virtualServices()
                   .withName(deploymentName)
                   .get()

will cause the exception

Expected behavior

No exception - expected to get VirtualService objects

Runtime

Kubernetes (vanilla)

Kubernetes API Server version

other (please specify in additional context)

Environment

other (please specify in additional context)

Fabric8 Kubernetes Client Logs

io.fabric8.kubernetes.client.KubernetesClientException: Error executing: GET at: https://172.21.0.1/apis/networking.istio.io/v1beta1/namespaces/statusmanager-pr-291/virtualservices/core--2-1-3--fegip. Cause: Do not know how to construct standard type id resolver for idType: DEDUCTION
	at io.fabric8.kubernetes.client.dsl.base.OperationSupport.requestException(OperationSupport.java:697)
	at io.fabric8.kubernetes.client.dsl.base.OperationSupport.requestException(OperationSupport.java:701)
	at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:566)
	at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:519)
	at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleGet(OperationSupport.java:488)
	at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleGet(OperationSupport.java:458)
	at io.fabric8.kubernetes.client.dsl.base.BaseOperation.handleGet(BaseOperation.java:696)
	at io.fabric8.kubernetes.client.dsl.base.BaseOperation.getMandatory(BaseOperation.java:182)
	at io.fabric8.kubernetes.client.dsl.base.BaseOperation.get(BaseOperation.java:149)
	at io.fabric8.kubernetes.client.dsl.base.BaseOperation.get(BaseOperation.java:83)
	at com.daimler.ivk.kbv.statusmanager.k8s.KubernetesService.createVirtualService(KubernetesService.java:178)
	at com.daimler.ivk.kbv.statusmanager.k8s.KubernetesService.createCoreDeployment(KubernetesService.java:68)
	at com.daimler.ivk.kbv.statusmanager.k8s.KubernetesService$$FastClassBySpringCGLIB$$b9d5f8f2.invoke(<generated>)
	at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
	at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:771)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
	at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749)
	at org.springframework.cloud.sleuth.annotation.NonReactorSleuthMethodInvocationProcessor.proceedUnderSynchronousSpan(NonReactorSleuthMethodInvocationProcessor.java:55)
	at org.springframework.cloud.sleuth.annotation.NonReactorSleuthMethodInvocationProcessor.process(NonReactorSleuthMethodInvocationProcessor.java:37)
	at org.springframework.cloud.sleuth.annotation.SleuthInterceptor.invoke(SleuthAdvisorConfig.java:192)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175)
	at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749)
	at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:691)
	at com.daimler.ivk.kbv.statusmanager.k8s.KubernetesService$$EnhancerBySpringCGLIB$$fe1ea9e.createCoreDeployment(<generated>)
	at com.daimler.ivk.kbv.statusmanager.service.StatusManagerService.lambda$execute$0(StatusManagerService.java:58)
	at <unknown class>.accept(Unknown Source)
	at java.base/java.util.ArrayList.forEach(Unknown Source)
	at com.daimler.ivk.kbv.statusmanager.service.StatusManagerService.execute(StatusManagerService.java:56)
	at com.daimler.ivk.kbv.statusmanager.service.StatusManagerService$$FastClassBySpringCGLIB$$991f96a8.invoke(<generated>)
	at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
	at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:771)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
	at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749)
	at org.springframework.cache.interceptor.CacheInterceptor.lambda$invoke$0(CacheInterceptor.java:53)
	at <unknown class>.invoke(Unknown Source)
	at org.springframework.cache.interceptor.CacheAspectSupport.invokeOperation(CacheAspectSupport.java:366)
	at org.springframework.cache.interceptor.CacheAspectSupport.execute(CacheAspectSupport.java:421)
	at org.springframework.cache.interceptor.CacheAspectSupport.execute(CacheAspectSupport.java:346)
	at org.springframework.cache.interceptor.CacheInterceptor.invoke(CacheInterceptor.java:61)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
	at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749)
	at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:691)
	at com.daimler.ivk.kbv.statusmanager.service.StatusManagerService$$EnhancerBySpringCGLIB$$f25b4141.execute(<generated>)
	at com.daimler.ivk.kbv.statusmanager.scheduler.StatusScheduler.execute(StatusScheduler.java:109)
	at com.daimler.ivk.kbv.statusmanager.scheduler.StatusScheduler$$FastClassBySpringCGLIB$$ca33d7a1.invoke(<generated>)
	at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
	at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:771)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
	at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749)
	at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:88)
	at org.springframework.cloud.sleuth.instrument.scheduling.TraceSchedulingAspect.traceBackgroundThread(TraceSchedulingAspect.java:76)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
	at java.base/java.lang.reflect.Method.invoke(Unknown Source)
	at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:644)
	at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:633)
	at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:70)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
	at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749)
	at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:95)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
	at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749)
	at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:691)
	at com.daimler.ivk.kbv.statusmanager.scheduler.StatusScheduler$$EnhancerBySpringCGLIB$$c66c9bab.execute(<generated>)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
	at java.base/java.lang.reflect.Method.invoke(Unknown Source)
	at org.springframework.scheduling.support.ScheduledMethodRunnable.run(ScheduledMethodRunnable.java:84)
	at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
	at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source)
	at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
	at java.base/java.lang.Thread.run(Unknown Source)
Caused by: java.lang.IllegalStateException: Do not know how to construct standard type id resolver for idType: DEDUCTION
	at com.fasterxml.jackson.databind.jsontype.impl.StdTypeResolverBuilder.idResolver(StdTypeResolverBuilder.java:281)
	at com.fasterxml.jackson.databind.jsontype.impl.StdTypeResolverBuilder.buildTypeDeserializer(StdTypeResolverBuilder.java:134)
	at com.fasterxml.jackson.databind.deser.BasicDeserializerFactory.findTypeDeserializer(BasicDeserializerFactory.java:1610)
	at com.fasterxml.jackson.databind.deser.BasicDeserializerFactory.findPropertyTypeDeserializer(BasicDeserializerFactory.java:1818)
	at com.fasterxml.jackson.databind.deser.BasicDeserializerFactory.resolveMemberAndTypeAnnotations(BasicDeserializerFactory.java:2186)
	at com.fasterxml.jackson.databind.deser.BeanDeserializerFactory.constructSettableProperty(BeanDeserializerFactory.java:799)
	at com.fasterxml.jackson.databind.deser.BeanDeserializerFactory.addBeanProps(BeanDeserializerFactory.java:522)
	at com.fasterxml.jackson.databind.deser.BeanDeserializerFactory.buildBeanDeserializer(BeanDeserializerFactory.java:234)
	at com.fasterxml.jackson.databind.deser.BeanDeserializerFactory.createBeanDeserializer(BeanDeserializerFactory.java:143)
	at io.fabric8.kubernetes.model.jackson.JsonUnwrappedDeserializer.<init>(JsonUnwrappedDeserializer.java:105)
	at io.fabric8.kubernetes.model.jackson.JsonUnwrappedDeserializer.createContextual(JsonUnwrappedDeserializer.java:112)
	at com.fasterxml.jackson.databind.DeserializationContext.handlePrimaryContextualization(DeserializationContext.java:665)
	at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.resolve(BeanDeserializerBase.java:508)
	at com.fasterxml.jackson.databind.deser.DeserializerCache._createAndCache2(DeserializerCache.java:293)
	at com.fasterxml.jackson.databind.deser.DeserializerCache._createAndCacheValueDeserializer(DeserializerCache.java:244)
	at com.fasterxml.jackson.databind.deser.DeserializerCache.findValueDeserializer(DeserializerCache.java:142)
	at com.fasterxml.jackson.databind.DeserializationContext.findNonContextualValueDeserializer(DeserializationContext.java:481)
	at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.resolve(BeanDeserializerBase.java:497)
	at com.fasterxml.jackson.databind.deser.DeserializerCache._createAndCache2(DeserializerCache.java:293)
	at com.fasterxml.jackson.databind.deser.DeserializerCache._createAndCacheValueDeserializer(DeserializerCache.java:244)
	at com.fasterxml.jackson.databind.deser.DeserializerCache.findValueDeserializer(DeserializerCache.java:142)
	at com.fasterxml.jackson.databind.DeserializationContext.findContextualValueDeserializer(DeserializationContext.java:458)
	at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.createContextual(CollectionDeserializer.java:181)
	at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.createContextual(CollectionDeserializer.java:26)
	at com.fasterxml.jackson.databind.DeserializationContext.handlePrimaryContextualization(DeserializationContext.java:665)
	at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.resolve(BeanDeserializerBase.java:508)
	at com.fasterxml.jackson.databind.deser.DeserializerCache._createAndCache2(DeserializerCache.java:293)
	at com.fasterxml.jackson.databind.deser.DeserializerCache._createAndCacheValueDeserializer(DeserializerCache.java:244)
	at com.fasterxml.jackson.databind.deser.DeserializerCache.findValueDeserializer(DeserializerCache.java:142)
	at com.fasterxml.jackson.databind.DeserializationContext.findNonContextualValueDeserializer(DeserializationContext.java:481)
	at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.resolve(BeanDeserializerBase.java:497)
	at com.fasterxml.jackson.databind.deser.DeserializerCache._createAndCache2(DeserializerCache.java:293)
	at com.fasterxml.jackson.databind.deser.DeserializerCache._createAndCacheValueDeserializer(DeserializerCache.java:244)
	at com.fasterxml.jackson.databind.deser.DeserializerCache.findValueDeserializer(DeserializerCache.java:142)
	at com.fasterxml.jackson.databind.DeserializationContext.findRootValueDeserializer(DeserializationContext.java:491)
	at com.fasterxml.jackson.databind.ObjectMapper._findRootDeserializer(ObjectMapper.java:4713)
	at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4522)
	at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3513)
	at io.fabric8.kubernetes.client.utils.Serialization.unmarshal(Serialization.java:312)
	at io.fabric8.kubernetes.client.utils.Serialization.unmarshal(Serialization.java:266)
	at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:558)
	... 74 common frames omitted

Additional context

kubernetes-client 5.11.1
kubernetes server version: v1.19
k8s running on ibm cloud

The text was updated successfully, but these errors were encountered:

manusa · 2022-01-03T15:37:00Z

/cc @Sgitario @metacosm

Hillelmed · 2022-01-03T16:28:44Z

@manusa
Hi, i have the same issue please check this build https://github.com/fabric8io/kubernetes-client/runs/4692531992?check_suite_focus=true

i wait for it ASAP :)

metacosm · 2022-01-03T20:09:38Z

Would it be possible to add the YAML for the VirtualService that's causing the error? Which version of Istio are you using?

chichaoatabc · 2022-01-04T09:47:10Z

The same exception occured on my system. When I get or create any istio crd(virtualservice\destinationrule\gateway...), the exception throws Cause: com.fasterxml.jackson.annotation.JsonTypeInfo$Id.DEDUCTION.
kubernetes-client 5.11.1
kubernetes server version: v1.21
istio:1.12.1

rbaeurle · 2022-01-04T13:15:59Z

At this time we call the client API there is no VirtualService at all. So I cannot provide any YAML.
Regarding the Istio version:
client version: 1.12.1
control plane version: 1.11.5
data plane version: 1.11.5 (4 proxies)

rohanKanojia · 2022-01-04T13:51:07Z

@rbaeurle : It would be really helpful if you could share a reproducer project with us.

rbaeurle · 2022-01-04T14:10:38Z

Unfortunately, this is a customer project. I.e. I would have to create a test project first and then put it in my private GitHub account ....

rbaeurle · 2022-01-05T07:36:46Z

Update:
I did some further investigations and checked the jackson-lib dependencies. I recognized that there was a version mismatch between the one I got due to my spring boot starter dependency and the one with a 'naked' kubernetes-client project. After adding the jackson dependencies explicitly to my project the IllegalStateException as described above didn't happen anymore.
Unfortunately I get a new one that is similar to the one described in #3696:

java.lang.IllegalStateException: cannot make a new request because the previous response is still open: please call response.close()
	at okhttp3.internal.connection.Transmitter.newExchange(Transmitter.java:164)
	at okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.java:41)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:142)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:117)
	at okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.java:94)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:142)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:117)
	at okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.java:93)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:142)
	at okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.java:88)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:142)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:117)
	at io.fabric8.kubernetes.client.okhttp.OkHttpClientBuilderImpl$InteceptorAdapter.intercept(OkHttpClientBuilderImpl.java:66)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:142)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:117)
	at io.fabric8.kubernetes.client.okhttp.OkHttpClientBuilderImpl$InteceptorAdapter.intercept(OkHttpClientBuilderImpl.java:62)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:142)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:117)
	at io.fabric8.kubernetes.client.okhttp.OkHttpClientBuilderImpl$InteceptorAdapter.intercept(OkHttpClientBuilderImpl.java:62)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:142)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:117)
	at io.fabric8.kubernetes.client.okhttp.OkHttpClientBuilderImpl$InteceptorAdapter.intercept(OkHttpClientBuilderImpl.java:62)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:142)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:117)
	at okhttp3.RealCall.getResponseWithInterceptorChain(RealCall.java:229)
	at okhttp3.RealCall.execute(RealCall.java:81)
	at io.fabric8.kubernetes.client.okhttp.OkHttpClientImpl.send(OkHttpClientImpl.java:138)
	at io.fabric8.kubernetes.client.dsl.base.OperationSupport.retryWithExponentialBackoff(OperationSupport.java:575)
	at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:554)
	at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:519)
	at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleGet(OperationSupport.java:488)
	at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleGet(OperationSupport.java:458)
	at io.fabric8.kubernetes.client.dsl.base.BaseOperation.handleGet(BaseOperation.java:696)
	at io.fabric8.kubernetes.client.dsl.base.BaseOperation.getMandatory(BaseOperation.java:182)
	at io.fabric8.kubernetes.client.dsl.base.BaseOperation.get(BaseOperation.java:149)
	at io.fabric8.kubernetes.client.dsl.base.BaseOperation.get(BaseOperation.java:83)
	at com.daimler.ivk.kbv.statusmanager.k8s.KubernetesService.createDeployment(KubernetesService.java:107)
	at com.daimler.ivk.kbv.statusmanager.k8s.KubernetesService.createCoreDeployment(KubernetesService.java:72)
	at com.daimler.ivk.kbv.statusmanager.k8s.KubernetesService$$FastClassBySpringCGLIB$$b9d5f8f2.invoke(<generated>)

If this is a duplicate to #3696 please feel free to close this ticket. On the other hand .. could this problem also be related to a dependency version mismatch?
Thanks for your support ...

metacosm · 2022-01-05T13:07:55Z

@rbaeurle Thank you for the update. The other issue is a separate one that should be fixed now.

rbaeurle closed this as completed Jan 4, 2022

rbaeurle reopened this Jan 4, 2022

metacosm closed this as completed Jan 5, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

IllegalStateException while reading VirtualServices #3709

IllegalStateException while reading VirtualServices #3709

rbaeurle commented Jan 3, 2022

manusa commented Jan 3, 2022

Hillelmed commented Jan 3, 2022

metacosm commented Jan 3, 2022

chichaoatabc commented Jan 4, 2022

rbaeurle commented Jan 4, 2022

rohanKanojia commented Jan 4, 2022

rbaeurle commented Jan 4, 2022

rbaeurle commented Jan 5, 2022

metacosm commented Jan 5, 2022

IllegalStateException while reading VirtualServices #3709

IllegalStateException while reading VirtualServices #3709

Comments

rbaeurle commented Jan 3, 2022

Describe the bug

Fabric8 Kubernetes Client version

Steps to reproduce

Expected behavior

Runtime

Kubernetes API Server version

Environment

Fabric8 Kubernetes Client Logs

Additional context

manusa commented Jan 3, 2022

Hillelmed commented Jan 3, 2022

metacosm commented Jan 3, 2022

chichaoatabc commented Jan 4, 2022

rbaeurle commented Jan 4, 2022

rohanKanojia commented Jan 4, 2022

rbaeurle commented Jan 4, 2022

rbaeurle commented Jan 5, 2022

metacosm commented Jan 5, 2022