Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Jet 4.3 fails when Kryo enabled #44

Open
TomaszGaweda opened this issue Nov 13, 2020 · 4 comments
Open

Jet 4.3 fails when Kryo enabled #44

TomaszGaweda opened this issue Nov 13, 2020 · 4 comments
Assignees
@jerrinot

Comments

@TomaszGaweda
Copy link

Consider following test:

import com.hazelcast.config.GlobalSerializerConfig;
import com.hazelcast.config.SerializationConfig;
import com.hazelcast.jet.Jet;
import com.hazelcast.jet.config.JetConfig;
import com.hazelcast.jet.pipeline.Pipeline;
import com.hazelcast.jet.pipeline.Sinks;
import com.hazelcast.jet.pipeline.Sources;
import info.jerrinot.subzero.Serializer;
import org.junit.jupiter.api.Test;

class ReproTest {

	@Test
	void repro() {
		var jc = new JetConfig();
		jc.getProperties().setProperty("hazelcast.logging.type", "slf4j");
		jc.getProperties().setProperty("hazelcast.phone.home.enabled", "false");

		var hazelcastConf = jc.getHazelcastConfig();
		hazelcastConf.setClusterName("serialization-test");
		hazelcastConf.setSerializationConfig(kryoSerializationConfiguration());

		var jet = Jet.newJetInstance(jc);

		var pipeline = Pipeline.create()
				.readFrom(Sources.list("abc"))
				.writeTo(Sinks.noop())
		.getPipeline();

		jet.newJob(pipeline).join();
	}


	private static SerializationConfig kryoSerializationConfiguration() {
		return new SerializationConfig()
				.setGlobalSerializerConfig(new GlobalSerializerConfig()
						.setClassName(Serializer.class.getName())
						.setOverrideJavaSerialization(true)
				);
	}

}

It works fine on Jet 4.2, fails on Jet 4.3. Serializer.class is SubZero's class.
Environment: Windows 10, Java 14.
@jerrinot jerrinot self-assigned this Nov 16, 2020
@jerrinot
Copy link
Owner

excellent reproducer, thanks! I will have a look today or tomorrow.

@jerrinot
Copy link
Owner

It's not working for me even in 4.2. The last Jet version where the test is passing is 4.1.1.

@jerrinot
Copy link
Owner

this is from Jet 4.2:

digraph DAG {
	"listSource(abc)" [localParallelism=1];
	"noop" [localParallelism=1];
	"listSource(abc)" -> "noop" [queueSize=1024];
}
HINT: You can use graphviz or http://viz-js.com to visualize the printed graph.
00:00  WARN: [kryo] Unable to load class com.hazelcast.jet.impl.connector.HazelcastReaders$$Lambda$1170/0x0000000800e89c40 with kryo's ClassLoader. Retrying with current..
14:21:13.397 [ERROR] [c.h.j.i.o.InitExecutionOperation] info.jerrinot.subzero.relocated.com.esotericsoftware.kryo.KryoException: Unable to find class: com.hazelcast.jet.impl.connector.HazelcastReaders$$Lambda$1170/0x0000000800e89c40
com.hazelcast.nio.serialization.HazelcastSerializationException: info.jerrinot.subzero.relocated.com.esotericsoftware.kryo.KryoException: Unable to find class: com.hazelcast.jet.impl.connector.HazelcastReaders$$Lambda$1170/0x0000000800e89c40
	at com.hazelcast.internal.serialization.impl.SerializationUtil.handleException(SerializationUtil.java:103)
	at com.hazelcast.internal.serialization.impl.AbstractSerializationService.readObject(AbstractSerializationService.java:290)
	at com.hazelcast.internal.serialization.impl.ByteArrayObjectDataInput.readObject(ByteArrayObjectDataInput.java:567)
	at com.hazelcast.jet.impl.processor.ProcessorSupplierFromSimpleSupplier.readData(ProcessorSupplierFromSimpleSupplier.java:56)
	at com.hazelcast.internal.serialization.impl.DataSerializableSerializer.readInternal(DataSerializableSerializer.java:160)
	at com.hazelcast.internal.serialization.impl.DataSerializableSerializer.read(DataSerializableSerializer.java:106)
	at com.hazelcast.internal.serialization.impl.DataSerializableSerializer.read(DataSerializableSerializer.java:51)
	at com.hazelcast.internal.serialization.impl.StreamSerializerAdapter.read(StreamSerializerAdapter.java:44)
	at com.hazelcast.internal.serialization.impl.AbstractSerializationService.readObject(AbstractSerializationService.java:284)
	at com.hazelcast.internal.serialization.impl.ByteArrayObjectDataInput.readObject(ByteArrayObjectDataInput.java:567)
	at com.hazelcast.jet.impl.execution.init.CustomClassLoadedObject$Serializer.read(CustomClassLoadedObject.java:125)
	at com.hazelcast.jet.impl.execution.init.CustomClassLoadedObject$Serializer.read(CustomClassLoadedObject.java:92)
	at com.hazelcast.internal.serialization.impl.StreamSerializerAdapter.read(StreamSerializerAdapter.java:44)
	at com.hazelcast.internal.serialization.impl.AbstractSerializationService.readObject(AbstractSerializationService.java:284)
	at com.hazelcast.internal.serialization.impl.ByteArrayObjectDataInput.readObject(ByteArrayObjectDataInput.java:567)
	at com.hazelcast.jet.impl.execution.init.CustomClassLoadedObject.read(CustomClassLoadedObject.java:57)
	at com.hazelcast.jet.impl.execution.init.VertexDef.readData(VertexDef.java:161)
	at com.hazelcast.internal.serialization.impl.DataSerializableSerializer.readInternal(DataSerializableSerializer.java:160)
	at com.hazelcast.internal.serialization.impl.DataSerializableSerializer.read(DataSerializableSerializer.java:106)
	at com.hazelcast.internal.serialization.impl.DataSerializableSerializer.read(DataSerializableSerializer.java:51)
	at com.hazelcast.internal.serialization.impl.StreamSerializerAdapter.read(StreamSerializerAdapter.java:44)
	at com.hazelcast.internal.serialization.impl.AbstractSerializationService.readObject(AbstractSerializationService.java:284)
	at com.hazelcast.internal.serialization.impl.ByteArrayObjectDataInput.readObject(ByteArrayObjectDataInput.java:567)
	at com.hazelcast.jet.impl.util.ImdgUtil.readList(ImdgUtil.java:150)
	at com.hazelcast.jet.impl.execution.init.ExecutionPlan.readData(ExecutionPlan.java:289)
	at com.hazelcast.internal.serialization.impl.DataSerializableSerializer.readInternal(DataSerializableSerializer.java:160)
	at com.hazelcast.internal.serialization.impl.DataSerializableSerializer.read(DataSerializableSerializer.java:106)
	at com.hazelcast.internal.serialization.impl.DataSerializableSerializer.read(DataSerializableSerializer.java:51)
	at com.hazelcast.internal.serialization.impl.StreamSerializerAdapter.read(StreamSerializerAdapter.java:44)
	at com.hazelcast.internal.serialization.impl.AbstractSerializationService.toObject(AbstractSerializationService.java:203)
	at com.hazelcast.jet.impl.execution.init.CustomClassLoadedObject.deserializeWithCustomClassLoader(CustomClassLoadedObject.java:66)
	at com.hazelcast.jet.impl.operation.InitExecutionOperation.deserializePlan(InitExecutionOperation.java:117)
	at com.hazelcast.jet.impl.operation.InitExecutionOperation.run(InitExecutionOperation.java:72)
	at com.hazelcast.spi.impl.operationservice.Operation.call(Operation.java:184)
	at com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.call(OperationRunnerImpl.java:228)
	at com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.run(OperationRunnerImpl.java:217)
	at com.hazelcast.spi.impl.operationexecutor.impl.OperationExecutorImpl.run(OperationExecutorImpl.java:406)
	at com.hazelcast.spi.impl.operationexecutor.impl.OperationExecutorImpl.runOrExecute(OperationExecutorImpl.java:433)
	at com.hazelcast.spi.impl.operationservice.impl.Invocation.doInvokeLocal(Invocation.java:590)
	at com.hazelcast.spi.impl.operationservice.impl.Invocation.doInvoke(Invocation.java:575)
	at com.hazelcast.spi.impl.operationservice.impl.Invocation.invoke0(Invocation.java:534)
	at com.hazelcast.spi.impl.operationservice.impl.Invocation.invoke(Invocation.java:236)
	at com.hazelcast.spi.impl.operationservice.impl.InvocationBuilderImpl.invoke(InvocationBuilderImpl.java:59)
	at com.hazelcast.jet.impl.MasterContext.invokeOnParticipant(MasterContext.java:261)
	at com.hazelcast.jet.impl.MasterContext.invokeOnParticipants(MasterContext.java:245)
	at com.hazelcast.jet.impl.MasterJobContext.lambda$tryStartJob$2(MasterJobContext.java:228)
	at com.hazelcast.jet.impl.JobCoordinationService.lambda$submitToCoordinatorThread$42(JobCoordinationService.java:978)
	at com.hazelcast.jet.impl.JobCoordinationService.submitToCoordinatorThread(JobCoordinationService.java:987)
	at com.hazelcast.jet.impl.JobCoordinationService.submitToCoordinatorThread(JobCoordinationService.java:977)
	at com.hazelcast.jet.impl.MasterJobContext.tryStartJob(MasterJobContext.java:187)
	at com.hazelcast.jet.impl.JobCoordinationService.tryStartJob(JobCoordinationService.java:887)
	at com.hazelcast.jet.impl.JobCoordinationService.lambda$submitJob$1(JobCoordinationService.java:246)
	at com.hazelcast.jet.impl.JobCoordinationService.lambda$submitToCoordinatorThread$42(JobCoordinationService.java:978)
	at com.hazelcast.jet.impl.JobCoordinationService.lambda$submitToCoordinatorThread$43(JobCoordinationService.java:997)
	at com.hazelcast.internal.util.executor.CompletableFutureTask.run(CompletableFutureTask.java:64)
	at com.hazelcast.internal.util.executor.CachedExecutorServiceDelegate$Worker.run(CachedExecutorServiceDelegate.java:217)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)
	at java.base/java.lang.Thread.run(Thread.java:832)
	at com.hazelcast.internal.util.executor.HazelcastManagedThread.executeRun(HazelcastManagedThread.java:64)
	at com.hazelcast.internal.util.executor.HazelcastManagedThread.run(HazelcastManagedThread.java:80)
Caused by: info.jerrinot.subzero.relocated.com.esotericsoftware.kryo.KryoException: Unable to find class: com.hazelcast.jet.impl.connector.HazelcastReaders$$Lambda$1170/0x0000000800e89c40
	at info.jerrinot.subzero.relocated.com.esotericsoftware.kryo.util.DefaultClassResolver.readName(DefaultClassResolver.java:160)
	at info.jerrinot.subzero.relocated.com.esotericsoftware.kryo.util.DefaultClassResolver.readClass(DefaultClassResolver.java:133)
	at info.jerrinot.subzero.relocated.com.esotericsoftware.kryo.Kryo.readClass(Kryo.java:693)
	at info.jerrinot.subzero.relocated.com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:804)
	at info.jerrinot.subzero.internal.strategy.GlobalKryoStrategy.readObject(GlobalKryoStrategy.java:28)
	at info.jerrinot.subzero.internal.strategy.KryoStrategy.read(KryoStrategy.java:85)
	at info.jerrinot.subzero.AbstractSerializer.read(AbstractSerializer.java:30)
	at com.hazelcast.internal.serialization.impl.StreamSerializerAdapter.read(StreamSerializerAdapter.java:44)
	at com.hazelcast.internal.serialization.impl.AbstractSerializationService.readObject(AbstractSerializationService.java:284)
	... 59 more
Caused by: java.lang.ClassNotFoundException: com.hazelcast.jet.impl.connector.HazelcastReaders$$Lambda$1170/0x0000000800e89c40
	at java.base/java.lang.Class.forName0(Native Method)
	at java.base/java.lang.Class.forName(Class.java:427)
	at info.jerrinot.subzero.relocated.com.esotericsoftware.kryo.util.DefaultClassResolver.readName(DefaultClassResolver.java:154)
	... 67 more
14:21:13.406 [ERROR] [c.h.j.i.MasterJobContext] Execution of job '0548-8cb8-8d00-0001', execution 0548-8cb8-8d01-0001 failed
	Start time: 2020-11-16T14:21:13.358
	Duration: 47 ms
	To see additional job metrics enable JobConfig.storeMetricsAfterJobCompletion
com.hazelcast.nio.serialization.HazelcastSerializationException: info.jerrinot.subzero.relocated.com.esotericsoftware.kryo.KryoException: Unable to find class: com.hazelcast.jet.impl.connector.HazelcastReaders$$Lambda$1170/0x0000000800e89c40
	at com.hazelcast.internal.serialization.impl.SerializationUtil.handleException(SerializationUtil.java:103)
	at com.hazelcast.internal.serialization.impl.AbstractSerializationService.readObject(AbstractSerializationService.java:290)
	at com.hazelcast.internal.serialization.impl.ByteArrayObjectDataInput.readObject(ByteArrayObjectDataInput.java:567)
	at com.hazelcast.jet.impl.processor.ProcessorSupplierFromSimpleSupplier.readData(ProcessorSupplierFromSimpleSupplier.java:56)
	at com.hazelcast.internal.serialization.impl.DataSerializableSerializer.readInternal(DataSerializableSerializer.java:160)
	at com.hazelcast.internal.serialization.impl.DataSerializableSerializer.read(DataSerializableSerializer.java:106)
	at com.hazelcast.internal.serialization.impl.DataSerializableSerializer.read(DataSerializableSerializer.java:51)
	at com.hazelcast.internal.serialization.impl.StreamSerializerAdapter.read(StreamSerializerAdapter.java:44)
	at com.hazelcast.internal.serialization.impl.AbstractSerializationService.readObject(AbstractSerializationService.java:284)
	at com.hazelcast.internal.serialization.impl.ByteArrayObjectDataInput.readObject(ByteArrayObjectDataInput.java:567)
	at com.hazelcast.jet.impl.execution.init.CustomClassLoadedObject$Serializer.read(CustomClassLoadedObject.java:125)
	at com.hazelcast.jet.impl.execution.init.CustomClassLoadedObject$Serializer.read(CustomClassLoadedObject.java:92)
	at com.hazelcast.internal.serialization.impl.StreamSerializerAdapter.read(StreamSerializerAdapter.java:44)
	at com.hazelcast.internal.serialization.impl.AbstractSerializationService.readObject(AbstractSerializationService.java:284)
	at com.hazelcast.internal.serialization.impl.ByteArrayObjectDataInput.readObject(ByteArrayObjectDataInput.java:567)
	at com.hazelcast.jet.impl.execution.init.CustomClassLoadedObject.read(CustomClassLoadedObject.java:57)
	at com.hazelcast.jet.impl.execution.init.VertexDef.readData(VertexDef.java:161)
	at com.hazelcast.internal.serialization.impl.DataSerializableSerializer.readInternal(DataSerializableSerializer.java:160)
	at com.hazelcast.internal.serialization.impl.DataSerializableSerializer.read(DataSerializableSerializer.java:106)
	at com.hazelcast.internal.serialization.impl.DataSerializableSerializer.read(DataSerializableSerializer.java:51)
	at com.hazelcast.internal.serialization.impl.StreamSerializerAdapter.read(StreamSerializerAdapter.java:44)
	at com.hazelcast.internal.serialization.impl.AbstractSerializationService.readObject(AbstractSerializationService.java:284)
	at com.hazelcast.internal.serialization.impl.ByteArrayObjectDataInput.readObject(ByteArrayObjectDataInput.java:567)
	at com.hazelcast.jet.impl.util.ImdgUtil.readList(ImdgUtil.java:150)
	at com.hazelcast.jet.impl.execution.init.ExecutionPlan.readData(ExecutionPlan.java:289)
	at com.hazelcast.internal.serialization.impl.DataSerializableSerializer.readInternal(DataSerializableSerializer.java:160)
	at com.hazelcast.internal.serialization.impl.DataSerializableSerializer.read(DataSerializableSerializer.java:106)
	at com.hazelcast.internal.serialization.impl.DataSerializableSerializer.read(DataSerializableSerializer.java:51)
	at com.hazelcast.internal.serialization.impl.StreamSerializerAdapter.read(StreamSerializerAdapter.java:44)
	at com.hazelcast.internal.serialization.impl.AbstractSerializationService.toObject(AbstractSerializationService.java:203)
	at com.hazelcast.jet.impl.execution.init.CustomClassLoadedObject.deserializeWithCustomClassLoader(CustomClassLoadedObject.java:66)
	at com.hazelcast.jet.impl.operation.InitExecutionOperation.deserializePlan(InitExecutionOperation.java:117)
	at com.hazelcast.jet.impl.operation.InitExecutionOperation.run(InitExecutionOperation.java:72)
	at com.hazelcast.spi.impl.operationservice.Operation.call(Operation.java:184)
	at com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.call(OperationRunnerImpl.java:228)
	at com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.run(OperationRunnerImpl.java:217)
	at com.hazelcast.spi.impl.operationexecutor.impl.OperationExecutorImpl.run(OperationExecutorImpl.java:406)
	at com.hazelcast.spi.impl.operationexecutor.impl.OperationExecutorImpl.runOrExecute(OperationExecutorImpl.java:433)
	at com.hazelcast.spi.impl.operationservice.impl.Invocation.doInvokeLocal(Invocation.java:590)
	at com.hazelcast.spi.impl.operationservice.impl.Invocation.doInvoke(Invocation.java:575)
	at com.hazelcast.spi.impl.operationservice.impl.Invocation.invoke0(Invocation.java:534)
	at com.hazelcast.spi.impl.operationservice.impl.Invocation.invoke(Invocation.java:236)
	at com.hazelcast.spi.impl.operationservice.impl.InvocationBuilderImpl.invoke(InvocationBuilderImpl.java:59)
	at com.hazelcast.jet.impl.MasterContext.invokeOnParticipant(MasterContext.java:261)
	at com.hazelcast.jet.impl.MasterContext.invokeOnParticipants(MasterContext.java:245)
	at com.hazelcast.jet.impl.MasterJobContext.lambda$tryStartJob$2(MasterJobContext.java:228)
	at com.hazelcast.jet.impl.JobCoordinationService.lambda$submitToCoordinatorThread$42(JobCoordinationService.java:978)
	at com.hazelcast.jet.impl.JobCoordinationService.submitToCoordinatorThread(JobCoordinationService.java:987)
	at com.hazelcast.jet.impl.JobCoordinationService.submitToCoordinatorThread(JobCoordinationService.java:977)
	at com.hazelcast.jet.impl.MasterJobContext.tryStartJob(MasterJobContext.java:187)
	at com.hazelcast.jet.impl.JobCoordinationService.tryStartJob(JobCoordinationService.java:887)
	at com.hazelcast.jet.impl.JobCoordinationService.lambda$submitJob$1(JobCoordinationService.java:246)
	at com.hazelcast.jet.impl.JobCoordinationService.lambda$submitToCoordinatorThread$42(JobCoordinationService.java:978)
	at com.hazelcast.jet.impl.JobCoordinationService.lambda$submitToCoordinatorThread$43(JobCoordinationService.java:997)
	at com.hazelcast.internal.util.executor.CompletableFutureTask.run(CompletableFutureTask.java:64)
	at com.hazelcast.internal.util.executor.CachedExecutorServiceDelegate$Worker.run(CachedExecutorServiceDelegate.java:217)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)
	at java.base/java.lang.Thread.run(Thread.java:832)
	at com.hazelcast.internal.util.executor.HazelcastManagedThread.executeRun(HazelcastManagedThread.java:64)
	at com.hazelcast.internal.util.executor.HazelcastManagedThread.run(HazelcastManagedThread.java:80)
Caused by: info.jerrinot.subzero.relocated.com.esotericsoftware.kryo.KryoException: Unable to find class: com.hazelcast.jet.impl.connector.HazelcastReaders$$Lambda$1170/0x0000000800e89c40
	at info.jerrinot.subzero.relocated.com.esotericsoftware.kryo.util.DefaultClassResolver.readName(DefaultClassResolver.java:160)
	at info.jerrinot.subzero.relocated.com.esotericsoftware.kryo.util.DefaultClassResolver.readClass(DefaultClassResolver.java:133)
	at info.jerrinot.subzero.relocated.com.esotericsoftware.kryo.Kryo.readClass(Kryo.java:693)
	at info.jerrinot.subzero.relocated.com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:804)
	at info.jerrinot.subzero.internal.strategy.GlobalKryoStrategy.readObject(GlobalKryoStrategy.java:28)
	at info.jerrinot.subzero.internal.strategy.KryoStrategy.read(KryoStrategy.java:85)
	at info.jerrinot.subzero.AbstractSerializer.read(AbstractSerializer.java:30)
	at com.hazelcast.internal.serialization.impl.StreamSerializerAdapter.read(StreamSerializerAdapter.java:44)
	at com.hazelcast.internal.serialization.impl.AbstractSerializationService.readObject(AbstractSerializationService.java:284)
	... 59 more
Caused by: java.lang.ClassNotFoundException: com.hazelcast.jet.impl.connector.HazelcastReaders$$Lambda$1170/0x0000000800e89c40
	at java.base/java.lang.Class.forName0(Native Method)
	at java.base/java.lang.Class.forName(Class.java:427)
	at info.jerrinot.subzero.relocated.com.esotericsoftware.kryo.util.DefaultClassResolver.readName(DefaultClassResolver.java:154)
	... 67 more
Exception in thread "main" java.util.concurrent.CompletionException: com.hazelcast.jet.JetException: com.hazelcast.nio.serialization.HazelcastSerializationException: info.jerrinot.subzero.relocated.com.esotericsoftware.kryo.KryoException: Unable to find class: com.hazelcast.jet.impl.connector.HazelcastReaders$$Lambda$1170/0x0000000800e89c40
	at java.base/java.util.concurrent.CompletableFuture.reportJoin(CompletableFuture.java:412)
	at java.base/java.util.concurrent.CompletableFuture.join(CompletableFuture.java:2108)
	at com.hazelcast.jet.Job.join(Job.java:137)
	at info.jerrinot.sandbox.kryobug.Main.main(Main.java:29)
Caused by: com.hazelcast.jet.JetException: com.hazelcast.nio.serialization.HazelcastSerializationException: info.jerrinot.subzero.relocated.com.esotericsoftware.kryo.KryoException: Unable to find class: com.hazelcast.jet.impl.connector.HazelcastReaders$$Lambda$1170/0x0000000800e89c40
	at com.hazelcast.jet.impl.JobCoordinationService.lambda$null$8(JobCoordinationService.java:320)
	at java.base/java.util.concurrent.CompletableFuture.uniHandle(CompletableFuture.java:930)
	at java.base/java.util.concurrent.CompletableFuture$UniHandle.tryFire(CompletableFuture.java:907)
	at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506)
	at java.base/java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2152)
	at com.hazelcast.jet.impl.util.NonCompletableFuture.internalCompleteExceptionally(NonCompletableFuture.java:59)
	at com.hazelcast.jet.impl.MasterJobContext.setFinalResult(MasterJobContext.java:493)
	at com.hazelcast.jet.impl.MasterJobContext.lambda$null$19(MasterJobContext.java:674)
	at com.hazelcast.jet.impl.util.ExceptionUtil.lambda$withTryCatch$0(ExceptionUtil.java:182)
	at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:859)
	at java.base/java.util.concurrent.CompletableFuture.uniWhenCompleteStage(CompletableFuture.java:883)
	at java.base/java.util.concurrent.CompletableFuture.whenComplete(CompletableFuture.java:2315)
	at com.hazelcast.jet.impl.MasterJobContext.lambda$finalizeJob$20(MasterJobContext.java:670)
	at com.hazelcast.jet.impl.JobCoordinationService.lambda$submitToCoordinatorThread$42(JobCoordinationService.java:978)
	at com.hazelcast.jet.impl.JobCoordinationService.lambda$submitToCoordinatorThread$43(JobCoordinationService.java:997)
	at com.hazelcast.internal.util.executor.CompletableFutureTask.run(CompletableFutureTask.java:64)
	at com.hazelcast.internal.util.executor.CachedExecutorServiceDelegate$Worker.run(CachedExecutorServiceDelegate.java:217)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)
	at java.base/java.lang.Thread.run(Thread.java:832)
	at com.hazelcast.internal.util.executor.HazelcastManagedThread.executeRun(HazelcastManagedThread.java:64)
	at com.hazelcast.internal.util.executor.HazelcastManagedThread.run(HazelcastManagedThread.java:80)
Caused by: com.hazelcast.nio.serialization.HazelcastSerializationException: info.jerrinot.subzero.relocated.com.esotericsoftware.kryo.KryoException: Unable to find class: com.hazelcast.jet.impl.connector.HazelcastReaders$$Lambda$1170/0x0000000800e89c40
	at com.hazelcast.internal.serialization.impl.SerializationUtil.handleException(SerializationUtil.java:103)
	at com.hazelcast.internal.serialization.impl.AbstractSerializationService.readObject(AbstractSerializationService.java:290)
	at com.hazelcast.internal.serialization.impl.ByteArrayObjectDataInput.readObject(ByteArrayObjectDataInput.java:567)
	at com.hazelcast.jet.impl.processor.ProcessorSupplierFromSimpleSupplier.readData(ProcessorSupplierFromSimpleSupplier.java:56)
	at com.hazelcast.internal.serialization.impl.DataSerializableSerializer.readInternal(DataSerializableSerializer.java:160)
	at com.hazelcast.internal.serialization.impl.DataSerializableSerializer.read(DataSerializableSerializer.java:106)
	at com.hazelcast.internal.serialization.impl.DataSerializableSerializer.read(DataSerializableSerializer.java:51)
	at com.hazelcast.internal.serialization.impl.StreamSerializerAdapter.read(StreamSerializerAdapter.java:44)
	at com.hazelcast.internal.serialization.impl.AbstractSerializationService.readObject(AbstractSerializationService.java:284)
	at com.hazelcast.internal.serialization.impl.ByteArrayObjectDataInput.readObject(ByteArrayObjectDataInput.java:567)
	at com.hazelcast.jet.impl.execution.init.CustomClassLoadedObject$Serializer.read(CustomClassLoadedObject.java:125)
	at com.hazelcast.jet.impl.execution.init.CustomClassLoadedObject$Serializer.read(CustomClassLoadedObject.java:92)
	at com.hazelcast.internal.serialization.impl.StreamSerializerAdapter.read(StreamSerializerAdapter.java:44)
	at com.hazelcast.internal.serialization.impl.AbstractSerializationService.readObject(AbstractSerializationService.java:284)
	at com.hazelcast.internal.serialization.impl.ByteArrayObjectDataInput.readObject(ByteArrayObjectDataInput.java:567)
	at com.hazelcast.jet.impl.execution.init.CustomClassLoadedObject.read(CustomClassLoadedObject.java:57)
	at com.hazelcast.jet.impl.execution.init.VertexDef.readData(VertexDef.java:161)
	at com.hazelcast.internal.serialization.impl.DataSerializableSerializer.readInternal(DataSerializableSerializer.java:160)
	at com.hazelcast.internal.serialization.impl.DataSerializableSerializer.read(DataSerializableSerializer.java:106)
	at com.hazelcast.internal.serialization.impl.DataSerializableSerializer.read(DataSerializableSerializer.java:51)
	at com.hazelcast.internal.serialization.impl.StreamSerializerAdapter.read(StreamSerializerAdapter.java:44)
	at com.hazelcast.internal.serialization.impl.AbstractSerializationService.readObject(AbstractSerializationService.java:284)
	at com.hazelcast.internal.serialization.impl.ByteArrayObjectDataInput.readObject(ByteArrayObjectDataInput.java:567)
	at com.hazelcast.jet.impl.util.ImdgUtil.readList(ImdgUtil.java:150)
	at com.hazelcast.jet.impl.execution.init.ExecutionPlan.readData(ExecutionPlan.java:289)
	at com.hazelcast.internal.serialization.impl.DataSerializableSerializer.readInternal(DataSerializableSerializer.java:160)
	at com.hazelcast.internal.serialization.impl.DataSerializableSerializer.read(DataSerializableSerializer.java:106)
	at com.hazelcast.internal.serialization.impl.DataSerializableSerializer.read(DataSerializableSerializer.java:51)
	at com.hazelcast.internal.serialization.impl.StreamSerializerAdapter.read(StreamSerializerAdapter.java:44)
	at com.hazelcast.internal.serialization.impl.AbstractSerializationService.toObject(AbstractSerializationService.java:203)
	at com.hazelcast.jet.impl.execution.init.CustomClassLoadedObject.deserializeWithCustomClassLoader(CustomClassLoadedObject.java:66)
	at com.hazelcast.jet.impl.operation.InitExecutionOperation.deserializePlan(InitExecutionOperation.java:117)
	at com.hazelcast.jet.impl.operation.InitExecutionOperation.run(InitExecutionOperation.java:72)
	at com.hazelcast.spi.impl.operationservice.Operation.call(Operation.java:184)
	at com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.call(OperationRunnerImpl.java:228)
	at com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.run(OperationRunnerImpl.java:217)
	at com.hazelcast.spi.impl.operationexecutor.impl.OperationExecutorImpl.run(OperationExecutorImpl.java:406)
	at com.hazelcast.spi.impl.operationexecutor.impl.OperationExecutorImpl.runOrExecute(OperationExecutorImpl.java:433)
	at com.hazelcast.spi.impl.operationservice.impl.Invocation.doInvokeLocal(Invocation.java:590)
	at com.hazelcast.spi.impl.operationservice.impl.Invocation.doInvoke(Invocation.java:575)
	at com.hazelcast.spi.impl.operationservice.impl.Invocation.invoke0(Invocation.java:534)
	at com.hazelcast.spi.impl.operationservice.impl.Invocation.invoke(Invocation.java:236)
	at com.hazelcast.spi.impl.operationservice.impl.InvocationBuilderImpl.invoke(InvocationBuilderImpl.java:59)
	at com.hazelcast.jet.impl.MasterContext.invokeOnParticipant(MasterContext.java:261)
	at com.hazelcast.jet.impl.MasterContext.invokeOnParticipants(MasterContext.java:245)
	at com.hazelcast.jet.impl.MasterJobContext.lambda$tryStartJob$2(MasterJobContext.java:228)
	at com.hazelcast.jet.impl.JobCoordinationService.lambda$submitToCoordinatorThread$42(JobCoordinationService.java:978)
	at com.hazelcast.jet.impl.JobCoordinationService.submitToCoordinatorThread(JobCoordinationService.java:987)
	at com.hazelcast.jet.impl.JobCoordinationService.submitToCoordinatorThread(JobCoordinationService.java:977)
	at com.hazelcast.jet.impl.MasterJobContext.tryStartJob(MasterJobContext.java:187)
	at com.hazelcast.jet.impl.JobCoordinationService.tryStartJob(JobCoordinationService.java:887)
	at com.hazelcast.jet.impl.JobCoordinationService.lambda$submitJob$1(JobCoordinationService.java:246)
	... 9 more
Caused by: info.jerrinot.subzero.relocated.com.esotericsoftware.kryo.KryoException: Unable to find class: com.hazelcast.jet.impl.connector.HazelcastReaders$$Lambda$1170/0x0000000800e89c40
	at info.jerrinot.subzero.relocated.com.esotericsoftware.kryo.util.DefaultClassResolver.readName(DefaultClassResolver.java:160)
	at info.jerrinot.subzero.relocated.com.esotericsoftware.kryo.util.DefaultClassResolver.readClass(DefaultClassResolver.java:133)
	at info.jerrinot.subzero.relocated.com.esotericsoftware.kryo.Kryo.readClass(Kryo.java:693)
	at info.jerrinot.subzero.relocated.com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:804)
	at info.jerrinot.subzero.internal.strategy.GlobalKryoStrategy.readObject(GlobalKryoStrategy.java:28)
	at info.jerrinot.subzero.internal.strategy.KryoStrategy.read(KryoStrategy.java:85)
	at info.jerrinot.subzero.AbstractSerializer.read(AbstractSerializer.java:30)
	at com.hazelcast.internal.serialization.impl.StreamSerializerAdapter.read(StreamSerializerAdapter.java:44)
	at com.hazelcast.internal.serialization.impl.AbstractSerializationService.readObject(AbstractSerializationService.java:284)
	... 59 more
Caused by: java.lang.ClassNotFoundException: com.hazelcast.jet.impl.connector.HazelcastReaders$$Lambda$1170/0x0000000800e89c40
	at java.base/java.lang.Class.forName0(Native Method)
	at java.base/java.lang.Class.forName(Class.java:427)
	at info.jerrinot.subzero.relocated.com.esotericsoftware.kryo.util.DefaultClassResolver.readName(DefaultClassResolver.java:154)
	... 67 more

It looks a bit differently on 4.3, that's to be expected given this change. However the root cause is probably the same.

@jerrinot
Copy link
Owner

this could possibly be related: this looks related: hazelcast/hazelcast-jet#2298

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@jerrinot jerrinot

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@jerrinot @TomaszGaweda