Update libraries #4

dartartem · 2020-07-07T14:35:09Z

No description provided.

dartartem · 2020-07-07T15:06:05Z

if use confluentinc/cp-zookeeper image, there will be error in proxy service:

proxy_1           | org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'subscriptionLoader': Invocation of init method failed; nested exception is java.lang.RuntimeException: org.apache.zookeeper.KeeperException$UnimplementedException: KeeperErrorCode = Unimplemented for /eventuate
proxy_1           | 	at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor.postProcessBeforeInitialization(InitDestroyAnnotationBeanPostProcessor.java:139) ~[spring-beans-5.1.7.RELEASE.jar!/:5.1.7.RELEASE]
proxy_1           | 	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyBeanPostProcessorsBeforeInitialization(AbstractAutowireCapableBeanFactory.java:414) ~[spring-beans-5.1.7.RELEASE.jar!/:5.1.7.RELEASE]
proxy_1           | 	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1770) ~[spring-beans-5.1.7.RELEASE.jar!/:5.1.7.RELEASE]
proxy_1           | 	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:593) ~[spring-beans-5.1.7.RELEASE.jar!/:5.1.7.RELEASE]
proxy_1           | 	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:515) ~[spring-beans-5.1.7.RELEASE.jar!/:5.1.7.RELEASE]
proxy_1           | 	at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:320) ~[spring-beans-5.1.7.RELEASE.jar!/:5.1.7.RELEASE]
proxy_1           | 	at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:222) ~[spring-beans-5.1.7.RELEASE.jar!/:5.1.7.RELEASE]
proxy_1           | 	at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:318) ~[spring-beans-5.1.7.RELEASE.jar!/:5.1.7.RELEASE]
proxy_1           | 	at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:199) ~[spring-beans-5.1.7.RELEASE.jar!/:5.1.7.RELEASE]
proxy_1           | 	at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:843) ~[spring-beans-5.1.7.RELEASE.jar!/:5.1.7.RELEASE]
proxy_1           | 	at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:877) ~[spring-context-5.1.7.RELEASE.jar!/:5.1.7.RELEASE]
proxy_1           | 	at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:549) ~[spring-context-5.1.7.RELEASE.jar!/:5.1.7.RELEASE]
proxy_1           | 	at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:142) ~[spring-boot-2.1.5.RELEASE.jar!/:2.1.5.RELEASE]
proxy_1           | 	at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:775) [spring-boot-2.1.5.RELEASE.jar!/:2.1.5.RELEASE]
proxy_1           | 	at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:397) [spring-boot-2.1.5.RELEASE.jar!/:2.1.5.RELEASE]
proxy_1           | 	at org.springframework.boot.SpringApplication.run(SpringApplication.java:316) [spring-boot-2.1.5.RELEASE.jar!/:2.1.5.RELEASE]
proxy_1           | 	at org.springframework.boot.SpringApplication.run(SpringApplication.java:1260) [spring-boot-2.1.5.RELEASE.jar!/:2.1.5.RELEASE]
proxy_1           | 	at org.springframework.boot.SpringApplication.run(SpringApplication.java:1248) [spring-boot-2.1.5.RELEASE.jar!/:2.1.5.RELEASE]
proxy_1           | 	at io.eventuate.tram.messaging.proxy.service.ProxyServiceMain.main(ProxyServiceMain.java:11) [classes!/:na]
proxy_1           | 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_171]
proxy_1           | 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_171]
proxy_1           | 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_171]
proxy_1           | 	at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_171]
proxy_1           | 	at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:48) [eventuate-tram-messaging-proxy-service-0.1.0-SNAPSHOT.jar:na]
proxy_1           | 	at org.springframework.boot.loader.Launcher.launch(Launcher.java:87) [eventuate-tram-messaging-proxy-service-0.1.0-SNAPSHOT.jar:na]
proxy_1           | 	at org.springframework.boot.loader.Launcher.launch(Launcher.java:50) [eventuate-tram-messaging-proxy-service-0.1.0-SNAPSHOT.jar:na]
proxy_1           | 	at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:51) [eventuate-tram-messaging-proxy-service-0.1.0-SNAPSHOT.jar:na]
proxy_1           | Caused by: java.lang.RuntimeException: org.apache.zookeeper.KeeperException$UnimplementedException: KeeperErrorCode = Unimplemented for /eventuate
proxy_1           | 	at io.eventuate.tram.messaging.proxy.service.SubscriptionPersistenceService.loadSubscriptionInfos(SubscriptionPersistenceService.java:65) ~[classes!/:na]
proxy_1           | 	at io.eventuate.tram.messaging.proxy.service.SubscriptionLoader.loadPersistentSubscriptions(SubscriptionLoader.java:28) ~[classes!/:na]
proxy_1           | 	at io.eventuate.tram.messaging.proxy.service.SubscriptionLoader.load(SubscriptionLoader.java:22) ~[classes!/:na]
proxy_1           | 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_171]
proxy_1           | 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_171]
proxy_1           | 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_171]
proxy_1           | 	at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_171]
proxy_1           | 	at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleElement.invoke(InitDestroyAnnotationBeanPostProcessor.java:363) ~[spring-beans-5.1.7.RELEASE.jar!/:5.1.7.RELEASE]
proxy_1           | 	at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleMetadata.invokeInitMethods(InitDestroyAnnotationBeanPostProcessor.java:307) ~[spring-beans-5.1.7.RELEASE.jar!/:5.1.7.RELEASE]
proxy_1           | 	at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor.postProcessBeforeInitialization(InitDestroyAnnotationBeanPostProcessor.java:136) ~[spring-beans-5.1.7.RELEASE.jar!/:5.1.7.RELEASE]
proxy_1           | 	... 26 common frames omitted
proxy_1           | Caused by: org.apache.zookeeper.KeeperException$UnimplementedException: KeeperErrorCode = Unimplemented for /eventuate
proxy_1           | 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:106) ~[zookeeper-3.5.4-beta.jar!/:3.5.4-beta-7f51e5b68cf2f80176ff944a9ebd2abbc65e7327]
proxy_1           | 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:54) ~[zookeeper-3.5.4-beta.jar!/:3.5.4-beta-7f51e5b68cf2f80176ff944a9ebd2abbc65e7327]
proxy_1           | 	at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:1450) ~[zookeeper-3.5.4-beta.jar!/:3.5.4-beta-7f51e5b68cf2f80176ff944a9ebd2abbc65e7327]
proxy_1           | 	at org.apache.curator.utils.ZKPaths.mkdirs(ZKPaths.java:308) ~[curator-client-4.2.0.jar!/:na]
proxy_1           | 	at org.apache.curator.framework.imps.ExistsBuilderImpl$2.call(ExistsBuilderImpl.java:230) ~[curator-framework-4.2.0.jar!/:4.2.0]
proxy_1           | 	at org.apache.curator.framework.imps.ExistsBuilderImpl$2.call(ExistsBuilderImpl.java:224) ~[curator-framework-4.2.0.jar!/:4.2.0]
proxy_1           | 	at org.apache.curator.connection.StandardConnectionHandlingPolicy.callWithRetry(StandardConnectionHandlingPolicy.java:64) ~[curator-client-4.2.0.jar!/:na]
proxy_1           | 	at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:100) ~[curator-client-4.2.0.jar!/:na]
proxy_1           | 	at org.apache.curator.framework.imps.ExistsBuilderImpl.pathInForeground(ExistsBuilderImpl.java:221) ~[curator-framework-4.2.0.jar!/:4.2.0]
proxy_1           | 	at org.apache.curator.framework.imps.ExistsBuilderImpl.forPath(ExistsBuilderImpl.java:206) ~[curator-framework-4.2.0.jar!/:4.2.0]
proxy_1           | 	at org.apache.curator.framework.imps.ExistsBuilderImpl.forPath(ExistsBuilderImpl.java:35) ~[curator-framework-4.2.0.jar!/:4.2.0]
proxy_1           | 	at org.apache.curator.framework.imps.CuratorFrameworkImpl.createContainers(CuratorFrameworkImpl.java:265) ~[curator-framework-4.2.0.jar!/:4.2.0]
proxy_1           | 	at io.eventuate.tram.messaging.proxy.service.SubscriptionPersistenceService.loadSubscriptionInfos(SubscriptionPersistenceService.java:48) ~[classes!/:na]
proxy_1           | 	... 35 common frames omitted

It seems it is because project uses zookeeper ttl feature (https://zookeeper.apache.org/doc/r3.5.4-beta/zookeeperProgrammers.html#TTL+Nodes) it should be enabled by specifying extendedTypesEnabled=true.

It should be done by using env variable ZOOKEEPER_EXTENDED_TYPES_ENABLED according https://docs.confluent.io/current/installation/docker/config-reference.html but it seems it does not work

cer · 2020-07-07T15:08:56Z

Which version of ZK implements extendedTypesEnabled=true.
Which version of ZK is cp-zookeeper?

dartartem · 2020-07-07T15:13:33Z

I tried to use 5.2.4 and latest (5.5.1).

TTL Nodes
Added in 3.5.3

cer · 2020-07-07T15:17:52Z

What's 5.2.4 and latest (5.5.1). - are those the versions of confluentinc/cp-zookeeper?

If so, then what is the version of ZK in confluentinc/cp-zookeeper:5.2.4?

dartartem · 2020-07-07T15:51:21Z

What's 5.2.4 and latest (5.5.1). - are those the versions of confluentinc/cp-zookeeper?

yes

If so, then what is the version of ZK in confluentinc/cp-zookeeper:5.2.4?

I found version of ZK in confluentinc/cp-zookeeper:5.5.1 by searching in running container.
it is 3.5.8
It seems I also found why property ZOOKEEPER_EXTENDED_TYPES_ENABLED does not work.
Container has file zookeeper.properties.template it seems it is used to explicitly define mappings,
content:


dataDir=/var/lib/zookeeper/data
dataLogDir=/var/lib/zookeeper/log

{# optional properties #}
{% set other_props = {
  'ZOOKEEPER_CLIENT_PORT' : 'clientPort',
  'ZOOKEEPER_TICK_TIME': 'tickTime',
  'ZOOKEEPER_GLOBAL_OUTSTANDING_LIMIT' : 'globalOutstandingLimit',
  'ZOOKEEPER_PRE_ALLOC_SIZE': 'preAllocSize',
  'ZOOKEEPER_SNAP_COUNT': 'snapCount',
  'ZOOKEEPER_TRACE_FILE': 'traceFile',
  'ZOOKEEPER_MAX_CLIENT_CNXNS' : 'maxClientCnxns',
  'ZOOKEEPER_CLIENT_PORT_ADDRESS' : 'clientPortAddress',
  'ZOOKEEPER_MIN_SESSION_TIMEOUT' : 'minSessionTimeout',
  'ZOOKEEPER_MAX_SESSION_TIMEOUT' : 'maxSessionTimeout',
  'ZOOKEEPER_FSYNC_WARNING_THRESHOLDMS' : 'fsync.warningthresholdms',
  'ZOOKEEPER_AUTOPURGE_SNAP_RETAIN_COUNT' : 'autopurge.snapRetainCount',
  'ZOOKEEPER_AUTOPURGE_PURGE_INTERVAL': 'autopurge.purgeInterval',
  'ZOOKEEPER_SYNC_ENABLED': 'syncEnabled',
  'ZOOKEEPER_ELECTION_ALG' : 'electionAlg',
  'ZOOKEEPER_INIT_LIMIT': 'initLimit',
  'ZOOKEEPER_LEADER_SERVES': 'leaderServes',
  'ZOOKEEPER_SYNC_LIMIT' : 'syncLimit',
  'ZOOKEEPER_CNX_TIMEOUT': 'cnxTimeout',
  'ZOOKEEPER_FORCE_SYNC': 'forceSync',
  'ZOOKEEPER_JUTE_MAX_BUFFER': 'jute.maxbuffer',
  'ZOOKEEPER_SKIP_ACL': 'skipACL',
  'ZOOKEEPER_QUORUM_LISTEN_ON_ALL_IPS': 'quorumListenOnAllIPs',
  'ZOOKEEPER_CLIENT_CNXN_SOCKET' : 'clientCnxnSocket',
  'ZOOKEEPER_SECURE_CLIENT_PORT' : 'secureClientPort',
  'ZOOKEEPER_SERVER_CNXN_SOCKET' : 'serverCnxnSocket',
  'ZOOKEEPER_X509_AUTHENTICATION_PROVIDER_SUPER_USER' : 'X509AuthenticationProvider.superUser',
  'ZOOKEEPER_SSL_AUTH_PROVIDER' : 'ssl.authProvider',
  'ZOOKEEPER_SSL_CLIENT_AUTH' : 'ssl.clientAuth',
  'ZOOKEEPER_SSL_KEYSTORE_LOCATION' : 'ssl.keyStore.location',
  'ZOOKEEPER_SSL_KEYSTORE_PASSWORD' : 'ssl.keyStore.password',
  'ZOOKEEPER_SSL_KEYSTORE_TYPE' : 'ssl.keyStore.type',
  'ZOOKEEPER_SSL_TRUSTSTORE_LOCATION' : 'ssl.trustStore.location',
  'ZOOKEEPER_SSL_TRUSTSTORE_PASSWORD' : 'ssl.trustStore.password',
  'ZOOKEEPER_SSL_TRUSTSTORE_TYPE' : 'ssl.trustStore.type',
  'ZOOKEEPER_SSL_ENABLED_PROTOCOLS' : 'ssl.enabledProtocols',
  'ZOOKEEPER_SSL_CONTEXT_SUPPLIER_CLASS' : 'ssl.context.supplier.class',
  'ZOOKEEPER_SSL_CIPHER_SUITES' : 'ssl.ciphersuites',
  'ZOOKEEPER_SSL_HOSTNAME_VERIFICATION' : 'ssl.hostnameVerification',
  'ZOOKEEPER_SSL_CRL' : 'ssl.crl',
  'ZOOKEEPER_SSL_OCPS' : 'ssl.ocsp',
  'ZOOKEEPER_SSL_HANDSHAKE_DETECTION_TIMEOUT_MILLIS' : 'ssl.handshakeDetectionTimeoutMillis',
  'ZOOKEEPER_SSL_QUORUM' : 'sslQuorum',
  'ZOOKEEPER_SSL_QUORUM_CLIENT_AUTH' : 'ssl.quorum.clientAuth',
  'ZOOKEEPER_SSL_QUORUM_KEYSTORE_LOCATION' : 'ssl.quorum.keyStore.location',
  'ZOOKEEPER_SSL_QUORUM_KEYSTORE_PASSWORD' : 'ssl.quorum.keyStore.password',
  'ZOOKEEPER_SSL_QUORUM_KEYSTORE_TYPE' : 'ssl.quorum.keyStore.type',
  'ZOOKEEPER_SSL_QUORUM_TRUSTSTORE_LOCATION' : 'ssl.quorum.trustStore.location',
  'ZOOKEEPER_SSL_QUORUM_TRUSTSTORE_PASSWORD' : 'ssl.quorum.trustStore.password',
  'ZOOKEEPER_SSL_QUORUM_TRUSTSTORE_TYPE' : 'ssl.quorum.trustStore.type',
  'ZOOKEEPER_SSL_QUORUM_ENABLED_PROTOCOLS' : 'ssl.quorum.enabledProtocols',
  'ZOOKEEPER_SSL_QUORUM_CIPHER_SUITES' : 'ssl.quorum.ciphersuites',
  'ZOOKEEPER_SSL_QUORUM_CONTEXT_SUPPLIER_CLASS' : 'ssl.quorum.context.supplier.class',
  'ZOOKEEPER_SSL_QUORUM_HOSTNAME_VERIFICATION' : 'ssl.quorum.hostnameVerification',
  'ZOOKEEPER_SSL_QUORUM_CRL' : 'ssl.quorum.crl',
  'ZOOKEEPER_SSL_QUORUM_OCPS' : 'ssl.quorum.ocsp',
  'ZOOKEEPER_SSL_QUORUM_HANDSHAKE_DETECTION_TIMEOUT_MILLIS' : 'ssl.quorum.handshakeDetectionTimeoutMillis',
  'ZOOKEEPER_SERVER_CNXN_FACTORY' : 'serverCnxnFactory',
  'ZOOKEEPER_AUTH_PROVIDER_X509' : 'authProvider.x509',
  'ZOOKEEPER_AUTH_PROVIDER_SASL' : 'authProvider.sasl',
  'ZOOKEEPER_CLIENT_PORT_UNIFICATION' : 'client.portUnification',
  'ZOOKEEPER_ADMIN_ENABLE_SERVER' : 'admin.enableServer',
  'ZOOKEEPER_ADMIN_SERVER_ADDRESS' : 'admin.serverAddress',
  'ZOOKEEPER_ADMIN_SERVER_PORT' : 'admin.serverPort',
  'ZOOKEEPER_ADMIN_IDLE_TIMEOUT' : 'admin.idleTimeout',
  'ZOOKEEPER_ADMIN_COMMAND_URL' : 'admin.commandURL'
 } -%}

{% for k, property in other_props.items() -%}
{% if env.get(k) != None -%}
{{property}}={{env[k]}}
{% endif -%}
{% endfor -%}

{% if env['ZOOKEEPER_SERVERS'] %}
{% set servers = env['ZOOKEEPER_SERVERS'].split(';') %}
{% for server in servers %}
server.{{ loop.index }}={{server}}
{% endfor %}
{% endif %}

{% if env['ZOOKEEPER_GROUPS'] %}
{% set groups = env['ZOOKEEPER_GROUPS'].split(';') %}
{% for group in groups %}
group.{{ loop.index }}={{group}}
{% endfor %}
{% endif %}

{% if env['ZOOKEEPER_WEIGHTS'] %}
{% set weights = env['ZOOKEEPER_WEIGHTS'].split(';') %}
{% for weight in weights %}
weight.{{ loop.index }}={{weight}}
{% endfor %}
{% endif %}

there is no such property

cer · 2020-07-07T15:55:39Z

You should submit an issue for the confluent-zookeeper project.

Just build a custom ZK image for this project.

dartartem · 2020-07-07T15:58:00Z

Chris,

You should submit an issue for the confluent-zookeeper project.

ok, will do

Just build a custom ZK image for this project.

I used custom zookeeper image from the eventuate local, is it ok?

#4: Updated libraries. Supported BUILD-SNAPSHOTS.

dartartem added a commit to dartartem/eventuate-tram-messaging-http that referenced this issue Jul 7, 2020

eventuate-tram#4: Updated libraries.

45162c8

cer added a commit that referenced this issue Jul 21, 2020

Merge pull request #5 from dartartem/master

21f6d23

#4: Updated libraries. Supported BUILD-SNAPSHOTS.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update libraries #4

Update libraries #4

dartartem commented Jul 7, 2020

dartartem commented Jul 7, 2020 •

edited

Loading

cer commented Jul 7, 2020

dartartem commented Jul 7, 2020

cer commented Jul 7, 2020

dartartem commented Jul 7, 2020

cer commented Jul 7, 2020

dartartem commented Jul 7, 2020

Update libraries #4

Update libraries #4

Comments

dartartem commented Jul 7, 2020

dartartem commented Jul 7, 2020 • edited Loading

cer commented Jul 7, 2020

dartartem commented Jul 7, 2020

cer commented Jul 7, 2020

dartartem commented Jul 7, 2020

cer commented Jul 7, 2020

dartartem commented Jul 7, 2020

dartartem commented Jul 7, 2020 •

edited

Loading