fix(docker):Update couchdb3.1 ubuntu20.04 liferay7.3.4 postgresql12 #1323

KoukiHama · 2021-07-26T15:13:17Z

Signed-off-by: Kouki Hama [email protected]

Please provide a summary of your changes here.

Which issue is this pull request belonging to and how is it solving it? (Refer to issue here)

Did you add or update any new dependencies that are required for your change?

Issue: #1322 #1331

Suggest Reviewer

You can suggest reviewers here with an @mention.

How To Test?

How should these changes be tested by the reviewer?
Have you implemented any additional tests?

Build docker image by using this commad

docker build -t sw360 .

Run docker

docker run -it -p 8090:8080 -p 5985:5984 -p 5435:5432 sw360

and set Liferay7.3.4 GUI

Checklist

Must:

All related issues are referenced in commit messages and in PR

KoukiHama · 2021-07-27T00:57:38Z

I have finished to update almost all, but I was not able to solve these Error which occurs in starting sw360
Do I need to set oauthclient setting like this commit in sw360vagrant?
sw360/sw360vagrant@2af52cb

2021-07-27 00:43:22 INFO Sw360CustomHeaderAuthenticationFilter:105 - AuthenticationFilter is NOT active!
2021-07-27 00:43:22 WARN AnnotationConfigEmbeddedWebApplicationContext:551 - Exception encountered during context initialization - cancelling refresh attempt: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'OAuthClientRepository' defined in file [/app/liferay-ce-portal-7.3.4-ga5/tomcat-9.0.33/webapps/authorization/WEB-INF/classes/org/eclipse/sw360/rest/authserver/client/persistence/OAuthClientRepository.class]: Bean instantiation via constructor failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.eclipse.sw360.rest.authserver.client.persistence.OAuthClientRepository]: Constructor threw exception; nested exception is org.ektorp.DbAccessException: 401:Unauthorized
URI: /sw360oauthclients/
Response Body:
{
"error" : "unauthorized",
"reason" : "You are not a server admin."
}
2021-07-27 00:43:22 ERROR SpringApplication:771 - Application startup failed
org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'OAuthClientRepository' defined in file [/app/liferay-ce-portal-7.3.4-ga5/tomcat-9.0.33/webapps/authorization/WEB-INF/classes/org/eclipse/sw360/rest/authserver/client/persistence/OAuthClientRepository.class]: Bean instantiation via constructor failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.eclipse.sw360.rest.authserver.client.persistence.OAuthClientRepository]: Constructor threw exception; nested exception is org.ektorp.DbAccessException: 401:Unauthorized
URI: /sw360oauthclients/
Response Body:
{
"error" : "unauthorized",
"reason" : "You are not a server admin."
}

JaideepPalit · 2021-07-27T09:57:57Z

Hi @KoukiHama . I guess you need to place application.yml in /etc/sw360/authorization/, with correct username and pwd

KoukiHama · 2021-07-27T12:02:29Z

Thank you @JaideepPalit .

I was able to remove ouath client error by adding application.yml in /etc/sw360/authorization/ !

But, other problem are happened ...

Component and Project pages are not visible. (I have already set user groups. )

In the container CouchDB3.1 works fine.

curl http://admin:[email protected]:5984
{"couchdb":"Welcome","version":"3.1.1","git_sha":"ce596c65d","uuid":"d3f19f9e1babc3dc92b3bd58d9f00ca4","features":["access-ready","partitioned","pluggable-storage-engines","reshard","scheduler"],"vendor":{"name":"The Apache Software Foundation"}}

Tomcat.log implies that some Credentials are incorrect. But I set user and password coucdb.properties in /etc/sw360
Should I set couchdb's user and password in other files aside from coucdb.propeties and application.yml?

couchdb.url = http://localhost:5984
couchdb.user = admin
couchdb.password = password
couchdb.database = sw360db
couchdb.usersdb = sw360users
couchdb.attachments = sw360attachments
lucenesearch.limit = 1000

Caused by: com.cloudant.client.org.lightcouch.CouchDbException: 401 Credentials are incorrect for server http://localhost:5984/_session.
at com.cloudant.client.org.lightcouch.CouchDbClient.execute(CouchDbClient.java:567) ~[cloudant-client-2.19.1.jar:?]
at com.cloudant.client.org.lightcouch.CouchDbClient.executeToInputStream(CouchDbClient.java:648) ~[cloudant-client-2.19.1.jar:?]
at com.cloudant.client.org.lightcouch.CouchDbClient.get(CouchDbClient.java:389) ~[cloudant-client-2.19.1.jar:?]
at com.cloudant.client.org.lightcouch.CouchDbClient.getAllDbs(CouchDbClient.java:243) ~[cloudant-client-2.19.1.jar:?]
at com.cloudant.client.api.CloudantClient.getAllDbs(CloudantClient.java:292) ~[cloudant-client-2.19.1.jar:?]
at org.eclipse.sw360.datahandler.cloudantclient.DatabaseInstanceCloudant.checkIfDbExists(DatabaseInstanceCloudant.java:37) ~[datahandler-13.4.0-SNAPSHOT.jar:?]
at org.eclipse.sw360.datahandler.cloudantclient.DatabaseInstanceCloudant.createDB(DatabaseInstanceCloudant.java:33) ~[datahandler-13.4.0-SNAPSHOT.jar:?]
at org.eclipse.sw360.datahandler.cloudantclient.DatabaseConnectorCloudant.(DatabaseConnectorCloudant.java:64) ~[datahandler-13.4.0-SNAPSHOT.jar:?]
at org.eclipse.sw360.fossology.config.FossologyConfig.configContainerRepository(FossologyConfig.java:41) ~[src-fossology-13.4.0-SNAPSHOT.jar:?]
at org.eclipse.sw360.fossology.config.FossologyConfig$$EnhancerBySpringCGLIB$$e2210a9a.CGLIB$configContainerRepository$3() ~[src-fossology-13.4.0-SNAPSHOT.jar:?]
at org.eclipse.sw360.fossology.config.FossologyConfig$$EnhancerBySpringCGLIB$$e2210a9a$$FastClassBySpringCGLIB$$c6b078a4.invoke() ~[src-fossology-13.4.0-SNAPSHOT.jar:?]
at org.springframework.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:244) ~[spring-core-5.2.7.RELEASE.jar:5.2.7.RELEASE]
at org.springframework.context.annotation.ConfigurationClassEnhancer$BeanMethodInterceptor.intercept(ConfigurationClassEnhancer.java:331) ~[spring-context-5.2.7.RELEASE.jar:5.2.7.RELEASE]
at org.eclipse.sw360.fossology.config.FossologyConfig$$EnhancerBySpringCGLIB$$e2210a9a.configContainerRepository() ~[src-fossology-13.4.0-SNAPSHOT.jar:?]
at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?]
at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:?]
at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?]
at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:154) ~[spring-beans-5.2.7.RELEASE.jar:5.2.7.RELEASE]
at org.springframework.beans.factory.support.ConstructorResolver.instantiate(ConstructorResolver.java:650) ~[spring-beans-5.2.7.RELEASE.jar:5.2.7.RELEASE]
at org.springframework.beans.factory.support.ConstructorResolver.instantiateUsingFactoryMethod(ConstructorResolver.java:483) ~[spring-beans-5.2.7.RELEASE.jar:5.2.7.RELEASE]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.instantiateUsingFactoryMethod(AbstractAutowireCapableBeanFactory.java:1338) ~[spring-beans-5.2.7.RELEASE.jar:5.2.7.RELEASE]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1177) ~[spring-bea:

JaideepPalit · 2021-07-27T15:43:23Z

Not sure , Could you try removing extra space from username and pwd
Since it is saying - 401 Credentials are incorrect for server

KoukiHama · 2021-07-27T18:38:57Z

@JaideepPalit Thank you! The issues was solved by your advice!

mcjaeger

I tested it with current master (import spdx BOM, add some SPDX files to created releases and generated OSS license info) -> all good.

Search does not work for me however, but ...

mcjaeger · 2021-07-27T19:22:24Z

scripts/docker-config/install_scripts/install_init_postgres_script.sh

+  curl -L https://www.postgresql.org/media/keys/ACCC4CF8.asc | apt-key add -
+  echo "deb http://apt.postgresql.org/pub/repos/apt focal-pgdg main" | tee -a /etc/apt/sources.list
+  apt-get update
+  DEBIAN_FRONTEND=noninteractive apt-get install postgresql-11 -y --no-install-recommends


for me, it is OK like installing from prostgresql public repos. Maybe postgresql-12 would also work (https://packages.ubuntu.com/focal/postgresql)

OK I have changed postgressql version from 11 to 12. And SW360 works well with postgresql12.

mcjaeger · 2021-07-27T20:04:01Z

BTW. about the search, it is

2021-07-27 20:03:13,703 WARN [DefaultHttpClient] Authentication error: Unable to respond to any of these challenges: {}
2021-07-27 20:03:13 ERROR LuceneAwareDatabaseConnector:163 - Retried with stale parameter.Getting error with reponse code=500

KoukiHama · 2021-07-28T06:38:11Z

For searching function and using couchdb-lucene, I planed to set [httpd_global_handlers] like CouchDB version2.x

[httpd_global_handlers]
_fti = {couch_httpd_proxy, handle_proxy_req, <<"http://127.0.0.1:8080/couchdb-lucene">>}

But I don't know how to set parameter of [httpd_global_handlers], which does not exists in default.ini and local.ini of CouchDB version3.1
https://github.com/apache/couchdb/blob/main/rel/overlay/etc/default.ini
https://github.com/apache/couchdb/blob/main/rel/overlay/etc/local.ini

P.S. [httpd_global_handlers] was removed.
apache/couchdb#1602

Signed-off-by: Kouki Hama <[email protected]>

KoukiHama · 2021-07-28T13:59:49Z

For GitHub action, I updated and cleaned up commit logs.

JaideepPalit · 2021-07-28T14:14:35Z

For search, I guess user name , pwd needs to be set in couchdb-lucene.ini with
https://github.com/eclipse/sw360/blob/c7b1335d4561c1736285d37baa61277c6420ae02/scripts/docker-config/install_scripts/build_couchdb_lucene.sh#L19

# Allow leading wildcard?
allowLeadingWildcard=true

# couchdb server mappings

[local]
url = http://username:pwd@localhost:5984/

Signed-off-by: Kouki Hama <[email protected]>

KoukiHama · 2021-07-28T16:21:47Z

@JaideepPalit Your guess is right!

By setting User and Password in couchdb-lucene.ini
Searching works fine!

I update code for changing couchdb-lucene.ini.
af44344

https://github.com/eclipse/sw360/blob/af443442a7c65e88a30e9df82fe89436f5a2c525/scripts/docker-config/install_scripts/build_couchdb_lucene.sh#L20

New one (with http://admin:password@localhost:5984/ )
Being able to search!!

Ref: Old one (without http://admin:password@localhost:5984/)
Not being able to search, return empty results.

mcjaeger · 2021-08-18T12:28:15Z

search works for me as well:

installation (build and run and LR setup) does not show issues

KoukiHama added WIP work in progress do not merge - нет! labels Jul 26, 2021

KoukiHama force-pushed the update_docker branch 4 times, most recently from 5e82af2 to de544f4 Compare July 26, 2021 22:58

KoukiHama added question Further information is requested help wanted Extra attention is needed and removed do not merge - нет! labels Jul 27, 2021

KoukiHama added needs code review needs general test This is general testing, meaning that there is no org specific issue to check for and removed WIP work in progress help wanted Extra attention is needed question Further information is requested labels Jul 27, 2021

mcjaeger reviewed Jul 27, 2021

View reviewed changes

KoukiHama added WIP work in progress and removed needs code review needs general test This is general testing, meaning that there is no org specific issue to check for labels Jul 28, 2021

KoukiHama changed the title ~~fix(docker):Update couchdb3.1 ubuntu20.04 liferay7.3.4 postgresql11~~ fix(docker):Update couchdb3.1 ubuntu20.04 liferay7.3.4 postgresql12 Jul 28, 2021

KoukiHama added needs code review needs general test This is general testing, meaning that there is no org specific issue to check for and removed WIP work in progress labels Jul 28, 2021

KoukiHama force-pushed the update_docker branch from dcb9f5a to 2d4997a Compare July 28, 2021 13:14

fix(docker):Update couchdb3.1 ubuntu20.04 liferay7.3.4 postgresql12

318d092

Signed-off-by: Kouki Hama <[email protected]>

KoukiHama force-pushed the update_docker branch from 9d4ae5a to 318d092 Compare July 28, 2021 13:57

KoukiHama mentioned this pull request Jul 28, 2021

Update version couchdb3.1 liferay7.3.4 ubuntu20.04 and postgres12 for docker build #1322

Closed

fix(script): add password and user in couchdb-lucene.ini

af44344

Signed-off-by: Kouki Hama <[email protected]>

mcjaeger merged commit faf273c into eclipse-sw360:master Aug 18, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix(docker):Update couchdb3.1 ubuntu20.04 liferay7.3.4 postgresql12 #1323

fix(docker):Update couchdb3.1 ubuntu20.04 liferay7.3.4 postgresql12 #1323

KoukiHama commented Jul 26, 2021 •

edited

Loading

KoukiHama commented Jul 27, 2021 •

edited

Loading

JaideepPalit commented Jul 27, 2021

KoukiHama commented Jul 27, 2021

JaideepPalit commented Jul 27, 2021

KoukiHama commented Jul 27, 2021 •

edited

Loading

mcjaeger left a comment

mcjaeger Jul 27, 2021

KoukiHama Jul 28, 2021 •

edited

Loading

mcjaeger commented Jul 27, 2021

KoukiHama commented Jul 28, 2021 •

edited

Loading

KoukiHama commented Jul 28, 2021 •

edited

Loading

JaideepPalit commented Jul 28, 2021

KoukiHama commented Jul 28, 2021 •

edited

Loading

mcjaeger commented Aug 18, 2021

fix(docker):Update couchdb3.1 ubuntu20.04 liferay7.3.4 postgresql12 #1323

fix(docker):Update couchdb3.1 ubuntu20.04 liferay7.3.4 postgresql12 #1323

Conversation

KoukiHama commented Jul 26, 2021 • edited Loading

Suggest Reviewer

How To Test?

Checklist

KoukiHama commented Jul 27, 2021 • edited Loading

JaideepPalit commented Jul 27, 2021

KoukiHama commented Jul 27, 2021

JaideepPalit commented Jul 27, 2021

KoukiHama commented Jul 27, 2021 • edited Loading

mcjaeger left a comment

Choose a reason for hiding this comment

mcjaeger Jul 27, 2021

Choose a reason for hiding this comment

KoukiHama Jul 28, 2021 • edited Loading

Choose a reason for hiding this comment

mcjaeger commented Jul 27, 2021

KoukiHama commented Jul 28, 2021 • edited Loading

KoukiHama commented Jul 28, 2021 • edited Loading

JaideepPalit commented Jul 28, 2021

KoukiHama commented Jul 28, 2021 • edited Loading

mcjaeger commented Aug 18, 2021

KoukiHama commented Jul 26, 2021 •

edited

Loading

KoukiHama commented Jul 27, 2021 •

edited

Loading

KoukiHama commented Jul 27, 2021 •

edited

Loading

KoukiHama Jul 28, 2021 •

edited

Loading

KoukiHama commented Jul 28, 2021 •

edited

Loading

KoukiHama commented Jul 28, 2021 •

edited

Loading

KoukiHama commented Jul 28, 2021 •

edited

Loading