8932 container base image #8933

poikilotherm · 2022-08-24T13:06:11Z

What this PR does / why we need it:

This PR adds a Maven submodule to build a base container image containing a tuned Payara app server and all things necessary to deploy Dataverse WAR in it.

Which issue(s) this PR closes:

Closes #8932

Special notes for your reviewer:

It might be debateable if this image should also do the heavy lifting for JHove config.
This pull request will be the first of more to come. It is necessary to talk about the review/QA process here, as discussed in-person. Looks like Spike: Backstop the local Dataverse QA process #8910 is related.
It does not yet contain a Github Actions to build and push the image to a registry.

TODOs:

Suggestions on how to test this:

Simply install Docker/... and follow the provided instructions within the new container guide.

Does this PR introduce a user interface change? If mockups are available, please link/include them here:
Nope.

Is there a release notes update needed for this change?:
I don't know yet. Maybe wait for having the Dataverse container ready and not just the base image.

Additional documentation:
Provided.

coveralls · 2022-08-24T13:08:32Z

Coverage remained the same at 19.997% when pulling 5122515 on poikilotherm:8932-ct-base-image into be48cfd on IQSS:develop.

Kris-LIBIS · 2022-08-24T20:21:10Z

Since you create the payara user in the Dockerfile, I would make the user (and group) ID configurable - aka as ARG. Setting this to the ID of the deployment user allows easy access to data on mounted volumes for backups and troubleshooting.

BTW, if you are interested to have a look at our generic docker images, the repository is public: https://github.com/libis/rdm-build.git. There I start from the Payara docker image and we change the user and group ID of the payara user if needed.

poikilotherm · 2022-08-25T21:44:28Z

Thanks @Kris-LIBIS for your feedback!

Maybe this base image is of help for you folks, too. I'll add the ARGS, sounds like a nice feature!

Browsing through the scripts I see you inherited from gdcc/dataverse-kubernetes so I'm just going to re-inherit some pretty nice ideas from you folks! 😄

carlsonp · 2022-09-01T01:21:49Z

I was able to successfully compile and run this image. I didn't try anything else at this point but it builds on my machine. Nice!

Will be replaced with a capability to make API endpoints for authentication providers read from MPCONFIG sources.

Payara 5 defaults to a "payara5" topmost dir, Payara 6 to "payara6". To avoid adding different directories in the assembly, cut the number from the directories name when unpacking. This does not prevent you from doing stupid things like not cleaning before switching the version leading to an unknown state of old and new libs, etc.

There was an ongoing discussion that the Docker Hub Image "openjdk" is not backed by any official supported project but complete goodwill of Oracle shipping their JRE/JDK. There is no "real" release of OpenJDK . There exist only real distributions like Oracle JDK, Eclipse Temurin, Azul JDK, AWS Corretto etc (see https://whichjdk.com). As for this reason the "openjdk" image has been deprecated, switching to Eclipse Temurin JRE here. See also: docker-library/openjdk#505

With the rise of Apple M1/M2 silicons, we need to provide ARM64 based images in addition to AMD64.

…as in Payara

Co-authored-by: Benjamin Peuch <[email protected]>

doc/sphinx-guides/source/container/index.rst

doc/sphinx-guides/source/container/base-image.rst

modules/container-base/README.md

doc/sphinx-guides/source/container/index.rst

pdurbin

@poikilotherm sorry to small chunk you, but this review is just for the docs. Overall, they look great!

doc/sphinx-guides/source/container/index.rst

modules/container-base/README.md

@atrisovic

Thanks @atrisovic @pdurbin

@pdurbin

As requested by review from @pdurbin, aligning image tag names.

mreekie · 2022-12-14T21:24:39Z

added to sprint Dec 15, 2022

poikilotherm · 2022-12-15T07:08:45Z

Todo inspired from @donsizemore #7530: make GC configurable

pdurbin

I'm approving this. All API tests are passing. I tweaked the docs a bit and ran mvn -Pct -f modules/container-base install as instructed in the new Container Guide.

It built the image, as expected:

$ docker images | head -2
REPOSITORY              TAG               IMAGE ID       CREATED          SIZE
gdcc/base               unstable          0a03fd50e8bf   11 minutes ago   588MB

@kcondon honestly I wouldn't worry too much about testing the mvn command above or any of the many configuration options that are documented in this PR (I didn't). This PR just lays the groundwork for a future "application image" (with Dataverse in it) to go on top of this base image. There are a few minor tweaks to our pom.xml but they shouldn't break anything. Again, all API tests are passing.

Thanks, @poikilotherm!

poikilotherm · 2022-12-16T19:52:18Z

I don't think I'll add the ZGC option for now, as it is strongly discouraged to use the Java 11 preview version of it.

Please go ahead and merge if you like it. 🙂

poikilotherm added Component: Containers Anything related to cloudy Dataverse, shipped in containers. Working Group: SWC labels Aug 24, 2022

poikilotherm marked this pull request as draft August 24, 2022 13:06

poikilotherm changed the title ~~8932 ct base image~~ 8932 container base image Aug 24, 2022

pdurbin self-assigned this Aug 26, 2022

poikilotherm force-pushed the 8932-ct-base-image branch 4 times, most recently from b449150 to ab5c03c Compare August 31, 2022 00:07

poikilotherm force-pushed the 8932-ct-base-image branch 3 times, most recently from 6d8571b to 4164116 Compare September 19, 2022 23:00

poikilotherm added 12 commits September 20, 2022 09:32

feat(ct-base): add new base container image in submodule

139f0f7

feat(ct-base): remove the esh tool

2319a47

Will be replaced with a capability to make API endpoints for authentication providers read from MPCONFIG sources.

chore(deps): update container plugin and payara version for containers

f0202cb

chore(deps): update Docker Maven Plugin to the latest release

76ea508

feat(ct-base): enable multiarch image build via docker buildx

f62dee2

With the rise of Apple M1/M2 silicons, we need to provide ARM64 based images in addition to AMD64.

chore(ct-base): add maintainer details to POM

72935d4

docs(ct-base): update OCI tag labels

17d8b53

feat(ct-base): add debug/develop mode script

0a9947b

deps(ct-base): update to jattach v2.1

2e812dc

chore(ct-base): add JMX to exposed ports and make it default enabled …

7e836c7

…as in Payara

docs(ct-base): apply some language tweaks to docs pages

4a79dcb

Co-authored-by: Benjamin Peuch <[email protected]>

johannes-darms reviewed Nov 9, 2022

View reviewed changes

doc/sphinx-guides/source/container/index.rst Show resolved Hide resolved

doc/sphinx-guides/source/container/base-image.rst Show resolved Hide resolved

atrisovic reviewed Nov 9, 2022

View reviewed changes

modules/container-base/README.md Outdated Show resolved Hide resolved

poikilotherm commented Nov 9, 2022

View reviewed changes

doc/sphinx-guides/source/container/index.rst Outdated Show resolved Hide resolved

pdurbin reviewed Nov 10, 2022

View reviewed changes

poikilotherm force-pushed the 8932-ct-base-image branch from 78e465c to e964d86 Compare November 10, 2022 15:41

docs(ct-base): incorporate review suggestions IQSS#8932

f1a64a8

Thanks @atrisovic @pdurbin

poikilotherm force-pushed the 8932-ct-base-image branch from e964d86 to f1a64a8 Compare November 10, 2022 15:43

style(ct-base): rephrase container image tags IQSS#8932

426f746

As requested by review from @pdurbin, aligning image tag names.

poikilotherm mentioned this pull request Nov 11, 2022

feat(upload): make upload file storage path configurable #6656 #8983

Merged

pdurbin added the Size: 3 A percentage of a sprint. 2.1 hours. label Dec 2, 2022

Merge branch 'develop' into 8932-ct-base-image

c63feb7

pdurbin removed their assignment Dec 13, 2022

Merge branch 'develop' into 8932-ct-base-image

aa7b8ee

pdurbin self-assigned this Dec 16, 2022

tweak docs IQSS#8932

5122515

pdurbin approved these changes Dec 16, 2022

View reviewed changes

pdurbin removed their assignment Dec 16, 2022

kcondon self-assigned this Dec 16, 2022

kcondon merged commit 71563c8 into IQSS:develop Dec 16, 2022

pdurbin added this to the 5.13 milestone Dec 21, 2022

GPortas mentioned this pull request Mar 3, 2023

Dataverse container-based development environment #9423

Closed

poikilotherm deleted the 8932-ct-base-image branch March 17, 2023 12:15

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

8932 container base image #8933

8932 container base image #8933

poikilotherm commented Aug 24, 2022 •

edited

Loading

coveralls commented Aug 24, 2022 •

edited

Loading

Kris-LIBIS commented Aug 24, 2022

poikilotherm commented Aug 25, 2022 •

edited

Loading

carlsonp commented Sep 1, 2022

pdurbin left a comment

mreekie commented Dec 14, 2022

poikilotherm commented Dec 15, 2022

pdurbin left a comment

poikilotherm commented Dec 16, 2022 •

edited

Loading

8932 container base image #8933

8932 container base image #8933

Conversation

poikilotherm commented Aug 24, 2022 • edited Loading

coveralls commented Aug 24, 2022 • edited Loading

Kris-LIBIS commented Aug 24, 2022

poikilotherm commented Aug 25, 2022 • edited Loading

carlsonp commented Sep 1, 2022

pdurbin left a comment

Choose a reason for hiding this comment

mreekie commented Dec 14, 2022

poikilotherm commented Dec 15, 2022

pdurbin left a comment

Choose a reason for hiding this comment

poikilotherm commented Dec 16, 2022 • edited Loading

poikilotherm commented Aug 24, 2022 •

edited

Loading

coveralls commented Aug 24, 2022 •

edited

Loading

poikilotherm commented Aug 25, 2022 •

edited

Loading

poikilotherm commented Dec 16, 2022 •

edited

Loading