GitHub actions

[blootsvoets]

Enabled GitHub actions. It requires repository secrets

DOCKER_HUB_USERNAME
DOCKER_HUB_ACCESS_TOKEN

to be set.

Changes to the POM file:

• The repositories have been slightly reordered to allow faster dependency fetching when there is no cache.
• The miredot plugin is disabled inside docker since the result is not used there
• The git-commit-id-plugin is disabled and instead is provided as part of the GitHub action.
• Java 11 is used for docker compilation, since Java 8 is not used during runtime and the miredot plugin is the only plugin not compatible with Java 11.
• Updated the war plugin, since the old version generated warnings on Java 11.

[blootsvoets]

The result of a Github Actions PR can be found here:
https://github.com/thehyve/OHDSI-WebAPI/pull/5/checks
Note that mvn test is failing, but this is also failing locally for me.

The result of a release action can be found here:
https://github.com/thehyve/OHDSI-WebAPI/actions/runs/370542273

And the release itself has the WebAPI.war asset added to it
https://github.com/thehyve/OHDSI-WebAPI/releases/tag/githubActionsTest

Tagging @leeevans and @anthonysena for their earlier review of the Dockerfile integration.

[leeevans]

@blootsvoets
Please update this pull request to handle the pom.xml conflict and I will go ahead and merge.

I've setup the GitHub repo secrets for this WebAPI & the Atlas repo and created the Docker Hub access token (thanks @msuchard for your help)

[leeevans]

@blootsvoets is there an easy way for folks to distinguish the Docker Hub CI build docker image from the Release build docker image? If not, should we use 'ohdsi/webapi-snapshot' for the CI build docker image name?

[blootsvoets]

Thanks @leeevans. I've merged the master branch. Note that I did add back the maven central repo to the pom.xml, because this decreases the war build time from 3:46 to 1:26 minutes, making it 2.6 times faster to build this way.

Snapshots are differentiated from releases by their tag. Releases have a tag that matches the Github Release, for example 2.8.0 and every new release will replace the latest tag. Commits to master get the master tag, whereas pull requests to master get the pr-<github-pr-number> tag.

[leeevans]

@blootsvoets thanks for updating with latest pom.xml file and answering my question about snapshot tracking.

[anthonysena]

Tagging @chrisknoll @leeevans - I was going to merge this set of changes in and noticed the addition of this repository. Isn't it now the case that any/all requests for dependencies will go to repo.ohdsi.org's nexus and when not found there, it will source them (presumably through maven central) and therefore this <repository> reference is no longer required?

[anthonysena]

Sorry, I should have re-read the PR. @blootsvoets mentioned:

Note that I did add back the maven central repo to the pom.xml, because this decreases the war build time from 3:46 to 1:26 minutes, making it 2.6 times faster to build this way.

Let me know if you have any concerns w/ this otherwise I'm fine to merge in these changes. Thanks!

[chrisknoll]

No we should use the ohdsdi.org repo since that bridges all the repos to a single endpoint, and will be even faster than just putting central first (since it will go through central connection first then to ohdsi), but with the updated repositories element, it only needs to go to ohdsi maven and it will have all the central maven artifacts cached.

For refrence, this is the only thing we need in pom:

  <repositories>
    <repository>
      <id>ohdsi</id>
      <name>repo.ohdsi.org</name>
      <url>http://repo.ohdsi.org:8085/nexus/content/groups/public</url>
    </repository>
  </repositories>

[blootsvoets]

Maven central uses a large CDN, so data is always stored relatively close to the person doing the build, and they have invested in fast networks and hardware to give the fastest possible response. This can empirically be verified by doing the docker build with and without central as first element, since this build cannot use the maven cache. It will also be the case for new contributors that do not yet have the dependencies cached yet. In terms of cost. So while indeed you don’t need central as the first repository, it does save time, costs and energy.

If you still prefer it to be removed, let me know and I’ll do it ASAP.

[chrisknoll]

That's an interesting point about the CDN, but I'm not sure it will make a big difference in practice. According to this reference from Apache, it is a best practice to use a repository manager with a proxy to the public maven repositories, and they call out the build performance when dealing with snapshots. But, I think you can also set up your own local settings.xml file to reference your own preferred maven repositories, described in this stack overflow post.

I want to be accomodating, but we've gone through a few iterations on configuring repositories in the pom.xml and using central. The biggest impact I've seen is that when doing local builds during development (and we're referencing snapshots) each build wants to make sure latest snapshot is available, and it would go out to a number of other repositories before finally getting to the ohdsi.org's repo and finding out that nothing has changed. This added at least 30-60 seconds to the build, which when you're doing multiple builds per hour, it really adds up. I haven't checked if we put central + ohdsi nexus into the pom if it leads to a slower build time...but using the single ohdsi nexus seems to be working ok.

So, did you want to try to see if using a pom.xml with central vs. without using it to determine if it has any impact on your docker build times? Also see if you can configure your own settings.xml to add central to it to see if you get the desired result of accessing central first before ohdsi nexus? If you are still experiencing problems even after using the ohdsi group http://repo.ohdsi.org:8085/nexus/content/groups/public, I'll check to see if adding the central to the pom.xml impacts build times.

[blootsvoets]

Alright, yes I did try the difference. I've now committed a change where the central repo is only used in the docker build. Would that alleviate your concerns? As mentioned in the first post, using central as a first repository reduces the WAR build time from 3:46 to 1:26.

[chrisknoll]

Yes, that works for me. Thanks.

[blootsvoets]

An unrelated question about repo.ohdsi.org: would it be possible to use HTTPS there, for example using letsencrypt? Gradle has already phased out support for non-HTTPS maven repositories mainly to guard against man in the middle attacks: https://blog.gradle.org/decommissioning-http
Maven central has discontinued HTTP support in the beginning of 2020.

[chrisknoll]

@leeevans maintains the host, and so I think he'd be the best to answer if he can set up an SSL cert and enable HTTPS on the server. I agree, we should be using SSL, but am also sensitive to the maintenance burden this could be put on @leeevans .

[leeevans]

@blootsvoets yes, I will be moving repo.ohdsi.org to HTTPS in the next couple of weeks. I'll coordinate with @anthonysena and @chrisknoll so the repo reference in the pom.xml can be updated at the appropriate time

[blootsvoets]

The POM has been modified as requested. @anthonysena or @chrisknoll would you please take another look?

[chrisknoll]

Looks good. @anthonysena : if your 'change request' is satisfied, go ahead and merge at your convenience.