-
Notifications
You must be signed in to change notification settings - Fork 565
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
build: docker images for latest release and main branch #2116
build: docker images for latest release and main branch #2116
Conversation
cdb5f66
to
12b01c9
Compare
This PR adds docker image building to the github actions workflow. Two images are built: - unstable: This is built from the current branch - latest: This is built from the version of rdflib in `docker/latest/requirements.txt`, but as this will be updated by dependabot it will essentially always be the latest version. The resulting image references are: - `ghcr.io/rdflib/rdflib:unstable` - `ghcr.io/rdflib/rdflib:latest` with an alias `ghcr.io/rdflib/rdflib:${rdflib_version}`, e.g. `ghcr.io/rdflib/rdflib:6.2.0`. As dependabot is conifgured for both pyton and docker dependencies, any change/update to dependencies will result in a new image being built. To avoid spurious changes in image digests the existing image is used as the cache source, so if the requirements.txt and Dockerfile for `ghcr.io/rdflib/rdflib:latest` has not changed then a new image should not be published for it even if the build runs. I have tested this in my forked repo, but there may be some kinks to still work out.
12b01c9
to
3585c67
Compare
@nicholascar this could be somewhat simpler with poetry, at the very least the redudant requirements.txt files could be avoided somewhat, but otherwise it would be the same for the most part. With poetry though you want to first export your dependencies to requirements.txt, something like: poetry export --without-hashes --dev --format requirements.txt > requirements.txt Then you would copy that requirements.txt file into the image. If that file is not stable (i.e. if it fluctuates randomly for the same poetry.lock) then the caching will not work too well. |
Image can be used as follow: $ echo '<ex:1> <ex:2> <ex:3> .' | docker run --rm -i ghcr.io/rdflib/rdflib:6.2.0 rdfpipe -i ntriples -o turtle -
@prefix ns1: <ex:> .
ns1:1 ns1:2 ns1:3 . |
Another way to use this: $ docker run --rm -i ghcr.io/rdflib/rdflib:6.2.0 python -c 'import rdflib; g = rdflib.Graph(); g.add((rdflib.RDF.type, rdflib.RDF.type, rdflib.RDF.type, )); print(g.serialize())'
@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
rdf:type a rdf:type . |
I do think for a long-term stable repo, it's probably always worth having the requirements.txt file built so non-Poetry users can use it with PIP, however we could happily do the more efficient image building with Poetry as you say. That's where we're going across the board (on other projects). Do you want to make this change? |
@jamiefeiss can you please pick up the patterns used here for Prez? |
I will create a discussion to check with others what they think sometime, I like poetry a lot and use it extensively, but I also don't want to make things harder for others so best to just see the level of support there is. |
@aucampia where do the images from this PR's creation scripts end up and how do people use them? I think that they might be private as I can't pull them from ghcr.io/rdflib/rdflib:latest etc. ("unauthorized"). I think we, or at least I!, need documentation on how to use the resultant images... |
The visibility needs to be changed here But I don't think I can do it.
I should add documentation and will when I can, hopefully this weekend or next week. |
Summary of changes
This PR adds docker image building to the github actions workflow.
Two images are built:
docker/latest/requirements.txt
, but as this will be updated bydependabot it will essentially always be the latest version.
The resulting image references are:
ghcr.io/rdflib/rdflib:unstable
ghcr.io/rdflib/rdflib:latest
with an aliasghcr.io/rdflib/rdflib:${rdflib_version}
, e.g.ghcr.io/rdflib/rdflib:6.2.0
.As dependabot is conifgured for both pyton and docker dependencies, any
change/update to dependencies will result in a new image being built.
To avoid spurious changes in image digests the existing image is used as
the cache source, so if the requirements.txt and Dockerfile for
ghcr.io/rdflib/rdflib:latest
has not changed then a new image shouldnot be published for it even if the build runs.
I have tested this in my forked repo, but there may be some kinks to
still work out.
Checklist
the same change.
so maintainers can fix minor issues and keep your PR up to date.