Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor container runtime interface #1414

Open
yuvipanda opened this issue Feb 16, 2025 · 3 comments
Open

Refactor container runtime interface #1414

yuvipanda opened this issue Feb 16, 2025 · 3 comments

Comments

@yuvipanda
Copy link
Collaborator

Our current container runtime interface has these as separate things:

  1. Build
  2. Run
  3. Push
  4. List images
  5. Inspect images

However, the following combinations are what happen a lot of the time:

  1. Build + Run (no push)
  2. Build + Push (No run)
  3. List Images

This was fine when we were doing this with the old docker build api, but I think causes issues now with docker buildx.

  1. The buildkit build cache is different from the image store now, so we must pass --load to buildx during build so our run finds the image without having to pull
  2. However, when we are only doing build + push, this adds unnecessary extra complexity that will literally never be used. It's a copy (IIRC) from buildkit cache to the image store, except the image store will never actually be used. This takes up disk space and nothing else. This is primarily a problem on binderhub installs
  3. I'd like us to eventually support running buildkit directly (which removes the need for a privileged pod), and that would only support build + push. This is actually the ideal for binderhub.

Proposal

I propose we simply add 'push' and 'run' as boolean args to build and deprecate the run and push methods. Additionally, if this feels weird, we can rename build to be something else.

Runtimes that don't support run (like buildkit) can simply error out when run is True.

@yuvipanda
Copy link
Collaborator Author

Opening as an issue instead of a PR particularly to get feedback from @manics who is the only other person I know who manages a container runtime right now :)

@manics
Copy link
Member

manics commented Feb 17, 2025

I think this is fine! I vaguely recall thinking about this when working on the original Container Engine abstraction PR, but wanted to minimise changes.

@yuvipanda
Copy link
Collaborator Author

I had to revert the buildx deployment on mybinder.org (jupyterhub/mybinder.org-deploy#3228) because the node's disk was consistently being filled up.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants