[RFC] Remote workers #7491
yuriy-yarosh
started this conversation in
Ideas
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Describe the feature you'd like to request
It would be nice to support buildkit remote workers using llb image build format, while using image docker layers to cache all the respective artifacts.
buildkit still serves the same purposes as turborepo, but for containers, by design.
Remote execution makes sense only for really long and complex tasks - current turborepo parallelization factor can get up to 60 threads on a big modern project, easily.
Describe the solution you'd like
Long and slow job chains can be shipped to a remote buildkit cluster as build contexts and get massive parallelization.
Remote build contexts are cached inside the buildkit daemon itself, so there's not much overhead in terms of communication.
It makes sense to duplicate the artifact cache locally as well, to minimize the respective communication overhead.
Describe alternatives you've considered
There's no other build IR similar to LLB to my knowledge.
Beta Was this translation helpful? Give feedback.
All reactions