Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

update twoliter and buildsys to put RPMs into per-package sub-directories #210

Merged
merged 5 commits into from
May 17, 2024

Conversation

webern
Copy link
Contributor

@webern webern commented May 10, 2024

Issue number:

Related #185
Related #73
Builds on #198

Description of changes:

In order to construct kits containing the correct RPMs, we need to stop using a flat build/rpms structure. Instead we need to put each RPM into a sub-directory. Then we can use the package-name to find the right RPMs to build repos with the correct RPMs on-the-fly when building packages, kits and variants.

Testing done:

  • Build a Bottlerocket Variant
  • unit tests are passing
  • Built a Bottlerocket variant from develop (flat RPM structure) then tried to do an incremental build without the twoliter: error if old rpm structure is discovered commit. Buildsys failed as expected with No matching package to install: 'bottlerocket-glibc-devel'
  • Tried again with twoliter: error if old rpm structure is discovered , got the helpful message: PLEASE RUN: cargo make clean
  • Tried again after a cargo make clean and the build succeeded

Terms of contribution:

By submitting this pull request, I agree that this contribution is dual-licensed under the terms of both the Apache License, version 2.0, and the MIT license.

@webern
Copy link
Contributor Author

webern commented May 10, 2024

Copy link
Contributor

@bcressey bcressey left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, just some minor notes.

tools/buildsys/src/builder.rs Outdated Show resolved Hide resolved
twoliter/embedded/Dockerfile.dockerignore Outdated Show resolved Hide resolved
twoliter/embedded/rpm2kit Outdated Show resolved Hide resolved
Comment on lines 20 to 23
# FIXME: should this be a versioned directory?
# - needs to change whenever the kit changes?
# - needs to be predictable for publishing to work?
# - needs to avoid ambiguity if we have other local builds of the same kit?
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we're emitting the right cargo:rerun-if-changed and cargo:rerun-if-env-changed directives in buildsys then we can drop this. Just need to make sure #198 (comment) is addressed.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the answer is as simple as rm -rf "${KIT_DIR}" since, if this script is running, then we know we are re-creating the kit. That's what I've pushed.

@webern webern added the kits Work relating to kits. label May 13, 2024
@webern webern marked this pull request as ready for review May 13, 2024 16:56
@webern webern changed the base branch from pr-198 to develop May 13, 2024 23:02
@webern webern force-pushed the package-dirs branch 2 times, most recently from 1d2df8b to 6eaabe9 Compare May 14, 2024 00:02
@webern
Copy link
Contributor Author

webern commented May 16, 2024

https://github.com/bottlerocket-os/twoliter/compare/3183c93da746776a9605b2008f2f036b1c22ba6c..b9d7639d19fe35c0ba44dd98f13782a1b30f8356

Add a check to Makefile.toml to tell the user when they need to run cargo make clean.

@webern
Copy link
Contributor Author

webern commented May 16, 2024

@bcressey this is ready for another review after adding the flat RPMs dir check.

webern and others added 5 commits May 17, 2024 10:20
Place output RPMs into a subdirectory using the package-name.
Signed-off-by: Ben Cressey <[email protected]>
Update the buildsys Dockerfile so that it works with RPM packages being
outputted to sub-directories, i.e. build/package-name/

Co-authored-by: Matt Briggs <[email protected]>
Co-authored-by: Ben Cressey <[email protected]>
Signed-off-by: Ben Cressey <[email protected]>
Add a check to Makefile.toml to see if the user has rpms in the
build/rpms directory. If so, this is likely a problem because the new
version of Buildsys will expect per-package subdirectories, and Cargo
will not know it needs to rebuild.

Raise an error that informs the user they need to run cargo make clean.
@webern
Copy link
Contributor Author

webern commented May 17, 2024

done

createrepo_c "${KIT_DIR}"
dnf --disablerepo '*' --repofrompath "kit,file:///${KIT_DIR}" repoquery --all
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just for my understanding, what does this do? is it just disabling the kit dnf repo by default so that it has to be explicitly enabled in the variant build?

@webern webern merged commit cbd0bd9 into develop May 17, 2024
1 check passed
@webern webern deleted the package-dirs branch May 17, 2024 19:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kits Work relating to kits.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants