Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add new dmsquash-live-autooverlay module #1991

Merged
merged 3 commits into from
Oct 15, 2022

Conversation

iammattcoleman
Copy link
Contributor

This pull request introduces a new module that creates a partition for overlayfs usage in the free space on the root filesystem's parent block device.

Ubuntu 20.04 (the main branch) and Fedora 36 (the fedora-36 branch) Kiwi image descriptions that produce ISO images that demonstrate this feature are available in this repository:
https://github.com/iammattcoleman/overlay-test

Checklist

  • I have tested it locally
  • I have reviewed and updated any documentation if relevant
  • I am providing new code and test(s) for it

@github-actions github-actions bot added modules Issue tracker for all modules test Issues related to testing labels Sep 27, 2022
@Conan-Kudo Conan-Kudo self-assigned this Sep 27, 2022
@Conan-Kudo
Copy link
Member

This looks great! I've approved the test run as you're a first-time contributor.

@iammattcoleman
Copy link
Contributor Author

Shell linting failed due to a couple instances of missing double quotes.

@iammattcoleman iammattcoleman marked this pull request as draft September 27, 2022 22:13
@iammattcoleman iammattcoleman force-pushed the create-overlay branch 5 times, most recently from b2089b8 to 1e681fe Compare September 27, 2022 23:30
@iammattcoleman iammattcoleman marked this pull request as ready for review September 27, 2022 23:31
@Conan-Kudo
Copy link
Member

I will run this once the Fedora datacenter maintenance window is over.

Copy link
Member

@Conan-Kudo Conan-Kudo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately, we're stuck in a catch-22, and while I can verify the failing test passes locally when I build the container and use it locally, it won't fix itself in here until after we merge it.

Otherwise, this looks good to merge. Thanks!

@Conan-Kudo Conan-Kudo enabled auto-merge (rebase) September 28, 2022 14:38
test/TEST-16-DMSQUASH/test.sh Outdated Show resolved Hide resolved
fi

live_dir=$(getarg rd.live.dir)
[ -z "$live_dir" ] && live_dir="LiveOS"
Copy link
Collaborator

@LaszloGombos LaszloGombos Sep 29, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This line seems repeat from https://github.com/dracutdevs/dracut/blob/master/modules.d/90dmsquash-live/dmsquash-live-root.sh#L21

Can we avoid repeating and risking making the two modules out of sync for the default value with later PRs ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This script must run before dmsquash-live-root.sh in order to work properly and this script won't always run since it's an optional module that complements 90dmsquash-live. So, both scripts will need a similar line, but it would be nice to have the constant ("LiveOS") in a single location.

Can you point me to some code that demonstrates sharing a constant between two related modules so I can follow the project's existing conventions?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To share a constant, we would need to introduce a shared file between dmsquash-live and dmsquash-live-autooverlay. I think it make sense to limit making changes in other modules as part of this module introduction PR, so perhaps we can do this refactoring as a follow-up PR.


overlayPartition=${blockDevice}$((currentPartitionCount + 1))

label=$(blkid --match-tag LABEL --output value "$rootDevice")
Copy link
Collaborator

@LaszloGombos LaszloGombos Sep 29, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it make sense to reuse devnames from 99base/dracut-lib.sh here ?
crypt and fs-lib modules need similar functionality and we call into devnames in those modules.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The approach I took was to gather all the data necessary to perform the task before making any destructive changes to the disk. That way it can avoid doing anything if it doesn't have all the information it needs.

devnames provides device names within /dev. This is gathering the root device's label and UUID for use in the createFilesystem() function on line 105, which creates a directory with a name that matches the default path format defined in dmsquash-live-root.sh. So, that's potentially another constant that should be shared between the two modules from a single place.

Copy link
Collaborator

@LaszloGombos LaszloGombos left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some review comments to better align with existing modules..

@LaszloGombos
Copy link
Collaborator

Unfortunately, we're stuck in a catch-22, and while I can verify the failing test passes locally

I find it a better option to just run the job under your personal github fork e.g. https://github.com/iammattcoleman/dracut/actions/workflows/manualtest.yml and post the log of at least one successful run. You might need to trigger container creation first manually under your personal github before triggering the manual test run.

auto-merge was automatically disabled October 3, 2022 22:33

Head branch was pushed to by a user without write access

@iammattcoleman iammattcoleman requested review from LaszloGombos and removed request for haraldh, johannbg and danimo October 3, 2022 22:38
Copy link
Collaborator

@LaszloGombos LaszloGombos left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes all look good to me, but perhaps the module is not yet complete, see additional comments.

Also if you have a chance, can you please also update the Gentoo container and add parted ?

@LaszloGombos
Copy link
Collaborator

LGTM, thanks for the timely interactions.

@Conan-Kudo Conan-Kudo enabled auto-merge (rebase) October 6, 2022 06:32
@Conan-Kudo Conan-Kudo disabled auto-merge October 6, 2022 06:35
@iammattcoleman
Copy link
Contributor Author

As I mentioned in the Matrix channel, please hold off on merging this: as @LaszloGombos's suggested, I ran the manual test in my fork and it failed:
https://github.com/iammattcoleman/dracut/actions/runs/3193095857/jobs/5211306428

While working on this, I was testing on bare metal (well, really a VM) and successfully running the test from within the TEST-16-DMSQUASH directory. I had not tried running the test from within the tests directory like the manual test GitHub Action.

After I saw the test fail in my fork, I reproduced it in a podman container locally. If I run make V=1 TESTS=16 clean check within /dracut/tests/, it fails. If I run make clean setup run within /dracut/tests/TEST-16-DMSQUASH/, it succeeds.

@iammattcoleman iammattcoleman force-pushed the create-overlay branch 2 times, most recently from 63d0e69 to c3240d9 Compare October 15, 2022 00:53
@iammattcoleman iammattcoleman force-pushed the create-overlay branch 2 times, most recently from 4ff9ddf to 1260587 Compare October 15, 2022 01:21
@github-actions github-actions bot added the github Issues related to .github label Oct 15, 2022
@Conan-Kudo Conan-Kudo enabled auto-merge (rebase) October 15, 2022 07:04
@Conan-Kudo Conan-Kudo requested a review from johannbg October 15, 2022 07:18
@Conan-Kudo Conan-Kudo disabled auto-merge October 15, 2022 17:45
@Conan-Kudo Conan-Kudo enabled auto-merge (rebase) October 15, 2022 17:45
@Conan-Kudo Conan-Kudo requested review from mrc0mmand and removed request for johannbg October 15, 2022 17:51
@Conan-Kudo Conan-Kudo merged commit a3c67d2 into dracutdevs:master Oct 15, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
github Issues related to .github modules Issue tracker for all modules test Issues related to testing
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants