rmdir: failed to remove '..../stage0/rootfs/debootstrap': Directory not empty

[DominoX]

Hi guys,

First try today with pi-gen. Didn't go well, but I'm real new to this so guessing I'm just doing something silly.

When running sudo ./build.sh, it spends about 6 minutes in the loop of 'Retrieving ...' followed by 'Validating ...' various files/packages, and then it dies with the error

rmdir: failed to remove '/home/pi/pi-gen/pi-gen/work/2019-10-26-Raspbian/stage0/rootfs/debootstrap': Directory not empty

MY SETUP:
--- Brand new RasPi 4 (4G version)
--- 32GB microSD
--- Latest Buster 4.19.75-v7l+ #1270 SMP Tue Sep 24 18:51:41 BST 2019
--- No other packages installed other than the prerequisites for pi-gen as per GIT home page

STEPS TO REPRODUCE:
--- SSH into raspi (default pi user, so default pwd)
--- git clone https://github.com/RPi-Distro/pi-gen.git
--- That creates a 'pi-gen' folder, so I cd to it
--- sudo ./build.sh
--- roughly 6 minutes retrieving/validating, then error

The ONLY line I have in my 'config' file is:
IMG_NAME='Raspbian'

Any advice on how I can troubleshoot this, or next step, much appreciated!!!

Dave

[EDIT] Reading further, I think this might be related to #294 The retrieving/validating steps in the screenshot there are idenical to mine, but my script quits immediately after displaying the rmdir error after the last retrieving/validating message ... wheras the script in that screenshot continues to run. WTF???

[jsnshrmn]

Also ran into this today on a 3b+. I attempted on master as well as the latest tagged release both with the host-side build and the docker build scripts.

[jsnshrmn]

when I rerun the docker script to continue, eg.
CONTINUE=1 ./build-docker.sh
I get

[23:18:10] Begin /pi-gen
[23:18:10] Begin /pi-gen/stage0
[23:18:10] Begin /pi-gen/stage0/prerun.sh
[23:18:10] End /pi-gen/stage0/prerun.sh
[23:18:10] Begin /pi-gen/stage0/00-configure-apt
[23:18:10] Begin /pi-gen/stage0/00-configure-apt/00-run.sh
install: target '/pi-gen/work/2019-10-27-Raspbian/stage0/rootfs/etc/apt/' is not a directory: No such file or directory

[XECDesign]

Could you give this a go?

diff --git a/scripts/common b/scripts/common
index 68fab54..e2048d9 100644
--- a/scripts/common
+++ b/scripts/common
@@ -17,8 +17,9 @@ bootstrap(){
        BOOTSTRAP_ARGS+=(--components "main,contrib,non-free")
        BOOTSTRAP_ARGS+=(--keyring "${STAGE_DIR}/files/raspberrypi.gpg")
        BOOTSTRAP_ARGS+=("$@")
+       printf -v BOOTSTRAP_STR '%q ' "${BOOTSTRAP_ARGS[@]}"
 
-       setarch linux32 capsh --drop=cap_setfcap -- "${BOOTSTRAP_CMD}" "${BOOTSTRAP_ARGS[@]}" || true
+       setarch linux32 capsh --drop=cap_setfcap -- -c "'${BOOTSTRAP_CMD}' $BOOTSTRAP_STR" || true
 
        if [ -d "$2/debootstrap" ]; then
                rmdir "$2/debootstrap"

[jsnshrmn]

The spice is flowing now (at least on docker). Haven't checked the host-based script.

[DominoX]

Could you give this a go?

diff --git a/scripts/common b/scripts/common
index 68fab54..e2048d9 100644
--- a/scripts/common
+++ b/scripts/common
@@ -17,8 +17,9 @@ bootstrap(){
        BOOTSTRAP_ARGS+=(--components "main,contrib,non-free")
        BOOTSTRAP_ARGS+=(--keyring "${STAGE_DIR}/files/raspberrypi.gpg")
        BOOTSTRAP_ARGS+=("$@")
+       printf -v BOOTSTRAP_STR '%q ' "${BOOTSTRAP_ARGS[@]}"
 
-       setarch linux32 capsh --drop=cap_setfcap -- "${BOOTSTRAP_CMD}" "${BOOTSTRAP_ARGS[@]}" || true
+       setarch linux32 capsh --drop=cap_setfcap -- -c "'${BOOTSTRAP_CMD}' $BOOTSTRAP_STR" || true
 
        if [ -d "$2/debootstrap" ]; then
                rmdir "$2/debootstrap"

Sorry guys. I'm just too new at this. What should I do with this code snippet? It doesn't look like regular commands I could execute via SSH in a bash shell?

[XECDesign]

It's a patch. If you're not familiar with patches, the simplest thing to do would be this:

1. Open scripts/common in an editor.
2. Find the line that looks like this:

setarch linux32 capsh --drop=cap_setfcap -- "${BOOTSTRAP_CMD}" "${BOOTSTRAP_ARGS[@]}" || true

3. Replace it with these two lines:

printf -v BOOTSTRAP_STR '%q ' "${BOOTSTRAP_ARGS[@]}"
setarch linux32 capsh --drop=cap_setfcap -- -c "'${BOOTSTRAP_CMD}' $BOOTSTRAP_STR" || true

4. Run a clean build.
5. Read up on patches when you have some time to kill. They will be come in handy.

[DominoX]

@XECDesign ... Thanks for the guidance about 'patches'. I learned something new today. It's a good day.

The change you suggested worked well. No longer aborting at the point all the packages are retrieving/validating.

I let it run. I had to go out. 4 hours later, came back and the raspi was completely dead. No SSH, no VNC. Connected to a screen while booting to diagnose issue. Saw the usual initial scrolling messages appear on the screen during the boot sequence, but then as it's about to launch the desktop, blank black screen with flashing cursor top-left corner ... and nothing else.

Mounted the microSD on a different working pi and looked around. Found /rootfs is 100% full with no free space.

I'm using a 32Gb microSD for the build. I guess that's too small. What size would work?

Thanks again,

Dave

[XECDesign]

I don't know off the top of my head, but pi-gen eats a lot more space than it should. A 64GB card will be enough, but I'd recommend using a USB HDD mount or at least a flash drive for such tasks.

[DominoX]

Thanks for the advice to move to a SSD. The build ran end-to-end this time. Total size used is just short of 58GB, so a 64GB microSD might have worked, but I'd be worried about the write wear-and-tear on the SD.

However, the resulting images created by the build are not working normally. I tried only the Buster-Full image, and it's broken in many ways. Eg:

• does not launch the raspi-config wizard at first boot to set locale, timezone, kbd, wifi country, etc. The official build from raspi.org always launches this wizard at first start-up.
• running `raspi-config' in SSH terminal does nothing. No app launches at all.
• using sudo in the terminal yields the error "sudo: /usr/bin/sudo must be owned by uid 0 and have the setuid bit set". Cannot elevate any cmds in the shell to run as sudo.

So basically, I can't do anything with this build. Am I right in assuming I can simply unzip images in 'deploy' folder (once build finished) and burn them to a SD card and then boot in raspi without any other steps/setup/config being needed? Or am I missing something?

Also, is there a full log of the build process anywhere? I could run build.sh > log, but is there an easier way to see the full results of each build stage? Reason I ask is because while the build was running (for hours and hours), I periodically glanced at the screen and noticed many cases where there were warnings and errors displayed, but the build continued anyway. Those errors included "cannot set LOCALE" quite a few times at different times throughout the build. Then there were a few "... not enough space ...", but that can't be on my setup since I'm running on a 1TB SSD with over 700GB free, and my SD card partitions are around 30% used. And too many others to mention. A log would sure help to go back and investigate each a bit more.

I guess we can go-ahead and close this issue 338 since the error/behavior I reported initially is fixed. The other stuff I'm facing is probably better addressed under its own issue number. Agree?

Best, Dave

[XECDesign]

Yes, that sounds like a separate issue. Please include what filesystem type you have on the ssd.

[DominoX]

@XECDesign - filesys on SSD = NTFS. Any clues about my earlier request?

Also, is there a full log of the build process anywhere? I could run build.sh > log, but is there an easier way to see the full results of each build stage? Reason I ask is because while the build was running (for hours and hours), I periodically glanced at the screen and noticed many cases where there were warnings and errors displayed, but the build continued anyway. Those errors included "cannot set LOCALE" quite a few times at different times throughout the build. Then there were a few "... not enough space ...", but that can't be on my setup since I'm running on a 1TB SSD with over 700GB free, and my SD card partitions are around 30% used. And too many others to mention. A log would sure help to go back and investigate each a bit more.

Best, Dave

[XECDesign]

I'm surprised it worked with NTFS at all. You have to use a fileystem like ext4 or at least something that supports unix permissions.

There's no log, but you can run something like ./build.sh |& tee build.log