Skip to content

Commit

Permalink
deblobbify, replacing the system folder with several other components
Browse files Browse the repository at this point in the history
  • Loading branch information
robertkirkman committed Jan 21, 2025
1 parent e3c861f commit 3b99512
Show file tree
Hide file tree
Showing 99 changed files with 805 additions and 2,460 deletions.
105 changes: 72 additions & 33 deletions .github/workflows/docker_build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,39 +13,79 @@ env:
CI: true
DOCKER_BUILDKIT: 1

# The GitHub ARM runner is capable of running
# /data/data/com.termux/files/usr/bin/apt-get upgrade
# natively during the update process for both aarch64
# and arm, demonstrating that it is compatible with
# 32-bit binaries and capable of completely
# replacing QEMU, but using it in the same workflow
# as the x86 runner appears to require two separate jobs.
jobs:
main:
runs-on: ubuntu-latest
ARM:
runs-on: ubuntu-24.04-arm
strategy:
matrix:
CPU_ARCH:
ARCHITECTURE:
- aarch64
- arm
- i686
- x86_64
steps:

- name: Clone repository
uses: actions/checkout@v4

- name: Setup binfmt_misc
if: (matrix.CPU_ARCH == 'aarch64') || (matrix.CPU_ARCH == 'arm')
run: docker run --rm --privileged aptman/qus -s -- -p aarch64 arm
- name: Set architecture variable
run: echo "ARCHITECTURE=${{ matrix.ARCHITECTURE }}" >> $GITHUB_ENV

- name: Build image
run: ./generate.sh

- name: Login to Docker Hub
if: github.ref == 'refs/heads/master' && github.event_name != 'pull_request' && github.repository == 'termux/termux-docker'
uses: docker/login-action@v3
with:
username: grimler
password: ${{ secrets.GRIMLER_DOCKER_TOKEN }}

- name: Push to Docker Hub
if: github.ref == 'refs/heads/master' && github.event_name != 'pull_request' && github.repository == 'termux/termux-docker'
run: docker push termux/termux-docker:${{ env.ARCHITECTURE }}

- name: Build images
- name: Export container as tar archive
if: always()
run: |
case '${{ matrix.CPU_ARCH }}' in
arm) SYSTEM_TYPE=arm; PLATFORM_TAG="linux/arm/v7";;
aarch64) SYSTEM_TYPE=arm; PLATFORM_TAG="linux/arm64";;
i686) SYSTEM_TYPE=x86; PLATFORM_TAG="linux/386";;
*) SYSTEM_TYPE=x86; PLATFORM_TAG="linux/amd64";;
esac
docker buildx build -t \
termux/termux-docker:${{ matrix.CPU_ARCH }} \
--platform "$PLATFORM_TAG" \
--build-arg BOOTSTRAP_ARCH=${{ matrix.CPU_ARCH }} \
--build-arg SYSTEM_TYPE="${SYSTEM_TYPE}" \
.
docker run \
--name termux-docker-${{ env.ARCHITECTURE }} \
termux/termux-docker:${{ env.ARCHITECTURE }} \
uname -a
docker stop termux-docker-${{ env.ARCHITECTURE }}
docker export -o termux-docker-${{ env.ARCHITECTURE }}.tar \
termux-docker-${{ env.ARCHITECTURE }}
sha256sum termux-docker-${{ env.ARCHITECTURE }}.tar
- name: Store tar archive
if: always()
uses: actions/upload-artifact@v4
with:
name: termux-docker-${{ env.ARCHITECTURE }}-${{ github.sha }}
path: termux-docker-${{ env.ARCHITECTURE }}.tar

x86:
runs-on: ubuntu-latest
strategy:
matrix:
ARCHITECTURE:
- x86_64
- i686
steps:

- name: Clone repository
uses: actions/checkout@v4

- name: Set architecture variable
run: echo "ARCHITECTURE=${{ matrix.ARCHITECTURE }}" >> $GITHUB_ENV

- name: Build image
run: ./generate.sh

- name: Login to Docker Hub
if: github.ref == 'refs/heads/master' && github.event_name != 'pull_request' && github.repository == 'termux/termux-docker'
Expand All @@ -57,28 +97,27 @@ jobs:
- name: Push to Docker Hub
if: github.ref == 'refs/heads/master' && github.event_name != 'pull_request' && github.repository == 'termux/termux-docker'
run: |
docker push termux/termux-docker:${{ matrix.CPU_ARCH }}
if [ ${{ matrix.CPU_ARCH }} = i686 ]; then
docker tag termux/termux-docker:i686 termux/termux-docker:latest
docker push termux/termux-docker:${{ env.ARCHITECTURE }}
if [ ${{ env.ARCHITECTURE }} = i686 ]; then
docker tag termux/termux-docker:${{ env.ARCHITECTURE }} termux/termux-docker:latest
docker push termux/termux-docker:latest
fi
- name: Export container as tar archive
if: always()
run: |
docker run \
--privileged \
--name termux-docker-${{ matrix.CPU_ARCH }} \
termux/termux-docker:${{ matrix.CPU_ARCH }} \
--name termux-docker-${{ env.ARCHITECTURE }} \
termux/termux-docker:${{ env.ARCHITECTURE }} \
uname -a
docker stop termux-docker-${{ matrix.CPU_ARCH }}
docker export -o termux-docker-${{ matrix.CPU_ARCH }}.tar \
termux-docker-${{ matrix.CPU_ARCH }}
sha256sum termux-docker-${{ matrix.CPU_ARCH }}.tar
docker stop termux-docker-${{ env.ARCHITECTURE }}
docker export -o termux-docker-${{ env.ARCHITECTURE }}.tar \
termux-docker-${{ env.ARCHITECTURE }}
sha256sum termux-docker-${{ env.ARCHITECTURE }}.tar
- name: Store tar archive
if: always()
uses: actions/upload-artifact@v4
with:
name: termux-docker-${{ matrix.CPU_ARCH }}-${{ github.sha }}
path: termux-docker-${{ matrix.CPU_ARCH }}.tar
name: termux-docker-${{ env.ARCHITECTURE }}-${{ github.sha }}
path: termux-docker-${{ env.ARCHITECTURE }}.tar
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
termux-docker-rootfs
135 changes: 47 additions & 88 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,91 +1,50 @@
##############################################################################
# Bootstrap Termux environment.
FROM scratch AS bootstrap

ARG BOOTSTRAP_VERSION=2023.02.19-r1%2Bapt-android-7
ARG BOOTSTRAP_ARCH=i686
ARG SYSTEM_TYPE=x86

# Docker uses /bin/sh by default, but we don't have it currently.
SHELL ["/system/bin/sh", "-c"]
ENV PATH /system/bin

# Copy libc, linker and few utilities.
COPY /system/$SYSTEM_TYPE /system

# Copy entrypoint script.
COPY /entrypoint.sh /entrypoint.sh
COPY /entrypoint_root.sh /entrypoint_root.sh

# Extract bootstrap archive and create symlinks.
ADD https://github.com/termux/termux-packages/releases/download/bootstrap-$BOOTSTRAP_VERSION/bootstrap-$BOOTSTRAP_ARCH.zip /bootstrap.zip
RUN busybox mkdir -p /data/data/com.termux/files && \
cd /data/data/com.termux/files && \
busybox mkdir ../cache ./usr ./home && \
busybox unzip -d usr /bootstrap.zip && \
busybox rm /bootstrap.zip && \
cd ./usr && \
busybox cat SYMLINKS.txt | while read -r line; do \
dest=$(echo "$line" | busybox awk -F '←' '{ print $1 }'); \
link=$(echo "$line" | busybox awk -F '←' '{ print $2 }'); \
busybox ln -s "$dest" "$link"; \
done && \
busybox rm SYMLINKS.txt && \
busybox ln -s /data/data/com.termux/files/usr /usr && \
busybox ln -s /data/data/com.termux/files/usr/bin /bin && \
busybox ln -s /data/data/com.termux/files/usr/tmp /tmp

# Link some utilities to busybox.
# Some utilities in $PREFIX are actually a wrapper of the same binary
# from /system/bin. See termux-tools/build.sh#L29.
RUN for tool in df mount ping ping6 su top umount; do \
busybox ln -s /system/bin/busybox /system/bin/$tool; \
done

# Set ownership and file access modes:
# * User content is owned by 1000:1000.
# * Termux file modes are set only for user.
# * Rest is owned by root and has 755/644 modes.
RUN busybox chown -Rh 0:0 /system && \
busybox chown -Rh 1000:1000 /data/data/com.termux && \
busybox ln -s /system/etc/passwd /etc/passwd && \
busybox ln -s /system/etc/group /etc/group && \
busybox find /system -type d -exec busybox chmod 755 "{}" \; && \
busybox find /system -type f -executable -exec busybox chmod 755 "{}" \; && \
busybox find /system -type f ! -executable -exec busybox chmod 644 "{}" \; && \
busybox find /data -type d -exec busybox chmod 755 "{}" \; && \
busybox find /data/data/com.termux/files -type f -o -type d -exec busybox chmod g-rwx,o-rwx "{}" \; && \
cd /data/data/com.termux/files/usr && \
busybox find ./bin ./lib/apt ./libexec -type f -exec busybox chmod 700 "{}" \;

# Install updates and cleanup when not building for arm.
ENV PATH /data/data/com.termux/files/usr/bin
RUN if [ ${SYSTEM_TYPE} = 'arm' ]; then exit; else \
/system/bin/mksh -T /dev/ptmx -c "/system/bin/dnsmasq -u root -g root --pid-file /dnsmasq.pid" && sleep 1 && \
su - system -c "/data/data/com.termux/files/usr/bin/apt update" && \
su - system -c "/data/data/com.termux/files/usr/bin/apt upgrade -o Dpkg::Options::=--force-confnew -yq" && \
rm -rf /data/data/com.termux/files/usr/var/lib/apt/* && \
rm -rf /data/data/com.termux/files/usr/var/log/apt/* && \
rm -rf /data/data/com.termux/cache/apt/* ;\
fi

##############################################################################
# Create final image.
FROM scratch

ENV ANDROID_DATA /data
ENV ANDROID_ROOT /system
ENV HOME /data/data/com.termux/files/home
ENV LANG en_US.UTF-8
ENV PATH /data/data/com.termux/files/usr/bin
ENV PREFIX /data/data/com.termux/files/usr
ENV TMPDIR /data/data/com.termux/files/usr/tmp
ENV TZ UTC

COPY --from=bootstrap / /

WORKDIR /data/data/com.termux/files/home
SHELL ["/data/data/com.termux/files/usr/bin/sh", "-c"]

ARG ROOTFS
ARG TERMUX_APP_PACKAGE
ARG TERMUX_BASE_DIR
ARG TERMUX_PREFIX

# Install generated rootfs containing bionic libc,
# toybox, mksh, iputils, dnsmasq, and termux bootstrap
COPY --chown=1000:1000 ${ROOTFS} /

# Docker uses /bin/sh by default, but we don't have it.
# This ENV PATH line selects /system/bin/sh.
# The final ENV PATH line later below then selects
# $TERMUX_PREFIX/bin/sh -> bash without having
# to add an additional SHELL line.
ENV PATH=/system/bin
SHELL ["sh", "-c"]

# Install updates and cleanup
# Start dnsmasq to resolve hostnames, and,
# for some reason the -c argument of toybox-su is not working,
# so this odd-looking script forces the update process
# to work using the -s argument of toybox-su instead, which is working.
RUN sh -T /dev/ptmx -c "$TERMUX_PREFIX/bin/dnsmasq -u root -g root --pid-file=/dnsmasq.pid" && \
sleep 1 && \
echo '#!/system/bin/sh' > /update.sh && \
echo "PATH=$TERMUX_PREFIX/bin" >> /update.sh && \
echo 'pkg update' >> /update.sh && \
echo 'apt-get upgrade -o Dpkg::Options::=--force-confnew -y' >> /update.sh && \
chmod +x /update.sh && \
su system -s /update.sh && \
rm -f /update.sh && \
rm -rf ${TERMUX_PREFIX}/var/lib/apt/* && \
rm -rf ${TERMUX_PREFIX}/var/log/apt/* && \
rm -rf /data/data/${TERMUX_APP_PACKAGE}/cache/apt/*

ENV ANDROID_DATA=/data
ENV ANDROID_ROOT=/system
ENV HOME=${TERMUX_BASE_DIR}/home
ENV LANG=en_US.UTF-8
ENV PATH=${TERMUX_PREFIX}/bin
ENV PREFIX=${TERMUX_PREFIX}
ENV TMPDIR=${TERMUX_PREFIX}/tmp
ENV TZ=UTC
ENV TERM=xterm

WORKDIR ${TERMUX_BASE_DIR}/home
ENTRYPOINT ["/entrypoint.sh"]
CMD ["/data/data/com.termux/files/usr/bin/login"]
CMD ["login"]
30 changes: 25 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,9 +49,12 @@ command:
docker run --rm --privileged aptman/qus -s -- -p aarch64 arm
```

Note that AArch64 and ARM containers work properly only in privileged
Note that AArch64 and ARM containers sometimes work properly only in privileged
mode. If you want your containers to have standard privileges, a custom
seccomp profile is required.
seccomp profile or a custom build of Docker might be required. The custom build
of Docker limits the customizations to purely what is necessary for
the `personality()` system call, leaving the security settings of all other system
calls untouched.

Variant with privileged container:

Expand All @@ -65,6 +68,23 @@ Variant with seccomp unconfined profile:
docker run -it --security-opt seccomp:unconfined termux/termux-docker:aarch64
```

Variant with custom build of Docker:

> [!NOTE]
> Example with Debian bookworm `armhf` host and the `docker.io` package. Assumes that [`deb-src` URIs](https://wiki.debian.org/Packaging/SourcePackage?action=show&redirect=SourcePackage#With_apt-get_source) and the [`devscripts` package](https://wiki.debian.org/Packaging#Suggested_tools_to_create_an_environment_for_packaging) are already installed, and that the current user is a member of the `docker` group.
```.sh
sudo apt build-dep docker.io
apt source docker.io
cp /path/to/termux-docker/custom-docker-with-unrestricted-personality.patch docker.io-*/debian/patches/
echo 'custom-docker-with-unrestricted-personality.patch' >> docker.io-*/debian/patches/series
cd docker.io-*/
DEB_BUILD_OPTIONS=nocheck debuild -b -uc -us
rm ../golang*
sudo apt install ../*.deb
docker run -it termux/termux-docker:arm
```

### Non-interactive execution of commands

You can run commands in non-interactive mode. Just append them to Docker
Expand Down Expand Up @@ -99,20 +119,20 @@ docker run -it --entrypoint /entrypoint_root.sh termux/termux-docker:latest
Docker:

```.sh
./build-all.sh
./generate.sh
```

Podman:

```.sh
./build-all.sh --podman
./generate.sh --podman
```

## Known issues

There a number of known issues which may not be resolved:

* ARM containers may require a custom seccomp profile to remove restrictions from
* ARM containers might require a custom seccomp profile or custom build of Docker to remove restrictions from the
`personality()` system call.

* When running certain multi threaded program in 32bit containers, the PIDs can
Expand Down
46 changes: 0 additions & 46 deletions build-all.sh

This file was deleted.

Loading

0 comments on commit 3b99512

Please sign in to comment.