Skip to content

Commit

Permalink
Allow to run as a devcontainer
Browse files Browse the repository at this point in the history
  • Loading branch information
felipecrs committed Mar 4, 2024
1 parent c0c8d22 commit f61ab27
Show file tree
Hide file tree
Showing 6 changed files with 183 additions and 36 deletions.
25 changes: 25 additions & 0 deletions .devcontainer/devcontainer.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
{
"build": {
"context": "..",
"dockerfile": "../Dockerfile"
},
"overrideCommand": false,
"mounts": [
"source=/var/run/docker.sock,target=/var/run/docker.sock,type=bind"
],
"runArgs": ["--network=host"],
"postCreateCommand": "pkgx install hadolint k3d helmfile werf kubectl",
"customizations": {
"vscode": {
"extensions": [
"timonwong.shellcheck",
"ms-azuretools.vscode-docker",
"exiasr.hadolint",
"foxundermoon.shell-format",
"esbenp.prettier-vscode",
"github.vscode-github-actions",
"ms-kubernetes-tools.vscode-kubernetes-tools"
]
}
}
}
6 changes: 1 addition & 5 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -43,11 +43,7 @@ jobs:
- name: Setup test dependencies
uses: pkgxdev/setup@v2
with:
+: >
k3d
helmfile
werf
kubectl
+: k3d helmfile werf kubectl

- name: Build
uses: docker/build-push-action@v5
Expand Down
4 changes: 3 additions & 1 deletion .vscode/extensions.json
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@
"ms-azuretools.vscode-docker",
"exiasr.hadolint",
"foxundermoon.shell-format",
"esbenp.prettier-vscode"
"esbenp.prettier-vscode",
"github.vscode-github-actions",
"ms-kubernetes-tools.vscode-kubernetes-tools"
]
}
6 changes: 2 additions & 4 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -44,15 +44,13 @@ ENV LANGUAGE="en_US:en"
ENV LC_ALL="en_US.UTF-8"
ENV TZ="Etc/UTC"

ENV CI="true"

RUN --mount=type=bind,source=scripts/prepare_image.sh,target=/prepare_image.sh \
/prepare_image.sh

COPY --from=rootfs / /

# use non-root user with sudo when needed
USER "${NON_ROOT_USER}:${NON_ROOT_USER}"
USER "${NON_ROOT_USER}"

WORKDIR "${AGENT_WORKDIR}"

Expand All @@ -70,4 +68,4 @@ ENV S6_SERVICES_GRACETIME="15000"
ENV S6_KEEP_ENV="1"

ENTRYPOINT [ "/entrypoint.sh" ]
CMD [ "jenkins-agent" ]
CMD []
156 changes: 134 additions & 22 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,38 +8,57 @@ A full fledged Docker in Docker image to act as a Jenkins Agent. Based on `ubunt
- Docker image: [`ghcr.io/felipecrs/jenkins-agent-dind`](https://github.com/felipecrs/jenkins-agent-dind/pkgs/container/jenkins-agent-dind)

> [!IMPORTANT]
> This image used to be uploaded to Docker Hub as `felipecrs/jenkins-agent` but it no longer is. Please update to the new tag `ghcr.io/felipecrs/jenkins-agent-dind:latest`.
> This image used to be uploaded to Docker Hub as `felipecrs/jenkins-agent` but it no longer is. Please update to the new tag `ghcr.io/felipecrs/jenkins-agent-dind`.
## Features

- Based on **Ubuntu 22.04 Jammy Jellyfish**: a more common OS to run your builds.
- Several common packages installed: run your builds without hassle.
- Fully working Docker in Docker: run your `docker build` commands isolated from the host Docker daemon.
- But also supports Docker on Docker: share the host's Docker daemon socket to avoid the overhead of running a nested Docker daemon. It is your choice.
- Act just as a Jenkins Agent out-of-the-box: run ephemeral build containers by using the Docker Plugin or Kubernetes Plugin on Jenkins. Works as the official `inbound-agent`.
- Includes [`pkgx`](https://pkgx.sh), a convenient package manager that allows you to easily install the necessary tools for your project. For example, you can use `pkgx install node@18` to install Node.js version 18.
- Facilitates debugging by providing an opt-in SSH server for your builds. Read more about it [here](#accessing-the-image-through-ssh).
- Facilitates debugging by providing an opt-in SSH server for your builds. Read more about it [here](#accessing-the-container-through-ssh).
- Can also be used as a [devcontainer](https://containers.dev/), ensuring both your development environment and your CI/CD environment are the same. Read more about it [here](#devcontainer).

## Usage

### Command line

Spin this agent in shell, if you want to play with it:
Spin this image in your terminal, if you want to play with it:

```sh
# -it: allows to interact with the container
# --rm: removes the container and its volumes after exiting
# --privileged: needed for running Docker in Docker
docker run -it --rm --privileged ghcr.io/felipecrs/jenkins-agent-dind bash
docker run -it --rm --privileged ghcr.io/felipecrs/jenkins-agent-dind
```

### Agent Template in Docker Cloud configuration on Jenkins
Alternatively, you can use the Docker on Docker mode:

```sh
# --volume: shares the host's Docker socket with the container
# --network=host: allows to access ports from other containers running on the host
docker run -it --rm --volume=/var/run/docker.sock:/var/run/docker.sock --network=host \
ghcr.io/felipecrs/jenkins-agent-dind
```

### Agent template with the [Docker Plugin](https://plugins.jenkins.io/docker-plugin/) on Jenkins

<details>
<summary>Click here to show</summary>

> [!WARNING]
> The image tag in this screenshot is outdated. The updated tag is `ghcr.io/felipecrs/jenkins-agent-dind:latest`.
> The image tag in this screenshot is outdated. The updated tag is `ghcr.io/felipecrs/jenkins-agent-dind`.
![Sample Agent Template configuration](https://user-images.githubusercontent.com/29582865/106769145-66379180-661b-11eb-93e3-5a7742eb46c0.png)

### Kubernetes Plugin Pod Template
</details>

### Pod template with the [Kubernetes Plugin](https://plugins.jenkins.io/kubernetes/) on Jenkins

<details>
<summary>Click here to show</summary>

The following is the Pod definition that you can use as a Pod template with the Kubernetes Plugin.

Expand All @@ -49,7 +68,7 @@ kind: Pod
spec:
containers:
- name: jnlp
image: ghcr.io/felipecrs/jenkins-agent-dind:latest
image: ghcr.io/felipecrs/jenkins-agent-dind
imagePullPolicy: Always
securityContext:
privileged: true
Expand All @@ -69,19 +88,27 @@ spec:
emptyDir: {}
```
### As a Jenkinsfile docker agent
</details>
When running as a Jenkinsfile docker agent, Jenkins will run the container as the host user instead of the default `jenkins` user.
### Jenkinsfile docker agent
But this image comes with [`fixuid`](https://github.com/boxboat/fixuid), which will automatically fix the user and group IDs of the `jenkins` user that comes with the image to match the host user.
<details>
<summary>Click here to show</summary>
This ensures file permissions are correct when running as a Jenkinsfile docker agent, as well as ensures `docker` from within the container still works.
When running as a `Jenkinsfile` docker agent, Jenkins will run the container as the host user instead of the default `jenkins` user.

This image comes with [`fixuid`](https://github.com/boxboat/fixuid), which will automatically fix the user and group IDs of the `jenkins` user that comes with the image to match the host user.

This ensures file permissions are correct when running as a `Jenkinsfile` docker agent, as well as ensures `docker` from within the container still works.

To run in Docker in Docker mode:

```groovy
// Jenkinsfile
pipeline {
agent {
docker {
image 'ghcr.io/felipecrs/jenkins-agent-dind:latest'
image 'ghcr.io/felipecrs/jenkins-agent-dind'
alwaysPull true
// --group-add=docker: is needed when using docker exec to run commands,
// which is what Jenkins does when running as a Jenkinsfile docker agent
Expand All @@ -98,19 +125,61 @@ pipeline {
}
```

### Accessing the image through SSH
Alternatively, you can use the Docker on Docker mode:

```groovy
// Jenkinsfile
pipeline {
agent {
docker {
image 'ghcr.io/felipecrs/jenkins-agent-dind'
alwaysPull true
args '--volume=/var/run/docker.sock:/var/run/docker.sock --network=host --group-add=docker'
}
}
stages {
stage('Verify docker works') {
steps {
sh 'docker version'
}
}
}
}
```

</details>

### Accessing the container through SSH

The image comes with SSHD installed and configured, but it does not start by default. To enable it, you need to add the `SSHD_ENABLED=true` environment variable when running the container.
<details>
<summary>Click here to show</summary>

This image comes with a SSH server installed and configured, but it comes disabled by default.

To enable it, you need to add the `SSHD_ENABLED=true` environment variable when running the container.

The SSHD server will run on port `22` and you can use the `jenkins` user to login, without any password.

#### Automatically expose SSH access for all builds
The image also comes with a convenience script at `/ssh-command/get.sh` that will output the SSH command to connect to the container, which you can use to connect to the container through SSH. Example:

The image comes with a convenience script at `/ssh-command/get.sh` that will output the SSH command to connect to the container, which you can use to connect to the container through SSH. Example:
```sh
docker run --rm -it --privileged \
-e SSHD_ENABLED=true \
-e NODE_NAME=$(hostname -I | awk '{ print $1 }') \
-e SSHD_PORT=2222 \
-p 2222:22 \
ghcr.io/felipecrs/jenkins-agent-dind \
/ssh-command/get.sh
```

![Example of SSH command](https://user-images.githubusercontent.com/29582865/203834385-1fb78d1d-5725-4074-8308-83a7b0ec818b.png)

##### Using Kubernetes Plugin
</details>

### Automatically expose SSH access for all builds with the Kubernetes Plugin

<details>
<summary>Click here to show</summary>

You can use a Kubernetes Pod Template to automatically expose SSH access for all builds.

Expand All @@ -131,7 +200,7 @@ metadata:
spec:
containers:
- name: jnlp
image: ghcr.io/felipecrs/jenkins-agent-dind:{{ $agentTag }}
image: ghcr.io/felipecrs/jenkins-agent-dind
imagePullPolicy: Always
env:
- name: SSHD_ENABLED
Expand Down Expand Up @@ -170,6 +239,7 @@ spec:
And here is an example of a Jenkinsfile:

```groovy
// Jenkinsfile
pipeline {
agent any
options {
Expand All @@ -188,11 +258,13 @@ pipeline {
It also works if you use a nested Docker agent:

```groovy
// Jenkinsfile
pipeline {
agent {
docker {
image 'felipecrs/fixdockergid:latest'
args '--volume=/ssh-command:/ssh-command --volume=/var/run/docker.sock:/var/run/docker.sock --group-add=docker'
image 'ghcr.io/felipecrs/jenkins-agent-dind'
alwaysPull true
args '--volume=/ssh-command:/ssh-command --volume=/var/run/docker.sock:/var/run/docker.sock --network=host --group-add=docker'
}
}
options {
Expand All @@ -208,9 +280,16 @@ pipeline {
}
```

##### Using as a Jenkinsfile docker agent
</details>

### Automatically expose SSH access for all builds as a Jenkinsfile docker agent

<details>
<summary>Click here to show</summary>

```groovy
// Jenkinsfile
// Generate an "unique" port for SSHD
env.SSHD_PORT = new Random(env.BUILD_TAG.hashCode()).nextInt(23000 - 22000) + 22000
Expand All @@ -235,3 +314,36 @@ pipeline {
}
}
```

</details>

### Devcontainer

It is a good practice to run your development environment in a container, so you can have a consistent environment across your team.

It is even better if you can run the same container in your CI/CD pipeline, so you can be sure that your build will behave the same way as your development environment.

As a Docker in Docker devcontainer:

```jsonc
// .devcontainer/devcontainer.json
{
"image": "ghcr.io/felipecrs/jenkins-agent-dind",
"overrideCommand": false,
"privileged": true
}
```

As a Docker on Docker devcontainer:

```jsonc
// .devcontainer/devcontainer.json
{
"image": "ghcr.io/felipecrs/jenkins-agent-dind",
"overrideCommand": false,
"mounts": [
"source=/var/run/docker.sock,target=/var/run/docker.sock,type=bind"
],
"runArgs": ["--network=host"]
}
```
22 changes: 18 additions & 4 deletions rootfs/entrypoint.sh
Original file line number Diff line number Diff line change
Expand Up @@ -8,16 +8,30 @@

set -eu

# Handle when no CMD is provided
if [[ $# -eq 0 ]]; then
# If JENKINS_URL is preset, assume we are running as a Kubernetes Pod template agent
if [[ -n "${JENKINS_URL:-}" ]]; then
set -- jenkins-agent
# Otherwise, if attached to a terminal, start a shell
elif [[ -t 0 ]]; then
set -- bash
# Otherwise, just keep the container running
else
set -- sleep infinity
fi
fi

uid="$(id -u)"
if [[ "${uid}" -eq 0 ]]; then
# If running as root, simply execute s6-overlay
export USER="root"
cmd=(/init_as_root)
set -- /init_as_root "$@"
else
# Otherwise, fix uid and gid, run s6-overlay as root and then drop
# privileges back to the user
export USER="${NON_ROOT_USER}"
cmd=(fixdockergid /init_as_root s6-setuidgid "${NON_ROOT_USER}")
export USER="${NON_ROOT_USER?}"
set -- fixdockergid /init_as_root s6-setuidgid "${NON_ROOT_USER}" "$@"
fi

exec -- "${cmd[@]}" "$@"
exec -- "$@"

0 comments on commit f61ab27

Please sign in to comment.