Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make imagecraft build an image #53

Merged
merged 20 commits into from
Aug 9, 2024
Merged
Show file tree
Hide file tree
Changes from 17 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 8 additions & 1 deletion docs/.custom_wordlist.txt
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,8 @@ classmethod
classvar
cli
CLI
cloud-init
Cloud-init
cls
cmake
CMake
Expand Down Expand Up @@ -148,6 +150,7 @@ implementers
IncompatibleSourceOptions
infos
ing
init
initialized
InvalidAction
InvalidActionException
Expand Down Expand Up @@ -318,6 +321,7 @@ PythonPlugin
PythonPluginProperties
PyYAML
PyYAML's
qemu
qmake
QmakePlugin
QmakePluginProperties
Expand Down Expand Up @@ -365,6 +369,7 @@ SnapInstallError
SnapPackage
SnapRefreshError
SnapSource
Snapstore
SnapUnavailable
SourceError
SourceHandler
Expand Down Expand Up @@ -397,6 +402,7 @@ Subpackages
subprocess
Subtractive
subtree
sudo
summarized
svn
symlink
Expand All @@ -411,6 +417,7 @@ Tox
txt
TypeError
ubuntu
UEFI
umount
Umount
unbuilt
Expand All @@ -428,8 +435,8 @@ unregistering
uri
URI
url
urls
URL's
urls
utilizing
utils
ValidationError
Expand Down
2 changes: 1 addition & 1 deletion docs/custom_conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -120,7 +120,7 @@
############################################################

# Links to ignore when checking links
linkcheck_ignore = ["http://127.0.0.1:8000"]
linkcheck_ignore = ["http://127.0.0.1:8000", "https://apt-repo.com"]

# Pages on which to ignore anchors
# (This list will be appended to linkcheck_anchors_ignore_for_url)
Expand Down
112 changes: 112 additions & 0 deletions docs/howto/basic_image.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,112 @@
.. _howto_buld_basic_image:

===================
Build a basic image
===================

These instructions describe how to build a ``classic`` server 24.04 AMD64 Ubuntu image with Imagecraft.

Prerequisites
-------------

- AMD64 machine with Ubuntu 18.04 or newer
- ``snapd`` installed
- 15GiB or more disk space to process the build and hold the resulting image

.. note:: Following instructions build an **AMD64** image on an **AMD64** machine. Building on another architecture would need several modifications not described on this page.

Install Imagecraft
~~~~~~~~~~~~~~~~~~

Imagecraft is only available as a snap in the `latest/edge` channel `from the Snapstore <https://snapcraft.io/imagecraft>`_. Install it with:

.. code-block::

sudo snap --classic --edge imagecraft
upils marked this conversation as resolved.
Show resolved Hide resolved

Verify that ``imagecraft`` is properly installed:

.. code-block::

imagecraft

.. important:: ``imagecraft`` requires **elevated permissions**. Run it with **root** privileges or using ``sudo``.


Prepare the configuration
-------------------------

Imagecraft configuration
~~~~~~~~~~~~~~~~~~~~~~~~

Save the following content as ``imagecraft.yaml``:

.. literalinclude:: code/basic_imagecraft.yaml
:language: yaml

.. note:: The name of the configuration file, ``imagecraft.yaml``, is **important** because Imagecraft uses it automatically.


Cloud-init configuration
~~~~~~~~~~~~~~~~~~~~~~~~

Prepare needed directories:

.. code-block::

mkdir -p cloud-init/var/lib/cloud/seed/nocloud
mkdir -p cloud-init/etc/cloud/cloud.cfg.d/

Write the following files in the ``cloud-init`` directory:

- ``cloud-init/var/lib/cloud/seed/nocloud/meta-data``

.. literalinclude:: code/cloud-init/meta-data
:language: yaml

- ``cloud-init/var/lib/cloud/seed/nocloud/user-data``

.. literalinclude:: code/cloud-init/user-data
:language: yaml

- ``cloud-init/etc/cloud/cloud.cfg.d/90_dpkg.cfg``

.. literalinclude:: code/cloud-init/90_dpkg.cfg
:language: yaml


Build the image
------------------------

.. code-block::

sudo imagecraft pack --destructive-mode

The resulting image file, ``pc.img``, is available in the current directory.


Run the image
--------------

Test the resulting image with QEMU.

Copy UEFI variables to a temporary directory:

.. code-block::

cp /usr/share/OVMF/OVMF_VARS.fd /tmp/OVMF_VARS.fd

Boot the resulting image with QEMU:

.. code-block:: none

qemu-system-x86_64 \
-accel kvm \
-m 2G \
-cpu host \
-smp 4 \
-drive if=pflash,format=raw,readonly=on,file=/usr/share/OVMF/OVMF_CODE.fd \
-drive if=pflash,format=raw,file=/tmp/OVMF_VARS.fd \
-drive file=pc.img,format=raw,index=0,media=disk

You should be able to log in with the user name and password defined in the cloud-init configuration.
43 changes: 43 additions & 0 deletions docs/howto/code/basic_imagecraft.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
name: ubuntu-server-amd64
version: 1
base: [email protected]
series: noble
platforms:
amd64:
build-for: [amd64]
build-on: [amd64]

package-repositories:
- type: apt
components: [main, restricted, universe, multiverse]
url: http://archive.ubuntu.com/ubuntu/
pocket: updates
used-for: build

parts:
gadget:
plugin: gadget
source: https://github.com/snapcore/pc-gadget.git
source-branch: classic
rootfs:
plugin: ubuntu-seed
ubuntu-seed-pocket: updates
ubuntu-seed-germinate:
urls:
- "git://git.launchpad.net/~ubuntu-core-dev/ubuntu-seeds/+git/"
branch: noble
vcs: true
names:
- server
- minimal
- standard
- cloud-image
ubuntu-seed-kernel: linux-image-generic
ubuntu-seed-extra-snaps: [snapd]
stage:
- -rootfs/etc/cloud/cloud.cfg.d/90_dpkg.cfg
cloud-init:
plugin: dump
source: cloud-init/
organize:
- '*': rootfs/
2 changes: 2 additions & 0 deletions docs/howto/code/cloud-init/90_dpkg.cfg
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
# to update this file, run dpkg-reconfigure cloud-init
datasource_list: [ NoCloud ]
2 changes: 2 additions & 0 deletions docs/howto/code/cloud-init/meta-data
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
dsmode: local
instance_id: ubuntu-server
7 changes: 7 additions & 0 deletions docs/howto/code/cloud-init/user-data
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
#cloud-config
chpasswd:
expire: true
users:
- name: ubuntu
password: ubuntu
type: text
2 changes: 2 additions & 0 deletions docs/howto/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -5,3 +5,5 @@ How-to guides

.. toctree::
:maxdepth: 1

basic_image
34 changes: 12 additions & 22 deletions docs/reference/code/example/imagecraft.yaml
Original file line number Diff line number Diff line change
@@ -1,53 +1,43 @@
name: ubuntu-server-amd64
version: 22.04
series: jammy
version: 1
base: [email protected]
series: noble
platforms:
amd64:
build-for: [amd64]
build-on: [amd64]

package-repositories:
- type: apt
components: [main, restricted]
components: [main, restricted, universe, multiverse]
url: http://archive.ubuntu.com/ubuntu/
flavor: ubuntu
pocket: proposed
pocket: updates
used-for: build
- type: apt
components: [main, multiverse]
used-for: run
- type: apt
ppa: canonical-foundations/ubuntu-image
used-for: build
- type: apt
ppa: canonical-foundations/ubuntu-image-private-test
auth: "username:password"
used-for: run

parts:
gadget:
plugin: gadget
source: https://github.com/snapcore/pc-gadget.git
source-branch: classic
gadget-type: git
gadget-target: server
rootfs:
plugin: ubuntu-seed
ubuntu-seed-pocket: updates
ubuntu-seed-germinate:
urls:
- "git://git.launchpad.net/~ubuntu-core-dev/ubuntu-seeds/+git/"
branch: jammy
branch: noble
vcs: true
names:
- server
- minimal
- standard
- cloud-image
ubuntu-seed-kernel: linux-image-generic
ubuntu-seed-extra-snaps: [core20, snapd]
ubuntu-seed-extra-packages: [hello-ubuntu-image-public]
ubuntu-seed-extra-snaps: [snapd]
stage:
- -etc/cloud/cloud.cfg.d/90_dpkg.cfg
- -rootfs/etc/cloud/cloud.cfg.d/90_dpkg.cfg
cloud-init:
plugin: dump
source: cloud-config/
source: cloud-init/
organize:
- '*': rootfs/
19 changes: 0 additions & 19 deletions imagecraft/application.py
Original file line number Diff line number Diff line change
Expand Up @@ -16,9 +16,6 @@

"""Main Imagecraft Application."""

from copy import deepcopy
from typing import Any

from craft_application import Application, AppMetadata, util
from overrides import override # type: ignore[reportUnknownVariableType]

Expand Down Expand Up @@ -52,19 +49,3 @@ def _configure_services(self, platform: str | None, build_for: str | None) -> No
platform=platform,
build_for=build_for,
)

@override
def _extra_yaml_transform(self, yaml_data: dict[str, Any]) -> dict[str, Any]:
# TODO: Apply extensions to the yaml

# The method documentation says to return a new yaml dict
new_yaml = deepcopy(yaml_data)

# Make sure not to stage or prime the gadget, as it should not be
# part of the rootfs - if the gadget is defined in the project.
if "gadget" in new_yaml["parts"]:
gadget = new_yaml["parts"]["gadget"]
gadget["override-stage"] = "true"
gadget["override-prime"] = "true"

return new_yaml
6 changes: 4 additions & 2 deletions imagecraft/image_definition.py
Original file line number Diff line number Diff line change
Expand Up @@ -106,6 +106,7 @@ class Rootfs(BaseModel):
pocket: str
mirror: str | None
seed: Seed
sources_list_deb822: bool

class Config:
"""Pydantic model configuration."""
Expand All @@ -121,7 +122,7 @@ class ImageDefinition(BaseModel):

name: str
display_name: str
revision: str
revision: int
class_: str = Field(alias="class")
architecture: str
series: str
Expand All @@ -141,7 +142,7 @@ class Config:
def __init__( # noqa: PLR0913
self,
series: str,
revision: str,
revision: int,
architecture: str,
pocket: str,
kernel: str | None,
Expand Down Expand Up @@ -177,6 +178,7 @@ def __init__( # noqa: PLR0913
names=seed_names,
pocket=seed_pocket,
),
sources_list_deb822=True, # Always set it to true for now
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We'll have to figure out how to approach this from the imagecraft POV. Maybe simply keying it on the base would be sufficient.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, this is "hack" for now. The "base" thing is a really tricky thing because the craft framework is expecting this base to exists and to be usable to build. How do we build a newer series on an older one in this case?

But otherwise I agree we could probably look at some value in the provided YAML and decide for the user if this should be set True/False.

Copy link
Collaborator Author

@upils upils Aug 8, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or we could use the version field?

),
)

Expand Down
11 changes: 11 additions & 0 deletions imagecraft/models/project.py
Original file line number Diff line number Diff line change
Expand Up @@ -153,6 +153,17 @@ def _validate_package_repositories(
validate_package_repositories(repositories)
return repositories

@validator("version") # pyright: ignore[reportUntypedFunctionDecorator]
@classmethod
def _validate_version(cls, version: str) -> str:
upils marked this conversation as resolved.
Show resolved Hide resolved
try:
int(version)
except ValueError as e:
raise CraftValidationError(
"Invalid version",
details="value is not an integer.",
) from e
return version

@validator("platforms", pre=True) # pyright: ignore[reportUntypedFunctionDecorator]
@classmethod
Expand Down
Loading
Loading