build: T6653: add build/manifest.json file

[c-po]

Change Summary

For a release workflow, we need to know the build artifact name.

We can save that data to build/manifest.json with a list of artifacts.

{
  "build_config": {
    "custom_apt_entry": [
      "deb [arch=amd64] https://repo.saltproject.io/py3/debian/11/amd64/3005 bullseye main"
    ],
    "custom_apt_key": [],
    "reuse_iso": null,
    "disk_size": 10,
    "build_by": "[email protected]",
    "build_comment": "",
    "build_type": "release",
    "architecture": "amd64",
    "debian_distribution": "bookworm",
    "debian_mirror": "http://deb.debian.org/debian",
    "debian_security_mirror": "http://deb.debian.org/debian-security",
    "debian_archive_areas": "main contrib non-free non-free-firmware",
    "vyos_mirror": "https://rolling-packages.vyos.net/current",
    "vyos_branch": "current",
    "release_train": "current",
    "kernel_version": "6.6.45",
    "bootloaders": "syslinux,grub-efi",
    "squashfs_compression_type": "xz -Xbcj x86 -b 256k -always-use-fragments -no-recovery",
    "website_url": "https://vyos.io",
    "support_url": "https://support.vyos.io",
    "bugtracker_url": "https://vyos.dev",
    "documentation_url": "https://docs.vyos.io/en/latest",
    "project_news_url": "https://blog.vyos.io",
    "additional_repositories": [
      "deb [arch=amd64] https://repo.saltproject.io/py3/debian/11/amd64/3005 bullseye main"
    ],
    "kernel_flavor": "amd64-vyos",
    "packages": [
      "qemu-guest-agent",
      "vyos-xe-guest-utilities",
      "grub2",
      "grub-pc",
      "vyos-linux-firmware",
      "vyos-intel-qat",
      "vyos-intel-ixgbe",
      "vyos-intel-ixgbevf",
      "mlnx-ofed-kernel-modules",
      "mlnx-tools",
      "openvpn-dco",
      "telegraf",
      "strace vim tmux git mc vyos-1x-smoketest",
      "hyperv-daemons",
      "vyos-1x-vmware"
    ],
    "image_format": [
      "iso"
    ],
    "architectures": {
      "amd64": {
        "packages": [
          "hyperv-daemons",
          "vyos-1x-vmware"
        ]
      }
    },
    "pbuilder_debian_mirror": "http://deb.debian.org/debian",
    "version": "1.5-foo-202408191937",
    "debug": false,
    "dry_run": false,
    "build_flavor": "generic",
    "build_dir": "build",
    "pbuilder_config": "build/pbuilderrc",
    "boot_settings": {
      "timeout": "5",
      "console_type": "tty",
      "console_num": "0",
      "console_speed": "115200",
      "bootmode": "normal"
    }
  },
  "artifacts": [
    "vyos-1.5-foo-202408191937-generic-amd64.iso"
  ],
  "pre_build_config": {
    "build_type": "release",
    "architecture": "amd64",
    "debian_distribution": "bookworm",
    "debian_mirror": "http://deb.debian.org/debian",
    "debian_security_mirror": "http://deb.debian.org/debian-security",
    "debian_archive_areas": "main contrib non-free non-free-firmware",
    "vyos_mirror": "https://rolling-packages.vyos.net/current",
    "vyos_branch": "current",
    "release_train": "current",
    "kernel_version": "6.6.45",
    "bootloaders": "syslinux,grub-efi",
    "squashfs_compression_type": "xz -Xbcj x86 -b 256k -always-use-fragments -no-recovery",
    "website_url": "https://vyos.io",
    "support_url": "https://support.vyos.io",
    "bugtracker_url": "https://vyos.dev",
    "documentation_url": "https://docs.vyos.io/en/latest",
    "project_news_url": "https://blog.vyos.io",
    "image_format": "iso",
    "packages": [
      "qemu-guest-agent",
      "vyos-xe-guest-utilities"
    ],
    "architectures": {
      "amd64": {
        "packages": [
          "hyperv-daemons",
          "vyos-1x-vmware"
        ]
      }
    },
    "build_by": "[email protected]",
    "pbuilder_debian_mirror": "http://deb.debian.org/debian",
    "version": "1.5-foo-202408191937",
    "build_comment": null,
    "debug": false,
    "dry_run": false,
    "custom_apt_entry": null,
    "custom_apt_key": null,
    "custom_package": [
      "strace vim tmux git mc vyos-1x-smoketest"
    ],
    "reuse_iso": null,
    "disk_size": null,
    "build_flavor": "generic"
  }
}

And get the resulting image (ISO) name using:

helix:~/vyos-build/build [artifacts-T6653] # jq -r .artifacts[0] manifest.json
vyos-1.5-foo-202408191937-generic-amd64.iso

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Code style update (formatting, renaming)
• Refactoring (no functional changes)
• Migration from an old Vyatta component to vyos-1x, please link to related PR inside obsoleted component
• Other (please describe):

Related Task(s)

• https://vyos.dev/T6653

Component(s) name

build scripts

Checklist:

• I have read the CONTRIBUTING document
• I have linked this PR to one or more Phabricator Task(s)
• My commit headlines contain a valid Task id
• My change requires a change to the documentation
• I have updated the documentation accordingly

[c-po]

@Mergifyio backport sagitta circinus

[github-actions]

👍
No issues in PR Title / Commit Title

[mergify]

backport sagitta circinus

✅ Backports have been created

• #746 build: T6653: add build/manifest.json file (backport #736) has been created for branch sagitta
• #747 build: T6653: add build/manifest.json file (backport #736) has been created for branch circinus

[Crushable1278]

Hello Mr. c-po,

This PR in its current state causes a failure exit code due to the build_artifact key not being present in the manifest dict.
Additionally, it will not capture any of the non-iso, non-raw file formats.

Error:

Traceback (most recent call last):
  File "/vyos/./build-vyos-image", line 654, in <module>
    manifest['build_artifact'].append(raw_image)
    ~~~~~~~~^^^^^^^^^^^^^^^^^^
KeyError: 'build_artifact'

If you change

               cmd(f"qemu-img convert -f raw -O {f} {image_opts} {raw_image} {target}")

        manifest['build_artifact'].append(raw_image)

to

                cmd(f"qemu-img convert -f raw -O {f} {image_opts} {raw_image} {target}")
                manifest['artifacts'].append(target)

        manifest['artifacts'].append(raw_image)

at the end of the script, I believe this works the way it was intended.

For example:

# cat build/manifest.json  | jq .artifacts
[
  "vyos-MyISO-amd64.iso",
  "vyos-MyISO-amd64.qcow2",
  "vyos-MyISO-amd64.vdi",
  "vyos-MyISO-amd64.vhdx",
  "vyos-MyISO-amd64.raw"
]

[c-po]

@Crushable1278 highly appreciated. Thanks - last minute changes are usually bad.

Discussed in Maintainer Meeting