Distrust the Infrastructure

[newbieAtGithub]

hi @tlaurion

do Heads & Core-boot distrust the infrastructure,
by signing all commit, before being pushed to Github ?

also during build by Circle CI,
does it verify each file first before build ?

thanks and regards,

[tlaurion]

You basically asked the same question under #111 #112 #113 #114 which, to me, are all subproblems of this issue: distrusting everything.

All commits are signed, you can review them on github, which is not specific to Heads project. Yes, those are signed before being pushed to github, where I sometimes push commits to heads-wiki directly from githuh GUI since its more convenient sometimes, but not for code changes. Github permits to falsely "sign" commits through github if user also has a public key pushed to github. Not sure of how that is done, but prior of those commits to be pushed and merged, I ask contributors to sign their commits and do myself.

also during build by Circle CI,
does it verify each file first before build ?

Take a look under modules/ directory over Heads main github.

There, you will see that modules specify either a version, or a commit id for github projects and then provide a checksum to verify against the tarball having been downloaded. Coreboot works the same. And also reauires signed commits.

Scripts under blobs include the same concept embedded into the scripts themselves and hashes to be validated against prior of continuing.

Now.

Local builds or Circleci goes through the same process and relies against git to clone Heads code repository.

This means that Heads, for a specific commit, will contain the same codebase unless modified prior of being compiled. If that happens, "-dirty" will be appended to the ROM name, clearly stating that the codebase that was used to build a ROM was not "clean" for that specific commit, also appended in the ROM name.

So yes. A ROM for a specific board has a commit id embedded in its filename, as documented in both build and download guide.

The point, here again, is that since Heads has reproducibility issues still, you cannot expect to build the same bit by not reproducible ROM as of today, as justified by still opened reproducibility issues in the regard. As said in other tickets, for the moment the issue (from memory) is linked to busybox, which impacts initrd and final rim images.

But on a same OS, same user, same buildstack (non-updated qubes template, for example) you should be able to reproduce the same boards ROMs for the same commit id.

If we were reusing a prebuilt debian-11 docker image today to build on circleci without updating the packages prior of building, we should be able to reproduce the same ROM, reusing the same docker image years from now, if of course the same tarballs can still be downloaded from the same URLs hardcoded under modules/

This also was a problem in the past.
So the idea here is to

• build a docker image in a reproducible way.
• change the build system to depend on it
• document its usage

And then, locally or through CircleCi, we should be able to build the same (bit by bit) reproducible ROMs based on same signed commits to produce the same libraries, binaries and final ROMs.

Until that moment, if you download from CircleCi, you trust CircleCI to not have been stopped in the middle if the builds to have a script modified.

Until reproducibility issues are resolved, the only total trust you could have is by building the ROMs yourself.

The end goal would he to have CircleCI raise issues for each commit in a separate project or something similar, that would give the ROM hashes for each compiled board image. With people able to report if what they built is different for whatever reason.

Hope that that answers your question.

[newbieAtGithub]

hi @tlaurion

thanks for explanation, i understand it better now,

i see that we have several infrastructures:

1. Development machine used by Heads' developers
2. Network between development machine & Github repo
3. Github repository
4. Network between Github & Circle CI
5. Circle CI
6. Network between Circle CI & Heads' user
7. Raspberry PI / Laptop / USB thumb drive
8. Target Laptop

and from your explanation above, we know how to distrust:

1. no option, but to trust Heads' developers & their development machine
2. network is distrusted, by signing commit
3. Github is distrusted, by signing commit
4. network is distrusted, by signing commit
5. Circle CI or Local Build, are distrusted, by requiring a signed commit & checksum verification
   Failure, in providing signed commit, or checksum verification, will result in "-dirty" ROM name,
   Success, in providing signed commit & checksum verification, will result in "-commit ID" ROM name
6. network is distrusted, by checksum verification, between:
   Circle CI generated hashes.txt, downloaded ROM & extracted ROM
7. Flashing media & tool (Rpi, Laptop, USB thumb drive), are distrusted, by checksum verification, means:
   we need to take backup of flashed image, either internally or externally,
   then perform checksum verification, in other laptop, between:
   Circle CI generated hashes.txt, backup-ed ROM & extracted ROM

Since there are reproducibility issues, Heads' build between Circle CI & local,
will produce different checksum, which mean both will produce hashes.txt, that contain different checksum,
therefore cannot compare / verify / validate local build, by using hashes.txt from Circle CI,
the only verification available, are the ROM name generated. -dirty vs -commit ID

is it complete & correct, or do i miss some points ?

thanks & regards,

[tlaurion]

I think this overcomplexify the problem and solution space.

Commits are signed, part of a pull request, and reviewed prior of merging, with regression testing. This is common to all open source projects.

One clones the Heads repo over CircleCI/locally. One starts a build. One can inspect hashes.txt locally vs the one over CircleCI. Realize which modules are not reproducible today and read issues on the matter.

One now has a ROM and hashes.txt he puts on USB thumb drive to upgrade internally/flashes externally. One not trusting his programmer(to modify what intentionally?) can reflash firmware internally.

You could rebuild the same ROM on same build machine (with same host build tools) and should expect the same hash for the final ROM. As explained in other issues, the problem of ROM reproducibility is that host environments differences can bleed in the builds. We would not have reproducibility issues if we were all using the same host to produce the ROMs in the same directories, using the same exact tools to produce the roms.

So at the end of the process, hashes.txt shows you individual hashes of everything packed in the firmware and for the firmware image itself:

• heads.cpio contains all the scripts and configs under initrd dir of github
• tools.cpio contains all the compiled binaries and libraries
• modules.cpio contains all kernel modules that were packed for a board
• all the above are compressed under initrd.cpio.xz
• coreboot does its stitching and includes the kernel and initrd as its payload.

hashes.txt gives hashes for all of those. At the end, a final hash is for the final ROM image. If that one is the same, it means that all the parts inside of it are still good.

Unfortunately, if you take a backup of your ROM from Heads, it will contain MRC cache, your gpg public keyring and config.user override. This ROM won't be exactly the same bit by bit copy of the ROM you initially flashed. This is why it is recommended to backup the ROM for inspection (all files under hashes.txt should still match) where reflashing the same firmware internally, keeping settings, should result in the same exact measurements.

The hashes.txt file is generated as part of Heads build process. It doesn't permit to distrust the infrastructure completely altogether, but it permits to isolate what file is not in expected state.

hashes.txt could and should be checked inside of Heads recovery shell as well.

[newbieAtGithub]

hi @tlaurion

okay, i think the information is complete & clear enough,
thanks a lot for explaining,

regards,