diff --git a/Boot.md b/Boot.md new file mode 100644 index 0000000..4f1ddf8 --- /dev/null +++ b/Boot.md @@ -0,0 +1,73 @@ + +Boot config files +=== + +A user has the option to make persistent modifications to the non-Qubes boot process by creating one or more of the following files: + +| file | description | +| ---- | ---- | +| kexec_menu.txt | contains multiple options for parameters to the kexec command | +| kexec_hashes.txt | a sha256sum file from within the respective boot directory | +| kexec_iso_add.txt | a sh variable to override the standard ISO kernel argument additions | +| kexec_iso_remove.txt | a sh variable to override the standard ISO kernel argument removals | +| kexec_default.$N.txt | specifies the default kexec parameters corresponding to the Nth menu option | +| kexec_default_hashes.txt | a sha256sum file for the default entry kexec file parameters | +| kexec_rollback.txt | a sha256sum of the TPM counter contents in the tmp directory | +| kexec_key_devices.txt | contains a list of "device uuid" combos for all LUKS devices to unlock | +| kexec_key_lvm.txt | contains the name of an LVM group to activate on boot | + +These can be placed in any of the following locations: + +| location | description | +| ---- | ---- | +| /boot/ | used during internal HD boot | +| /media/ | used during standard USB boot | +| /media/kexec_iso/$ISO_FILENAME/ | used during USB boot from a particular ISO file | + +These files are only used if there is an appropriate signature for them in `kexec.sig` covering all `kexec*.txt` in that location. This can be generated by running `kexec-sign-config -p /boot/`, etc. These files are copied by `kexec-check-config` to `/tmp/kexec/` only there's a valid signature. From there the boot routines reference only the configs in `/tmp/kexec`. + +If there is no persistent `kexec_menu.txt`, the boot directory will be searched for grub/syslinux-like configurations and it will be generated on-the-fly (for any of the HD/USB/USB-ISO locations). Creating a persistent `kexec_menu.txt` can be useful to limit the options displayed or to make custom persistent alterations to xen or kernel params. + +`kexec_menu.txt` has a simple layout with a single line per boot option: + +``` +description 1|elf|kernel /vmlinuz... |initrd /initramfs... |append ... +description 2|multiboot|kernel ... |module ... |module ... +description 3|xen|kernel /xen... |module /vmlinuz... | module /initramfs... +``` + +This is a sample `kexec_menu.txt` covering the expected options (derived from grub.cfg): + +``` +Ubuntu|elf|kernel /vmlinuz-4.8.0-58-generic|initrd /initrd.img-4.8.0-58-generic|append root=/dev/mapper/ubuntu--vg-root ro quiet splash crashkernel=384M-:128M crashkernel=384M-:128M +Memory test (memtest86+, serial console 115200)|elf|kernel /memtest86+.bin|append console=ttyS0,115200n8 +Qubes, with Xen hypervisor|multiboot|kernel /xen-4.6.5.gz placeholder |module /vmlinuz-4.4.67-13.pvops.qubes.x86_64 placeholder root=/dev/mapper/luks-UUID ro rd.qubes.hide_all_usb|module /initramfs-4.4.67-13.pvops.qubes.x86_64.img +``` + +If there is a persistent `kexec_hashes.txt`, a non-default boot will fail when the file hashes don't match the expected values. By default, no such checks are made. + +When booting from an ISO file on a USB drive, it must be signed by a valid key in the Heads ROM and the boot process will fail if invalid. The `kexec_iso_add.txt` and `kexec_iso_remove.txt` are useful to inject the appropriate kernel arguments to allow it to load properly. ISOs for Debian require that `kexec_iso_add.txt` contains to load properly: + +``` +findiso=${ISO_PATH} +``` + +Take a look at http://mbusb.aguslr.com/howto.html for more variations on the distro-specific ISO mounting command lines requirements. By default Heads uses two variants of this when booting from ISO where a `kexec_iso_add.txt` is not specified: + +``` +fromiso=/dev/disk/by-uuid/$DEV_UUID/$ISO_PATH iso-scan/filename=/${ISO_PATH} +``` + +Note that currently, any multiboot entry is interpreted as a Xen-variant and `kexec-boot` overrides the arguments to the multiboot kernel with custom arguments. A user can manually specify `multiboot` entries to override the default behavior by creating a custom `kexec_menu.txt`. + +If a user wishes to require that file hashes be checked for a succesful non-recovery boot, they may set the `CONFIG_BOOT_REQ_HASH=y` in their respective Heads config file. + +As as convenience mechanism, a user may select a boot option to always be used in the future, assuming that the boot parameters and file hashes have not changed. This can be done by running `kexec-save-default` manually or directly from the boot menu. This works for any boot location (HD/USB/USB ISO) but does modify the respective `/boot/` or `/media/` filesystems. An entry index is maintained so that if the options are being derived from the live `grub.cfg` (i.e. no persistent `kexec_menu.txt`) and when there is a change to the underlying grub parameters, the boot will fail and require the user to resign/revalidate the settings. This is useful to detect changes to the primary kernel/initramfs (for example in the Qubes case when the primary entry is first). + + +If a user wishes to require that a TPM counter be set for rollback prevention, they may set the `CONFIG_BOOT_REQ_ROLLBACK=y` in their respective Heads config file. When this is true, standard boot will only succeed if: + +1) Booting from an verified ISO +2) Booting from a mount point that has a valid `kexec_rollback.txt` in its parameter directory + +The simplest way to achieve this is to set a default boot option as this updates the rollback counter by default. diff --git a/Emulating-Heads.md b/Emulating-Heads.md index 2a79680..e198ab5 100644 --- a/Emulating-Heads.md +++ b/Emulating-Heads.md @@ -1,17 +1,17 @@ Generate the `qemu.rom` image: ``` -make BOARD=qemu +make BOARD=qemu-coreboot ``` Boot it in qemu: ``` -qemu-system-x86_64 -machine q35 -bios qemu.rom +build/make-4.2/make BOARD=qemu-coreboot run ``` Issues with emulation: * TPM is not available * Xen won't start dom0 correctly, but it is sufficient to test that the `initrd.cpio` file was correctly generated * This also lets us test Xen patches for legacy-free systems -* SATA controller sometimes takes minutes to timeout? \ No newline at end of file +* SATA controller sometimes takes minutes to timeout? diff --git a/Installing-Heads.md b/Installing-Heads.md index 95f6376..c81d7c8 100644 --- a/Installing-Heads.md +++ b/Installing-Heads.md @@ -32,31 +32,35 @@ Using a chip clip and a [SPI programmer](https://trmm.net/SPI_Flash), dump the e ![Flashing x230 SPI flash](https://farm3.staticflickr.com/2889/33186538873_c1290ca6ec_z_d.jpg) -Ok, now comes the time to write the 4MB `x230.coreboot.bin` file to SPI2 chip. With my programmer and minicom, I hit i to verify that the flash chip signature is correctly read a few times, and then send `u0 400000`↵ to initiate the upload. I then drop to a shell with Control-A J and finally send the file with `pv x230.rom > /dev/ttyACM0`↵. A minute later, I resume minicom and hit i again to check that the chip is still responding. +Ok, now comes the time to write the 4MB `x230-flash.bin` file to SPI2 chip (or `build/x230-flash/coreboot.rom` if you've built it locally). With my programmer and minicom, I hit i to verify that the flash chip signature is correctly read a few times, and then send `u0 400000`↵ to initiate the upload. I then drop to a shell with Control-A J and finally send the file with `pv x230.rom > /dev/ttyACM0`↵. A minute later, I resume minicom and hit i again to check that the chip is still responding. Move the clip to the SPI1 chip and flash the 8 MB `x230.me.bin` (TODO: document how to produce this with me cleaner -> [Clean the ME firmware](Clean-the-ME-firmware)). This time you'll send the command `u0 800000`↵. This will wipe out the official Intel firmware, leaving only a stub of it to bring up the Sandybridge CPU before shutting down the ME. As far as I can tell there are no ill effects other than an inability to power off the machine without using the power switch. Finally, remove the programmer, connect the power supply and try to reboot. -If all goes well, you should see the keyboard LED flash, and within a second the Heads splash screen appear. It currently drops you immediately into the shell, since the boot script portion has not yet been implemented. If it doesn't work, well, sorry about that. Please let me know what the symptoms are or what happened during the flashing. +If all goes well, you should see the keyboard LED flash, and within a second the Heads recovery splash screen will appear. It currently drops you immediately into the shell, to allow you to flash the full 12MB `x230.bin` Heads rom (or `build/x230/coreboot.rom` if you've built it locally). If it doesn't work, well, sorry about that. Please let me know what the symptoms are or what happened during the flashing. -Congratulations! You now have a Coreboot + Heads Linux machine. Adding your own signing key, installing Qubes and configuring tpmtotp are the next steps and need to be written. +Congratulations! You now have a Coreboot + Heads Linux machine. Adding your own signing key, installing Qubes and configuring tpmtotp are the next steps. -Adding your PGP key +On insert a USB drive containing the 12MB Heads rom and mount it using: + +``` +mount-usb +``` + +This will load the USB kernel modules and mount your drive at `/media`. + +Generating your PGP key === -Heads uses your own GPG key to sign updates and as a result it needs the -key stored in the `initrd.cpio` file that is built as part of the ROM image. -In the future ([issue #182](https://github.com/osresearch/heads/issues/182)) -it might be possible to install your own key from inside the recovery -shell; this would be preferable for many reasons. +If you're using a new Yubikey, you'll need to generate your key files. If you +already have the public and private key stubs for your Yubikey, please proceed +to the next section. -Insert your Yubikey into the build machine and in the `heads` build directory, -clean out the old files, then invoke GPG's the "Card Edit" function with +Insert your Yubikey into the x230, then invoke GPG's the "Card Edit" function with it targetting the local directory: ``` -rm ./initrd/.gnupg/* -gpg --card-edit --homedir ./initrd/.gnupg +gpg --homedir=/media/gnupg/ --card-edit ``` Go into "Admin" mode and generate a new key inside the Yubikey: @@ -70,10 +74,44 @@ Since this key can be replaced by replacing the ROM, it is not necessary to make a backup unless you want to. This will prompt you for the admin pin (`12345678` by default) and then the existing pin (`123456`). Follow the other prompts and eventually -you should have a key in `initrd/.gnupg/` +you should have a key in `/media/gnupg/`. + +Create a single file containing the public key and the private key stubs +for this Yubikey (the secret key lives only in the Yubikey). + +``` +gpg --homedir=/media/gnupg/ --export -a > /media/gnupg/public.key +gpg --homedir=/media/gnupg/ --export-secret-keys -a > /media/gnupg/private_stub.key +``` + +Adding your PGP key +=== +Heads uses your own GPG key to sign updates and as a result it needs the +key stored in the ROM image before flashing the full Heads ROM. + +Add your key to the Heads ROM using the following command: +``` +cbfs -o /media/x230.rom -a "heads/initrd/.gnupg/keys/public.key" -f /media/gnupg/public.key +cbfs -o /media/x230.rom -a "heads/initrd/.gnupg/keys/private_stub.key" -f /media/gnupg/private_stub.key +``` + +Any name can be used as long as the it is preceded by `heads/initrd/.gnupg/keys/` +and you can used a combined the public and private stubs (by `cat`ing them together). + +After these files are added to the `/media/x230.rom`, you should flash the full ROM: +``` +flashrom-x230.sh /media/x230.com +``` +Once `flashrom` is complete, reboot (using the `reboot` command) +and now you should now be back in the Heads runtime. It should +display a message that is is unable to unseal TOTP. + +Because the reproducible flash has an empty MRC cache, you need to +reboot one more time so that the PCR values as they would be going +forward. Configuring the TPM === @@ -94,7 +132,7 @@ tpmtotp ![TPMTOTP QR code](https://pbs.twimg.com/media/Cr8x7f6WEAEbBdq.jpg) -Once you own the TPM and have the final version of the `x230.rom` flashed, run `seal-totp` to generate a random secret, seal it with the current TPM PCR values and store the sealed value in the TPM's NVRAM. This will generate a QR code that you can scan with your google authenticator application and use to validate that the boot block, rom stage and Linux payload are un-altered. +Once you own the TPM, run `seal-totp` to generate a random secret, seal it with the current TPM PCR values and store the sealed value in the TPM's NVRAM. This will generate a QR code that you can scan with your google authenticator application and use to validate that the boot block, rom stage and Linux payload are un-altered. ![TPMTOTP output](https://farm8.static.flickr.com/7564/28580109172_5bd759f336.jpg) @@ -102,17 +140,60 @@ On the next boot, or if you run `unseal-totp`, the script will extract the seale This does not eliminate all firmware attacks (such as evil maid ones that replace the SPI flash chip), but when combined with the WP# pin and BP bits should eliminate a software only attack. +Generic OS Installation +=== + +1. Insert OS installation media into one of the USB3 ports (on the left side) + +The Heads boot process supports standard OS bootable media (where the USB drive contains the installation media which as created using `dd` or `unetbootin` etc.) as well as booting directly from verified ISOs on a plain old partition. + +For example, if the USB drive has a single partition, you can put the ISO image along with a trusted signature in the root directory: + +``` +/Qubes-R4.0-x86_64.iso +/Qubes-R4.0-x86_64.iso.asc +/Fedora-Workstation-Live-x86_64-27-1.6.iso +/Fedora-Workstation-Live-x86_64-27-1.6.iso.sig +/tails-amd64-3.7.iso +/tails-amd64-3.7.iso.sig +``` + +Each ISO is verified before booting so that you can be sure Live distros and installation media are not tammpered with, so this route is preferred when available. You can also sign the ISO with your own key: + +``` +gpg --output .sig --detach-sig +``` + +Some distros require additional options to boot properly directly from ISO. See [Boot config files](/Boot.md) for more information. + +2. Boot from USB by either running `usb-scan` or reboot into USB boot mode (hit 'u' before the normal boot) + + +3. Select the install boot option for your distro of choice and work through the standard OS installation procedures (including setting up LUKS disk encryption if desired) + + +4. Reboot and your new boot options should be available to be chosen by selecting 'm' at the boot screen + +If you want to set a default option so that you don't have to choose at every boot, you can do so from the menu by selecting 'd' on the confirmation screen. You will also be able to seal your disk encryption key using the TPM allowing you to use ensure only a boot password and the proper PCR state can unlock this yet. + +(\*) Ubuntu/Debian Note: These systems don't read `/etc/crypttab` in their initrd, so you need to adjust the crypttab in the OS and `update-initramfs -u` to have it attempt to use the injected key. Due to oddities in the cryptroot hooks, you also need keyscript to be in `/etc/crypttab` even as a no-op `/bin/cat`: + +`sda5_crypt UUID=XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX /secret.key luks,keyscript=/bin/cat` + +(Credit to http://www.pavelkogan.com/2015/01/25/linux-mint-encryption/ for this trick). + + Installing Qubes === ![Heads splash screen](https://farm3.staticflickr.com/2890/33999478295_5738432e82_z_d.jpg) -Boot into the recovery shell (hit 'r' at the prompt before the normal startup script tries to run) and plug the USB stick with the R3.2 install media into one of the USB3 ports (on the left side of the x230) and run the helper script to start the Qubes installer: +Plus in the USB stick with the R4.0 install media into one of the USB3 ports (on the left side of the x230) and boot into USB mode (hit 'u' at the prompt), then boot using this option: ``` -qubes-install +2. Install Qubes R4.0 [kernel /isolinux/xen.gz console=none] ``` -If that completes with no errors with will launch the Xen hypervisor from the x230's ROM image and start the Qube's installer. The first few seconds are run with an archaic video mode, so things appear a little wrid, but once the dom0 kernel initializes the graphics it should look right. +Wait a bit and the Qubes installer should get into graphical mode once dom0 kernel initializes graphics. ![Qubes partitioning](https://farm3.staticflickr.com/2858/33156102504_7fa25661c4_z_d.jpg) @@ -128,52 +209,28 @@ You will need it again shortly, so don't lose it yet. ![Signing Qubes binaries in /boot](https://farm3.staticflickr.com/2939/33999478305_f0f61a4408_z_d.jpg]) -Once Qubes has finished installing, you'll need to reboot into the Heads recovery shell. The first reboot will fail with errors about "`/boot/boot.hashes does not exist`", since it doesn't.. The first step is to copy the Heads Xen to the `/boot` drive, sign them and let Qubes finish its initialization. - -``` -mount -o rw,remount /boot -cp /bin/xen.gz /boot/xen-4.6.4-heads.gz -mount -o ro,remount /boot -qubes-boot /boot/xen-4.6.4-heads.gz /boot/vm... /boot/initramfs... -``` - -You will need to input the disk recovery key here (almost for the last time), -and this should start the final stage of the Qubes installer. Under -'Configure Qubes` you should select `Create USB qube holding all USB controllers` so that they are protected from outside devices. This step takes a little -while as the templates are configured... - -Eventually this will be done and you can click "Finish", then Qubes will -give you a login screen with your login password. +Once Qubes has finished installing, you'll need to reboot and select the 'Boot menu' option by hitting 'm'. -Now we need to make a small change to have Qubes' initramfs correctly find the TPM encrypted keys. Open a dom0 terminal (click on the Q in the upper left and select `Terminal Emulator`). In this shell run this perl command and rebuild the initrd with dracut (takes several seconds): - -TODO: how to force all of crypttab to be emitted? +Select the first boot option: ``` -sudo perl -pi -e 's: none: /secret.key' /etc/crypttab -sudo dracut --force +1. Qubes, with Xen hypervisor [...] ``` +Then make this the default boot entry by hitting 'd'. This will also allow you to seal the disk encryption key. -Reboot by selecting "Logout" and "Reboot" and you should be back to the -Heads recovery shell with the `boot.hashes` error. Insert your Yubikey and run -(hit tab to autocomplete the file names): +If you choose to add the disk encryption key to the TPM, you'll need to specify which LUKS volume. A default Qubes install will work if you leave the 'Encrypted LVM group?' response blank and enter `/dev/sda2` when asked about 'Encrypted devices?'. For more details see the TPM Disk encryption keys section below. You'll then be asked to enter the disk recovery key as well as the new boot password you'll use to unseal that key. +Once the key sealing process is complete, it will ask you to insert your GPG card then enter your PIN to sign the config. After this, it will reboot back to the main boot menu. Disconnect your GPG card otherwise Qubes might think you have a USB keyboard. -``` -qubes-update /boot/xen-4.6.4-heads.gz /boot/vmlinux... /boot/initramfs... -``` +To start Heads now (and in the future), just hit 'y' for default boot. -This should prompt you for the TPM owner password to create the new -counter (only the first time), then for your GPG card's password. -It will output `/boot/boot.hashes` and `/boot/boot.hashes.asc` -with the signed hashes of the executables. +This should start the final stage of the Qubes installer. Under +'Configure Qubes' you should select `Create USB qube holding all USB controllers` so that they are protected from outside devices. This step takes a little +while as the templates are configured... -Lastly you'll need to seal the disk encryption keys with a disk unlock key -that you will enter everytime. Run `seal-key` and it will prompt you for -the disk recovery key (the long one you entered above), a disk unlock -key that you will enter on every boot, and on your first run also the -TPM owner password to create the NVRAM space. +Eventually this will be done and you can click "Finish", then Qubes will +give you a login screen with your login password. Installing extra software @@ -229,25 +286,23 @@ TPM Disk encryption keys --- The keys are currently derived only from the user passphrase, which is expanded via the LUKS expansion algorithm to increase the time to brute force it. For extra protection it is possible to store the keys in the TPM so that they will only be released if the PCRs match. -*This section is an early draft* +If you want to use the TPM to seal a secret used to unlock your LUKS volumes: -There are two tools in the Heads ROM image for working with the TPM keys. `seal-key` will generate a new key, seal it with the current PCRs and add a TPM passphrase, then store it into the TPM NVRAM. `unseal-key` will extract it from the NVRAM and request the user passphrase to decrypt/unseal it. If the PCRs do not match, the TPM will reject the attempt (and hopefully dump keys after too many tries?). +1. Enter recovery mode +2. Ensure that your the boot devices is mounted: `mount -o ro /dev/sda1 /boot` or whatever is appropriate +3. Insert your GPG card +4. Run `kexec-save-key -p /boot/ ...` with the followed by options appropriate to your OS. The key will be installed in all devices in the LVM volume group as well as any other devices specified after the `-l` option. -To setup the drive encryption, generate and seal a new key. Then unseal it to create `/tmp/secret.key` in the initial ramdisk. Delete the old keys from the root, home and swap partitions (can this use disk labels?): +Examples for the `kexec-save-key` parameters: -``` -cryptsetup luksKillSlot /dev/sda2 1 -cryptsetup luksKillSlot /dev/sda3 1 -cryptsetup luksKillSlot /dev/sda5 1 -``` +| Installation Type | Command | +| ---- | ---- | +| Previous Heads installation | `kexec-save-key -p /boot/ -l qubes_dom0` | +| Default Qubes / Default Fedora 25 | `kexec-save-key -p /boot/ /dev/sda2` | +| Default Ubuntu 16.04 / Debian 9 (\*) | `kexec-save-key -p /boot/ /dev/sda5` | -Then add the (now cleartext) key to each partition: +5. Reboot and you will be prompted for your boot password when that device is used to boot in the future. -``` -cryptsetup luksAddKey /dev/sda2 /tmp/secret.key -cryptsetup luksAddKey /dev/sda3 /tmp/secret.key -cryptsetup luksAddKey /dev/sda5 /tmp/secret.key -``` NOTE: should the new LUKS headers be measured and the key re-sealed with those parameters? This is what the Qubes AEM setup uses and is probably a good idea (although we've already attested to the state of the firmware). diff --git a/Keys.md b/Keys.md index 1e8d4d7..afbcb64 100644 --- a/Keys.md +++ b/Keys.md @@ -91,8 +91,7 @@ TPM PCRs The actual assignment needs to be updated in the code; there are outstanding issues ( [MRC cache](https://github.com/osresearch/heads/issues/150), -[SMM reloc](https://github.com/osresearch/heads/issues/13), -[double measurement](https://github.com/osresearch/heads/issues/15) +[SMM reloc](https://github.com/osresearch/heads/issues/13) ) that need to be resolved as well. Until then this is a rough draft of how Heads uses the TPM PCRs. 0: Boot block @@ -108,3 +107,5 @@ The actual assignment needs to be updated in the code; there are outstanding iss 5: Heads Linux kernel modules 6: Drive LUKS headers + +7: Heads user-specific config files diff --git a/Upgrading.md b/Upgrading.md index e01f931..c529942 100644 --- a/Upgrading.md +++ b/Upgrading.md @@ -92,4 +92,6 @@ This needs the TPM owner password to be able to define the NVRAM space. Resealing the disk encryption keys --- -TODO. Yes, I know this is the difficult part. Sorry I haven't written it yet. +When you get to the standard boot menu and after you verify the TOTP, select 'm' to +go to the full boot menu. Select the option you want (usually the first), +make it the default by hitting 'd' and also say 'y' when asked to reseal the disk keys. \ No newline at end of file diff --git a/index.md b/index.md index d20bc5f..44c2340 100644 --- a/index.md +++ b/index.md @@ -5,6 +5,7 @@ Using Heads * [Installing Heads](/Installing-Heads) on an x230 Thinkpad * [Upgrading Heads](/Upgrading), including how to generate your TOTP token * [Keys, Passwords and PCRs in Heads](/Keys) (rough draft) +* [Boot config files](/Boot) * Qubes specific configurations (to be written) * Server specific configurations (to be written)