Save each partition as individual binaries

[chris-subtlebytes]

Fixes #714

[SergioGasquez]

I guess we need to decide a solution, I like what @ivmarkov suggested here: #714 (comment)

I would try to avoid having major breaking changes as we recently released 3.0

[chris-subtlebytes]

What are our options?

1. Add new commands save-firmware, save-part-table, and save-bootloader.

   From a user point of view, it becomes unwieldy to juggle three different commands especially because the partition table and partition table offset needs to be specified for each one. I already needed to juggle a handful while signing and encrypting. Remembering to update three different commands would have caused many more errors on my end. I was modifying the partition table to add nvs-keys, modifying sdkconfig.defaults to change bootloader flags, and messing about with the actual firmware image all at the same time. Having many commands that all need the same flags (bootloader and partition table) increases the odds that one of them has an incorrect value when the user is running each one separately.

2. Break compatibility by making --merge work as per docs and flag.

   Most users are going to use espflash flash, which isn't affected. I didn't need to save-image until I needed to sign and encrypt the bootloader and app image. Nobody can do that today because the functionality is broken. Every use case for save-image I can think of usually requires the individual parts, such as OTAs or inspecting the ELF.

   In my opinion, this still beta software that should be expected to break for bugfixes. If anybody has strong opinions on stability, I would hope they use esptool. The most impact a user would have is to add --merge to their command.

3. Update the default for --merge to true, but that's a bit tricky according to Add support for automatic negation flags clap-rs/clap#815

   This keeps the default behavior and docs the same.

4. Rename the --merge flag to --separate-bins
   This is similar to reversing the default value for --merge but still causes a minor breakage. I don't think anyone is using --merge because it's broken.

5. Is there any inspiration we can draw from esptool? I haven't used it much.

[ivmarkov]

What are our options?

1. Add new commands save-firmware, save-part-table, and save-bootloader.
   From a user point of view, it becomes unwieldy to juggle three different commands especially because the partition table and partition table offset needs to be specified for each one.

I don't think this statement is correct.
(Please check my understanding below, and if you think I'm wrong on any of those, if you can show the relevant code in espflash or ESP-IDF, really important to get this straight. :-) )

1. save-bootloader would not need neither the partition table, nor the partition table offset.

While save-bootloader can patch the built-in bootloader binary image to support a different flash size, turns out - just checked this morning - it cannot patch the built-in bootloader to support a different partition table offset than the default 0x8000 (@SergioGasquez right?).

Also, if you need a bootloader with a different partition table offset than 0x8000 OR a bootloader with secure boot capabilities (as I have doubts the built-in one is compiled with secure boot capabilities), save-bootloader is useless to you. I - for one - am instead using the custom-built bootloader that you always get with every single esp-idf-*-based binary crate, out of the box.

save-bootloader might use a user-supplied partition table offset to only check whether the built-in bootloader fits before the parrtition table start, but that's it.

2. save-firmware does not need the partition table itself or its offset either for the same reasons: it cannot patch an already compiled app image to a different partition table offset. The app image needs to know the partition table offset (but not the partition table CSV or BIN!) during OTA (updated: and also for all esp_partition_t-based APIs of course). The only way to do it is to compile the app image from the beginning with a new value for CONFIG_PARTITION_TABLE_OFFSET (if you use the esp-idf-* crates) or whatever ad-hoc method you use for baremetal, as OTA updates are not yet "officially" supported there, with esp-hal.

save-firmware might take advantage of a user-provided partition-table to check if the app image being converted to .BIN fits in the factory or one of the OTA partitions of the table, but that's it. And the current check for this is anyway a bit heuristical, as save-image can't actually know against which partition in the partition table to check the size of the app image anyway.

3. Obviously, save-part-table does not need to know the partition table offset either.

I already needed to juggle a handful while signing and encrypting. Remembering to update three different commands would have caused many more errors on my end. I was modifying the partition table to add nvs-keys, modifying sdkconfig.defaults to change bootloader flags, and messing about with the actual firmware image all at the same time. Having many commands that all need the same flags (bootloader and partition table) increases the odds that one of them has an incorrect value when the user is running each one separately.

I get it that it is difficult (I'm suffering myself through this), but we also have to be crystal clear among the three of us (@SergioGasquez included) what we need and what we don't / won't need/do w.r.t. support for Secure Boot in espflash.

[chris-subtlebytes]

Apologies for taking so long to respond; I'll try to be more active. Do we want this discussion in the PR or the issue?

save-bootloader would not need neither the partition table, nor the partition table offset.

If you want to support encryption, which is looks like #718 is going to do, then you will need the offset of the partition you're saving. See python -m espsecure encrypt_flash_data, which requires --address. If omitted: $ espsecure encrypt_flash_data: error: the following arguments are required: --address/-a because of how the encryption works.

Also, if you need a bootloader with a different partition table offset than 0x8000 OR a bootloader with secure boot capabilities (as I have doubts the built-in one is compiled with secure boot capabilities), save-bootloader is useless to you

This was annoying to encounter because I was setting bootloader-changing sdkconfig.defaults settings and scratching my head wondering why they're not working. I'm using --bootloader to get the one that's built anyway: espflash save-image --merge --chip esp32c3 --bootloader target/riscv32imc-esp-espidf/release/bootloader.bin --partition-table partitions.csv --partition-table-offset 0x000c000 target/riscv32imc-esp-espidf/release/my-project /tmp/espflash/my-image.bin

So I imagine espflash save-bootloader would need to be espflash save-bootloader --bootloader /target/*/*/bootloader.bin for anyone to use properly. It's still necessary to save the image like this because it corrects the header and SHA.

we also have to be crystal clear among the three of us what we need and what we don't / won't need/do w.r.t. support for Secure Boot in espflash.

Here's my full set of commands (using my current proposed commit), which I think touches most aspects aside from secure boot v1:

# step 1 build and save
$ cargo build --release
# save all image partitions as per proposed patch
$ espflash save-image --chip esp32c3 target/riscv32imc-esp-espidf/release/my-project path/to/my-project.bin

# step 2 generate signing keys
# do three times because there are three key efuse slots and three key revocation efuses
# you could add this to espflash, but would people really trust it over openssl? I suppose it may be more convenient but we're talking about security.
# for ESP32, ESP32-S2, ESP32-S3, ESP32-C3
$ openssl genrsa -out my-project_rsa3072_secure_boot_signing_key_0.pem 3072
# for ESP32-C2, ESP32-C6, ESP32-H2, ESP32-P4
$ openssl ecparam -name prime256v1 -genkey -noout -out my-project_p256_secure_boot_signing_key_0.pem

# step 3 sign binaries
# sign the image and bootloader from step 1 with the key generated in step 2 using espsecure.py.
# imo this could be folded into save-image with either `--sign0 path --sign1 path --sign2 path` or `--sign path0 --sign path1 --sign path2`
$ python -m espsecure sign_data FILE_FROM_ESPFLASH_SAVEIMAGE --version 2 --keyfile my-project_KEYTYPE_secure_boot_signing_key_0.pem
$ python -m espsecure sign_data path/to/bootloader.bin --version 2 --keyfile my-project_KEYTYPE_secure_boot_signing_key_0.pemmy-project_KEYTYPE_secure_boot_signing_key_1.pem my-project_KEYTYPE_secure_boot_signing_key_2.pem

# step 4 generate encryption key and burn it into efuse
# (usually only for development purposes as the ESP should do this internally for release devices)
$ openssl rand -out path/to/my-project_aes128_development_flash_encryption_key.key 32
$ python -m espefuse --port PORT --chip CHIP burn_key BLOCK_KEY3 path/to/my-project_aes128_development_flash_encryption_key.key XTS_AES_128_KEY

# step 4.5 encrypt binaries
# espflash could add `--encrypt` to `save-image`. Sometimes other partitions (nvs-keys, FAT, etc) need to be encrypted as well, so there needs to be 2 ways of doing it.
$ python -m espsecure encrypt_flash_data --keyfile ~/.secrets/my-project/my-project_aes128_development_flash_encryption_key.key --aes_xts --address 0x0010000 --out images/0x0010000_my-project.bin.encrypted images/0x0010000_my-project.bin

# step 5 generate hash of signing keys
# needs to be done once per signing key
# this could be added to espflash as a new command
$ python -m espsecure extract_public_key --version 2 --keyfile path/to/my-project_KEYTYPE_secure_boot_signing_key_X.pem path/to/my-project_KEYTYPE_secure_boot_signing_key_X.pub.pem
$ python -m espsecure digest_sbv2_public_key --keyfile path/to/my-project_KEYTYPE_secure_boot_signing_key_X.pub.pem -o path/to/my-project_KEYTYPE_secure_boot_signing_key_X.pub.digest
$ python -m espefuse --port PORT --chip CHIP burn_key BLOCK_KEYX path/to/my-project_KEYTYPE_secure_boot_signing_key_X.pub.digest SECURE_BOOT_DIGESTX

# step 6 upload all partitions to device
# this step could definitely be improved by espflash as we have to run the command per each partition (bootloader, partition table, app). Care needs to be taken to still handle the bespoke partitions too.
$ espflash write-bin 0xADDRESS path/to/0x0000000_my-project.bin # bootloader
$ espflash write-bin 0xADDRESS path/to/0x000c000_my-project.bin # partition table
$ espflash write-bin 0xADDRESS path/to/0x0010000_my-project.bin # app
# extra flashes sometimes needed
# note that if the flash is encrypted, reading an erased region will look like valid (albeit random) data so encrypted devices always need to write encrypted zeroes after erasing for some features to work.
$ espflash write-bin 0xADDRESS path/to/0x000d000_my-project.bin # ota data, which sometimes needs to be written with zeroes to boot into the newly flashed factory partition instead of ota2
$ espflash write-bin 0xADDRESS path/to/0x03ed000_my-project.bin # nvs keys, which needs to be written with zeroes for it to internally generate a new key for an nvs partition.

# Wait about a minute for the plaintext flash image to self-encrypt. Interrupting before it's finished can cause corruption.
# If the device does not support RTS/DTR, manually enter download mode and run espflash with --before no-reset.

# verify that secure boot and flash encryption are enabled
$ python -m esptool get_security_info

#718 comments:

we are currently discussing a v4 release

Is there a chat room or new issue open for this? I still need to upload a lot of changes / bug reports / feature requests for esp-idf-[sys, svc] and if espflash v4 is related to those then maybe I can at least offload my ideas there. I think at this point I may have one of the most complex esp-idf projects so I've hit a lot of rough areas (which is expected).

[ivmarkov]

Thanks a lot for your feedback.

I've put comments below so that we can compare with the way I do these steps.

Apologies for taking so long to respond; I'll try to be more active. Do we want this discussion in the PR or the issue?

save-bootloader would not need neither the partition table, nor the partition table offset.

If you want to support encryption, which is looks like #718 is going to do, then you will need the offset of the partition you're saving. See python -m espsecure encrypt_flash_data, which requires --address. If omitted: $ espsecure encrypt_flash_data: error: the following arguments are required: --address/-a because of how the encryption works.

The --address parameter of the encrypt_flash_data command however is not the address of the partition table, but the starting address of the partition you are encrypting. This is so because encryption is position-dependent. For most chips the BL start address is 0x0; for some - it is 0x1000. In any case, espflash is well aware of the chip type and knows the bootloader address.

With that said, I don't think we should be having save-bootloader do encryption. If we do this, we should also make it do signing, as signing must happen before encryption.

Where do we stop?

I strongly believe each command should do just one simple thing. save-bootloader should just generate an unencrypted binary image of a bootloader so that you don't have to do it yourself. Not that I believe once encryption and secure boot are in play, this is not very realistic (that is, to use a pre-generated bootloader image) but anyway.

If you need to sign and/or encrypt it, you might use different commands (which - btw - and for the most part - wouldn't even care if they are encrypting a bootloader, an app image or some data partition). Just like the existing commands from Esapressif's Python tooling.

Also, if you need a bootloader with a different partition table offset than 0x8000 OR a bootloader with secure boot capabilities (as I have doubts the built-in one is compiled with secure boot capabilities), save-bootloader is useless to you

This was annoying to encounter because I was setting bootloader-changing sdkconfig.defaults settings and scratching my head wondering why they're not working. I'm using --bootloader to get the one that's built anyway: espflash save-image --merge --chip esp32c3 --bootloader target/riscv32imc-esp-espidf/release/bootloader.bin --partition-table partitions.csv --partition-table-offset 0x000c000 target/riscv32imc-esp-espidf/release/my-project /tmp/espflash/my-image.bin

So I imagine espflash save-bootloader would need to be espflash save-bootloader --bootloader /target/*/*/bootloader.bin for anyone to use properly. It's still necessary to save the image like this because it corrects the header and SHA.

The header and the SHA would only need to be corrected if the flash-size is changed inside the bootloader header.
Given that you use your custom-built bootloader, I do not understand why espflash should do that for you in the first place?
Just put CONFIG_ESPTOOLPY_FLASHSIZE_8MB=y in your sdkconfig.defaults and then you don't have to deal with save-bootloader at all.

# step 2 generate signing keys
# do three times because there are three key efuse slots and three key revocation efuses
# you could add this to espflash, but would people really trust it over openssl? I suppose it may be more convenient but we're talking about security.
# for ESP32, ESP32-S2, ESP32-S3, ESP32-C3
$ openssl genrsa -out my-project_rsa3072_secure_boot_signing_key_0.pem 3072
# for ESP32-C2, ESP32-C6, ESP32-H2, ESP32-P4
$ openssl ecparam -name prime256v1 -genkey -noout -out my-project_p256_secure_boot_signing_key_0.pem

1. You need just one signing key
2. This signing key should be global and generated once only. Having a signing key per device is a bad idea. How do you send a signed OTA image to the device after that? By keeping all signed device-specific keys? So this is more like Step 0 or Step -1.

Step 2 is to only extract the public key from the signing key as it is needed during efuse further below.

UPDATE: Forgot to say that an alternative to 3 signing keys (too much hassle) is that you just close the remaining two slots. If I'm not mistaken, the efuse command would even do it if you allow it to... or was it the bootloader...

# step 3 sign binaries
# sign the image and bootloader from step 1 with the key generated in step 2 using espsecure.py.
# imo this could be folded into save-image with either `--sign0 path --sign1 path --sign2 path` or `--sign path0 --sign path1 --sign path2`
$ python -m espsecure sign_data FILE_FROM_ESPFLASH_SAVEIMAGE --version 2 --keyfile my-project_KEYTYPE_secure_boot_signing_key_0.pem
$ python -m espsecure sign_data path/to/bootloader.bin --version 2 --keyfile my-project_KEYTYPE_secure_boot_signing_key_0.pemmy-project_KEYTYPE_secure_boot_signing_key_1.pem my-project_KEYTYPE_secure_boot_signing_key_2.pem

Agreed

# step 4 generate encryption key and burn it into efuse
# (usually only for development purposes as the ESP should do this internally for release devices)
$ openssl rand -out path/to/my-project_aes128_development_flash_encryption_key.key 32
$ python -m espefuse --port PORT --chip CHIP burn_key BLOCK_KEY3 path/to/my-project_aes128_development_flash_encryption_key.key XTS_AES_128_KEY

Agreed. But not sure it is only necessary for development purposes. If you want to preserve the DL mode with security enabled and be capable of uploading new images via UART, you need to keep all encryption keys for all devices you have ever created (as I'll do). One never knows.

# step 4.5 encrypt binaries
# espflash could add `--encrypt` to `save-image`. Sometimes other partitions (nvs-keys, FAT, etc) need to be encrypted as well, so there needs to be 2 ways of doing it.
$ python -m espsecure encrypt_flash_data --keyfile ~/.secrets/my-project/my-project_aes128_development_flash_encryption_key.key --aes_xts --address 0x0010000 --out images/0x0010000_my-project.bin.encrypted images/0x0010000_my-project.bin

Agreed, but you already see how your save-image idea starts to break? encryption is after signing and does not care what is being encrypted. I don't think adding magic flags to save-image/save-bootloader/whatever-elf-to-bin cmd we come up with will make it clearer.

I think we should just follow the division of work (and general command layout) that the Espressif Python tooling did. It had stand the test of time at least.

# step 5 generate hash of signing keys
# needs to be done once per signing key
# this could be added to espflash as a new command
$ python -m espsecure extract_public_key --version 2 --keyfile path/to/my-project_KEYTYPE_secure_boot_signing_key_X.pem path/to/my-project_KEYTYPE_secure_boot_signing_key_X.pub.pem
$ python -m espsecure digest_sbv2_public_key --keyfile path/to/my-project_KEYTYPE_secure_boot_signing_key_X.pub.pem -o path/to/my-project_KEYTYPE_secure_boot_signing_key_X.pub.digest
$ python -m espefuse --port PORT --chip CHIP burn_key BLOCK_KEYX path/to/my-project_KEYTYPE_secure_boot_signing_key_X.pub.digest SECURE_BOOT_DIGESTX

An easier way is to use burn_key_digest instead of burn_key and skip the whole "generate hash from public key" thing, because burn_key_digest tales the public key rather than its sha-256 hash in the first place.

# step 6 upload all partitions to device
# this step could definitely be improved by espflash as we have to run the command per each partition (bootloader, partition table, app). Care needs to be taken to still handle the bespoke partitions too.
$ espflash write-bin 0xADDRESS path/to/0x0000000_my-project.bin # bootloader
$ espflash write-bin 0xADDRESS path/to/0x000c000_my-project.bin # partition table
$ espflash write-bin 0xADDRESS path/to/0x0010000_my-project.bin # app
# extra flashes sometimes needed
# note that if the flash is encrypted, reading an erased region will look like valid (albeit random) data so encrypted devices always need to write encrypted zeroes after erasing for some features to work.

Agreed, but you can also just merge all binaries together, then encrypt the whole thing, and then just tell espflash to write-bin the big elephant from address 0x0. Or help develop espfactory which does that and much more.

$ espflash write-bin 0xADDRESS path/to/0x000d000_my-project.bin # ota data, which sometimes needs to be written with zeroes to boot into the newly flashed factory partition instead of ota2

Agreed. An alternative is to just do espflash erase-flash before flashing, however "erase-flash" does not work with secure download mode enabled - neither with espflash which does not support secure DL mode at all, nor with esptool.

$ espflash write-bin 0xADDRESS path/to/0x03ed000_my-project.bin # nvs keys, which needs to be written with zeroes for it to internally generate a new key for an nvs partition.

Agreed if you start with an empty NVS partition. For suffciently-complex projects that do have device-specific data this might not be the case (it is not for me). In these cases you need to pre-generate and pre-encrypt both the NVS partition table contents, as well as the NVS keys partition.

# Wait about a minute for the plaintext flash image to self-encrypt. Interrupting before it's finished can cause corruption.

I don't think relying on the bootloader to do it is ideal. It is best to use CONFIG_SECURE_FLASH_REQUIRE_ALREADY_ENABLED=y and pre-encrypt all data yourself. This way the bootloader does not have to do the pre-encrypt process itself.
And - if you leave secure DL mode enabled you can continue re-flashing your pre-encrypted images, given that you keep the encryption key around. Because this bootloader-induced encryption works the first time only.

# If the device does not support RTS/DTR, manually enter download mode and run espflash with --before no-reset.

Why is this necessary?

# verify that secure boot and flash encryption are enabled
$ python -m esptool get_security_info

Agreed!

Is there a chat room or new issue open for this? I still need to upload a lot of changes / bug reports / feature requests for esp-idf-[sys, svc] and if espflash v4 is related to those then maybe I can at least offload my ideas there. I think at this point I may have one of the most complex esp-idf projects so I've hit a lot of rough areas (which is expected).

The chat is called esp-rs and is a Matrix chat (linked from most projects).

Feel free to open issues (and PRs) at the esp-idf-* projects.

[chris-subtlebytes]

The --address parameter of the encrypt_flash_data command however is not the address of the partition table, but the starting address of the partition you are encrypting.

If we have a save command for each (bootloader, partition table, app image), those offsets are all described in the partition table, then you'd need to manually supply the offset for each partition or use the partition table. That was the point I was trying to make if there was a need to support encryption. And yes, the bootloader won't need a specified offset.

I don't think we should be having save-bootloader do encryption. If we do this, we should also make it do signing, as signing must happen before encryption.

Sure, but I don't see why espflash couldn't sign as well. Of course these may not be features for a while or ever, but I don't think we should design the save-image commands to limit us from the opportunity. I'm not saying having separate commands for each partition is still compatible with signing and encryption, but it would get unwieldy passing the same flags in for each one.

put CONFIG_ESPTOOLPY_FLASHSIZE_8MB=y in your sdkconfig.defaults and then you don't have to deal with save-bootloader

Fair, but I don't want to have a separate sdkconfig.defaults for every device I support. I'm sure I could come up with a workaround but using the same commands for each partition type was convenient.

You need just one signing key. This signing key should be global and generated once only. Having a signing key per device is a bad idea.

I think you misunderstand. Most ESPs support more than one signature key slot and a corresponding set of efuses for key revocation.

This is a mitigation in case the private signing key is leaked. Say my first signing key gets compromised, I can make a new image that revokes it and secure boot will burn the efuse and only use slots 2 and 3 for verification going forward. It happens :/ https://arstechnica.com/gadgets/2022/12/samsungs-android-app-signing-key-has-leaked-is-being-used-to-sign-malware/

Agreed, but you already see how your save-image idea starts to break? encryption is after signing and does not care what is being encrypted.

I still maintain that we could add a --sign flag. And yes I know how encryption and signing work; note the order of step 3 (sign) and step 4 (encrypt). I don't see how my idea is starting to break - please expand.

An easier way is to use burn_key_digest instead of burn_key

neat. maybe I'll use that moving forward.

Agreed, but you can also just merge all binaries together, then encrypt the whole thing, and then just tell espflash to write-bin

If espflash can do the whole thing (pad, sign parts, merge, encrypt, etc) all in one shot without any intermediate commands, that would be amazing.

Agreed. An alternative is to just do espflash erase-flash

If you erase-flash on an encrypted partition, the regular reads it will be reading encrypted raw flash zeroes which will be essentially noise to the reader. You need to generate a file of zeroes, encrypt it, then write the encrypted section. Because this is only a scenario for development, I don't see how secure download mode matters because you could just disable it.

I don't think relying on the bootloader to do it is ideal. It is best to use CONFIG_SECURE_FLASH_REQUIRE_ALREADY_ENABLED=y and pre-encrypt all data yourself.

You have a different threat model than me. If I distribute the software, why do I also need to be able to extract the consumer's data? I'm relying on the ESP to internally generate it's own unique encryption key so that the consumer can feel confident that it's nearly impossible (without breaking ESP's security model) to exfiltrate the data. In fact, the APIs I have for my project explicitly do not expose the data generated on device.

My instructions are tailored for building a release build.

# If the device does not support RTS/DTR, manually enter download mode and run espflash with --before no-reset.
Why is this necessary?

This wasn't too relevant to the discussion here but it made its way in from my copy/paste. It's a nice little tip for anyone that stumbles across this though.

---

I feel like your response was written from the perspective that I don't know what I'm doing. I provided the encryption instructions so we could look over what could/should be added to espflash and the merits of the different approaches for commands and flags.

It almost sounds like you're arguing FOR a single command instead of separate save-bootloader, save-parition-table, save-app-image in case espflash can support generating a signed, (sometimes encrypted) single fat binary.

Imagine a future like this:

espflash flash \
--chip chip \
--partition-table path/to/table --partition-table-offset 0x1234 \
--sign0 path/to/key0 --sign1 path/to/key1 --sign2 path/to/key2 \
--encrypt path/to/encryption-key \
--partition 'partition_name:path/to/contents' # in case you have custom otadata, nvs, nvs-keys, fat, etc., matched up with part name in the partition table.

And nearly identical save-image

espflash save-image \
--chip chip \
--partition-table path/to/table --partition-table-offset 0x1234 \
--sign0 path/to/key0 --sign1 path/to/key1 --sign2 path/to/key2 \
--encrypt path/to/encryption-key \
--partition 'partition_name:path/to/contents' \ # in case you have custom otadata, nvs, nvs-keys, fat, etc., matched up with part name in the partition table.
path/to/out/ # the output directory of the image, merged or otherwise.

(--partition just as a potential mechanism to specify the data necessary to build a merged image. The flag format is illustrative and not a serious proposal)

With #727 reading env and config for the missing flags, it could even be simplified to espflash flash and espflash save-image [--merged] path/to/out.

My understanding is that the equivalent with separate commands would look like this. Please correct me if you have different ideas.

espflash save-bootloader \
--chip chip \
--sign0 path/to/key0 --sign1 path/to/key1 --sign2 path/to/key2 \
--encrypt path/to/encryption-key
path/to/out/

espflash save-partition-table \
--chip chip \
--partition-table path/to/table --partition-table-offset 0x1234 \
--sign0 path/to/key0 --sign1 path/to/key1 --sign2 path/to/key2 \
--encrypt path/to/encryption-key
path/to/out/

espflash save-app-image \
--chip chip \
--offset 0x1234 \ # or --partition-table?
--sign0 path/to/key0 --sign1 path/to/key1 --sign2 path/to/key2 \
--encrypt path/to/encryption-key
path/to/out/

And without signing or encryption:

espflash save-bootloader \
--chip chip \
path/to/out/

espflash save-partition-table \
--chip chip \
--partition-table path/to/table --partition-table-offset 0x1234 \
path/to/out/

espflash save-app-image \
--chip chip \
--offset 0x1234 \ # or --partition-table?
path/to/out/

[SergioGasquez]

Hi! Sorry for the late reply! And thanks for the discussion, I will try to catch up/do some investigation and provide some input.

it cannot patch the built-in bootloader to support a different partition table offset than the default 0x8000 (@SergioGasquez right?).

It cant at the moment, but if we know where this happens in the bootloader we may be able to patch it.

Is there a chat room or new issue open for this? I still need to upload a lot of changes / bug reports / feature requests for esp-idf-[sys, svc] and if espflash v4 is related to those then maybe I can at least offload my ideas there. I think at this point I may have one of the most complex esp-idf projects so I've hit a lot of rough areas (which is expected).

As Ivan mentioned, most of the things are discussed in the Matrix chat or in the community meetings. We have a v4 gh milestone with the current issues we want to tackle for [email protected] and we are already starting to work towards it, so we can already merge breaking changes, like this one.

I still cant decide if we want to go the 3 commands or the 1 command road, to be honest, but we want to make sure this is also compatible with esp-hal, where we dont have sdkconfig.

[ivmarkov]

Hi! Sorry for the late reply! And thanks for the discussion, I will try to catch up/do some investigation and provide some input.

it cannot patch the built-in bootloader to support a different partition table offset than the default 0x8000 (@SergioGasquez right?).

It cant at the moment, but if we know where this happens in the bootloader we may be able to patch it.

I think this is pretty much impossible, as the partition offset is just a C #define and as such is hard-coded in gazillion of places in both the bootloader, as well as the app image. There is just no single, predictable place in these where this offset is recorded, that you can patch "post-factum".

Is there a chat room or new issue open for this? I still need to upload a lot of changes / bug reports / feature requests for esp-idf-[sys, svc] and if espflash v4 is related to those then maybe I can at least offload my ideas there. I think at this point I may have one of the most complex esp-idf projects so I've hit a lot of rough areas (which is expected).

As Ivan mentioned, most of the things are discussed in the Matrix chat or in the community meetings. We have a v4 gh milestone with the current issues we want to tackle for [email protected] and we are already starting to work towards it, so we can already merge breaking changes, like this one.

I still cant decide if we want to go the 3 commands or the 1 command road, to be honest, but we want to make sure this is also compatible with esp-hal, where we dont have sdkconfig.

I think the topic is deep enough that I'll bring it for the next ESP-RS mtg/discussion on Feb 13. Will try to do a short summary writeup on all issues, not just this one (the list keeps piling up a bit).

[ivmarkov]

@chris-subtlebytes If you can attend the next mtg I would really appreciate that. This stuff is so complex, that we need every person with experience in that we can get. :)

Also, I haven't addressed your last comment yet because I think a live meeting might actually be the easier way to resolve these issues, as I think they are also getting a bit stylistic or philosophical (as in whether we should follow the command layout / overall approach of the tried-and-tested esptool, or whether espflash should risk it a bit and invent something a bit on its own).

[SergioGasquez]

Just for reference, the meeting we are talking about is esp-rs/rust#251, happening 13th of February 2025 - 5 pm CET/GMT+1, does that work for you @chris-subtlebytes? If it doesnt work for you we should probably come up with another way to properly discuss this