Allow compressed flash updates #6820
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
earlephilhower
force-pushed
the
eboot-to-O2
branch
2 times, most recently
from
8ebe220 to
83adc5d
Compare
earlephilhower
changed the title
|
This PR can now automatically decompress GZIP'd .BIN files and run them. Prior versions had an incorrectly low code size due to me forgetting to set the "use gzip" flag. D'oh. Real memory usage is 0xee0, leaving ~260 bytes free in the 4K sector. |
earlephilhower
changed the title
Modified the bootloader to be able to take stored updates in compressed GZIP format (i.e. the output of "gzip -9 xxx.bin") and decompress them on-the-fly to their final destination. This can work for apps and for filesystems (when used with the 2-step update option). Allow eboot to be built using -Os/2 optimizations by fixing some portions which failed when any optimizations were used. Add -Wall and use data and function sections to reduce size. Use -Os to minimize size. Remove obsolete esptool-ck calls to build a .ROM image, we don't use it. Move all uninitted variables to RAM from IRAM, allowing 8-bit access. Hook in @d-a-v and @pfalcon's uzlib port to actually do the decompression. Do not use any CRC checking which saves space. Since we have overwritten all of flash by the time we know id the CRC matches, there's nothing we could have done anyway. Adjust the Updater class to support GZIP files and not attempt to patch them. Bootloader builds to 0xd90 out of 0xfff bytes.
earlephilhower
force-pushed
the
eboot-to-O2
branch
from
080eb8f to
67ba90d
Compare
|
Fixes #6614 |
d-a-v
approved these changes
Dec 4, 2019
There was a problem hiding this comment.
This gives a 🚀 bright future 🚀 for esp8285 and more generally 1M or 2M setups
I tested with the ESP8266HttpUpdate library (with the below changes) and it works very well.
Thanks alot @earlephilhower !
--- a/libraries/ESP8266httpUpdate/src/ESP8266httpUpdate.cpp
+++ b/libraries/ESP8266httpUpdate/src/ESP8266httpUpdate.cpp
@@ -370,7 +370,7 @@ HTTPUpdateResult ESP8266HTTPUpdate::handleUpdate(HTTPClient& http, const String&
}
// check for valid first magic byte
- if(buf[0] != 0xE9) {
+ if(buf[0] != 0xE9 && buf[0] != 0x1f) {
DEBUG_HTTP_UPDATE("[httpUpdate] Magic header does not start with 0xE9\n");
_setLastError(HTTP_UE_BIN_VERIFY_HEADER_FAILED);
http.end();
@@ -378,14 +378,16 @@ HTTPUpdateResult ESP8266HTTPUpdate::handleUpdate(HTTPClient& http, const String&
}
- uint32_t bin_flash_size = ESP.magicFlashChipSize((buf[3] & 0xf0) >> 4);
+ if (buf[0] == 0xe9) {
+ uint32_t bin_flash_size = ESP.magicFlashChipSize((buf[3] & 0xf0) >> 4);
- // check if new bin fits to SPI flash
- if(bin_flash_size > ESP.getFlashChipRealSize()) {
- DEBUG_HTTP_UPDATE("[httpUpdate] New binary does not fit SPI Flash size\n");
- _setLastError(HTTP_UE_BIN_FOR_WRONG_FLASH);
- http.end();
- return HTTP_UPDATE_FAILED;
+ // check if new bin fits to SPI flash
+ if(bin_flash_size > ESP.getFlashChipRealSize()) {
+ DEBUG_HTTP_UPDATE("[httpUpdate] New binary does not fit SPI Flash size\n");
+ _setLastError(HTTP_UE_BIN_FOR_WRONG_FLASH);
+ http.end();
+ return HTTP_UPDATE_FAILED;
+ }
}
}
if(runUpdate(*tcp, len, http.header("x-MD5"), command)) {
|
Thanks, @d-a-v. I tried the httpupdateserver, not the httpupdate path and I'll be sure to include your patch in the next push. |
|
Just curious, how should the bin be compressed in order to be uncompressed with this bootloader? |
|
It should be able to parse any valid gzip file. I just use |
For now, because there are some self-test failures with @d-a-v's esp8266 branch (whose cool new features we don't actually use in eboot now) start with pfalcon's 2.9 release and add the 2 patches (clcidx to code from IRAM/RODATA, and the Windows test file renaming) needed to build and run successfully.
This was referenced Dec 18, 2019
|
Are there any working examples of compressed for file system update (LittleFS) OTA using ESP8266httpUpdate? I can't seem to find any. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
As part of the compressed OTA image work, we will need to be able to
build the bootloader using -O2 to minimize code size for the
decompressor.
It was as -O0 which does no code optimization whatsoever because there
was a bit of quasi-assembly that was optimized out with any other
setting. Building uzlib with -O0 is probably a bad thing and will wind
up much larger than with -O2.
Rewrite the jump-to-app code in pure ASM to avoid the optimization
issue.
Add -Wall and clean up warnings it generates.
Remove call to esptool-ck and generation of unused eboot.bin application
image since it is not used by the Arduino core.