image too big when building with additional packages and Gluon based on OpenWrt 22.03

[rotanid]

after the merge of #2426 there's much additional space needed by Gluon, leading to the impossibility to include stuff like usb kernel modules if you want the builds to succeed generating images.

the actual error messages vary, sometimes there's "image too big" but sometimes there's only the obscure bash error bash: line 1: 8: Bad file descriptor followed by Failed to open firmware file

the issue was observed when building ath79-generic images for devices with 8MB flash

maybe we need to try to strip new features off of the newer kernel, like we did in the past when we wanted to fit Gluon onto 4MB flash

[rotanid]

with the recent optimizations in #2508 #2509 and #2510 the issue is not that prominent anymore. thanks @NeoRaider @blocktrron

[neocturne]

I'd still like to look into the excessive size increase of the luaposix package.

[maurerle]

I just ran into this issue for the RE450 and the build process gave quite well feedback, that the image is too big and therefore no file is created.

The bash file then tries to copy from gluon-build/openwrt/build_dir/target-mips_24kc_musl/linux-ath79_generic/tmp/... to gluon-build/output which does not work, as the file is not there.

Then it terminates and subsequent images (which are not affected from this) are also not showing up in the output folder.

I could not find where it is copied, but it would be much better if this is more failsafe to copy the other images?

[rotanid]

no problem with gluon per-se, look at #2581 (comment)

[maurerle]

Thank you for your help with my site.mk!

Yet, I would still suggest that the build script should not stop copying the other successful images which were build successfully, If one image can't be found?

More specifically - catch the error copying a non-existing file in L54 inside the loop per device:

gluon/scripts/copy_output.lua

Lines 48 to 54 in ac24b70

	for _, image in ipairs(images) do
	clean(image, image.image)
	local destdir, destname = image:dest_name(image.image)
	local source = image_source(image)
	lib.exec {'cp', source, destdir..'/'..destname}

In my case, the build log looked like this:

 make[3] target/install
 make[4] -C target/linux install
 make[7] -C target/linux/ath79/image/lzma-loader compile loader.elf
    WARNING: Image file /gluon-build/openwrt/build_dir/target-mips_24kc_musl/linux-ath79_generic/tmp/openwrt-ath79-generic-dlink_dir-825-b1-squashfs-sysupgrade.bin is too big: 6750212 > 6356992
    WARNING: Image file /gluon-build/openwrt/build_dir/target-mips_24kc_musl/linux-ath79_generic/tmp/openwrt-ath79-generic-tplink_re450-v1-squashfs-sysupgrade.bin is too big:  > 6160384
 make[3] package/index
 make[3] json_overview_image_info
 make[3] checksum
make[1]: Leaving directory '/gluon-build/openwrt'
cp: cannot stat 'openwrt/bin/targets/ath79/generic/openwrt-ath79-generic-dlink_dir-825-b1-squashfs-sysupgrade.bin': No such file or directory
openwrt/staging_dir/hostpkg/bin/lua: scripts/target_lib.lua:116: assertion failed!
stack traceback:
        [C]: in function 'assert'
        scripts/target_lib.lua:116: in function <scripts/target_lib.lua:114>
        (tail call): ?
        scripts/copy_output.lua:54: in main chunk
        [C]: ?
make: *** [Makefile:194: all] Error 1

which stopped copying the other images

[mkg20001]

collecting all the missing files, copying those that exist and then printing an error about which files were missing and which targets they were from. that would be the stopgap, maybe I'll take a look.

[Djfe]

did you take a look at the size increase of the luaposix package, yet? @NeoRaider
what @mkg20001 mentioned would still be nice to have. Should I open a separate issue for that, so we don't forget about it?