Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Application core dumps when connecting to mdash if file system or file IO is used #8

Open
jimarcher opened this issue Apr 18, 2020 · 2 comments
Open

Application core dumps when connecting to mdash if file system or file IO is used #8

jimarcher opened this issue Apr 18, 2020 · 2 comments

Comments

@jimarcher
Copy link

Hard to pin own exactly which library causes this problem, but the executive summary is that if an ESP32 application uses either a file system on SPI flash (as with the ESP32-WROOM-32U module) and/or uses the c language file IO functions to talk to a file system created by Mongoose, then a core dump quickly happens after boot.

Code to reproduce is below, with backtraces. This application will run reliably until an MDash token is added. It then core dumps almost immediately.

Call to set mdash token (token edited)

mos config-set dash.enable=true dash.token=dOS0j99ABCDEFGE4B3mLxxx

mos.yml

    author: mongoose-os
    description: A Mongoose OS app skeleton
    version: 1.0

    libs_version: ${mos.version}
    modules_version: ${mos.version}
    mongoose_os_version: ${mos.version}

    # Optional. List of tags for online search.
    tags:
      - c

    # List of files / directories with C sources. No slashes at the end of dir names.
    sources:
      - src

    # List of dirs. Files from these dirs will be copied to the device filesystem
    filesystem:
      - fs

    build_vars:
      ESP_IDF_SDKCONFIG_OPTS: ${build_vars.ESP_IDF_SDKCONFIG_OPTS} CONFIG_ESPTOOLPY_FLASHSIZE_4MB=}
      ESP_IDF_SDKCONFIG_OPTS: ${build_vars.ESP_IDF_SDKCONFIG_OPTS} CONFIG_ESPTOOLPY_FLASHSIZE_16MB=y CONFIG_ESPTOOLPY_FLASHSIZE="16MB"}
      ESP_IDF_EXTRA_PARTITION: fs_ext,data,spiffs,,12288K

    config_schema:
    - ["fstab.fs0.dev", "fs_ext"]
    - ["fstab.fs0.type", "LFS"]
    - ["fstab.fs0.opts", '{"bs": 4096}']
    - ["fstab.fs0.path", "/data"]
    - ["fstab.fs0.create", true]

    - ["wifi.sta.enable", true]
    - ["wifi.sta.ssid", "Jim LAN"]
    - ["wifi.sta.pass", "password"]

    libs:
      - origin: https://github.com/mongoose-os-libs/ca-bundle
      - origin: https://github.com/mongoose-os-libs/rpc-service-config
      - origin: https://github.com/mongoose-os-libs/rpc-service-fs
      - origin: https://github.com/mongoose-os-libs/rpc-uart
      - origin: https://github.com/mongoose-os-libs/dash
      - origin: https://github.com/mongoose-os-libs/rpc-service-ota
      - origin: https://github.com/mongoose-os-libs/wifi

      # Added for file system
      - origin: https://github.com/mongoose-os-libs/fstab
      - origin: https://github.com/mongoose-os-libs/vfs-fs-lfs

    # Used by the mos tool to catch mos binaries incompatible with this file format
    manifest_version: 2017-09-29

main.cpp

    /*
    boot: Partition Table:�[0m
    [Apr 13 23:04:26.809] �[0;32mI (65) boot: ## Label            Usage          Type ST Offset   Length   Flags�[0m
    [Apr 13 23:04:26.809] �[0;32mI (73) boot:  0 nvs              WiFi data        01 02 00009000 00004000 00000000�[0m
    [Apr 13 23:04:26.809] �[0;32mI (82) boot:  1 otadata          OTA data         01 00 0000d000 00002000 00000000�[0m
    [Apr 13 23:04:26.820] �[0;32mI (90) boot:  2 app_0            OTA app          00 10 00010000 00180000 00000000�[0m
    [Apr 13 23:04:26.825] �[0;32mI (98) boot:  3 fs_0             FS               01 82 00190000 00040000 00000000�[0m
    [Apr 13 23:04:26.836] �[0;32mI (106) boot:  4 app_1            OTA app          00 11 001d0000 00180000 00000000�[0m
    [Apr 13 23:04:26.842] �[0;32mI (115) boot:  5 fs_1             FS               01 82 00350000 00040000 00000000�[0m
    [Apr 13 23:04:26.853] �[0;32mI (123) boot:  6 fs_ext           FS               01 82 00390000 00c00000 00000000�[0m
    [Apr 13 23:04:26.859] �[0;32mI (131) boot: End of partition table�[0m
    */

    #include "mgos.h"
    #include "mgos_system.h"

    #include <stdio.h>
    #include <string.h>
    #include <sys/unistd.h>
    #include <sys/stat.h>
    #include "esp_err.h"
    #include "esp_log.h"

    static void timerHeapCb(void *args)
    {
      LOG(LL_INFO, ("Heap Size: %d", mgos_get_heap_size()));
      LOG(LL_INFO, ("Free Heap Size: %d", mgos_get_free_heap_size()));
    }

    static void timerCb(void *args)
    {
      static int counter = 0;
      LOG(LL_INFO, ("Timer called: %d", counter));

      FILE* f = fopen("/data/log.txt", "a");
      if (f == NULL)
      {
        LOG(LL_INFO, ("%s","Failed to open file for writing"));
        return;
      }
      fprintf(f, "Hello World - AAA! %d\n", counter++);
      fclose(f);
    }

    enum mgos_app_init_result mgos_app_init(void)
    {
      LOG(LL_INFO, ("%s", "We are in the Entry point!!!!!"));
      mgos_set_timer(2500, MGOS_TIMER_REPEAT, timerCb, NULL);
      mgos_set_timer(300, MGOS_TIMER_REPEAT, timerHeapCb, NULL);
      return MGOS_APP_INIT_SUCCESS;
    }

Back trace:

    Jims-MacBook-Pro-486:objs jim$ /Users/jim/Library/Arduino15/packages/esp32/tools/xtensa-esp32-elf-gcc/1.22.0-80-g6c4433a-5.2.0/bin/xtensa-esp32-elf-addr2line -pfiaC -e File_System_Test_App.elf 0x4000c472 0x4015c027 0x40157ec6 0x4017aeda 0x4017a1a4 0x4017a2ae 0x4017d1be 0x40179cf6 0x40179d46 0x4016c129 0x4016d6d5 0x4016e87a 0x401704a0 0x401704cf 0x40181fdd 0x4016a2d9 0x40084002 0x400841c9

    0x4000c472: ?? ??:0

    0x4015c027: mbedtls_platform_zeroize at /data/tmp/mos_prebuild/tmp/mos_libs/mbedtls/mbedtls/library/platform_util.c:78

    0x40157ec6: mbedtls_mpi_zeroize at /data/tmp/mos_prebuild/tmp/mos_libs/mbedtls/mbedtls/library/bignum.c:763

     (inlined by) mbedtls_mpi_free at /data/tmp/mos_prebuild/tmp/mos_libs/mbedtls/mbedtls/library/bignum.c:109

    0x4017aeda: ecdh_free_internal at /data/tmp/mos_prebuild/tmp/mos_libs/mbedtls/mbedtls/library/ecdh.c:284

     (inlined by) mbedtls_ecdh_free at /data/tmp/mos_prebuild/tmp/mos_libs/mbedtls/mbedtls/library/ecdh.c:321

    0x4017a1a4: mbedtls_ssl_handshake_free at /data/tmp/mos_prebuild/tmp/mos_libs/mbedtls/mbedtls/library/ssl_tls.c:7798

    0x4017a2ae: ssl_handshake_wrapup_free_hs_transform at /data/tmp/mos_prebuild/tmp/mos_libs/mbedtls/mbedtls/library/ssl_tls.c:7798

     (inlined by) mbedtls_ssl_handshake_wrapup at /data/tmp/mos_prebuild/tmp/mos_libs/mbedtls/mbedtls/library/ssl_tls.c:6604

    0x4017d1be: mbedtls_ssl_handshake_client_step at /data/tmp/mos_prebuild/tmp/mos_libs/mbedtls/mbedtls/library/ssl_cli.c:3763

    0x40179cf6: mbedtls_ssl_handshake_step at /data/tmp/mos_prebuild/tmp/mos_libs/mbedtls/mbedtls/library/ssl_tls.c:7798

    0x40179d46: mbedtls_ssl_handshake at /data/tmp/mos_prebuild/tmp/mos_libs/mbedtls/mbedtls/library/ssl_tls.c:7798

    0x4016c129: mg_ssl_if_handshake at /data/tmp/mos_prebuild/tmp/cesanta/mos-libs/mongoose/build/objs/moslib/mongoose/src/mg_net.c:1155

    0x4016d6d5: mg_ssl_handshake at /data/tmp/mos_prebuild/tmp/cesanta/mos-libs/mongoose/build/objs/moslib/mongoose/src/mg_net.c:1155

    0x4016e87a: mg_recv_tcp at /data/tmp/mos_prebuild/tmp/cesanta/mos-libs/mongoose/build/objs/moslib/mongoose/src/mg_net.c:1155

     (inlined by) mg_do_recv at /data/tmp/mos_prebuild/tmp/cesanta/mos-libs/mongoose/build/objs/moslib/mongoose/src/mg_net.c:582

    0x401704a0: mg_if_can_recv_cb at /data/tmp/mos_prebuild/tmp/cesanta/mos-libs/mongoose/build/objs/moslib/mongoose/src/mg_net.c:1155

     (inlined by) mg_ev_mgr_lwip_process_signals at /data/tmp/mos_prebuild/tmp/cesanta/mos-libs/mongoose/build/objs/moslib/common/platforms/lwip/mg_lwip_ev_mgr.c:60

    0x401704cf: mg_lwip_if_poll at /data/tmp/mos_prebuild/tmp/cesanta/mos-libs/mongoose/build/objs/moslib/mongoose/src/mg_net.c:1155

    0x40181fdd: mg_mgr_poll at /data/tmp/mos_prebuild/tmp/cesanta/mos-libs/mongoose/build/objs/moslib/mongoose/src/mg_net.c:1155

    0x4016a2d9: mongoose_poll at /data/tmp/mos_prebuild/tmp/cesanta/mos-libs/mongoose/src/mgos_mongoose.c:239

    0x40084002: mgos_mg_poll_cb at /data/fwbuild-volumes/latest/apps/File_System_Test_App/esp32/build_contexts/build_ctx_917480771/deps/freertos/src/mgos_freertos.c:257

    0x400841c9: mgos_task at /data/fwbuild-volumes/latest/apps/File_System_Test_App/esp32/build_contexts/build_ctx_917480771/deps/freertos/src/mgos_freertos.c:257

    Jims-MacBook-Pro-486:objs jim$

Another backtrace:

    Jims-MacBook-Pro-486:objs jim$ /Users/jim/Library/Arduino15/packages/esp32/tools/xtensa-esp32-elf-gcc/1.22.0-80-g6c4433a-5.2.0/bin/xtensa-esp32-elf-addr2line -pfiaC -e File_System_Test_App.elf 0x4009234f 0x40081d31 0x40082215 0x4000bec7 0x4012f384 0x4012f40f 0x4012f463 0x40132ee7 0x40137f00 0x4013bcf6 0x4012f161

    0x4009234f: multi_heap_free_impl at /opt/Espressif/esp-idf/components/heap/multi_heap.c:699

    0x40081d31: heap_caps_free at /opt/Espressif/esp-idf/components/heap/heap_caps.c:365

    0x40082215: _free_r at /opt/Espressif/esp-idf/components/newlib/syscalls.c:42

    0x4000bec7: ?? ??:0

    0x4012f384: mem_free at /opt/Espressif/esp-idf/components/lwip/lwip/src/core/mem.c:176

    0x4012f40f: do_memp_free_pool at /opt/Espressif/esp-idf/components/lwip/lwip/src/core/memp.c:231

    0x4012f463: memp_free at /opt/Espressif/esp-idf/components/lwip/lwip/src/core/memp.c:231

    0x40132ee7: pbuf_free at /opt/Espressif/esp-idf/components/lwip/lwip/src/core/pbuf.c:1331

    0x40137f00: ip4_input at /opt/Espressif/esp-idf/components/lwip/lwip/src/core/ipv4/ip4.c:454

    0x4013bcf6: ethernet_input at /opt/Espressif/esp-idf/components/lwip/lwip/src/netif/ethernet.c:184

    0x4012f161: tcpip_thread at /opt/Espressif/esp-idf/components/lwip/lwip/src/api/tcpip.c:483

    Jims-MacBook-Pro-486:objs jim$
@jimarcher
Copy link
Author

@nliviu reports a workaround for this is to switch to single core:

build_vars:
  ESP_IDF_SDKCONFIG_OPTS: >
    ${build_vars.ESP_IDF_SDKCONFIG_OPTS} 
      CONFIG_ESPTOOLPY_FLASHSIZE_4MB=
      CONFIG_ESPTOOLPY_FLASHSIZE_16MB=y
      CONFIG_ESPTOOLPY_FLASHSIZE=16MB
      CONFIG_FREERTOS_UNICORE=y

This workaround did work for me. Of course it leaves Mongoose back to single core.

@rojer
Copy link
Contributor

rojer commented Apr 21, 2020

so it crashes in mbedtls with multicore enabled? interesting.
thanks for reporting, i will investigate.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@rojer @jimarcher