libbpf-tools/offcputime: improve stack trace format

[ekyooo]

Could you please let me what do you think about changing offcputime's stack trace format to?:

Add additional information and change format of backtrace

• add symbol base offset, dso name, dso base offset
• symbol and dso info is included if it's available in target binary
• changed format:
  INDEX ADDR [SYMBOL+OFFSET] (MODULE+OFFSET)

Before:

  # offcputime
      psiginfo
      vscanf
      __snprintf_chk
      [unknown]
      [unknown]
      [unknown]
      [unknown]
      [unknown]
      sd_event_exit
      sd_event_dispatch
      sd_event_run
      [unknown]
      __libc_start_main
      [unknown]
      -                systemd-journal (204)
          1

      xas_load
      xas_find
      filemap_map_pages
      __handle_mm_fault
      handle_mm_fault
      do_page_fault
      do_translation_fault
      do_mem_abort
      do_el0_ia_bp_hardening
      el0_ia
      xas_load
      --
  failed to get syms
      -                PmLogCtl (138757)
        1

After:

  # offcputime
      #0  0xffffffc01018b7e8 __arm64_sys_clock_nanosleep+0x0
      #1  0xffffffc01009a93c el0_svc_handler+0x34
      #2  0xffffffc010084a08 el0_svc+0x8
      #3  0xffffffc01018b7e8 __arm64_sys_clock_nanosleep+0x0
      --
      #4  0x0000007fa0bffd14 clock_nanosleep+0x94 (/usr/lib/libc-2.31.so+0x9ed14)
      #5  0x0000007fa0c0530c nanosleep+0x1c (/usr/lib/libc-2.31.so+0xa430c)
      #6  0x0000007fa0c051e4 sleep+0x34 (/usr/lib/libc-2.31.so+0xa41e4)
      #7  0x000000558a5a9608 flb_loop+0x28 (/usr/bin/fluent-bit+0x52608)
      #8  0x000000558a59f1c4 flb_main+0xa84 (/usr/bin/fluent-bit+0x481c4)
      #9  0x0000007fa0b85124 __libc_start_main+0xe4 (/usr/lib/libc-2.31.so+0x24124)
      #10 0x000000558a59d828 _start+0x34 (/usr/bin/fluent-bit+0x46828)
      -                fluent-bit (1238)
          1

      #0  0xffffffc01027daa4 generic_copy_file_checks+0x334
      #1  0xffffffc0102ba634 __handle_mm_fault+0x8dc
      #2  0xffffffc0102baa20 handle_mm_fault+0x168
      #3  0xffffffc010ad23c0 do_page_fault+0x148
      #4  0xffffffc010ad27c0 do_translation_fault+0xb0
      #5  0xffffffc0100816b0 do_mem_abort+0x50
      #6  0xffffffc0100843b0 el0_da+0x1c
      #7  0xffffffc01027daa4 generic_copy_file_checks+0x334
      --
  failed to get syms
      #8  0x0000007f8dc12648
      #9  0x0000007f8dc0aef8
      #10 0x0000007f8dc1c990
      #11 0x0000007f8dc08b0c
      #12 0x0000007f8dc08e48
      #13 0x0000007f8dc081c8
      -                PmLogCtl (2412)
          1

You can see the code change for this in this patch.
5a07dfe

[yonghong-song]

I think it is a good for certain use cases, esp. you might get an address for a [unknown] entry and you can check it manually with the binary to find out what is going on. bcc tool wakeuptime.py has an option for this purpose:

parser.add_argument("-v", "--verbose", action="store_true",
    help="show raw addresses")

I think we can add -v option to offcputime (in libbpf-tools) and offcputime.py (in traditional bcc tools).
Could you send two patches for this? One is for libbpf-tools and the other is for bcc python-based tools? Thanks.

[ekyooo]

Okay. I'll send two patches for libbpf-tools and python-based tool.

• For the -v option, libbpf-tools uses -v in the sense of "verbose level (debug) log". So after adding this the option has the additional meaning of "verbose output format". Do you agree?
• Is 'tp_btf/sched_switch' not supported in linux v5.4? I check before setting up qemu for a higher Linux version.

Thank you for your opinion.