Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

libbpf-tools/offcputime: Add dso info and symbol offset to backtrace for -v option #4288

Merged
merged 1 commit into from
Oct 27, 2022

Conversation

ekyooo
Copy link
Contributor

@ekyooo ekyooo commented Oct 22, 2022

Add additional information and change format of backtrace

  • add symbol base offset, dso name, dso base offset
  • symbol and dso info is included if it's available in target binary
  • changed format: INDEX ADDR [SYMBOL+OFFSET] (MODULE+OFFSET)

Print backtrace of ip if it failed to get syms.

Before:

  # offcputime -v
      psiginfo
      vscanf
      __snprintf_chk
      [unknown]
      [unknown]
      [unknown]
      [unknown]
      [unknown]
      sd_event_exit
      sd_event_dispatch
      sd_event_run
      [unknown]
      __libc_start_main
      [unknown]
      -                systemd-journal (204)
          1

      xas_load
      xas_find
      filemap_map_pages
      __handle_mm_fault
      handle_mm_fault
      do_page_fault
      do_translation_fault
      do_mem_abort
      do_el0_ia_bp_hardening
      el0_ia
      xas_load
      --
  failed to get syms
      -                PmLogCtl (138757)
        1

After:

  # offcputime -v
      #0  0xffffffc01018b7e8 __arm64_sys_clock_nanosleep+0x0
      #1  0xffffffc01009a93c el0_svc_handler+0x34
      #2  0xffffffc010084a08 el0_svc+0x8
      #3  0xffffffc01018b7e8 __arm64_sys_clock_nanosleep+0x0
      --
      #4  0x0000007fa0bffd14 clock_nanosleep+0x94 (/usr/lib/libc-2.31.so+0x9ed14)
      #5  0x0000007fa0c0530c nanosleep+0x1c (/usr/lib/libc-2.31.so+0xa430c)
      #6  0x0000007fa0c051e4 sleep+0x34 (/usr/lib/libc-2.31.so+0xa41e4)
      #7  0x000000558a5a9608 flb_loop+0x28 (/usr/bin/fluent-bit+0x52608)
      #8  0x000000558a59f1c4 flb_main+0xa84 (/usr/bin/fluent-bit+0x481c4)
      #9  0x0000007fa0b85124 __libc_start_main+0xe4 (/usr/lib/libc-2.31.so+0x24124)
      #10 0x000000558a59d828 _start+0x34 (/usr/bin/fluent-bit+0x46828)
      -                fluent-bit (1238)
          1

      #0  0xffffffc01027daa4 generic_copy_file_checks+0x334
      #1  0xffffffc0102ba634 __handle_mm_fault+0x8dc
      #2  0xffffffc0102baa20 handle_mm_fault+0x168
      #3  0xffffffc010ad23c0 do_page_fault+0x148
      #4  0xffffffc010ad27c0 do_translation_fault+0xb0
      #5  0xffffffc0100816b0 do_mem_abort+0x50
      #6  0xffffffc0100843b0 el0_da+0x1c
      #7  0xffffffc01027daa4 generic_copy_file_checks+0x334
      --
      #8  0x0000007f8dc12648 [unknown]
      #9  0x0000007f8dc0aef8 [unknown]
      #10 0x0000007f8dc1c990 [unknown]
      #11 0x0000007f8dc08b0c [unknown]
      #12 0x0000007f8dc08e48 [unknown]
      #13 0x0000007f8dc081c8 [unknown]
    -                PmLogCtl (2412)
        1

Fixed: #3884
Signed-off-by: Eunseon Lee [email protected]

…for -v option

Add additional information and change format of backtrace
- add symbol base offset, dso name, dso base offset
- symbol and dso info is included if it's available in target binary
- changed format:
INDEX ADDR [SYMBOL+OFFSET] (MODULE+OFFSET)

Print backtrace of ip if it failed to get syms.

Before:
  # offcputime -v
    psiginfo
    vscanf
    __snprintf_chk
    [unknown]
    [unknown]
    [unknown]
    [unknown]
    [unknown]
    sd_event_exit
    sd_event_dispatch
    sd_event_run
    [unknown]
    __libc_start_main
    [unknown]
    -                systemd-journal (204)
        1

    xas_load
    xas_find
    filemap_map_pages
    __handle_mm_fault
    handle_mm_fault
    do_page_fault
    do_translation_fault
    do_mem_abort
    do_el0_ia_bp_hardening
    el0_ia
    xas_load
    --
failed to get syms
      -                PmLogCtl (138757)
        1

After:
  # offcputime -v
    #0  0xffffffc01018b7e8 __arm64_sys_clock_nanosleep+0x0
    iovisor#1  0xffffffc01009a93c el0_svc_handler+0x34
    iovisor#2  0xffffffc010084a08 el0_svc+0x8
    iovisor#3  0xffffffc01018b7e8 __arm64_sys_clock_nanosleep+0x0
    --
    iovisor#4  0x0000007fa0bffd14 clock_nanosleep+0x94 (/usr/lib/libc-2.31.so+0x9ed14)
    iovisor#5  0x0000007fa0c0530c nanosleep+0x1c (/usr/lib/libc-2.31.so+0xa430c)
    iovisor#6  0x0000007fa0c051e4 sleep+0x34 (/usr/lib/libc-2.31.so+0xa41e4)
    iovisor#7  0x000000558a5a9608 flb_loop+0x28 (/usr/bin/fluent-bit+0x52608)
    iovisor#8  0x000000558a59f1c4 flb_main+0xa84 (/usr/bin/fluent-bit+0x481c4)
    iovisor#9  0x0000007fa0b85124 __libc_start_main+0xe4 (/usr/lib/libc-2.31.so+0x24124)
    iovisor#10 0x000000558a59d828 _start+0x34 (/usr/bin/fluent-bit+0x46828)
    -                fluent-bit (1238)
        1

    #0  0xffffffc01027daa4 generic_copy_file_checks+0x334
    iovisor#1  0xffffffc0102ba634 __handle_mm_fault+0x8dc
    iovisor#2  0xffffffc0102baa20 handle_mm_fault+0x168
    iovisor#3  0xffffffc010ad23c0 do_page_fault+0x148
    iovisor#4  0xffffffc010ad27c0 do_translation_fault+0xb0
    iovisor#5  0xffffffc0100816b0 do_mem_abort+0x50
    iovisor#6  0xffffffc0100843b0 el0_da+0x1c
    iovisor#7  0xffffffc01027daa4 generic_copy_file_checks+0x334
    --
    iovisor#8  0x0000007f8dc12648 [unknown]
    iovisor#9  0x0000007f8dc0aef8 [unknown]
    iovisor#10 0x0000007f8dc1c990 [unknown]
    iovisor#11 0x0000007f8dc08b0c [unknown]
    iovisor#12 0x0000007f8dc08e48 [unknown]
    iovisor#13 0x0000007f8dc081c8 [unknown]
    -                PmLogCtl (2412)
        1

Fixed: iovisor#3884
Signed-off-by: Eunseon Lee <[email protected]>
@brendangregg
Copy link
Member

Why include the offset in MODULE+OFFSET? I don't really mind, but I want to know if there's a precedent.

Note that stackcount added -s for symbol offsets, but since this adds more fields I'm ok with it being -v.

@ekyooo
Copy link
Contributor Author

ekyooo commented Oct 22, 2022

Why include the offset in MODULE+OFFSET? I don't really mind, but I want to know if there's a precedent.

Note that stackcount added -s for symbol offsets, but since this adds more fields I'm ok with it being -v.

I think Leaksanitizer can be a precedent for this format. (https://github.com/google/sanitizers/wiki/AddressSanitizerLeakSanitizer)

Direct leak of 5 byte(s) in 1 object(s) allocated from:
    #0 0x44f2de in malloc /usr/home/hacker/llvm/projects/compiler-rt/lib/asan/asan_malloc_linux.cc:74
    #1 0x464e86 in Baz() (/usr/home/hacker/a.out+0x464e86)  //  MODULE+OFFSET
    #2 0x464fb4 in main (/usr/home/hacker/a.out+0x464fb4)  //  MODULE+OFFSET
    #3 0x7f7e760b476c in __libc_start_main /build/buildd/eglibc-2.15/csu/libc-start.c:226

We can also use the module name and module offset instead of symbol information if symbol resolution fails as in the example below.

                 #2 0x7f97fea90c (/usr/lib/libcbe.so+0x1baf90c)                                                                                                                    
                 #3 0x7f99250430  (/usr/lib/libcbe.so+0x2e15430)                                                                                                                    
                 #4 0x7f99250ae8 (/usr/lib/libcbe.so+0x2e15ae8)                                                                                                                    
                 #5 0x7f9925ac40 (/usr/lib/libcbe.so+0x2e1fc40)                                                                                                                    
                 #6 0x7f9925afc0 (/usr/lib/libcbe.so+0x2e1ffc0)                                                                                                                    
                 #7 0x7f9921c8d4 (/usr/lib/libcbe.so+0x2de18d4)   

Also, if a module has multiple static functions with the same name, we can use MODULE OFFSET to differentiate them as I think.

Thank you.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

libbpf-tools/offcputime: improve stack trace format
3 participants