Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reduce libstdc++/libc++ bloat in statically-linked executables #689

Open
strager opened this issue Apr 15, 2022 · 4 comments
Open

Reduce libstdc++/libc++ bloat in statically-linked executables #689

strager opened this issue Apr 15, 2022 · 4 comments
Labels
performance Slowness or potential optimization

Comments

@strager
Copy link
Collaborator

strager commented Apr 15, 2022

I think the biggest binary size win would be avoiding statically linking with the C++ runtimes.

On Linux, for example, I noticed that statically linking the C++ runtime is a problem because exception support gets pulled in, even if we don't use exceptions. Exception support in libstdc++ includes a C++ symbol demangler which is bloated.

We've gotten good binary size reductions in the past by avoiding some parts of the C++ standard libraries. See commits 909fae2 and dfbb182 for example.

@strager strager added the performance Slowness or potential optimization label Apr 15, 2022
@strager
Copy link
Collaborator Author

strager commented Apr 18, 2022

Audit of a Linux build (libstdc++):

  • std::__throw_length_error is called by std::vector.
  • std::__throw_out_of_range_fmt is called by std::string::substr.
  • std::__throw_logic_error is called by std::string::string.
  • std::__throw_bad_alloc is called by various things, including std::vector and std::unordered_map.
  • operator new and operator delete are called by plenty of things.
  • std::__detail::_Prime_rehash_policy::_M_next_bkt, std::__detail::_Prime_rehash_policy::_M_need_rehash, and std::_Hash_bytes are called by std::unordered_map.
  • __cxa_atexit, __cxa_guard_acquire, and __cxa_guard_release are called by file_output_stream::get_stderr, file_output_stream::get_stdout, basic_configuration_filesystem::canonicalize_path, parser::check_jsx_attribute, and boost::container::dtl::singleton_default<boost::container::pmr::new_delete_resource_imp>::instance.

I used this script to correlate imported (undefined) symbols with their callers:

# Usage: objdump -xd build-size/quick-lint-js | python slurp.py | c++filt

import collections
import re
import sys

callers = collections.defaultdict(set)
current_symbol = "???"

for line in sys.stdin:
    line = line.rstrip("\n")
    match = re.match(r"^[0-9a-f]{16} <(?P<symbol>.*)>:$", line)
    if match is not None:
        current_symbol = match.group("symbol")
    match = re.match(r".*(call|jmp).*<(?P<symbol>(_Z|__cxa).*@plt)>$", line)
    if match is not None:
        called_symbol = match.group("symbol")
        callers[called_symbol].add(current_symbol)

for callee in sorted(callers.keys()):
    print(callee)
    for caller in sorted(callers[callee]):
        print(f"\t{caller}")
    print()

@strager
Copy link
Collaborator Author

strager commented Apr 18, 2022

A plan:

  • Replace std::string file paths with a custom class.
  • Replace other uses of std::string with arrays or vector or something.
  • Replace std::vector with boost or another implementation.
  • Implement our own lazy-loading singleton.
  • If necessary: replace std::unordered_map with boost or another implementation.

@AbleBacon
Copy link

For std::vector, it may be sufficient just to override the the default allocator, since Allocator::allocate() is one of the main things that can throw exceptions. That way, you don't have to rip std::vector out completely. The same could be helpful for std::string and std::unordered_map as well.

@strager
Copy link
Collaborator Author

strager commented May 21, 2022

@AbleBacon I recall some range checks too (e.g. for std::vector<>::at), not just std::bad_alloc.

strager added a commit that referenced this issue Oct 26, 2023
Reduce our use of std::vector to reduce binary bloat:
#689
strager added a commit that referenced this issue Oct 26, 2023
Reduce our use of std::string/String8 to reduce binary bloat:
#689
strager added a commit that referenced this issue Oct 26, 2023
Reduce our use of std::string/String8 to reduce binary bloat:
#689
strager added a commit that referenced this issue Oct 26, 2023
Reduce our use of std::vector to reduce binary bloat:
#689
strager added a commit that referenced this issue Oct 27, 2023
Reduce our use of std::vector to reduce binary bloat:
#689
strager added a commit that referenced this issue Oct 27, 2023
Reduce our use of std::string/String8 to reduce binary bloat:
#689
strager added a commit that referenced this issue Oct 27, 2023
Reduce our use of std::string/String8 to reduce binary bloat:
#689
strager added a commit that referenced this issue Oct 27, 2023
Reduce our use of std::vector to reduce binary bloat:
#689
strager added a commit that referenced this issue Oct 27, 2023
Reduce our use of std::vector to reduce binary bloat:
#689
strager added a commit that referenced this issue Oct 27, 2023
Reduce our use of std::vector to reduce binary bloat:
#689
strager added a commit that referenced this issue Oct 27, 2023
Reduce our use of std::string/String8 to reduce binary bloat:
#689
strager added a commit that referenced this issue Oct 27, 2023
Reduce our use of std::string/String8 to reduce binary bloat:
#689
strager added a commit that referenced this issue Oct 27, 2023
Reduce our use of std::vector to reduce binary bloat:
#689
strager added a commit that referenced this issue Oct 27, 2023
Reduce our use of std::vector to reduce binary bloat:
#689
strager added a commit that referenced this issue Oct 27, 2023
Reduce our use of std::vector to reduce binary bloat:
#689
strager added a commit that referenced this issue Oct 27, 2023
Reduce our use of std::vector in non-test code to reduce binary bloat:
#689
strager added a commit that referenced this issue Oct 27, 2023
Reduce our use of std::vector in non-test code to reduce binary bloat:
#689
strager added a commit that referenced this issue Oct 27, 2023
Reduce our use of std::vector to reduce binary bloat:
#689
strager added a commit that referenced this issue Oct 27, 2023
Reduce our use of std::vector to reduce binary bloat:
#689
strager added a commit that referenced this issue Oct 27, 2023
Reduce our use of std::vector in non-test code to reduce binary bloat:
#689
strager added a commit that referenced this issue Oct 27, 2023
Reduce our use of std::vector to reduce binary bloat:
#689
strager added a commit that referenced this issue Oct 27, 2023
Reduce our use of std::vector to reduce binary bloat:
#689
strager added a commit that referenced this issue Oct 27, 2023
Reduce our use of std::vector in non-test code to reduce binary bloat:
#689
strager added a commit that referenced this issue Oct 27, 2023
Reduce our use of std::vector to reduce binary bloat:
#689
strager added a commit that referenced this issue Nov 6, 2023
Reduce our use of std::vector to reduce binary bloat:
#689
strager added a commit that referenced this issue Nov 6, 2023
Reduce our use of std::vector to reduce binary bloat:
#689
strager added a commit that referenced this issue Nov 6, 2023
Reduce our use of std::vector to reduce binary bloat:
#689
strager added a commit that referenced this issue Nov 6, 2023
Reduce our use of std::vector to reduce binary bloat:
#689
strager added a commit that referenced this issue Nov 6, 2023
Reduce our use of std::vector to reduce binary bloat:
#689
strager added a commit that referenced this issue Nov 6, 2023
Reduce our use of std::vector to reduce binary bloat:
#689
strager added a commit that referenced this issue Nov 6, 2023
Reduce our use of std::vector to reduce binary bloat:
#689
strager added a commit that referenced this issue Nov 6, 2023
Reduce our use of std::vector to reduce binary bloat:
#689
strager added a commit that referenced this issue Nov 6, 2023
Reduce our use of std::vector to reduce binary bloat:
#689
strager added a commit that referenced this issue Nov 6, 2023
Reduce our use of std::vector to reduce binary bloat:
#689
strager added a commit that referenced this issue Nov 6, 2023
Reduce our use of std::vector to reduce binary bloat:
#689
strager added a commit that referenced this issue Nov 6, 2023
Reduce our use of std::vector to reduce binary bloat:
#689
strager added a commit that referenced this issue Nov 6, 2023
Reduce our use of std::vector to reduce binary bloat:
#689
strager added a commit that referenced this issue Nov 7, 2023
Reduce our use of std::vector to reduce binary bloat:
#689

Because Debug_Server now uses Vector internally, make the vector
instrumentation tests for Debug_Server tolerant of extra vector
profile data.
strager added a commit that referenced this issue Nov 7, 2023
Reduce our use of std::vector to reduce binary bloat:
#689
strager added a commit that referenced this issue Nov 7, 2023
Reduce our use of std::vector to reduce binary bloat:
#689
strager added a commit that referenced this issue Nov 7, 2023
Reduce our use of std::vector to reduce binary bloat:
#689
strager added a commit that referenced this issue Nov 7, 2023
Reduce our use of std::vector to reduce binary bloat:
#689

Because Debug_Server now uses Vector internally, make the vector
instrumentation tests for Debug_Server tolerant of extra vector
profile data.
strager added a commit that referenced this issue Nov 7, 2023
Reduce our use of std::vector to reduce binary bloat:
#689
strager added a commit that referenced this issue Nov 7, 2023
Reduce our use of std::vector to reduce binary bloat:
#689
strager added a commit that referenced this issue Nov 7, 2023
Reduce our use of std::vector to reduce binary bloat:
#689
strager added a commit that referenced this issue Nov 7, 2023
Reduce our use of std::vector to reduce binary bloat:
#689
strager added a commit that referenced this issue Nov 7, 2023
Reduce our use of std::vector to reduce binary bloat:
#689
strager added a commit that referenced this issue Nov 7, 2023
Reduce our use of std::vector to reduce binary bloat:
#689
strager added a commit that referenced this issue Nov 8, 2023
Reduce our use of std::vector to reduce binary bloat:
#689
strager added a commit that referenced this issue Nov 8, 2023
Reduce our use of std::vector to reduce binary bloat:
#689
strager added a commit that referenced this issue Nov 8, 2023
Reduce our use of std::vector to reduce binary bloat:
#689
strager added a commit that referenced this issue Nov 21, 2023
Reduce our use of std::vector to reduce binary bloat:
#689
strager added a commit that referenced this issue Nov 21, 2023
Reduce our use of std::vector to reduce binary bloat:
#689
strager added a commit that referenced this issue Nov 21, 2023
Reduce our use of std::vector to reduce binary bloat:
#689
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
performance Slowness or potential optimization
Projects
None yet
Development

No branches or pull requests

2 participants