Skip to content

Various materials about Profile Guided Optimization and other similar stuff like AutoFDO, Bolt, etc.

License

Notifications You must be signed in to change notification settings

zamazan4ik/awesome-pgo

Repository files navigation

awesome-pgo

Various materials about Profile Guided Optimization (PGO) and other similar stuff like AutoFDO, Bolt, etc.

!!!ARTICLE!!!

There is an (unfinished) article about all the details about PGO, PLO, etc. - link. With high chance, it will answer (almost) all your questions about PGO and PLO.

How to fail with PGO?

The PGO implementer

Theory (a little bit)

Also, you could find PDO (Profile Directed Optimization), FDO (Feedback Driven Optimization), FBO (Feedback Based Optimization), PDF (Profile Directed Feedback), PBO (Profile Based Optimization) - do not worry, that's just a PGO but with a different name.

Additionally, I need to mention Link-Time Optimization (LTO) since usually PGO is applied after LTO (since usually LTO is easier to enable and it brings significant performance and/or binary size improvements). PGO does not replace LTO but complements it. More information about LTO can be found in lto.md.

PGO Showcases

Here I collect links to the articles/benchmarks/etc. with PGO on multiple projects (with numbers!).

Browsers

Compilers and interpreters

Developer tooling

Operating systems

Virtual machines

Databases

Logging

Proxy

Other

Projects with already integrated PGO into their build scripts

Below you can find some examples of where and how PGO is integrated into different projects.

Project-specific documentation about PGO

Here we collect projects where PGO is described as an optimization option in the documentation:

PGO support in programming languages and compilers

Possibly other compilers support PGO too. If you know any, please let me know.

PGO support in build systems

Here we collect and track PGO integrations into build systems:

  • Cargo: No built-in support but there is awesome cargo-pgo
  • Bazel: Supports (command-line reference)
  • CMake: No support yet (GitLab issue)
  • Meson: Supports (b_pgo in the docs).
  • SCons: No support yet (GitHub discussion)
  • YPKG (Solus): The addition of the profile key the in the package.yml format will perform an automatic PGO build docs. (Example)
  • boulder (Serpent OS): The addition of the workload key in the stone.yaml format will perform an automatic PGO build TODO: no docs available?

Are we PGO yet?

Check "are_we_pgo_yet.md" file in the repo to check the PGO status in a project.

BOLT showcases

Here I collect all results by applying LLVM BOLT to the projects (with numbers).

Projects with already integrated BOLT into their build scripts

Are we BOLT yet?

Just a list of BOLT-related issues in different projects. So you can estimate the BOLT state in your favorite open-source product.

PGO adoption across Linux distros

Even if PGO is supported by a project, it does not mean that your favorite Linux distro builds this project with PGO enabled. For this there are a lot of reasons: maintainer burden (because we are humans (yet)), build machines burden (in general you need to compile twice), reproducibility issues (like profile is an additional input to the build process and you need to make it reproducible), a maintainer just don't know about PGO, etc.

So here I will try to collect information about the PGO status across the Linux distros for the projects that support PGO in the upstream. If you didn't find your distro - don't worry! Just check it somehow (probably in some chats/distros' build systems, etc.) and report it here (e.g. via Issues) - I will add it to the list.

  • GCC:
    • Note: PGO for GCC usually is not enabled for all architectures since it requires too much from the build systems
    • Debian: yes
    • Ubuntu: same as Debian
    • RedHat: Yes. And that is the reason why PGO is enabled for GCC in all RedHat-based distros.
    • Fedora: yes
    • Rocky Linux: yes
    • Alma Linux: yes
    • NixOS: no
    • OpenSUSE: yes, see line 2414
    • Solus: yes
  • Clang:
    • Binaries from LLVM are already PGO-optimized (according to the note about using "stage2" build - it's PGO optimized build)
    • RedHat (CentOS Stream): no
    • Fedora: no
    • AlmaLinux: no
    • Rocky Linux: no
    • NixOS: no
    • Arch Linux: sent an email to the Clang maintainer in Arch Linux - no response yet
    • Solus: yes
  • Rustc:
  • CPython:
    • Fedora: yes. Also, check this discussion. I guess other RedHat-based distro builds are the same for this package (however I didn't check it but Rocky Linux is the same).
    • Solus: yes
  • LDC:

BOLT adoption across Linux distros

Here we track LLVM BOLT enablement across various projects in various OS-specific build scripts:

Meta-issues about PGO and LLVM BOLT usage in different OSs and package managers:

Useful links

My talks about PGO

I spent some time on giving talks about PGO at different conferences. Multiple times people were asking me to share slides/videos. Here they are.

Few notes about them:

  • None of this talks contains all the material from this repository since it's impossible to cover all PGO nuances during a talk at a conference. That's why reading the article is highly recommended anyway.

  • You can see that multiple talks have the same or almost the same content. That's fine since different conferences usually have different people that didn't listen talks from other conferences. For you it means that you don't need to watch / listen all the presentations. But if you want - you are welcome!

  • TechSpot 2024 "Performance meetup" (Warsaw, Poland), “Profile-Guided Optimization (PGO): 101” (video, slides)

  • Winglang Community Meeting March 2024 (Online), “Profile-Guided Optimization (PGO) for Winglang” (slides)

  • Tech Internals Conf 2024 (Cyprus), “Profile-Guided Optimization (PGO): (Ab)using it for fun and profit” (video, slides)

  • OxidizeConf 2024 (Berlin, Germany), “Profile-Guided Optimization (PGO) in Rust - 101” (video, slides)

  • RustFest 2024 (Zurich, Switzerland), “Profile-Guided Optimization (PGO) in Rust: unknown parts” (slides)

  • p99conf 2024 (Online), “Profile-Guided Optimization (PGO): (Ab)using it for Fun and Profit” (video, slides)

  • RustLab 2024 (Florence, Italy), “Profile-Guided Optimization (PGO) for Rust applications: Practical Guide” (video, slides)

Related projects

Contribute

If you have an example where PGO shines (and where doesn't) - please open an issue and/or PR to the repo. It's important to collect as many as possible showcases about PGO!

About

Various materials about Profile Guided Optimization and other similar stuff like AutoFDO, Bolt, etc.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published