Profile-Guided Optimization (PGO) benchmark results

[zamazan4ik]

(initially was posted as a Reddit comment)

Hi!

I decided to test Profile-Guided Optimization (PGO) effects on the library performance (as I already did for many other applications - check https://github.com/zamazan4ik/awesome-pgo ). Unfortunately, in the  https://github.com/lovasoa/serde-sqlite-jsonb repo both Issues and Discussions are disabled so I leave my results here. Such information can be valuable for users who want to extract additional performance from the library.

Test environment

• Fedora 39
• Linux kernel 6.8.4
• AMD Ryzen 9 5900x
• 48 Gib RAM
• SSD Samsung 980 Pro 2 Tib
• Compiler: Rustc 1.76
• library version: the latest for now from the master branch on commit 6088de7023a4cd1b8c72eca3459dd4cfd5ae8d50
• Disabled Turbo boost (for better results consistency across runs)

Benchmark

For benchmark purposes, I use built-in benchmarks. Release bench is done with taskset -c 0 cargo bench, PGO instrumentation - with taskset -c 0 cargo pgo bench, PGO optimization - with taskset -c 0 cargo pgo optimize bench. cargo-pgo is used for performing all PGO-related routines. As a PGO training workload, the same benchmark was used. taskset is used for reducing OS scheduler noise.

All tests are done on the same machine, done multiple times (results are the same), with the same background "noise" (as much as I can guarantee of course) - the results are reproducible at least on my machine.

Results

Here are the results:

• Release: https://gist.github.com/zamazan4ik/dea1f376ede6237d68cc0f9503a3dde2
• PGO-optimized compared to Release: https://gist.github.com/zamazan4ik/1c0f3c03712d3efa452e565f496f1b65
• (just for reference) PGO-instrumented compared to Release: https://gist.github.com/zamazan4ik/63416a4c6ee3d4b13a6933864caf5a92

At least to the simple benchmarks above, PGO measurably improves the library's performance. However, I understand that these benchmarks can be too synthetic.

Maybe putting some note about PGO effects on the library performance somewhere in the README is worth it. Also, it will be nice if you open Issues and/or Discussion in the repo ;)

Moved this content from Reddit to repo to discuss it further here (just in case).