Add benchmarks for rinja

[GuillaumeGomez]

rinja is a askama fork made by me and another askama maintainer. More information about why and the main differences can be found here.

Since rinja diverged "enough", I think it's worth adding it to the benchmarks list. By running it locally, I got:

Found 8 outliers among 100 measurements (8.00%)
  8 (8.00%) high mild

cmp_render/minijinja    time:   [9.1311 µs 9.1745 µs 9.2243 µs]
Found 7 outliers among 100 measurements (7.00%)
  6 (6.00%) high mild
  1 (1.00%) high severe
cmp_render/tera         time:   [11.926 µs 11.945 µs 11.967 µs]
Found 12 outliers among 100 measurements (12.00%)
  4 (4.00%) high mild
  8 (8.00%) high severe
cmp_render/liquid       time:   [17.181 µs 17.301 µs 17.432 µs]
Found 5 outliers among 100 measurements (5.00%)
  1 (1.00%) high mild
  4 (4.00%) high severe
cmp_render/handlebars   time:   [10.564 µs 10.622 µs 10.700 µs]
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high severe
cmp_render/rinja        time:   [1.7499 µs 1.7508 µs 1.7520 µs]
Found 20 outliers among 100 measurements (20.00%)
  20 (20.00%) high mild
cmp_render/askama       time:   [2.1653 µs 2.1697 µs 2.1747 µs]

I didn't add the numbers in the PR though but I can update them with mine. Not sure how you prefer to do it.

In any case, thanks for maintaining this project and this bench comparison list!

[zamazan4ik]

I found this Reddit post and decided to perform some Profile-Guided Optimization (PGO) benchmarks to check - is it possible or not to improve the libraries' performance with PGO. Below are the results. If you are interested in other PGO-related benchmarks for other projects, you can check the awesome-pgo repo.

Test environment

• Fedora 40
• Linux kernel 6.10.11
• AMD Ryzen 9 5900x
• 48 Gib RAM
• SSD Samsung 980 Pro 2 Tib
• Compiler - Rustc 1.81.0
• Libraries version: this branch
• Disabled Turbo boost

Benchmark

For PGO optimization I use cargo-pgo tool. Release bench results I got with taskset -c 0 cargo bench --bench comparison command. The PGO training phase is done with taskset -c 0 cargo pgo bench -- --bench comparison, PGO optimization phase - with taskset -c 0 cargo pgo optimize bench -- --bench comparison.

taskset -c 0 is used to reduce the OS scheduler's influence on the results. All measurements are done on the same machine, with the same background "noise" (as much as I can guarantee).

Results

I got the following results:

• Release: https://gist.github.com/zamazan4ik/79b0607f544b4fdfc758587120a9a5fa
• PGO optimized compared to Release: https://gist.github.com/zamazan4ik/2a7ef042e57e5705a62c4d7ef82535e8
• (just for reference) PGO instrumented compared to Release: https://gist.github.com/zamazan4ik/44f32b7cf8f6d96228346a9f782258cf

According to the results ("PGO optimized compared to Release"), PGO measurably improves the libraries' performance.

Further steps

At the very least, the libraries' users can find this performance report and decide to enable PGO for their applications if they care about performance in their workloads. Maybe a small note somewhere in the documentation (the README file?) in the corresponding libraries repos will be enough to raise awareness about this possible performance improvement.

[mitsuhiko]

Thanks. I will try to run them when convenient on the machine I ran them last time.