Add script to benchmark compilation times

[sveitser]

Docs on cargo profiles: https://doc.rust-lang.org/cargo/reference/profiles.html#debug

Find some tradeoffs for how long compilation takes and how convenient it is to spot issues. Ideally we would have all debug info and the fastest compilation times but that's not possible.

For the CI we definitely don't need debuggers to work but it would be nice to have tracebacks because it's very time consuming to re-run it with a different profile. For local development we should IMO have tracebacks enabled by default for running tests and running the demo. Maybe it should also be possible to attach a debugger without having to re-compile after running tests.

Time it takes to run cargo build in seconds:

debug-none-strip-none = 205
debug-line-tables-only-strip-none = 231 # tracebacks with line numbers
debug-limited-strip-none = 235
debug-full-strip-none = 318
debug-none-strip-debuginfo = 204
debug-line-tables-only-strip-debuginfo = 224 # minimal useful tracebacks
debug-limited-strip-debuginfo = 228
debug-none-strip-symbols = 203 # what we currently use
debug-line-tables-only-strip-symbols = 226
debug-limited-strip-symbols = 247
debug-full-strip-symbols = 331

Currently the results show that in order to get minimally useful tracebacks we take about a 10% hit in compilation time. For tracebacks with line numbers it's closer to 15%.

In order to get any meaningful tracebacks we need at least debug = line-tables-only and strip = debuginfo. This gives tracebacks that don't include line numbers except the place in the code where the panic occured. I think this is what release does by default.

cargo run --profile debug-line-tables-only-strip-debuginfo  
thread 'main' panicked at src/main.rs:59:10:
called `Option::unwrap()` on a `None` value
stack backtrace:
   0: rust_begin_unwind
   1: core::panicking::panic_fmt
   2: core::panicking::panic
   3: core::option::unwrap_failed
   4: rust_playground::my_fun
   5: rust_playground::main
   6: core::ops::function::FnOnce::call_once

In order to get tracebacks with all files and line numbers we need strip = "none"

cargo run --profile debug-line-tables-only-strip-none 
called `Option::unwrap()` on a `None` value
stack backtrace:
   0: rust_begin_unwind
             at /rustc/4d669fb34e7db6f3825d01e4c59b7996f0531431/library/std/src/panicking.rs:681:5
   1: core::panicking::panic_fmt
             at /rustc/4d669fb34e7db6f3825d01e4c59b7996f0531431/library/core/src/panicking.rs:75:14
   2: core::panicking::panic
             at /rustc/4d669fb34e7db6f3825d01e4c59b7996f0531431/library/core/src/panicking.rs:145:5
   3: core::option::unwrap_failed
             at /rustc/4d669fb34e7db6f3825d01e4c59b7996f0531431/library/core/src/option.rs:2009:5
   4: unwrap<()>
             at /nix/store/fi8ydppm3dpk73gabvhwmsbjqwzalvzg-rust-nightly-complete-with-components-2024-12-09/lib/rustlib/src/rust/library/core/src/option.rs:972:21
   5: my_fun
             at ./src/main.rs:59:5
   6: rust_playground::main
             at ./src/main.rs:63:5
   7: core::ops::function::FnOnce::call_once
             at /nix/store/fi8ydppm3dpk73gabvhwmsbjqwzalvzg-rust-nightly-complete-with-components-2024-12-09/lib/rustlib/src/rust/library/core/src/ops/function.rs:250:5
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

Code I used for this is

fn my_fun() {
    None.unwrap()
}

fn main() {
    my_fun()
}

cc @tbro