Start-up time benchmarks

[pzmarzly]

There are some requests/second benchmarks on the web, but I was curious in how fast Rocket server starts. On cheap SATA3 SSD and Intel i5 6300HQ "hello-world" example needs about 6 ms both in debug and release mode. Of course this number will go up in actual application, which requires to load more data from disk, may load some shared libraries or connect to some database. However, I think that even in such cases the startup time will be much better than it is for servers written in interpreted/JIT-compiled languages.

I think some people may find this information useful, especially when deciding whether to build a scale-to-zero cluster (e.g. with https://github.com/knative/serving).

I think it would be nice to have such benchmark somewhere on the website, ideally compared with other frameworks, with CGI "hack" (#97, #132) and with Unix sockets backend when it lands (#545).

I've seen author's stance on benchmarks #710 (comment) but while I agree that the difference between 50,000 reqs/s and 100,000 reqs/s is almost always irrelevant (most sites will never get that much traffic), the difference between 6ms and 600ms server startup time can hopefully make some new opportunities (and be a point in Rocket vs Flask / Rust vs Python debates 😉)

Using ts from moreutils for timestamping (not the best method, I should have used some proper benchmarking tool to measure startup time without disk cache and with warm cache):

$ cargo +nightly run 2>&1 | ts '[%M:%.S]'
[21:45.952937]     Finished dev [unoptimized + debuginfo] target(s) in 0.22s
[21:45.971598]      Running `/home/pzmarzly/Rocket/target/debug/hello_world`
[21:45.976198] Configured for development.
[21:45.976340]     => address: localhost
[21:45.976414]     => port: 8000
[21:45.976482]     => log: normal
[21:45.976522]     => workers: 8
[21:45.976576]     => secret key: generated
[21:45.976615]     => limits: forms = 32KiB
[21:45.976653]     => keep-alive: 5s
[21:45.976692]     => tls: disabled
[21:45.976731] Mounting /:
[21:45.976783]     => GET / (hello)
[21:45.976823] Rocket has launched from http://localhost:8000

[jebrosen]

Given that startup time is usually heavily dependent on I/O and database connections, widely variable, and usually one of the least important metrics when it comes to performance, I don't think it's worth paying special attention to.

If Rocket does ever decide to advertise benchmark results (as it has in the past, but stopped because they were somewhat misleading and not the easiest to interpret), maybe this can be considered then.