Speed up Diesel compilation

[Nutomic]

Diesel is currently very slow to compile, because it only consists of a single crate, which Rust compiles on a single thread. You can see that in the graph below.

The best solution is to split the project into multiple separate crates. This is what we did with Lemmy, and in my experience it also helped a lot to optimize the code better. I wrote an explanation here for generating and interpreting the build graph. The Rust documentation also has some info on it.

If anything about this is unclear to you, I will try my best to answer your questions.

cc @dessalines

[weiznich]

Thanks for coming here to raise this request. I'm well aware that splitting up projects into multiple crates can improve compile times. Code organization wise this would be something that would be interesting for diesel as well, but it's not something that's high on my priority list (read that as not for the 2.0 release and not for a potential 2.1 release). That written I will likely not stop anyone from working on this as long as it does not infer to much with my own work. In this case I would expect a proposal first that outlines which part belongs to which newly created crate and how we can maintain the "same" public API afterwards.

That all written: I don't believe it will have that large impacts on your compile time. I think you don't you understand how different diesel is to the normal compiler workload. This means the "normal" strategies to optimize compile times won't have much effect here.
I've done some investigation what exactly takes that much time to process for rustc some days back, while adding diesel to the rustc-pref test suite. See here for the results. Notably rustc spends a lot of time on expanding macros and doing type checking and for the 128-columns feature to check module privacy. This behaviour seems to be not linear with the crate size. (See rust-lang/rust#81262 and rust-lang/rust#81263 for the corresponding rustc issues). I did some investigations back then and found that the macro expansion is mostly caused by this macro call, which cannot be split up into multiple crates due to the orphan rule (At least not if you want to keep some logical order). Especially those two sub-marcos are really sensitive to compile time performance (See rust-lang/rust#75542 for details and a much worse version). I've tried to rewrite this in #2748 by using a proc macro attribute, but that turned out to be even slower than the macro by example expansion. It seems like syn/quote is just not up the task of generating much code fast. At that point I stopped working on it as this is not that important for the 2.0 release. Again if someone things they could do that better feel free to pick that PR and improve it. As other side node that means that diesels compile time performance is really sensitive to which feature flags are passed. Disabling the 32-columns feature will definitively speed up compilation by 2x.

[weiznich]

As an additional note: All of the points mentioned above about macro expansions and column counts are basically workarounds for a missing language feature: variadic generics. I expect that having them would fix a large part of the "compilation is slow" issue as the compiler won't need to evaluate basically the same trait impl over and over again.