Add feature flags to reduce dependencies

[traviscross]

We have 223 unique external direct and transitive dependencies. Many
of these are needed only for serving files via HTTP. Others are
needed only for making outbound HTTP/S requests. In certain
situations, such as when building Zola as part of a continuous
integration pipeline, these features are unnecessary, and we would
prefer a faster and simpler build.

In this commit, we add serve and request feature flags to Zola.
The serve flag controls whether the embedded HTTP server is built
and whether the zola serve command is available. The request flag
controls whether we support making outbound HTTP/S requests, such as
for link checking. Both of these features are enabled by default.

With both of these features disabled, such as when building with
--no-default-features, we require only 160 unique external
dependencies, a 28% reduction. Notably, we're able to drop both
openssl and tokio. Dropping openssl in particular can help with
producing statically-linked builds.

With the serve flag, we depend on 210 unique external crates. With
the request flag, we depend on 199 unique external crates.

[traviscross]

In this commit, we have updated the installation documentation to reflect the available feature flags and their use.

We added these flags as part of work to support building Zola statically, so this has some relevance to issue #316. However, the benefit of having these feature flags likely stands on its own.

[traviscross]

We have ensured that all tests pass on Zola and on all components in the workspace in all feature configurations. Further, we've tested building, serving, and checking the documentation in each configuration where the operation is relevant.

[Keats]

I'm not super keen on that to be honest. I think features for Zola are only useful for some developers to avoid potential build issues, like sass-rs which is hard to build on some systems. The rest will always be built with all features enabled so there's no point splitting them.

Imo, a feature for sass (enabled by default) should be added but that's it.

[traviscross]

Hi @Keats; thanks for sharing your concerns. Understand and agree that most people building or using this on their local systems will want all features enabled (I do too!). The use-case here is that we're building Zola to be run as part of a continuous integration system for building a site. The embedded HTTP server and link checker don't serve a purpose in that application. They are, in this application, actively unwanted because we don't want our CI systems accepting inbound or making outbound network requests in any case.

So even aside from reducing dependencies, that's worth it to us and probably to others using it in a similar manner.

On the dependency side, this has two main positive effects:

• Removing OpenSSL from the dependency tree makes Zola easier to build reliably and in a statically-linked manner, which is particularly important for CI.
• Removing the things that make or receive network requests and OpenSSL reduce Zola's attack surface and make security review and vulnerability tracking easier for the resulting binary, which is important for some people.

This patch is also a stepping stone to getting Zola building statically in its full configuration. Testing and working through the problems was made easier by this. As you mention, sass-rs is the other tricky thing to build. (I didn't carve that one out because it seemed more central to Zola's functionality than the embedded server or link checker.)

I'm expecting to submit further PRs to sass-rs and to Zola to get Zola building statically in its full configuration.

If you start to see the value after giving some thought to the use-case and benefits I've described, great. (I recognize the appeal may be to a niche group.) If not, let me know, and I'll close the PR, and we'll just maintain this patch out of tree. If you want, I'm also happy to submit a PR with a feature flag for sass, or to roll it into this PR if you think that would be better.

[Keats]

Why build Zola from source everytime on CI though? It takes a good 5-10min on release mode vs grabbing it from one of the many package managers or just wget from the release page.

I think the only non-static part is reqwest right? I wanted to use the rustls feature (https://zola.discourse.group/t/remove-dependency-on-preinstalled-openssl/221) to get rid of openssl but it seems like some sites do not work with it; it might be ok for a SSG though?

[traviscross]

Hi @Keats; we don't build it every time we build the site. We build it whenever our Zola branch changes and cache the built artifact. That cached artifact is used to build the site. As far as why we build it ourselves rather than using a distributed binary, the immediate motivation is that we're testing in-progress improvements to Zola, such as work toward a solution to #519. We'll be submitting those patches upstream as they become ready for your review.

Also, while our use-case is CI, and I focused a lot on that above, I think anyone running Zola on a server in any capacity would want to strip out at least the embedded HTTP server for the (at least perceived) security benefits mentioned. Sometimes it can be useful to trust that Zola (or any binary) is providing only a strict input -> output function.

Regarding static builds, sass-rs' dependency on libstdc++ is perhaps the bigger challenge than OpenSSL. That precludes the simple use the musl-gcc, which Rust uses by default when building statically with musl. I have managed to get Zola to build statically using musl-cross-make and some changes to Zola/sass-rs.

To get there, I found it useful to get OpenSSL out of the way, hence the origin story of this PR. However, I actually expect that getting Zola to build statically with OpenSSL won't be a big deal. We'll add a feature flag to Zola that will cause openssl-sys to build its vendored version of OpenSSL, and that will need to be passed when building statically. As you say, using rustls could be another option. It may be worth adding feature flags to select which is used (there are pros and cons to each). I can include that with the PRs I'll be submitting for the remainder of the static build work.

[traviscross]

Also, it may be worth mentioning, most of the Cargo.toml plumbing in this PR is groundwork that would be needed anyway for plumbing openssl or rustls feature flags to where they are needed.

[traviscross]

Apropos of the issue raised in the discourse group, the correct end result for Zola's own binary releases is probably to build them statically against OpenSSL, and perhaps against rustls when it's a bit more mature. That would avoid not just the conflict that user hit, but any conflicts with system libraries. It would also allow the Zola releases to be installed on non-gnu hosts such as Alpine Linux (which is a favorite of people using Docker... such as for CI).

Of course, that doesn't mean that the vendored OpenSSL needs to be the default for cargo build. That choice can be made separately, and it may be reasonable to use the system OpenSSL by default there.

(Actually, perhaps the ideal would be if the openssl-sys crate were to automatically fall back to its vendored copy when libssl-dev isn't available, like sass-sys does with libsass-dev.)

[Keats]

Thanks for the background info.

For rustls I think it's mature right now, it's just that it intentionally doesn't support every algorithms on purpose, which means it might not work with some sites.
The main issue I have with the reqwest feature gating is that we changes the load_data function. reqwest is only used for zola check (a command that you probably want to run on CI, with HTTP request to check external links) and that function which both becomes something very different with or without the flag on.

I was planning to add the sass-rs feature for the next Zola version since it's such a pain to build at times and prevent people from contributing.

Rather than adding a feature gate for reqwest, I am more thinking of trying to switch it to use rustls and see if people report unexpected failures; we get rid of openssl entirely and maybe can help improve rustls with issues/bugs that way. If too many people report issues, we can back down and look again at switching to the static openssl version. I don't think I want to make that choice selectable, if someone really needs another version of reqwest, they can fork it and it's a few chars diff.

[traviscross]

Thanks @Keats for the further thoughts. You're right about load_data. I hadn't fully considered what that function does with respect to outbound requests (or else I would have at least documented the changes there).

With respect to rustls and sass, that makes sense. I'm working up a commit series that switches to rustls, includes the sass feature flag, and rethinks the request feature a bit.

[Keats]

Closing that for now as it has way too many conflicts and is better to start from scratch.

[Keats]

@traviscross do you have an idea when/if you'll get to that?