prost-build: protoc path at runtime

[oll3]

We've built a small tool for pre-generating a Rust crate from a tree of protobuf files. Instead of generating the protobuf/gRPC code at build time of a project that project adds a dependency of the pre-generated protocol crate which may contain code for a number of protobuf/gRPC API's. This to simplify the process of creating a new project and speed up the build time.

However I noticed that when our tool is being built on another host than were it's later executed (to do the actual generation of the crate) then the path to protoc may no longer be valid. Depending on the environment where the tool is built prost-build will either try run one of the bundled protoc, but which are no longer bundled or a protoc selected by build time environment $PROTOC.

Could one introduce something like an env var at build time to tell Prost to use protoc available in the runtime PATH instead of using the baked in path?

[danburkert]

@oll3 prost-build will use protoc if set in the environment, see: https://docs.rs/prost-build/0.6.1/prost_build/#sourcing-protoc.

[oll3]

@oll3 prost-build will use protoc if set in the environment, see: https://docs.rs/prost-build/0.6.1/prost_build/#sourcing-protoc.

Yes, but prost-build uses this variable at build time...
This cli tool we've built (or is trying to build) is having prost-build as a dependency (not a build-dependency) and uses prost-build to generate code at runtime. The tool it self doesn't include the source generated by prost-build, it only generates source with help of prost-build (and tonic), and also structures the generated modules and generates a Cargo.toml for the protocol crate.

Usage of the tool looks something like:

gen-crate --crate-name my-protocol-crate --proto-root-tree some/dir

Where the directory given is scanned recursively to build a module tree from all packages found, and a Cargo.toml, a source tree of generated protobuf/gRPC files and a lib.rs which list all modules is spitted out.

The problem is that at this time, when the cli tool executes, prost-build has already decided to use whatever path the cli tool had to protoc when the tool was compiled . Everything works fine if the tool is built and executed on the same host, but if we build the tool and executes it on another host where protoc is in a different location it won't be found. Do you understand my case? :)

I can see that this might be outside the scope of Prost usage but wanted to ask anyway if this could be something supported.

[danburkert]

I see. That sounds more like the way a traditional protoc plugin is structured, which prost doesn't currently provide. There are a bunch of potential workarounds, but none of them are particularly clean, eg using relative paths and/or copying the bundled protoc along with you executable.

[oll3]

Yes, exactly. The copy workaround is what I consider at the moment, but feels quite ugly.
Could it be an option to introduce a way of telling prost-build to just use protoc from the system path at runtime?

[danburkert]

Yes, I was actually surprised to see that setting env PROTOC=protoc doesn't work, due to some probably over zealous checking in prost-build: https://github.com/danburkert/prost/blob/master/prost-build/build.rs#L35-L40. If removing that check makes it work I'm happy with that as a change.

[oll3]

This seems to work ok, at least as long as I also set the include directory to a directory that exists on the target.

[dfreese]

For what it's worth, put together a plugin in #313, but ran into a similar issue with the need for protoc needing to be set in PROTOC and exist at build time.

[danburkert]

Is this still an issue with prost 0.7 and #283?

[oll3]

Is this still an issue with prost 0.7 and #283?

Seems to be fixed @danburkert , thanks!