Make it possible to run a nested Bazel in a repository rule and keep the Bazel server around

[lberki]

Description of the feature request:

Clover has the issue that in order to determine the source code that needs to be built, they need a binary written in Rust.

Repository fetching currently can't depend on executing Bazel actions (and it would be a momentous change to make that so) so the next best thing is to run a nested Bazel instance within a repository rule. This works, however, as it is, it cannot keep a Bazel server alive by design: we run repository rules under process-wrapper (see here), which makes sure that no processes are left alive using child subreapers. Obviously, this means that the Bazel server cannot be left alive, either.

I don't particularly like poking a hole in this otherwise nice architectural boundary, but the alternatives are not nice, either:

• They could just run Cargo to build a Rust binary (thereby not re-using the same Rust code in the outer Bazel invocation, less caching, etc.)
• They could stop using Rust and replace it with something that does not need to be compiled (but this looks like a "tail wags the dog" kind of bugfix)
• We could implement repositories depending on actions (very tempting, but also something we'd probably regret if done without a lot of thought and we could regret it even in the best case)

So the best alternative I know of is to find a way to allow nested Bazel servers while limiting collateral damage in a clever way.

Which category does this issue belong to?

No response

What underlying problem are you trying to solve with this feature?

No response

Which operating system are you running Bazel on?

No response

What is the output of bazel info release?

No response

If bazel info release returns development version or (@non-git), tell us how you built Bazel.

No response

What's the output of git remote get-url origin; git rev-parse master; git rev-parse HEAD ?

No response

Have you found anything relevant by searching the web?

No response

Any other information, logs, or outputs that you want to share?

No response

[Wyverald]

This is a bit out of my depth. Is there a specific solution you would recommend? Like ... somehow turn the child process into a daemon here if we detect it to be a Bazel server process?

cc @matts1

[matts1]

@Wyverald I wrote a patch locally that adds a flag to repository_ctx.execute that, if set, disables these lines. Does that sound reasonable? @lberki doesn't like it, but in our meeting couldn't really think of a better option.

[lberki]

I was more thinking along the lines of explicitly telling the outer Bazel to run an inner Bazel because telling software explicitly what to do in the first place is usually a much better option than making it figure that out from circumstantial evidence.

Overnight, I had a thought that might even be classified as "productive". Let me start a GitHub discussion about it before it falls out of my brain. Yesterday, my stance about the flag @matts1 mentioned was that I don't like it but I don't see a better option, but now I think I do.

[lberki]

Here is the proposal: #20464

[matts1]

If we're not willing to add a flag to repository_ctx.execute that disables child subreaping, I've just thought of an alternative that we could use in the meantime with no changes to bazel itself.

We could add to tools/bazel to ensure that we start the nested bazel server if it isn't already started. This would ensure that the child subprocess reaping would reap the client, but the server would be running outside of the process wrapper.

[matts1]

Update: I successfully made the nested bazel server persist using crrev.com/c/5107290.

I'm happy to mark this as closed - I'm reasonably comfortable with the current solution.

[lberki]

Well this is genius; do I understand correctly that what you are doing is to start up a Bazel server "outside", which can then be accessed from within a process-wrapper process tree?

[matts1]

That's correct, yes

[lberki]

Well then I'd classify this as an "evil genius" kind of hack. Let's continue the discussion on #20464 because that particular solution doesn't look very easy to maintain.