wanted: io_grpc_grpc_java

[alexeagle]

Module location

https://github.com/grpc/grpc-java

Link to bzlmod issue in the module's repository

grpc/grpc-java#9559

[meteorcloudy]

Do we really need grpc-java as a Bazel module, or should Bazel users just fetch the jar dependencies via rules_jvm_external?

[alexeagle]

Yes, I think we do, because it has the java_grpc_library rule which relies on the protoc plugin in that repo.

[alexeagle]

Another resolution for this is to relocate the java_grpc_library rule to somewhere like rules_java or contrib_rules_jvm - it doesn't seem like maintainers on grpc/grpc-java are excited to continue providing Bazel rules from there, and I think they probably shouldn't be maintaining any. @shs96c will know what to do :)

[meteorcloudy]

Yes, I would also like to see the Bazel APIs to be moved out of grpc-java if they are not getting well maintained there.

[shs96c]

GRPC-related things feel like a better fit for one of the (many) grpc providing rulesets.

[alexeagle]

Do you have a recommendation of which one? None of them are canonical and last time the SIG touched this in https://bazel-contrib.github.io/SIG-rules-authors/proto-grpc.html we just made a spreadsheet because things were so confusing.

[ejona86]

it doesn't seem like maintainers on grpc/grpc-java are excited to continue providing Bazel rules from there

I am currently keeping the rules roughly in-sync with those inside Google for Blaze. The story may be different for other languages, but for Java there's been no change to the rule support.

Bzlmod is a different discussion. It looks invasive and a repeated cost as we make releases. So it would take some concerted effort to figure out what the heck is going on and what will be necessary going forward. We accept PRs for new stuff, and maintain what is there already. A PR isn't a sure win, but it generally does make things more clear as to what is going on. I don't see the point of forking if you haven't tried a PR upstream yet.

Do we really need grpc-java as a Bazel module, or should Bazel users just fetch the jar dependencies via rules_jvm_external?

Given you would still need the protoc plugin, which is a native C++ binary, I don't think rules_jvm_external is much of a solution. You would still need the cc_binary from the grpc-java repository.

[plobsing]

Do we really need grpc-java as a Bazel module, or should Bazel users just fetch the jar dependencies via rules_jvm_external?

I've been maintaining a private fork implementing pretty much this concept. I had a completely different reason to do this: the WORKSPACE idioms recommended by grpc-java include overriding certain Maven dependencies with built-from-source versions (IO_GRPC_GRPC_JAVA_OVERRIDE_TARGETS). It seems these overrides do not participate in coursier's version resolution but override whatever version does get resolved (even if newer), which can lead to incompatible JARs being used. By plumbing grpc-java version information down into coursier, I was able to convert what had been a recurring, difficult to diagnose, runtime error into a much clearer pinner-execution-time error. Or often, it just selected compatible versions and there were no errors. The information necessary to do proper version resolution is also sufficient to pull the JARs from Maven Central, so there was not much point continuing to build them from source after that.

A few subtle things I learned from this exercise:

• Public build targets under @io_grpc_grpc_java need to be maintained. Other code expects to reference into it (e.g. grpc-kotlin does). This can be achieved by setting up alias rules.
• It is likely best to continue adding all grpc-java artifacts to the maven_install call rather than just those strictly necessary at runtime by generated code; grpc-java JARs have exact-version constraints among themselves and coursier is not always smart enough to pick the same version for all of them (e.g. if it wasn't told about certain packages explicitly).

While not a bzlmod module today, I think it would be pretty trivial to convert what I have into one - aside from the ruleset it just references prebuilt assets from Maven Central.

Given you would still need the protoc plugin, which is a native C++ binary, I don't think rules_jvm_external is much of a solution. You would still need the cc_binary from the grpc-java repository.

There are prebuilt versions of the protoc plugin on Maven Central; I believe the Maven protobuf plugin uses them. I've been pulling them in my WORKSPACE file using http_file:

http_file(
    name = "protoc-gen-grpc-java-{platform}".format(platform = platform),
    executable = True,
    sha256 = sha256,
    urls = [
        "https://repo1.maven.org/maven2/io/grpc/protoc-gen-grpc-java/{version}/protoc-gen-grpc-java-{version}-{platform}.exe".format(
            version = version,
            platform = platform,
        ),
    ],
)

Bzlmod is a different discussion. It looks invasive and a repeated cost as we make releases. So it would take some concerted effort to figure out what the heck is going on and what will be necessary going forward. We accept PRs for new stuff, and maintain what is there already. A PR isn't a sure win, but it generally does make things more clear as to what is going on. I don't see the point of forking if you haven't tried a PR upstream yet.

I'm interested in making grpc-java work for bzlmod consumers, and I'd prefer to do that in coordination with upstream. But as I see it, because of issues relating to version resolution mentioned above, having the ruleset live in the same repository as the sourcecode for the JARs is a net negative; an easy protection against aliasing between from-source and Maven-published artifacts is to just not make the sources available in the first place. For that reason, I'd like to see this hosted in a new repo rather than having it be a PR against the existing grpc/grpc-java repo. Could/should that repo live under the gRPC org (and if so what's the protocol to set that up?) or should it live elsewhere?

I am also somewhat concerned about the toil associated with releases; my current workflow is partially manual. It takes a few minutes to do now that I've figured out what needs doing, which is fine for the couple times a year my project updates its version of this dependency, but is a bit much for the couple times a month grpc-java cuts a release. I believe this is a scriptable task, and I've been tinkering on one although I don't have anything working so far.

[keith]

#1699