Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Prost and Tonic rules. #2033

Merged
merged 31 commits into from
Jun 30, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
6cb92b2
Setup Prost and Tonic rules.
freeformstu Jun 25, 2023
2c6d472
Regenerate documentation
freeformstu Jun 25, 2023
fced3e7
Add more tests.
freeformstu Jun 27, 2023
09d8142
Add more tests and address feedback.
freeformstu Jun 27, 2023
c1f3d99
Regenerate documentation
freeformstu Jun 27, 2023
a603dc2
Add to proto docs page.
freeformstu Jun 27, 2023
cb0ab09
Bump min supported bazel version.
freeformstu Jun 27, 2023
a1714e1
buildifier
freeformstu Jun 27, 2023
f3ccc01
Always enable backtracing.
freeformstu Jun 27, 2023
a58749c
Add more info to failing rename.
freeformstu Jun 27, 2023
d7e7312
Set min rust version to 1.62.0
freeformstu Jun 27, 2023
d67c839
Handle rust keywords as package names.
freeformstu Jun 27, 2023
e03d107
exclude windows from prost toolchain support.
freeformstu Jun 27, 2023
e901991
buildifier
freeformstu Jun 27, 2023
c4fe6aa
redundant
freeformstu Jun 27, 2023
0f80615
Use prost-types to parse the file descriptor set.
freeformstu Jun 28, 2023
2870e43
Cleanup and more tests.
freeformstu Jun 28, 2023
6554778
Move prost-types to toolchain definition.
freeformstu Jun 28, 2023
7e82121
fix rustfmt
freeformstu Jun 28, 2023
00682b3
Add example of building protos with complex imports
illicitonion Jun 28, 2023
6ceafbb
impl Display
freeformstu Jun 28, 2023
e848353
Fix all tests
freeformstu Jun 28, 2023
3c91f2c
Add rust checks for the complex import protos.
freeformstu Jun 28, 2023
e08d9ad
Address feedback
freeformstu Jun 29, 2023
d3ea3d7
Fix buildifier
freeformstu Jun 29, 2023
74e5b08
Depend on remote-apis repo.
freeformstu Jun 30, 2023
9bd48e1
Remove bazel remote apis due to file length and transitive dependency…
freeformstu Jun 30, 2023
b9b2872
Update patch and docs.
freeformstu Jun 30, 2023
5d64e3c
Regenerate documentation
freeformstu Jun 30, 2023
38b9989
Regenerate documentation
freeformstu Jun 30, 2023
7ac0bd1
Update docs.
freeformstu Jun 30, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions .bazelci/presubmit.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ aspects_flags: &aspects_flags
- "--config=rustfmt"
- "--config=clippy"
min_rust_version_shell_commands: &min_rust_version_shell_commands
- sed -i 's|^rust_register_toolchains(|rust_register_toolchains(versions = ["1.59.0"],\n|' WORKSPACE.bazel
- sed -i 's|^rust_register_toolchains(|rust_register_toolchains(versions = ["1.62.0"],\n|' WORKSPACE.bazel
illicitonion marked this conversation as resolved.
Show resolved Hide resolved
nightly_flags: &nightly_flags
- "--//rust/toolchain/channel=nightly"
nightly_aspects_flags: &nightly_aspects_flags
Expand Down Expand Up @@ -205,15 +205,15 @@ tasks:
ubuntu1804:
name: "Min Bazel Version"
# If updating the minimum bazel version, please also update /docs/index.md
bazel: "5.2.0"
bazel: "6.0.0"
illicitonion marked this conversation as resolved.
Show resolved Hide resolved
platform: ubuntu1804
build_targets: *default_linux_targets
test_targets: *default_linux_targets
coverage_targets: *default_linux_targets
post_shell_commands: *coverage_validation_post_shell_commands
ubuntu1804_with_aspects:
name: "Min Bazel Version With Aspects"
bazel: "5.2.0"
bazel: "6.0.0"
platform: ubuntu1804
build_targets: *default_linux_targets
test_targets: *default_linux_targets
Expand Down
8 changes: 6 additions & 2 deletions WORKSPACE.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -10,14 +10,18 @@ load("@rules_rust//crate_universe:repositories.bzl", "crate_universe_dependencie

crate_universe_dependencies(bootstrap = True)

load("@rules_rust//proto:repositories.bzl", "rust_proto_repositories")
load("@rules_rust//proto:repositories.bzl", "rust_proto_dependencies", "rust_proto_register_toolchains")

rust_proto_repositories()
rust_proto_dependencies()

rust_proto_register_toolchains()

load("@rules_rust//proto:transitive_repositories.bzl", "rust_proto_transitive_repositories")

rust_proto_transitive_repositories()

register_toolchains("@rules_rust//proto/prost/private/...")

load("@rules_rust//bindgen:repositories.bzl", "rust_bindgen_dependencies", "rust_bindgen_register_toolchains")

rust_bindgen_dependencies()
Expand Down
2 changes: 1 addition & 1 deletion crate_universe/private/crates_vendor.bzl
Original file line number Diff line number Diff line change
Expand Up @@ -346,7 +346,7 @@ bazel run //3rdparty:crates_vendor -- --repin
```

Under the hood, `--repin` will trigger a [cargo update](https://doc.rust-lang.org/cargo/commands/cargo-update.html)
call against the generated workspace. The following table describes how to controll particular values passed to the
call against the generated workspace. The following table describes how to control particular values passed to the
`cargo update` command.

| Value | Cargo command |
Expand Down
2 changes: 2 additions & 0 deletions docs/BUILD.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -121,6 +121,8 @@ PAGES = dict([
"rust_proto_repositories",
"rust_proto_transitive_repositories",
"rust_proto_toolchain",
"rust_prost_library",
"rust_tonic_library",
],
),
page(
Expand Down
2 changes: 1 addition & 1 deletion docs/crate_universe.md
Original file line number Diff line number Diff line change
Expand Up @@ -462,7 +462,7 @@ bazel run //3rdparty:crates_vendor -- --repin
```

Under the hood, `--repin` will trigger a [cargo update](https://doc.rust-lang.org/cargo/commands/cargo-update.html)
call against the generated workspace. The following table describes how to controll particular values passed to the
call against the generated workspace. The following table describes how to control particular values passed to the
`cargo update` command.

| Value | Cargo command |
Expand Down
40 changes: 40 additions & 0 deletions docs/flatten.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@
* [rust_library](#rust_library)
* [rust_library_group](#rust_library_group)
* [rust_proc_macro](#rust_proc_macro)
* [rust_prost_library](#rust_prost_library)
* [rust_proto_library](#rust_proto_library)
* [rust_proto_repositories](#rust_proto_repositories)
* [rust_proto_toolchain](#rust_proto_toolchain)
Expand All @@ -43,6 +44,7 @@
* [rust_stdlib_filegroup](#rust_stdlib_filegroup)
* [rust_test](#rust_test)
* [rust_test_suite](#rust_test_suite)
* [rust_tonic_library](#rust_tonic_library)
* [rust_toolchain](#rust_toolchain)
* [rust_toolchain_repository](#rust_toolchain_repository)
* [rust_toolchain_repository_proxy](#rust_toolchain_repository_proxy)
Expand Down Expand Up @@ -1792,6 +1794,25 @@ Registers the default toolchains for the `rules_rust` [bindgen][bg] rules.
| <a id="rust_bindgen_register_toolchains-register_toolchains"></a>register_toolchains | Whether or not to register toolchains. | `True` |


<a id="rust_prost_library"></a>

## rust_prost_library

<pre>
rust_prost_library(<a href="#rust_prost_library-name">name</a>, <a href="#rust_prost_library-kwargs">kwargs</a>)
</pre>

A rule for generating a Rust library using Prost.

**PARAMETERS**


| Name | Description | Default Value |
| :------------- | :------------- | :------------- |
| <a id="rust_prost_library-name"></a>name | The name of the target. | none |
| <a id="rust_prost_library-kwargs"></a>kwargs | Additional keyword arguments for the underlying <code>rust_prost_library</code> rule. | none |


<a id="rust_proto_repositories"></a>

## rust_proto_repositories
Expand Down Expand Up @@ -1996,6 +2017,25 @@ rust_test_suite(
| <a id="rust_test_suite-kwargs"></a>kwargs | Additional keyword arguments for the underyling [rust_test](#rust_test) targets. The <code>tags</code> argument is also passed to the generated <code>test_suite</code> target. | none |


<a id="rust_tonic_library"></a>

## rust_tonic_library

<pre>
rust_tonic_library(<a href="#rust_tonic_library-name">name</a>, <a href="#rust_tonic_library-kwargs">kwargs</a>)
</pre>

A rule for generating a Rust library using Prost and Tonic.

**PARAMETERS**


| Name | Description | Default Value |
| :------------- | :------------- | :------------- |
| <a id="rust_tonic_library-name"></a>name | The name of the target. | none |
| <a id="rust_tonic_library-kwargs"></a>kwargs | Additional keyword arguments for the underlying <code>rust_tonic_library</code> rule. | none |


<a id="rust_toolchain_repository"></a>

## rust_toolchain_repository
Expand Down
2 changes: 1 addition & 1 deletion docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -93,7 +93,7 @@ Failure to do so will result in rules attempting to match a `stable` toolchain w

## Supported bazel versions

The oldest version of Bazel the `main` branch is tested against is `5.2.0`. Previous versions may still be functional in certain environments, but this is the minimum version we strive to fully support.
The oldest version of Bazel the `main` branch is tested against is `6.0.0`. Previous versions may still be functional in certain environments, but this is the minimum version we strive to fully support.

We test these rules against the latest rolling releases of Bazel, and aim for compatibility with them, but prioritise stable releases over rolling releases where necessary.

Expand Down
170 changes: 162 additions & 8 deletions docs/rust_proto.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,15 +6,22 @@
* [rust_proto_repositories](#rust_proto_repositories)
* [rust_proto_transitive_repositories](#rust_proto_transitive_repositories)
* [rust_proto_toolchain](#rust_proto_toolchain)
* [rust_prost_library](#rust_prost_library)
* [rust_tonic_library](#rust_tonic_library)


## Overview

These build rules are used for building [protobufs][protobuf]/[gRPC][grpc] in [Rust][rust] with Bazel.

There are two rule sets. The first ruleset defines the `rust_proto_library` and `rust_grpc_library`
rules which generate Rust code using the [`rust-protobuf`] dependencies. The second ruleset defines
the `rust_prost_library` and `rust_tonic_library` rules which generate Rust code using the [`prost`]
and [`tonic`] dependencies respectively.

[rust]: http://www.rust-lang.org/
[protobuf]: https://developers.google.com/protocol-buffers/
[grpc]: https://grpc.io
[`rust-protobuf`]: https://github.com/stepancheg/rust-protobuf/

See the [protobuf example](../examples/proto) for a more complete example of use.

Expand All @@ -33,17 +40,21 @@ load("@rules_rust//proto:transitive_repositories.bzl", "rust_proto_transitive_re
rust_proto_transitive_repositories()
```

This will load crate dependencies of protobuf that are generated using
[crate_universe](./crate_universe.md) inside the rules_rust
repository. However, using those dependencies might conflict with other uses
of [crate_universe](./crate_universe.md). If you need to change
those dependencies, please see the [dedicated section below](#custom-deps).
This will load the required dependencies for the Proto, Prost, and Tonic rules. It will also
register a default toolchain for the `rust_proto_library` and `rust_grpc_library` rules. The
`prost` and `tonic` rules do not specify a default toolchain in order to avoid mismatched
dependency issues.

To customize the `rust_proto_library` and `rust_grpc_library` toolchain, please see the section
[Customizing `rust-protobuf` Dependencies](#custom-proto-deps).

To setup the `prost` and `tonic` toolchain, please see the section [Customizing `prost` and `tonic` Dependencies](#custom-prost-deps).

For additional information about Bazel toolchains, see [here](https://docs.bazel.build/versions/master/toolchains.html).

## <a name="custom-deps">Customizing dependencies
## <a name="custom-proto-deps">Customizing `rust-protobuf` Dependencies

These rules depends on the [`protobuf`](https://crates.io/crates/protobuf) and
These rules depend on the [`protobuf`](https://crates.io/crates/protobuf) and
the [`grpc`](https://crates.io/crates/grpc) crates in addition to the [protobuf
compiler](https://github.com/google/protobuf). To obtain these crates,
`rust_proto_repositories` imports the given crates using BUILD files generated with
Expand Down Expand Up @@ -114,6 +125,111 @@ but due to [bazelbuild/bazel#6889](https://github.com/bazelbuild/bazel/issues/68
all dependencies added via the toolchain ends-up being in the wrong
configuration.

## <a name="custom-prost-deps">Customizing `prost` and `tonic` Dependencies

These rules depend on the [`prost`] and [`tonic`] dependencies. To setup the necessary toolchain
for these rules, you must define a toolchain with the [`prost`], [`prost-types`], [`tonic`],[`protoc-gen-prost`], and [`protoc-gen-tonic`] crates as well as the [`protoc`] binary.

[`prost`]: https://crates.io/crates/prost
[`prost-types`]: https://crates.io/crates/prost-types
[`protoc-gen-prost`]: https://crates.io/crates/protoc-gen-prost
[`protoc-gen-tonic`]: https://crates.io/crates/protoc-gen-tonic
[`tonic`]: https://crates.io/crates/tonic
[`protoc`]: https://github.com/protocolbuffers/protobuf

To get access to these crates, you can use the [crate_universe](./crate_universe.md) repository
rules. For example:

```python
load("//crate_universe:defs.bzl", "crate", "crates_repository")

crates_repository(
name = "crates_io",
annotations = {
"protoc-gen-prost": [crate.annotation(
gen_binaries = ["protoc-gen-prost"],
patch_args = [
"-p1",
],
patches = [
# Note: You will need to use this patch until a version greater than `0.2.2` of
# `protoc-gen-prost` is released.
"@rules_rust//proto/prost/private/3rdparty/patches:protoc-gen-prost.patch",
],
)],
"protoc-gen-tonic": [crate.annotation(
gen_binaries = ["protoc-gen-tonic"],
)],
},
cargo_lockfile = "Cargo.Bazel.lock",
mode = "remote",
packages = {
"prost": crate.spec(
version = "0",
),
"prost-types": crate.spec(
version = "0",
),
"protoc-gen-prost": crate.spec(
version = "0",
),
"protoc-gen-tonic": crate.spec(
version = "0",
),
"tonic": crate.spec(
version = "0",
),
},
repository_name = "rules_rust_prost",
tags = ["manual"],
)
```

You can then define a toolchain with the `rust_prost_toolchain` rule which uses the crates
defined above. For example:

```python
load("@rules_rust//proto/prost:defs.bzl", "rust_prost_toolchain")
load("@rules_rust//rust:defs.bzl", "rust_library_group")

rust_library_group(
name = "prost_runtime",
deps = [
"@crates_io//:prost",
],
)

rust_library_group(
name = "tonic_runtime",
deps = [
":prost_runtime",
"@crates_io//:tonic",
],
)

rust_prost_toolchain(
name = "prost_toolchain_impl",
prost_plugin = "@crates_io//:protoc-gen-prost__protoc-gen-prost",
prost_runtime = ":prost_runtime",
prost_types = "@crates_io//:prost-types",
proto_compiler = "@com_google_protobuf//:protoc",
tonic_plugin = "@crates_io//:protoc-gen-tonic__protoc-gen-tonic",
tonic_runtime = ":tonic_runtime",
)

toolchain(
name = "prost_toolchain",
toolchain = "default_prost_toolchain_impl",
toolchain_type = "//proto/prost:toolchain_type",
)
```

Lastly, you must register the toolchain in your `WORKSPACE` file. For example:

```python
register_toolchains("//toolchains:prost_toolchain")
```


<a id="rust_grpc_library"></a>

Expand Down Expand Up @@ -260,6 +376,25 @@ See @rules_rust//proto:BUILD for examples of defining the toolchain.
| <a id="rust_proto_toolchain-protoc"></a>protoc | The location of the <code>protoc</code> binary. It should be an executable target. | <a href="https://bazel.build/concepts/labels">Label</a> | optional | <code>@com_google_protobuf//:protoc</code> |


<a id="rust_prost_library"></a>

## rust_prost_library

<pre>
rust_prost_library(<a href="#rust_prost_library-name">name</a>, <a href="#rust_prost_library-kwargs">kwargs</a>)
</pre>

A rule for generating a Rust library using Prost.

**PARAMETERS**


| Name | Description | Default Value |
| :------------- | :------------- | :------------- |
| <a id="rust_prost_library-name"></a>name | The name of the target. | none |
| <a id="rust_prost_library-kwargs"></a>kwargs | Additional keyword arguments for the underlying <code>rust_prost_library</code> rule. | none |


<a id="rust_proto_repositories"></a>

## rust_proto_repositories
Expand Down Expand Up @@ -292,3 +427,22 @@ This macro should be called immediately after the `rust_proto_repositories` macr



<a id="rust_tonic_library"></a>

## rust_tonic_library

<pre>
rust_tonic_library(<a href="#rust_tonic_library-name">name</a>, <a href="#rust_tonic_library-kwargs">kwargs</a>)
</pre>

A rule for generating a Rust library using Prost and Tonic.

**PARAMETERS**


| Name | Description | Default Value |
| :------------- | :------------- | :------------- |
| <a id="rust_tonic_library-name"></a>name | The name of the target. | none |
| <a id="rust_tonic_library-kwargs"></a>kwargs | Additional keyword arguments for the underlying <code>rust_tonic_library</code> rule. | none |


Loading