proto3 & grpc

[ahjohannessen]

As I understand it everything is suddenly an optional field in proto3, perhaps for reasons that are understandable considering languages such as Java and C#.

However, if messages are used to communicate via gRPC in a Scala setting then it becomes very cumbersome to ensure required fields on messages, because suddenly you also need to pattern match wrt. Option even when you know that field is required.

Would it make sense to build in required as some sort of extension to ScalaPB when using proto3 that causes the code generator to generate non-optional fields on case classes? I can see that falls a bit apart when used in a mixed setting.

Secondly, if the above is a bad idea, would it work if one defines messages that need that sort of semantics in a proto file that uses syntax v2 and then defines gRPC in a different proto that uses syntax v3 and imports the one with syntax v2?

[eiennohito]

1. It is completely possible to mix v2 and v3 style files. I do it personally. Protobuf spec supports it, so it is a bug if something does not work. map<> does not work with v2 syntax though.

2. With v3 only messages are wrapped with option. Primitive fields stay as they are.

[ahjohannessen]

@eiennohito Uh, that is nice :) I can see this from the docs:

However, proto2 enums cannot be used in proto3 syntax.

What is the deal with that. Seems to compile when I import a proto2 message type that uses enum into a proto3 file with a grpc service?

Does it only mean that I cannot use an enum directly in another proto3 message, but otherwise fine if I import a message with proto2 that uses an enum defined in its own file?

[eiennohito]

Probably so.
I did not use proto3 enums in my project.

By the way, I have already talked about the topic you have raised in #40.
It was refused that time. I started using v2 protobufs because of that mostly.

[thesamet]

@ahjohannessen would the solution suggested at #40 would work for you? That is, add field-level option that tells the generator not to wrap a value in an option. Something like:

option Message msg = 1 [(scalapb).no_box = true]

This would apply to any field in proto2, and for proto3 this is relevant only to messages since primitives are not boxed.

[ahjohannessen]

@thesamet In a pure scala environment I suppose that would be fine - however it is not so much an issue as I can define messages in proto2 and use those with services generated in proto3.

However, what I think is more interesting to investigate is the generated service code with respect to being able to use a scalaz Task instead of a scala Future plus making the code more Scala / FP like.

[ahjohannessen]

Furthermore, being able to bridge between scalaz-stream Process (soon fs2 Stream) and StreamObserver would be awesome :)

I suppose that requires some sort of pluggability in the code generator?

[thesamet]

Perhaps the scalaz-stream suggestion should go into its own separate github issue, with more details? It seems like the discussion here is diverging to two paths (scalaz-streams, and option boxing), so if there's interest in #40 let's continue there?

[ahjohannessen]

Yep, agree 👍