Develop code generation for gRPC Services. #2

timburks · 2016-10-25T23:26:46Z

Currently there is no code generation for gRPC services. This needs to be added either to the swift-proto plugin or to a separate plugin.

jcanizales · 2016-10-26T01:35:33Z

As a separate plugin, per Thomas' reasoning in apple/swift-protobuf#118 :)

johndpope · 2016-11-04T17:26:25Z

From the look of objective-c generator / it's intermingled into visual studio at some point.
https://github.com/grpc/grpc/blob/master/vsprojects/grpc_objective_c_plugin/grpc_objective_c_plugin.vcxproj

objective-c generator
https://github.com/bazelbuild/bazel/blob/master/third_party/grpc/src/compiler/objective_c_plugin.cc

Part 1)
Need to choose language to spit out swift service class
Does it make most sense to use swift? or cpp?
Could we cherry pick from Apple's implementation?
https://github.com/jcanizales/swift-protobuf-plugin/blob/master/Sources/FileGenerator.swift thoughts?

Part 2)
create a canonical gist with http2 service layer to plug into this various protobuf swift files.
https://github.com/grpc/grpc/blob/master/src/objective-c/examples/SwiftSample/ViewController.swift

https://github.com/nathanborror/swift-grpc

johndpope · 2016-11-06T20:13:22Z

I've been looking into different widgets to get job done -
I'd say we'd need a protobuf parser as first step - which led me to this grammar file
https://raw.githubusercontent.com/notak/tinyproto/36e44b3c04a55d754bf1125d4e527423c659fe9b/src/main/antlr4/me/taks/proto/Protobuf.g4

so - if we throw this at antlr-swift
https://github.com/janyou/ANTLR-Swift-Target
we'd have a protobuf parser/lexar/tokenizer in swift - which would spit out a full AST
from there we'd expose all the methods / service calls for respective proto file

I think biting the bullet and going Swift is nice scenic route. c++ while it will work - will be hard(er) for future offshoot libraries to hot wire. (like Alamofire swapping in their libraries)
The other alternative is to cherry pick Apple's protobuf source code.

We can then plug into antlr swift runtime
https://github.com/janyou/Antlr-Swift-Runtime.git

DRAFTED
https://github.com/johndpope/Antlr-Swift-Runtime/tree/master/Test/Test

jcanizales · 2016-11-06T23:25:03Z

Oh, but protoc will parse the .proto files for us already. The usual setup is protoc executes your plugin in a separate process, writes a serialized CodeGeneratorRequest to its stdin, and then reads a serialized CodeGeneratorResponse from its stdout.

So the plugin can be written in Swift alright, and just needs to conform to that protocol to get the parsing for free.

johndpope · 2016-11-07T18:44:20Z

then cherry picking Apple's work is best option.
https://github.com/apple/swift-protobuf/blob/bf53b0b56ca611c8fa634006c846a87bb6f2d079/Sources/protoc-gen-swift/Context.swift

https://github.com/apple/swift-protobuf/blob/bf53b0b56ca611c8fa634006c846a87bb6f2d079/Reference/google/protobuf/compiler/plugin.pb.swift

Then we just need the canonical example.
IF proto file looks like this

service MasterService {
  // Creates a session.
  rpc CreateSession(CreateSessionRequest) returns (CreateSessionResponse);

  // Extends a session.
  rpc ExtendSession(ExtendSessionRequest) returns (ExtendSessionResponse);

  // Prepares future partial run calls.
  rpc PartialRunSetup(PartialRunSetupRequest) returns (PartialRunSetupResponse);

  // Drives the graph computation.
  rpc RunStep(RunStepRequest) returns (RunStepResponse);

  // Closes a session.
  rpc CloseSession(CloseSessionRequest) returns (CloseSessionResponse);

  // List the devices usable by the master.
  rpc ListDevices(ListDevicesRequest) returns (ListDevicesResponse);

  // Close all existing sessions.
  rpc Reset(ResetRequest) returns (ResetResponse);
}

then ....

fill in the gaps

....

johndpope · 2016-11-28T19:57:52Z

This Edge framework for server side swift seems like nice path forward (albeit not suitable for ios)
https://github.com/SwiftOnEdge/Edge

johndpope · 2016-12-20T19:53:32Z

So - need to commit to specific interoperable network library.
https://www.reddit.com/r/swift/comments/57m49o/how_to_make_a_network_request_on_linux/
Presumably - the limitations of urlsession across different platforms is still the case?

I found this - a swifty version of libcurl wrapped around IBMs' swift libcurl.
https://github.com/dmcyk/SwiftyCurl/

@jcanizales - did you envisage the RxLibrary having a life beyond objective-c? I read in comments - you said it should eventually be separated from grpc library.
or should a swifty alternative be sought? rxswift ? I'm thinking the use case being swift server to server communications for microservices.

jcanizales · 2016-12-30T17:20:41Z

RxLibrary, like many others, is nicer to use from Swift than ObjC. And it can be used standalone by just depending on the gRPC-RxLibrary cocoapod. But Swift's generics can be used to create a still nicer Swift version of the library.

I've had in my mind, for a while, to create a doc comparing proposed surfaces for Swift gRPC: one using RxSwift, the other ReactiveSwift, and what we have today from importing the ObjC library.

Between RxSwift and ReactiveSwift, I personally think having different types for "hot" and "cold" streams is a better idea. But from the POV of the user, it's best if it matches whatever rx library you're already using in your app. That means that either:

We pick one of RxSwift or ReactiveSwift, and write or find an adapter library to get one from the other.
We use RxLibrary (or write a more Swifty version), and write adapter libraries for the other two.

johndpope · 2016-12-30T22:36:37Z

fyi Kitura/CCurl#10
IBM's banking on fully deprecating ccurl as a result of Apple presumably getting around to rewriting nsurlsession for server side (hopefully by swift4?) - we could lock that in and cross fingers it will be future proof (in the future).
The only reservation I have with using rxlibrary is it's introducing objective-c which we superseded in the design up until this point. I've heard good things about rxswift.
Whereas reactiveswift has reactivecocoa dependencies (& with carthage it is a bit of pain)
KevinVitale/ReactiveJSON#7
for the document with pros + cons - you could throw a gist up or share google sheet doc.

timburks · 2017-01-07T03:32:08Z

With this commit, there's a code generating plugin in the Plugins directory that generates client and server stubs for all four API types (Unary, Server Streaming, Client Streaming, Bidi Streaming). The plugin uses templates in the Plugins/templates directory and currently these must be in the same directory where the plugin is run. Templates use Kyle Fuller's Stencil package and will hopefully be easy to modify with improvements or for other calling styles. The current generated code surface closely follows the Go grpc generator.

johndpope · 2017-04-25T21:06:03Z

@timburks - thanks for your efforts here - pleased to say I got the service classes spat out (untested) for tensorflow. https://github.com/nubbel/swift-tensorflow/

Contains the following commits: - Refactor gRPC decoding into dedicated codec classes. - Start work on GRPCServerHandler. - Add a "unary call handler" and use that for the tests. - Refactoring starting a GRPC server into a dedicated class. - Fix sending unary responses. - Add a handler for client-streaming calls. - Also implement bidirectional-streaming calls. - Make sure to flush in server-streaming calls after each sent message. - Add the missing test cases to `allTests`. - Refactor `StatusSendingHandler` into its own class. - Rename `GRPCServerHandler` to `GRPCChannelHandler`. - Remove a FIXME. - Add a few more comments. - Attach the actual call handlers as channel handlers instead of manually forwarding messages to them. # This is the commit message grpc#2: Remove SwiftGRPCNIO's dependency on SwiftGRPC and move the responsibility for encoding GRPC statuses to HTTP1ToRawGRPCServerCoded. # This is the commit message grpc#3: Temporarily disable two test cases that are failing at the moment. # This is the commit message grpc#4: Add SwiftGRPCNIO as an exposed library. # This is the commit message grpc#5: Another try at getting CI to work with SwiftGRPCNIO. # This is the commit message grpc#6: More dependency fixes. # This is the commit message grpc#7: Add `SwiftGRPCNIO.EchoServerTests` to LinuxMain.swift. # This is the commit message grpc#8: Fix a string comparison in `.travis-install.sh`. # This is the commit message grpc#9: Add nghttp2 to the list of CI dependencies. # This is the commit message grpc#10: Another try with installing nghttp2 via brew. # This is the commit message grpc#11: Another try at using libnghttp2-dev under Ubuntu 14.04. # This is the commit message grpc#12: More Travis fixes. # This is the commit message grpc#13: One last try. # This is the commit message grpc#2: Disable two more tests for now, as they sometimes fail on CI. # This is the commit message grpc#3: Make Carthage debug builds verbose. # This is the commit message grpc#4: Only use SwiftGRPC-Carthage.xcodeproj for Carthage builds.

* Proposal to fix #362. Allows user to setup a channel with a client certificate and key but still use default root certificates. * fix #362. Respect PR critique. make roots_pem() public. Update Channel init. * fix #362: make roots_pem() return force-unwrapped String. Change Channel API accordingly. * Remove an obsolete comment. * Minor whitespace tweak #1 * Minor whitespace tweak #2

timburks changed the title ~~Code generation for gRPC Services~~ Develop code generation for gRPC Services. Oct 25, 2016

timburks added the kind/enhancement Improvements to existing feature. label Oct 25, 2016

johndpope mentioned this issue Oct 26, 2016

Enhancement - protoc - service definitions - generate gRPC service definitions apple/swift-protobuf#118

Closed

johndpope mentioned this issue Nov 2, 2016

Swift API tensorflow/tensorflow#19

Closed

This was referenced Nov 4, 2016

An exercise for the reader - HTTP Archive files ijoshsmith/json2swift#3

Closed

objective-c - Examples grpc/grpc#8645

Closed

This was referenced Dec 20, 2016

sample gist - code dmcyk/SwiftyCurl#3

Closed

swift + tensorflow pathways forward - Dec 2016 nubbel/swift-tensorflow#5

Closed

timburks closed this as completed Jan 7, 2017

maznikoff mentioned this issue Aug 17, 2018

Segmentation fault: 11 #280

Closed

pbrazdil added a commit to pbrazdil/grpc-swift that referenced this issue Oct 4, 2019

try grpc#2

0b6f9e6

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Develop code generation for gRPC Services. #2

Develop code generation for gRPC Services. #2

timburks commented Oct 25, 2016

jcanizales commented Oct 26, 2016 •

edited

Loading

johndpope commented Nov 4, 2016 •

edited

Loading

johndpope commented Nov 6, 2016 •

edited

Loading

jcanizales commented Nov 6, 2016

johndpope commented Nov 7, 2016 •

edited by jcanizales

Loading

johndpope commented Nov 28, 2016 •

edited

Loading

johndpope commented Dec 20, 2016 •

edited

Loading

jcanizales commented Dec 30, 2016 •

edited

Loading

johndpope commented Dec 30, 2016

timburks commented Jan 7, 2017

johndpope commented Apr 25, 2017

Develop code generation for gRPC Services. #2

Develop code generation for gRPC Services. #2

Comments

timburks commented Oct 25, 2016

jcanizales commented Oct 26, 2016 • edited Loading

johndpope commented Nov 4, 2016 • edited Loading

johndpope commented Nov 6, 2016 • edited Loading

jcanizales commented Nov 6, 2016

johndpope commented Nov 7, 2016 • edited by jcanizales Loading

johndpope commented Nov 28, 2016 • edited Loading

johndpope commented Dec 20, 2016 • edited Loading

jcanizales commented Dec 30, 2016 • edited Loading

johndpope commented Dec 30, 2016

timburks commented Jan 7, 2017

johndpope commented Apr 25, 2017

jcanizales commented Oct 26, 2016 •

edited

Loading

johndpope commented Nov 4, 2016 •

edited

Loading

johndpope commented Nov 6, 2016 •

edited

Loading

johndpope commented Nov 7, 2016 •

edited by jcanizales

Loading

johndpope commented Nov 28, 2016 •

edited

Loading

johndpope commented Dec 20, 2016 •

edited

Loading

jcanizales commented Dec 30, 2016 •

edited

Loading