-
Notifications
You must be signed in to change notification settings - Fork 175
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[FEAT] add spark-connect protocol #3189
Conversation
CodSpeed Performance ReportMerging #3189 will not alter performanceComparing Summary
|
1d61321
to
15ed80d
Compare
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #3189 +/- ##
==========================================
- Coverage 78.60% 77.90% -0.71%
==========================================
Files 637 638 +1
Lines 78225 79207 +982
==========================================
+ Hits 61491 61707 +216
- Misses 16734 17500 +766
|
src/spark-connect-script/build.rs
Outdated
@@ -0,0 +1,28 @@ | |||
fn main() -> std::io::Result<()> { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
let place this crate under filepaths:
src/scripts/spark-connect-generation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
This PR adds a script-based approach for generating the Spark Connect protobuf code, eliminating the need for developers to have the protobuf compiler (protoc) installed locally.
Key changes:
spark-connect-script
crate that usestonic-build
to compile the .proto files into Rust code at build timesrc/spark-connect/src/lib.rs
This approach makes it easier for developers to work with the Spark Connect protocol since they don't need to manage protoc installation and versioning. The generated code is checked into the repo and can be reviewed/updated as needed.
Dependencies added: