Initial Mid-level FlightClient
#3378
Conversation
alamb
changed the title
FlightClientFlightClient
| /// calling [`Self::into_inner`] and using the [`FlightDataStream`] | ||
| /// directly. | ||
| #[derive(Debug)] | ||
| pub struct FlightRecordBatchStream { |
There was a problem hiding this comment.
This code is not tested yet (it is covered by tests in the IOx repo). I plan to add tests for it as a follow on PR
|
Marking as draft as I work out conflicts with #3359 |
Co-authored-by: Raphael Taylor-Davies <[email protected]>
| ) | ||
| })?; | ||
|
|
||
| arrow_ipc::reader::read_dictionary( |
There was a problem hiding this comment.
I believe there was some discussion about improving the efficiency of IPC reads, etc. This is now the place to do so.
| // specific language governing permissions and limitations | ||
| // under the License. | ||
|
|
||
| /// Errors for the Apache Arrow Flight crate |
There was a problem hiding this comment.
This follows the same model as ArrowError as I did not want to innovate a new Error paradigm (e.g. thiserror as part of this PR)
|
|
||
| //! Integration test for "mid level" Client | ||
|
|
||
| mod common { |
There was a problem hiding this comment.
here are the "missing" tests for arrow-flight -- I think this framework will let us bang out the tests for the flight / flight sql functionality
|
Benchmark runs are scheduled for baseline = af07998 and contender = 17b3210. 17b3210 is a master commit associated with this PR. Results will be available as each benchmark for each run completes. |
Which issue does this PR close?
Towards #3371
Rationale for this change
I am building a FlightSQL client. There are many lower level details such as RecordBatch reconstruction that are not FlightSQL specific. Not only are they easier to test as a separate layer, the mid level client is useful on its own (for writing clients for other Arrow Flight based APIs) so I am contributing it here.
More details are on #3371
If this basic approach is accepted, I plan:
FlightClientWhat changes are included in this PR?
FlightClient,FlightErrormodules, originally written for IOx in https://github.com/influxdata/influxdb_iox/pull/6427Are there any user-facing changes?
These are new features
cc @tustvold @Dandandan @avantgardnerio