Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[C++][Python] Basic conversion of RecordBatch to Arrow Tensor #40059

Closed
Tracked by #40058
AlenkaF opened this issue Feb 13, 2024 · 0 comments · Fixed by #40064
Closed
Tracked by #40058

[C++][Python] Basic conversion of RecordBatch to Arrow Tensor #40059

AlenkaF opened this issue Feb 13, 2024 · 0 comments · Fixed by #40064
Assignees
@AlenkaF
Labels
Component: C++ Component: Python Type: enhancement
Milestone
16.0.0

Comments

@AlenkaF
Copy link
Member

AlenkaF commented Feb 13, 2024

Describe the enhancement requested

This issue is a part of #40058 and adds RecordBatch → Tensor basic conversion with:

  • One data type (all columns having for example an int32 data type)
  • No missing values (only NaN)
  • Column-major layout (write data column by column)

Component(s)

C++, Python
@AlenkaF AlenkaF mentioned this issue Feb 13, 2024
Open
12 tasks
@AlenkaF AlenkaF self-assigned this Feb 13, 2024
@github-actions github-actions bot mentioned this issue Feb 13, 2024
Merged
jorisvandenbossche pushed a commit that referenced this issue Mar 5, 2024
@AlenkaF @bkietz
GH-40059: [C++][Python] Basic conversion of RecordBatch to Arrow Tens…
3ba6d28 
…or (#40064)

### Rationale for this change

There is no method currently in Arrow C++ to convert `Table` or `RecordBatch` to a `Tensor`. In #40058 we are proposing to add the conversion and this PR starts with the basic implementation for `RecordBatch`.

### What changes are included in this PR?

Basic conversion `RecordBatch` → `Tensor` is added together with Python bindings. The implementation details are:

- One data type (all columns having for example an `int32` data type) support.
- No missing values support (only `NaN`).
- Column-major layout of the resulting `Tensor`.

### Are these changes tested?

Yes.

### Are there any user-facing changes?

No.
* Closes: #40059

Lead-authored-by: AlenkaF <[email protected]>
Co-authored-by: Alenka Frim <[email protected]>
Co-authored-by: Benjamin Kietzman <[email protected]>
Signed-off-by: Joris Van den Bossche <[email protected]>
@jorisvandenbossche jorisvandenbossche added this to the 16.0.0 milestone Mar 5, 2024
mapleFU pushed a commit to mapleFU/arrow that referenced this issue Mar 7, 2024
@AlenkaF @bkietz @mapleFU
apacheGH-40059: [C++][Python] Basic conversion of RecordBatch to Arro…
021866a 
…w Tensor (apache#40064)

### Rationale for this change

There is no method currently in Arrow C++ to convert `Table` or `RecordBatch` to a `Tensor`. In apache#40058 we are proposing to add the conversion and this PR starts with the basic implementation for `RecordBatch`.

### What changes are included in this PR?

Basic conversion `RecordBatch` → `Tensor` is added together with Python bindings. The implementation details are:

- One data type (all columns having for example an `int32` data type) support.
- No missing values support (only `NaN`).
- Column-major layout of the resulting `Tensor`.

### Are these changes tested?

Yes.

### Are there any user-facing changes?

No.
* Closes: apache#40059

Lead-authored-by: AlenkaF <[email protected]>
Co-authored-by: Alenka Frim <[email protected]>
Co-authored-by: Benjamin Kietzman <[email protected]>
Signed-off-by: Joris Van den Bossche <[email protected]>
thisisnic pushed a commit to thisisnic/arrow that referenced this issue Mar 8, 2024
@AlenkaF @bkietz @thisisnic
apacheGH-40059: [C++][Python] Basic conversion of RecordBatch to Arro…
89f1a3d 
…w Tensor (apache#40064)

### Rationale for this change

There is no method currently in Arrow C++ to convert `Table` or `RecordBatch` to a `Tensor`. In apache#40058 we are proposing to add the conversion and this PR starts with the basic implementation for `RecordBatch`.

### What changes are included in this PR?

Basic conversion `RecordBatch` → `Tensor` is added together with Python bindings. The implementation details are:

- One data type (all columns having for example an `int32` data type) support.
- No missing values support (only `NaN`).
- Column-major layout of the resulting `Tensor`.

### Are these changes tested?

Yes.

### Are there any user-facing changes?

No.
* Closes: apache#40059

Lead-authored-by: AlenkaF <[email protected]>
Co-authored-by: Alenka Frim <[email protected]>
Co-authored-by: Benjamin Kietzman <[email protected]>
Signed-off-by: Joris Van den Bossche <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@AlenkaF AlenkaF

Labels
Component: C++ Component: Python Type: enhancement
Projects
None yet
Milestone
16.0.0
Development

Successfully merging a pull request may close this issue.

GH-40059: [C++][Python] Basic conversion of RecordBatch to Arrow Tensor AlenkaF/arrow
2 participants
@jorisvandenbossche @AlenkaF