Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce IDL-based Tensor/Image APIs for python. #3188

Merged
merged 13 commits into from
Sep 4, 2023

Conversation

jleibs
Copy link
Member

@jleibs jleibs commented Sep 1, 2023

What

Implement the assorted init / to_arrow helpers necessary to make tensors and images work.

In particular, needed to override TensorData.__init__ to provide better auto-validation and more flexibility.

Examples of valid constructions for the TensorData

TensorData(array=[[1,2,3],[4,5,6]])
TensorData(array=[[1,2,3],[4,5,6]], names=['a', 'b'])
TensorData(buffer=[1,2,3,4,5,6], shape=[TensorDimension(2, name='a'), TensorDimension(3, name='b')]

Additionally, the Tensor or Image archetypes can be derived directly from an array-type,

Tensor([[1,2,3],[4,5,6]])
Tensor(TensorData(array=[[1,2,3],[4,5,6]]))

Image([[1,2,3],[4,5,6]])
Image(TensorData(array=[[1,2,3],[4,5,6]]))

Needed to add the ability to override init for archetypes in order to patch in validation on image shapes.

Part of: #2792

Checklist

@jleibs jleibs marked this pull request as ready for review September 2, 2023 00:51
@jleibs jleibs added codegen/idl 🐍 Python API Python logging API labels Sep 2, 2023
@teh-cmc teh-cmc self-requested a review September 4, 2023 07:41
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

rename to image_simple.py once rebased!

docs/code-examples/image_simple_v2.py Outdated Show resolved Hide resolved
@jleibs jleibs merged commit 54875bd into main Sep 4, 2023
@jleibs jleibs deleted the jleibs/python_image_apis branch September 4, 2023 13:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants