Tensor (and its many friends) migration to archetypes
#2792
Labels
🏹 arrow
concerning arrow
🌊 C++ API
C/C++ API specific
codegen/idl
🐍 Python API
Python logging API
🦀 Rust API
Rust logging API
Milestone
Comments
teh-cmc
added
🐍 Python API
This was referenced Jul 24, 2023
3 tasks
teh-cmc
added a commit
that referenced
this issue
Sep 1, 2023
### What This PR introduces the new IDL schemas for Tensors and Images. There is a fairly significant change to how we structure our Tensor-related archetypes. Whereas before we used a single Archetype for both Tensors and Images, they are now split into two archetypes that share a common `TensorData` component. `TensorMeaning` is now gone and will be replaced with per-image-type archetypes: `Image`, `DepthImage`, and `SegmentationImage` This PR only introduces the IDL and the Rust APIs. Additional work required for: #2792 - Python APIs - C++ APIs - Round-trip tests - DepthImage - SegmentationImage - Support for f16 types - View parts / heuristics / etc Example of new Image usage: ``` let mut image = Array::<u8, _>::zeros((200, 300, 3).f()); image.slice_mut(s![.., .., 0]).fill(255); image.slice_mut(s![50..150, 50..150, 0]).fill(0); image.slice_mut(s![50..150, 50..150, 1]).fill(255); let image = Image::try_from(image)?; MsgSender::from_archetype("image", &image)?.send(&rec_stream)?; ``` ## New Archetypes: - Tensor - TensorData - Image - TensorData ## New Components - TensorData - TensorData ## New Datatypes - TensorData - TensorId - TensorDimension - TensorBuffer * Fixes #1992 ### Checklist * [x] I have read and agree to [Contributor Guide](https://github.com/rerun-io/rerun/blob/main/CONTRIBUTING.md) and the [Code of Conduct](https://github.com/rerun-io/rerun/blob/main/CODE_OF_CONDUCT.md) * [x] I've included a screenshot or gif (if applicable) * [x] I have tested [demo.rerun.io](https://demo.rerun.io/pr/3032) (if applicable) - [PR Build Summary](https://build.rerun.io/pr/3032) - [Docs preview](https://rerun.io/preview/pr%3Ajleibs%2Fhope_tensors/docs) - [Examples preview](https://rerun.io/preview/pr%3Ajleibs%2Fhope_tensors/examples) --------- Co-authored-by: Clement Rey <[email protected]>
3 tasks
jleibs
added a commit
that referenced
this issue
Sep 4, 2023
### What Implement the assorted init / to_arrow helpers necessary to make tensors and images work. In particular, needed to override `TensorData.__init__` to provide better auto-validation and more flexibility. Examples of valid constructions for the TensorData ``` TensorData(array=[[1,2,3],[4,5,6]]) TensorData(array=[[1,2,3],[4,5,6]], names=['a', 'b']) TensorData(buffer=[1,2,3,4,5,6], shape=[TensorDimension(2, name='a'), TensorDimension(3, name='b')] ``` Additionally, the `Tensor` or `Image` archetypes can be derived directly from an array-type, ``` Tensor([[1,2,3],[4,5,6]]) Tensor(TensorData(array=[[1,2,3],[4,5,6]])) Image([[1,2,3],[4,5,6]]) Image(TensorData(array=[[1,2,3],[4,5,6]])) ``` Needed to add the ability to override init for archetypes in order to patch in validation on image shapes. Part of: #2792 ### Checklist * [x] I have read and agree to [Contributor Guide](https://github.com/rerun-io/rerun/blob/main/CONTRIBUTING.md) and the [Code of Conduct](https://github.com/rerun-io/rerun/blob/main/CODE_OF_CONDUCT.md) * [x] I've included a screenshot or gif (if applicable) * [x] I have tested [demo.rerun.io](https://demo.rerun.io/pr/3188) (if applicable) - [PR Build Summary](https://build.rerun.io/pr/3188) - [Docs preview](https://rerun.io/preview/dcd01eb045ca3e737c55d6893c2c4203a96141e4/docs) <!--DOCS-PREVIEW--> - [Examples preview](https://rerun.io/preview/dcd01eb045ca3e737c55d6893c2c4203a96141e4/examples) <!--EXAMPLES-PREVIEW--> - [Recent benchmark results](https://ref.rerun.io/dev/bench/) - [Wasm size tracking](https://ref.rerun.io/dev/sizes/)
4 tasks
emilk
added a commit
that referenced
this issue
Sep 7, 2023
* Part of #2792 ### What Introduces DepthImage and SegmentationImage as additional archetypes, along with round-trip-tests and code-examples. Deletes all of the old Image/Tensor related components. Updates all the necessary places to derive meaning from the indicator components. This still leaves Images ViewPartSystem as a monolithic system, but splits up the process calls into the 3 separate ArchetypeViews. There's a lot of duplicate code here that could definitely be factored out. Still Needed for feature-parity with the old APIs. - F16 Support I've added it as TODO:s on #2792 Things to investigate: - [x] Roud-trip test is panicking during comparison of `annotation_context_segmentation` although both code-examples run fine on their own. ### Checklist * [x] I have read and agree to [Contributor Guide](https://github.com/rerun-io/rerun/blob/main/CONTRIBUTING.md) and the [Code of Conduct](https://github.com/rerun-io/rerun/blob/main/CODE_OF_CONDUCT.md) * [x] I've included a screenshot or gif (if applicable) * [x] I have tested [demo.rerun.io](https://demo.rerun.io/pr/3237) (if applicable) - [PR Build Summary](https://build.rerun.io/pr/3237) - [Docs preview](https://rerun.io/preview/632bdf6b5baf0c9120795898ad4950b0be2f9be9/docs) <!--DOCS-PREVIEW--> - [Examples preview](https://rerun.io/preview/632bdf6b5baf0c9120795898ad4950b0be2f9be9/examples) <!--EXAMPLES-PREVIEW--> - [Recent benchmark results](https://ref.rerun.io/dev/bench/) - [Wasm size tracking](https://ref.rerun.io/dev/sizes/) --------- Co-authored-by: Emil Ernerfeldt <[email protected]>
|
Split C++ out into its own issue: |
3 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Turn into sub-issues as needed.
jpeg_qualityf16C++ supportExtensionsUnit test (constructors, serialization)re_componentsThe text was updated successfully, but these errors were encountered: