Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable pickling for tensor and tensor based geometry #5509

Merged
merged 15 commits into from
Sep 25, 2022

Conversation

yuecideng
Copy link
Collaborator

@yuecideng yuecideng commented Sep 7, 2022

This PR makes Tensor, TensorMap and tensor based geometry pickling.

Current behaviours:

  • Tensor will be on the same device after de-serialization.
  • Non contiguous tensor will be converted to contiguous tensor after de-serilization.

This change is Reviewable

@update-docs
Copy link

update-docs bot commented Sep 7, 2022

Thanks for submitting this pull request! The maintainers of this repository would appreciate if you could update the CHANGELOG.md based on your changes.

@yuecideng yuecideng requested a review from yxlao September 7, 2022 04:29
@yxlao
Copy link
Collaborator

yxlao commented Sep 8, 2022

LGTM after adding python unit tests.

@yuecideng
Copy link
Collaborator Author

  • Add python unit tests for pickling support.
  • Minor change PointCloud::ToString() output to be consistent with TriangleMesh:
PointCloud on CPU:0 [2 points (Float32)].
Attributes: normals (dtype = Float32, shape = {2, 3}).

@yxlao

.gitignore Outdated
@@ -90,3 +90,6 @@ docs/Doxyfile
docs/getting_started.rst
docs/docker.rst
docs/tensorboard.md

# test
*.pkl
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove this

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

utility::LogError(
"Invalid state! Expecting a tuple of size 2.");
}
const Device& device = t[0].cast<Device>();
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use Device::IsAvailable(). For example, when pickling, we might store CUDA:1, but another computer may only have CUDA:0. Same for others.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

[](py::tuple t) {
if (t.size() != 2) {
utility::LogError(
"Invalid state! Expecting a tuple of size "
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add a comment: "Cannot unpickle Device." It is useful for the user to know which class went wrong. Same for others.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

const Device& device = t[0].cast<Device>();
if (device.IsCUDA() && !core::cuda::IsAvailable()) {
utility::LogWarning(
"CUDA is not available, tensor will be "
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Change this comment as well. Same for others.

@pytest.mark.parametrize("device", list_devices())
def test_pickle(device):
img = o3d.t.geometry.Image(o3c.Tensor.ones((10, 10, 3), o3c.uint8, device))
pickle.dump(img, open("img.pkl", "wb"))
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use

import tempfile

with tempfile.TemporaryDirectory() as temp_dir:
    pickle_path = xxx

Same for others.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

@yuecideng yuecideng requested a review from yxlao September 23, 2022 02:48
@yxlao yxlao merged commit 7a25f3d into master Sep 25, 2022
@yxlao yxlao deleted the yueci/tgeometry-pickle branch September 25, 2022 12:41
@reyanshsolis
Copy link
Collaborator

reyanshsolis commented Sep 27, 2022

Great work. It would be very helpful to add a brief documentation example on the Tensor page.
TODO: Check newly added features that are missing documentation and make a priority-based checklist for the same.

@yuecideng
Copy link
Collaborator Author

It would be very helpful to add a brief documentation example on the Tensor page

Sounds good, I will add it later.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants