Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Initial commit of primitive_array type #34

Merged
merged 6 commits into from
Dec 17, 2021
Merged

Conversation

liambenson
Copy link
Contributor

@liambenson liambenson commented Dec 13, 2021

Fixes: DATA-556

Description of Changes

Using pre-allocated RosValues to hold each primitive within an array can be very expensive for large arrays of small data types.
This change makes it so that we create a RosValue on the stack to point to an element of these arrays whenever the items are accessed. This involves a few steps:

  1. RosValue::Pointers now have two 'underlying definitions': one where we track the location in the dynamic vector, and one where we simply store a RosValue
  2. Added a primitive_array type which stores primitives (not including strings) which do not have RosValues pre-allocated for them.

Test Plan

Verified that a large amount of memory was no longer being used by reading a bag full of CompressedImage messages.

Copy link
Contributor

@brendangeck brendangeck left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

General feedback: This is now getting to the point where it's difficult for me to "simulate" the codepath in my head as I review the PR. I think in cases like these, I'd prefer we write some tests to confirm this logic will continue to work after future changes.

lib/ros_value.h Outdated
} else {
primitive_info_.~primitive_info_t();
}
}
RosValue operator=(const RosValue&) {
throw std::runtime_error("Can't call this");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel like we should have a better error message than this

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I actually want to write this, thanks for catching.

@liambenson
Copy link
Contributor Author

General feedback: This is now getting to the point where it's difficult for me to "simulate" the codepath in my head as I review the PR. I think in cases like these, I'd prefer we write some tests to confirm this logic will continue to work after future changes.

@brendangeck I can definitely add some more tests around arrays specifically. The existing tests do a decent job of covering the primitive_array type, but not so much normal arrays and some other edge cases.

@liambenson liambenson merged commit 9d3adde into master Dec 17, 2021
@liambenson liambenson deleted the liam/primitive_array branch December 17, 2021 01:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants