Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Data Refactor #535

Merged
merged 1 commit into from
Oct 17, 2022
Merged

Data Refactor #535

merged 1 commit into from
Oct 17, 2022

Conversation

tylerflex
Copy link
Collaborator

@tylerflex tylerflex commented Oct 4, 2022

Made a new branch to start the data refactor from scratch.

To Do List:

  • reorganize N2F: move near2far.py to components/, merge monitor_data.py and `monitor_data_n2f.py
  • clean up imports: move all td namespace imports to tidy3d/__init__.py, move Symmetry to types.py.
  • simplify file IO: exclude data from .json, write components as string in hdf5.
  • add frozen data fields: replace MonitorData methods (eg. apply_symmery() with properties using stored).
  • Simplify DataArray subclassing from xr.DataArray
  • introduce Dataset abstraction layer, test if circular imports are avoided.
  • fix sim_data: make monitor_data a property and add tuple of MonitorData, fix the getitem and other operations.
  • improve test coverage for data components.
  • make updater work with new changes.
  • fix up backend: add fields into the MonitorData constructors.
  • final clean up / documentation

Backend Changes Needed

  • when the data is empty and constructing the DataArray on the backend, use np.zeros() with the correct shape (including a 0).
  • need to pass symmetry, symmetry_center, and grid_expanded to FieldData -like objects when they are created. If left out, it will just assume symmetry=(0,0,0).
  • there is no longer a .load_from_handle() classmethod, it’s used a couple times in the backend, we need to find a different approach.
  • the coords must be aligned with the shape of the DataArray.data. Sometimes this might not be the case, I had to fix a few times in the front end.

@tylerflex tylerflex force-pushed the feature/data_refactor_1_8 branch 18 times, most recently from c3ca4a8 to 5e02971 Compare October 10, 2022 10:35
@tylerflex tylerflex force-pushed the feature/data_refactor_1_8 branch 6 times, most recently from a68e7d5 to c592a11 Compare October 14, 2022 09:32
@tylerflex tylerflex force-pushed the feature/data_refactor_1_8 branch from da3d09e to f729508 Compare October 17, 2022 14:11
@tylerflex tylerflex merged commit 424d35a into develop Oct 17, 2022
@tylerflex tylerflex deleted the feature/data_refactor_1_8 branch October 17, 2022 14:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant