copy reference pixels during dqinit #1121
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
dqinit
adds data views to the resulting datamodel which when saved triggers asdf to save the base array resulting in large file sizes. Similar to #1098 this PR adds copies of those arrays to the output datamodel to avoid larger-than-needed file sizes.Without this PR running the
elp
benchmarks (each step in `elp) with a 330Mb input file produces files with sizes as follows:Note the jump to 2.2 G for the
refpix
output.With this PR the file sizes are:
Note that the jump in file size is not seen until later in the pipeline because the base array saved for the views in dqinit is shared until later when the base array is replaced.
Regtests run with no errors: https://plwishmaster.stsci.edu:8081/job/RT/job/Roman-Developers-Pull-Requests/623/
Checklist
CHANGES.rst
under the corresponding subsection