Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

JP-2071: Added a boolean switch to turn off calculations for one group ramp #75

Closed
wants to merge 15 commits into from

Conversation

kmacdonald-stsci
Copy link
Collaborator

This PR resolves spacetelescope/jwst#6013

A boolean switch has been added to STCAL ramp fitting to suppress ramps that have only one good group. These ramps will be treated as having zero good groups. This is done by finding the one good group in the ramp, then setting the DQ flag for that ramp to DO_NOT_USE.

Two tests have been added to test this new feature.

@codecov
Copy link

codecov bot commented Feb 11, 2022

Codecov Report

Merging #75 (33d8b58) into main (47aec18) will increase coverage by 0.07%.
The diff coverage is 96.15%.

Impacted file tree graph

@@            Coverage Diff             @@
##             main      #75      +/-   ##
==========================================
+ Coverage   88.63%   88.70%   +0.07%     
==========================================
  Files          16       16              
  Lines        2262     2285      +23     
==========================================
+ Hits         2005     2027      +22     
- Misses        257      258       +1     
Impacted Files Coverage Δ
src/stcal/ramp_fitting/ols_fit.py 93.77% <ø> (ø)
src/stcal/ramp_fitting/ramp_fit.py 94.64% <96.00%> (+0.52%) ⬆️
src/stcal/ramp_fitting/ramp_fit_class.py 100.00% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 47aec18...33d8b58. Read the comment docs.

Copy link
Contributor

@dmggh dmggh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm still reviewing, but I have a few comments/questions so far.

Copy link
Collaborator

@hbushouse hbushouse left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a bunch of comments and questions for this first-pass review.

good_groups = np.zeros(intdq.shape, dtype=int)

# Compute the number of good groups in each ramp
good_groups[intdq == 0] = 1
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure that relying on intdq being exactly zero to indicate a good group is entirely appropriate. It's possible that for some flag values to be present (non-zero) that are to be treated as information only (e.g. a warning), but does not render the data unusable. It should be only cases where a group has the DO_NOT_USE flag set that they should be considered completely bad. This assumes that any group that has either the SATURATED or JUMP flags set also has DO_NOT_USE set. We would need to check that.

Copy link
Collaborator Author

@kmacdonald-stsci kmacdonald-stsci Feb 15, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will require discussion because this is how segments are computed at

mask_2d[gdq_sect_r != 0] = False # RE-exclude bad group dq values

The assumption that if a flag is non-zero then the group is not used has always been the assumption for this code.

ngood_groups = good_groups.sum(axis=0)

# For each pixel that has only one good group, mark those groups as
# DO_NOT_USE, making the ramp effectively zero good groups.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this DO_NOT_USE setting only being applied to some internal data array, for use solely within the slope calculations? We don't want those flags to get set in the original input data to the step (i.e. we don't want to make them permanent).

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The modification to the input DQ flags is not saved. In ramp_fit_step.py the process function runs ramp_fit, then the input ramp model is closed without updating or saving.

@kmacdonald-stsci
Copy link
Collaborator Author

This PR does not correctly address the feature request.

@kmacdonald-stsci kmacdonald-stsci deleted the jp_2071 branch March 3, 2022 00:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add ramp_fit parameter to turn on/off one-group slope calculations
3 participants