Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Al-479: Add ramp fitting to STCAL #6

Merged
merged 4 commits into from
May 17, 2021

Conversation

kmacdonald-stsci
Copy link
Collaborator

Ramp fitting code moved from JWST to STCAL.

…. The addition of this code to STCAL works fine with JWST ramp fitting tests, including regression tests.
@kmacdonald-stsci kmacdonald-stsci requested review from nden and dmggh May 13, 2021 12:54
dmggh
dmggh previously approved these changes May 14, 2021
Copy link
Contributor

@dmggh dmggh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks fine to me.

@jdavies-st
Copy link
Collaborator

jdavies-st commented May 14, 2021

Running the jwst regression tests with this branch and the branch for spacetelescope/jwst#6023 together in the same job works well. 🎉

Passing results here:

https://plwishmaster.stsci.edu:8081/job/RT/job/jdavies-dev/241/

Clean up the PEP8 issues in this PR, and this can be merged.

jdavies-st
jdavies-st previously approved these changes May 14, 2021
Copy link
Collaborator

@jdavies-st jdavies-st left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me.

Comment on lines +1 to +5
DO_NOT_USE = 2**0 # Bad pixel. Do not use.
SATURATED = 2**1 # Pixel saturated during exposure.
JUMP_DET = 2**2 # Jump detected during exposure
NO_GAIN_VALUE = 2**19 # Gain cannot be measured
UNRELIABLE_SLOPE = 2**24 # Slope variance large (i.e., noisy pixel)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We are going to want to figure out a way to make sure that JWST and Roman use the same DQ mnemonics, and a way to formalize that in a test across both pipelines. Otherwise the shared code model will fall apart.

CHANGES.rst Outdated Show resolved Hide resolved
src/stcal/ramp_fitting/ols_fit.py Show resolved Hide resolved
# final group. Similarly, if leading groups 1 though N have all pixels
# flagged as DO_NOT_USE, those groups will be ignored by ramp fitting, and
# the input model arrays will be resized appropriately. If all pixels in
# all groups are flagged, return None for the models.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@hbushouse Is this specific to MIRI? I realize this is how the code was in jwst. Should this logic be applied to all instruments?



def ramp_fit(model, buffsize, save_opt, readnoise_2d, gain_2d,
algorithm, weighting, max_cores):
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it feasible to add the mission/instrument specific dqflags as a parameter to ramp_fit and avoid hardcoding the values for the two missions here. They can be passed as a

dqflags = {'group_dq: {...}. 
           'pixel_dq': {...}
          }

or a dict of only the flags needed by ramp_fitting

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will have to be discussed what the best way is to get this information to ramp fit in STCAL. Changing the ramp_fit function will mean changing all the tests, but that might be the best way to make this function general use for multiple pipelines.

src/stcal/ramp_fitting/ols_fit.py Outdated Show resolved Hide resolved
src/stcal/ramp_fitting/ols_fit.py Outdated Show resolved Hide resolved
src/stcal/ramp_fitting/ols_fit.py Outdated Show resolved Hide resolved
src/stcal/ramp_fitting/ols_fit.py Outdated Show resolved Hide resolved
src/stcal/ramp_fitting/ols_fit.py Outdated Show resolved Hide resolved
src/stcal/ramp_fitting/ols_fit.py Outdated Show resolved Hide resolved
@nden
Copy link
Collaborator

nden commented May 16, 2021

Many of the comments are related to docstrings. The main functional one is the suggestion to pass the dqflags as a parameter to ramp_fit in the ramp_fit_step code so that they can be mission specific.

@kmacdonald-stsci
Copy link
Collaborator Author

Regarding the docstrings, wherever there was an ndarray, I moved the dimension and type to the description portion for the parameter and marked the type as simply ndarray.

@nden
Copy link
Collaborator

nden commented May 17, 2021

Given the two approvals in the comments I am going to merge this.
The remaining issues will be resolved in separate PRs.

@nden nden merged commit d721016 into spacetelescope:main May 17, 2021
@kmacdonald-stsci kmacdonald-stsci deleted the al_479_add_ramp_fit branch May 17, 2021 19:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants