Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Silence resample test warning by using non-zero data #8251

Merged

Conversation

jdavies-st
Copy link
Collaborator

@jdavies-st jdavies-st commented Jan 31, 2024

Currently there's a warning when running one of the resample unit tests.

$ pytest jwst/resample/tests/test_resample_step.py::test_nirspec_wcs_roundtrip
========================================================== test session starts ===========================================================
platform darwin -- Python 3.12.0, pytest-7.4.4, pluggy-1.3.0
crds_context: jwst_1188.pmap
rootdir: /Users/jdavies/dev/jwst
configfile: pyproject.toml
plugins: ci_watson-0.6.2, asdf-3.0.1, cov-4.1.0, jwst-1.13.4.dev36+g5a0835817, doctestplus-1.1.0, requests-mock-1.11.0, xdist-3.5.0
collected 1 item                                                                                                                         

jwst/resample/tests/test_resample_step.py .                                                                                        [100%]

============================================================ warnings summary ============================================================
jwst/resample/tests/test_resample_step.py::test_nirspec_wcs_roundtrip
  /Users/jdavies/dev/jwst/jwst/resample/resample_spec.py:144: RuntimeWarning: invalid value encountered in scalar divide
    wmean_s = np.sum(sd[good_s]) / total

jwst/resample/tests/test_resample_step.py::test_nirspec_wcs_roundtrip
  /Users/jdavies/dev/jwst/jwst/resample/resample_spec.py:145: RuntimeWarning: invalid value encountered in scalar divide
    wmean_l = np.sum(ld[good_s]) / total

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
===================================================== 1 passed, 2 warnings in 2.72s ======================================================

To find out what's causing it, we convert warnings to errors and print the local values of the variables (pruning the output here for readability).

$ pytest jwst/resample/tests/test_resample_step.py::test_nirspec_wcs_roundtrip -l -W error
========================================================== test session starts ===========================================================
platform darwin -- Python 3.12.0, pytest-7.4.4, pluggy-1.3.0
crds_context: jwst_1188.pmap
rootdir: /Users/jdavies/dev/jwst
configfile: pyproject.toml
plugins: ci_watson-0.6.2, asdf-3.0.1, cov-4.1.0, jwst-1.13.4.dev36+g5a0835817, doctestplus-1.1.0, requests-mock-1.11.0, xdist-3.5.0
collected 1 item                                                                                                                         

jwst/resample/tests/test_resample_step.py F                                                                                        [100%]

================================================================ FAILURES ================================================================
_______________________________________________________ test_nirspec_wcs_roundtrip _______________________________________________________

    def test_nirspec_wcs_roundtrip(nirspec_rate):
        im = AssignWcsStep.call(nirspec_rate)
        im = Extract2dStep.call(im)
        for slit in im.slits:
            _set_photom_kwd(slit)
>       im = ResampleSpecStep.call(im)

jwst/resample/tests/test_resample_step.py:228: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../miniconda3/envs/jwst-dev/lib/python3.12/site-packages/stpipe/step.py:653: in call
../../miniconda3/envs/jwst-dev/lib/python3.12/site-packages/stpipe/step.py:479: in run
jwst/resample/resample_spec_step.py:98: in process
jwst/resample/resample_spec_step.py:134: in _process_multislit
jwst/resample/resample_spec.py:91: in __init__
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

    def build_nirspec_output_wcs(self, refmodel=None):
        """
        Create a spatial/spectral WCS covering footprint of the input
        """
        all_wcs = [m.meta.wcs for m in self.input_models if m is not refmodel]
        if refmodel:
            all_wcs.insert(0, refmodel.meta.wcs)
        else:
            refmodel = self.input_models[0]
    
        # make a copy of the data array for internal manipulation
        refmodel_data = refmodel.data.copy()
        # renormalize to the minimum value, for best results when
        # computing the weighted mean below
        refmodel_data -= np.nanmin(refmodel_data)
    
        # save the wcs of the reference model
        refwcs = refmodel.meta.wcs
    
        # setup the transforms that are needed
        s2d = refwcs.get_transform('slit_frame', 'detector')
        d2s = refwcs.get_transform('detector', 'slit_frame')
        s2w = refwcs.get_transform('slit_frame', 'world')
    
        # estimate position of the target without relying on the meta.target:
        # compute the mean spatial and wavelength coords weighted
        # by the spectral intensity
        bbox = refwcs.bounding_box
        grid = wcstools.grid_from_bounding_box(bbox)
        _, s, lam = np.array(d2s(*grid))
        sd = s * refmodel_data
        ld = lam * refmodel_data
        good_s = np.isfinite(sd)
        if np.any(good_s):
            total = np.sum(refmodel_data[good_s])
>           wmean_s = np.sum(sd[good_s]) / total
E           RuntimeWarning: invalid value encountered in scalar divide
...
good_s     = array([[False, False, False, ..., False, False, False],
       [False, False, False, ..., False, False, False],
      ...True],
       [ True,  True,  True, ...,  True,  True,  True],
       [False, False, False, ..., False, False, False]])
...
refmodel_data = array([[0., 0., 0., ..., 0., 0., 0.],
       [0., 0., 0., ..., 0., 0., 0.],
       [0., 0., 0., ..., 0., 0., 0.],
    ..., 0., 0., ..., 0., 0., 0.],
       [0., 0., 0., ..., 0., 0., 0.],
       [0., 0., 0., ..., 0., 0., 0.]], dtype=float32)
...
sd         = array([[nan, nan, nan, ..., nan, nan, nan],
       [nan, nan, nan, ..., nan, nan, nan],
       [nan, nan, nan, ..., na...-0., -0., ..., -0., -0., -0.],
       [-0., -0., -0., ..., -0., -0., -0.],
       [nan, nan, nan, ..., nan, nan, nan]])
...
total      = 0.0

jwst/resample/resample_spec.py:144: RuntimeWarning
======================================================== short test summary info =========================================================
FAILED jwst/resample/tests/test_resample_step.py::test_nirspec_wcs_roundtrip - RuntimeWarning: invalid value encountered in scalar divide
=========================================================== 1 failed in 2.73s ============================================================

So it's a divide-by-zero warning because the sum total of the flux in the slit is zero. This is the part of the code where the mean spatial location of the spectrum is found by flux-weighting the spatial WCS. But in the test, the flux was all zeros.

This PR fixes the input data to the test to be non-zero.

One could probably handle this exceedingly unlikely case in the runtime code as well, i.e. check for total == 0, but this is probably not realistic for any real data.

Checklist for maintainers

  • added entry in CHANGES.rst within the relevant release section
  • updated or added relevant tests
  • updated relevant documentation
  • added relevant milestone
  • added relevant label(s)
  • ran regression tests, post a link to the Jenkins job below.
    How to run regression tests on a PR
  • Make sure the JIRA ticket is resolved properly

Copy link

codecov bot commented Jan 31, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (557c3fa) 75.18% compared to head (4ca53ee) 75.18%.

Additional details and impacted files
@@           Coverage Diff           @@
##           master    #8251   +/-   ##
=======================================
  Coverage   75.18%   75.18%           
=======================================
  Files         470      470           
  Lines       38549    38549           
=======================================
  Hits        28983    28983           
  Misses       9566     9566           
Flag Coverage Δ *Carryforward flag
nightly 77.37% <ø> (ø) Carriedforward from 557c3fa

*This pull request uses carry forward flags. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@hbushouse hbushouse added this to the Build 10.2 milestone Feb 5, 2024
Copy link
Collaborator

@hbushouse hbushouse left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good.

@hbushouse hbushouse merged commit 4d2f08c into spacetelescope:master Feb 5, 2024
32 of 33 checks passed
@jdavies-st jdavies-st deleted the fix-nirspec-resample-warning branch February 5, 2024 13:36
emolter pushed a commit to emolter/jwst that referenced this pull request Feb 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants