JP-3601: Fix background interval edge case

[melanieclarke]

Resolves JP-3601

Closes #8431

This PR fixes a crash in an extraction background edge case, in which both lower and upper limits for the background region are outside the valid area for some data range.

Checklist for maintainers

• added entry in CHANGES.rst within the relevant release section
• updated or added relevant tests
• updated relevant documentation
• added relevant milestone
• added relevant label(s)
• ran regression tests, post a link to the Jenkins job below.
  How to run regression tests on a PR
• Make sure the JIRA ticket is resolved properly

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 56.41%. Comparing base (6580914) to head (7b8f283).
Report is 4 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #8433      +/-   ##
==========================================
+ Coverage   56.38%   56.41%   +0.03%     
==========================================
  Files         387      387              
  Lines       38716    38733      +17     
==========================================
+ Hits        21830    21852      +22     
+ Misses      16886    16881       -5     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[melanieclarke]

In testing, the initial simple fix was a little too simple: it did not handle correctly the case where part of a pixel falls within the specified region, or the case where an empty interval is specified, but the limits are within range of the data. The updated version should now handle both of those cases correctly, as well as fixing the original crash.

[nden]

I am not familiar with extract_1d at that level of detail but the changes seem to address the ticket. Is there a reasonable tests to add?

I started a regression test run here https://plwishmaster.stsci.edu:8081/job/RT/job/JWST-Developers-Pull-Requests/1408

[melanieclarke]

Thanks @nden - I'll see if I can add some unit tests to cover the relevant cases.

[melanieclarke]

@nden - unit tests added for a couple relevant cases in background and source extraction.

Looking at the regression tests, it looks like the test_nirspec_fs_spec3[s00024-s1600a1-x1d] failure is the only relevant change. For this spectral extraction, the WCS is invalid: all wavelengths are set to NaN and the extraction region is out of range. The old version defaulted the flux to 0.0 and the errors to NaN in this case; the new version defaults the flux to NaN and the errors to 0.0. I think the new version is closer to the intended default values.

[hbushouse]

Looks good to me. I'll take your word that the new regtest result is the more appropriate result.

[melanieclarke]

Looks good to me. I'll take your word that the new regtest result is the more appropriate result.

For posterity - the new result matches the default return value for the _extract_src_flux function in extract1d.py when no good data points are found:
flux, var_poisson, var_rnoise, var_flat, bkg_flux, bkg_var_poisson, bkg_var_rnoise, bkg_var_flat, npixels, weight = np.nan, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0
So, I think the result now matches the intent of the code, at least.