JP-3153: post-commissioning AMI3 updates#7862

Merged

zacharyburnett merged 211 commits intospacetelescope:masterfrom rcooper295:amianalyze

Mar 25, 2024

+2,949-1,732

Commits on Mar 25, 2024

Added keywords read in from datamodel and stored in NIRISS object for oifits writing, additional input args for flexibility

rcooper295
authored and
braingram
committed
commented out now-unused code including throughput ref file use, to be deleted later.

rcooper295
authored and
braingram
committed
Updated filter/source spectrum combination, expand data dimensions if 2D input, pass new args to instrument_data.NIRISS

rcooper295
authored and
braingram
committed
Run on 3d files: fringefitter contains two methods fit_fringes_all and fit_fringes_single_integration. Models are fit to each slice in single_integration, then are returned to fit_fringes_all to be…

rcooper295
authored and
braingram
committed
functions for filter/source combination, etc copied from ImPlaneIA

rcooper295
authored and
braingram
committed
added step args affine2d, bandpass, src, firstfew, chooseholes, usebp (tbc)

rcooper295
authored and
braingram
committed
updates to weighted_operations, matrix_operations to use bad pixel mask

rcooper295
authored and
braingram
committed
fixed typos

rcooper295
authored and
braingram
committed
Pipeline adaptation of fourier bad pixel fix added, and usage of dq mask to ignore bad pixels in fringe fitting (use_bp) option added. Models produced are identical to current ImPlaneIA

rcooper295
authored and
braingram
committed
additional bad pixels >10 sigma from median image added to DQ array for correction and/or ignoring in fringefitting

rcooper295
authored and
braingram
committed
first commit of code prepare data to write to stdatamodels AmiOIModel

rcooper295
authored and
braingram
committed
changes to test new datamodel. made residual calculation consistent between weighted and unweighted matrix operations (data - model).

rcooper295
authored and
braingram
committed
Changes to write out AmiOIModel, multi still in progress, cleaning needed

rcooper295
authored and
braingram
committed
uncommented asn rule changes to use 3d inputs for AMI exposures

rcooper295
authored and
braingram
committed
minor code and comment cleanup

rcooper295
authored and
braingram
committed
pistons and pist_err columns now included in oifits array extension

rcooper295
authored and
braingram
committed
removed unused Calibrate class. Some aspects may be added back later

rcooper295
authored and
braingram
committed
Added new CalibOifits class, using new AmiOIModel structure

rcooper295
authored and
braingram
committed
replaced use of AmiLgModel with AmiOIModel in ami_normalize step

rcooper295
authored and
braingram
committed
modify dtype of oi array for calibrated oifits files

rcooper295
authored and
braingram
committed
add calibrator name to calibrated oifits header

rcooper295
authored and
braingram
committed
Changes to try to write out to new amilgfitmodel

rcooper295
authored and
braingram
committed
Updates to write out multi-integration oifits files

rcooper295
authored and
braingram
committed
add skip=True to spec block of ami_average step to turn it off in the ami3 pipeline by default

rcooper295
authored and
braingram
committed
update input arguments in spec, models returned from step call

rcooper295
authored and
braingram
committed
print statements replaced with logging, minor comment cleanup

rcooper295
authored and
braingram
committed
style fixes: removed unused imports, etc

rcooper295
authored and
braingram
committed
Docstrings and minor cleanup

rcooper295
authored and
braingram
committed
more minor style changes

rcooper295
authored and
braingram
committed
even more minor style changes

rcooper295
authored and
braingram
committed
removed parentheses in spec block that caused parsing error

rcooper295
authored and
braingram
committed
WIP updates to calwebb_ami3 to use restructured flow skipping average step

rcooper295
authored and
braingram
committed
Updated ami_analyze documentation

rcooper295
authored and
braingram
committed
More RTD updates to ami_analyze and ami_normalize

rcooper295
authored and
braingram
committed
WIP address docs build issue caused by missing WEBBPSF_PATH

rcooper295
authored and
braingram
committed
removed references to throughput ref file in docs

rcooper295
authored and
braingram
committed
RTD updates about AMI products (WIP)

rcooper295
authored and
braingram
committed
fix 'malformed tables'

rcooper295
authored and
braingram
committed
calwebb_ami3 pipeline working with updated steps

rcooper295
authored and
braingram
committed
ami3 pipeline and ami average docs updates

rcooper295
authored and
braingram
committed
copy input header keywords to all three outputs

rcooper295
authored and
braingram
committed
updated docs with agreed-upon file suffixes and extensions for OIFITS products

rcooper295
authored and
braingram
committed
one more RTD update

rcooper295
authored and
braingram
committed
file suffixes updated in calwebb_ami3

rcooper295
authored and
braingram
committed
set ami suffix based on result index
braingram
committed
Save amianalyze outputs with correct suffixes by overriding save_model method

rcooper295
authored and
braingram
committed
update ci to use webbpsf cache like romancal
braingram
committed
remove unused import
braingram
committed
add new deps
braingram
committed
attempt to get webbpsf cache to download
braingram
committed
bump astroquery min
braingram
committed
Updated ami3 reg test to follow restructured stage and test new outputs

rcooper295
authored and
braingram
committed
added webbpsf path to jenkinsRT for testing

rcooper295
authored and
braingram
committed
update regtest for new files
braingram
committed
use simpson to avoid warnings
braingram
committed
remove is comparison using flatdqm
braingram
committed
Correct NaNs in median image used to find rotation. Add 'fudge' factor to avoid singularity in centered psf models as in ImPlaneIA.

rcooper295
authored and
braingram
committed
Use updated throughput reference files instead of WebbPSF data files

rcooper295
authored and
braingram
committed
Use new nrm reference file for pupil mask used in bp_fix instead of WebbPSF data file

rcooper295
authored and
braingram
committed
Replaced use of verbose argument with logging in filter source spectrum functions

rcooper295
authored and
braingram
committed
Fixed typo, STA_INDEX now 1-indexed as required for OIFITS

rcooper295
authored and
braingram
committed
Removed use of astroquery to get target information, now using input model metadata

rcooper295
authored and
braingram
committed
style fixes, removed unused imports

rcooper295
authored and
braingram
committed
pass WEBBPSF_PATH to test env
braingram
committed
refine test error matches
braingram
committed
replace DataModel check with datamodel.open
braingram
committed
add test for even oversample
braingram
committed
add test for get_filt_spec
braingram
committed
add get_src_spec test
braingram
committed
prefix private cdmatrix_to_sky with underscore
braingram
committed
cleanup spacing, remove commented code
braingram
committed
remove verbose and plot
braingram
committed
more code cleanup
braingram
committed
simplify oifits
braingram
committed
fix style
braingram
committed
clean up oifits docstrings
braingram
committed
add minimal fake model
braingram
committed
remove astroquery
braingram
committed
add a few more unit tests
braingram
committed
update tests
braingram
committed
remove unused import
braingram
committed
remove previous CI changes for webbpsf data
braingram
committed
update a few variable names
braingram
committed
docs updates to use throughput and nrm reference files

rcooper295
authored and
braingram
committed
remove use of src (source spectral type) input argument that relied on stsynphot data files. Now using flat spectrum

rcooper295
authored and
braingram
committed
remove stsynphot dependency and unit tests

rcooper295
authored and
braingram
committed
undoing mask definition change that hasn't been tested enough

rcooper295
authored and
braingram
committed
style fixes

rcooper295
authored and
braingram
committed
RTD style fix

rcooper295
authored and
braingram
committed
update suffix lists for ami
braingram
committed
mock nrm ref in unit test
braingram
committed
updated science products descriptions with correct suffixes

rcooper295
authored and
braingram
committed
changelog entry

rcooper295
authored and
braingram
committed
docs formatting fix

rcooper295
authored and
braingram
committed
RTD product types updated

rcooper295
authored and
braingram
committed
Update docs/jwst/ami_analyze/description.rst

authored and
braingram
committed
Apply suggestions from code review

authored and
braingram
committed
Small tweaks in response to review, mostly docstrings

rcooper295
authored and
braingram
committed
Apply more small suggestions from code review

authored and
braingram
committed
Moved bandpass logic to utils function handle_bandpass() to enable unit testing at some point

rcooper295
authored and
braingram
committed
Simplified (removed) unnecessary use of fudge factor in hexagon transfrom

rcooper295
authored and
braingram
committed
Further code cleanup in response to review

rcooper295
authored and
braingram
committed
Style fixes

rcooper295
authored and
braingram
committed
clean up FringeFitter kwargs
braingram
committed
add weighted_operations unit test
braingram
committed
update test data to avoid divide by zero
braingram
committed
add rcond to lstsq to silence FutureWarning
braingram
committed
Added keywords read in from datamodel and stored in NIRISS object for oifits writing, additional input args for flexibility

rcooper295
authored and
braingram
committed
commented out now-unused code including throughput ref file use, to be deleted later.

rcooper295
authored and
braingram
committed
Updated filter/source spectrum combination, expand data dimensions if 2D input, pass new args to instrument_data.NIRISS

rcooper295
authored and
braingram
committed
Run on 3d files: fringefitter contains two methods fit_fringes_all and fit_fringes_single_integration. Models are fit to each slice in single_integration, then are returned to fit_fringes_all to be…

rcooper295
authored and
braingram
committed
functions for filter/source combination, etc copied from ImPlaneIA

rcooper295
authored and
braingram
committed
added step args affine2d, bandpass, src, firstfew, chooseholes, usebp (tbc)

rcooper295
authored and
braingram
committed
updates to weighted_operations, matrix_operations to use bad pixel mask

rcooper295
authored and
braingram
committed
fixed typos

rcooper295
authored and
braingram
committed
Pipeline adaptation of fourier bad pixel fix added, and usage of dq mask to ignore bad pixels in fringe fitting (use_bp) option added. Models produced are identical to current ImPlaneIA

rcooper295
authored and
braingram
committed
additional bad pixels >10 sigma from median image added to DQ array for correction and/or ignoring in fringefitting

rcooper295
authored and
braingram
committed
first commit of code prepare data to write to stdatamodels AmiOIModel

rcooper295
authored and
braingram
committed
changes to test new datamodel. made residual calculation consistent between weighted and unweighted matrix operations (data - model).

rcooper295
authored and
braingram
committed
Changes to write out AmiOIModel, multi still in progress, cleaning needed

rcooper295
authored and
braingram
committed
minor code and comment cleanup

rcooper295
authored and
braingram
committed
pistons and pist_err columns now included in oifits array extension

rcooper295
authored and
braingram
committed
removed unused Calibrate class. Some aspects may be added back later

rcooper295
authored and
braingram
committed
Added new CalibOifits class, using new AmiOIModel structure

rcooper295
authored and
braingram
committed
replaced use of AmiLgModel with AmiOIModel in ami_normalize step

rcooper295
authored and
braingram
committed
modify dtype of oi array for calibrated oifits files

rcooper295
authored and
braingram
committed
add calibrator name to calibrated oifits header

rcooper295
authored and
braingram
committed
Changes to try to write out to new amilgfitmodel

rcooper295
authored and
braingram
committed
Updates to write out multi-integration oifits files

rcooper295
authored and
braingram
committed
update input arguments in spec, models returned from step call

rcooper295
authored and
braingram
committed
print statements replaced with logging, minor comment cleanup

rcooper295
authored and
braingram
committed
style fixes: removed unused imports, etc

rcooper295
authored and
braingram
committed
Docstrings and minor cleanup

rcooper295
authored and
braingram
committed
more minor style changes

rcooper295
authored and
braingram
committed
even more minor style changes

rcooper295
authored and
braingram
committed
removed parentheses in spec block that caused parsing error

rcooper295
authored and
braingram
committed
WIP updates to calwebb_ami3 to use restructured flow skipping average step

rcooper295
authored and
braingram
committed
Updated ami_analyze documentation

rcooper295
authored and
braingram
committed
More RTD updates to ami_analyze and ami_normalize

rcooper295
authored and
braingram
committed
WIP address docs build issue caused by missing WEBBPSF_PATH

rcooper295
authored and
braingram
committed
RTD updates about AMI products (WIP)

rcooper295
authored and
braingram
committed
fix 'malformed tables'

rcooper295
authored and
braingram
committed
calwebb_ami3 pipeline working with updated steps

rcooper295
authored and
braingram
committed
ami3 pipeline and ami average docs updates

rcooper295
authored and
braingram
committed
copy input header keywords to all three outputs

rcooper295
authored and
braingram
committed
updated docs with agreed-upon file suffixes and extensions for OIFITS products

rcooper295
authored and
braingram
committed
one more RTD update

rcooper295
authored and
braingram
committed
file suffixes updated in calwebb_ami3

rcooper295
authored and
braingram
committed
set ami suffix based on result index
braingram
committed
Save amianalyze outputs with correct suffixes by overriding save_model method

rcooper295
authored and
braingram
committed
update ci to use webbpsf cache like romancal
braingram
committed
remove unused import
braingram
committed
add new deps
braingram
committed
attempt to get webbpsf cache to download
braingram
committed
bump astroquery min
braingram
committed
Updated ami3 reg test to follow restructured stage and test new outputs

rcooper295
authored and
braingram
committed
added webbpsf path to jenkinsRT for testing

rcooper295
authored and
braingram
committed
update regtest for new files
braingram
committed
use simpson to avoid warnings
braingram
committed
remove is comparison using flatdqm
braingram
committed
Correct NaNs in median image used to find rotation. Add 'fudge' factor to avoid singularity in centered psf models as in ImPlaneIA.

rcooper295
authored and
braingram
committed
Use updated throughput reference files instead of WebbPSF data files

rcooper295
authored and
braingram
committed
Use new nrm reference file for pupil mask used in bp_fix instead of WebbPSF data file

rcooper295
authored and
braingram
committed
Replaced use of verbose argument with logging in filter source spectrum functions

rcooper295
authored and
braingram
committed
Fixed typo, STA_INDEX now 1-indexed as required for OIFITS

rcooper295
authored and
braingram
committed
Removed use of astroquery to get target information, now using input model metadata

rcooper295
authored and
braingram
committed
style fixes, removed unused imports

rcooper295
authored and
braingram
committed
pass WEBBPSF_PATH to test env
braingram
committed
refine test error matches
braingram
committed
replace DataModel check with datamodel.open
braingram
committed
add test for even oversample
braingram
committed
add test for get_filt_spec
braingram
committed
add get_src_spec test
braingram
committed
prefix private cdmatrix_to_sky with underscore
braingram
committed
cleanup spacing, remove commented code
braingram
committed
remove verbose and plot
braingram
committed
more code cleanup
braingram
committed
simplify oifits
braingram
committed
fix style
braingram
committed
clean up oifits docstrings
braingram
committed
add minimal fake model
braingram
committed
remove astroquery
braingram
committed
add a few more unit tests
braingram
committed
update tests
braingram
committed
remove unused import
braingram
committed
remove previous CI changes for webbpsf data
braingram
committed
update a few variable names
braingram
committed
docs updates to use throughput and nrm reference files

rcooper295
authored and
braingram
committed
remove use of src (source spectral type) input argument that relied on stsynphot data files. Now using flat spectrum

rcooper295
authored and
braingram
committed
remove stsynphot dependency and unit tests

rcooper295
authored and
braingram
committed
undoing mask definition change that hasn't been tested enough

rcooper295
authored and
braingram
committed
style fixes

rcooper295
authored and
braingram
committed
RTD style fix

rcooper295
authored and
braingram
committed
update suffix lists for ami
braingram
committed
mock nrm ref in unit test
braingram
committed
updated science products descriptions with correct suffixes

rcooper295
authored and
braingram
committed
docs formatting fix

rcooper295
authored and
braingram
committed
Update docs/jwst/ami_analyze/description.rst

authored and
braingram
committed
Apply suggestions from code review

authored and
braingram
committed
Small tweaks in response to review, mostly docstrings

rcooper295
authored and
braingram
committed
Apply more small suggestions from code review

authored and
braingram
committed
Moved bandpass logic to utils function handle_bandpass() to enable unit testing at some point

rcooper295
authored and
braingram
committed
Simplified (removed) unnecessary use of fudge factor in hexagon transfrom

rcooper295
authored and
braingram
committed
Further code cleanup in response to review

rcooper295
authored and
braingram
committed
Style fixes

rcooper295
authored and
braingram
committed
Remove 'short summary' header from all docstrings and other minor docstring improvements

rcooper295
authored and
braingram
committed
removed unused module to get webbpsf filter info and associated test

rcooper295
authored and
braingram
committed
style fix, remove unused import

rcooper295
authored and
braingram
committed
small documentation update

rcooper295
authored and
braingram
committed
remove unused import
braingram
committed
replace changelog entry
braingram
committed