having trouble simulating an image for a NIRcam observation... #823
Comments
|
Hmm. It's not obvious to me from the logs why the Inst keyword is missing. Can you send one of your yaml files? If there are details in the yaml that you'd rather not post here, you can also create a new ticket at the JWST Help desk (jwsthelp.stsci.edu). If you select the "JWST General" or "NIRCam" categories, I'll get the ticket. |
|
Dear Bryan:
Many thanks for your prompt response --- happy to share the yaml files --
see attached.
cheers,
kalvir dhuga
…On Fri, Jan 6, 2023 at 5:23 PM Bryan Hilbert ***@***.***> wrote:
Hmm. It's not obvious to me from the logs why the Inst keyword is missing.
Can you send one of your yaml files? If there are details in the yaml that
you'd rather not post here, you can also create a new ticket at the JWST
Help desk (jwsthelp.stsci.edu). If you select the "JWST General" or
"NIRCam" categories, I'll get the ticket.
—
Reply to this email directly, view it on GitHub
<#823 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/A5CM5VS67W3VHEGZ2AYNUKDWRATETANCNFSM6AAAAAATS3SM5I>
.
You are receiving this because you authored the thread.Message ID:
***@***.***>
|
|
HI Kalvir, |
|
Dear Bryan: Okay, file attached; kalvir |
|
Ah I think I see the problem. You don't have any source catalogs specified in the yaml files. Sorry, the code should explicitly look for that, but it doesn't. Anyway, you can specify source catalog names when running the yaml generator: https://mirage-data-simulator.readthedocs.io/en/latest/yaml_generator.html#source-catalogs |
|
Or, of course, you can manually add your catalogs to the yaml files. |
|
Dear Bryan:
i just tried the following insert in the script (just prior to the yaml
generator command:
catalogs = {'NGC5408X-1': {'nircam': {'point_source': '
ptsrc_for_dummy-jet-prop_observations_001.cat',
'galaxy': '
galaxies_for_dummy-jet-prop_observations_001.cat',
},
},
}
Unfortunately it generates the same 'Inst' error. Thinking that the inst
key is case sensitive I also tried 'NIRCAM', and 'NIRcam' -- all gave the
same error.
kalvir
…On Sat, Jan 7, 2023 at 7:54 PM Bryan Hilbert ***@***.***> wrote:
Or, of course, you can manually add your catalogs to the yaml files.
—
Reply to this email directly, view it on GitHub
<#823 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/A5CM5VVJGTKHMIHTFSG53ITWRGNTRANCNFSM6AAAAAATS3SM5I>
.
You are receiving this because you authored the thread.Message ID:
***@***.***>
|
|
The error indicates that "Inst" is not in the yaml file at all. Are you using glob to produce a list of yaml files to run? The log makes it look like it's treating observation_list.yaml as one of your exposure yaml files. Could that be it? If you ignore that file does it work? |
|
Dear Bryan:
My apologies for dropping off the radar in the last week or so --- life
just got in the way. I wanted to let you know that your suggestion of
inputting the catalogs by hand was a good one and i did manage in the end
to produce some images. That's the good news. The not so good news is that
the final images (read from the *linear.fits files) are not quite what I
expected. Two things (in my opinion) seem to be out place:
a) way too many what I think are cosmic rays and not real astrophysical
sources, and
b) the images are identical for all the different targets/exposure settings
Regarding the first point I have tried changing the library (SUNMAX to
SUNMIN) and the scale factor (from 1.0 to 0.0) for cosmic rays in the input
file but apparently with little effect. Regarding the second point, I have
tried running the script for one exposure at a time for two different
targets with different catalogs but again the output fields seem to be
identical as if somehow the program is generating the background for one
fixed catalog for all observations.
At this stage I'm stumped for ideas, any suggestions you can offer would be
most appreciated.
Again, many thanks for your previous help; I certainly wouldn't have gotten
as far as I have without your input.
cheers,
kalvir dhuga
ps. I can send you whatever files you think might help to diagnose the
problem.
…On Sat, Jan 7, 2023 at 1:03 PM Kalvir Dhuga ***@***.***> wrote:
Dear Bryan:
i just tried the following insert in the script (just prior to the yaml
generator command:
catalogs = {'NGC5408X-1': {'nircam': {'point_source': '
ptsrc_for_dummy-jet-prop_observations_001.cat',
'galaxy': '
galaxies_for_dummy-jet-prop_observations_001.cat',
},
},
}
Unfortunately it generates the same 'Inst' error. Thinking that the inst
key is case sensitive I also tried 'NIRCAM', and 'NIRcam' -- all gave the
same error.
kalvir
On Sat, Jan 7, 2023 at 7:54 PM Bryan Hilbert ***@***.***>
wrote:
> Or, of course, you can manually add your catalogs to the yaml files.
>
> —
> Reply to this email directly, view it on GitHub
> <#823 (comment)>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/A5CM5VVJGTKHMIHTFSG53ITWRGNTRANCNFSM6AAAAAATS3SM5I>
> .
> You are receiving this because you authored the thread.Message ID:
> ***@***.***>
>
|
|
I haven't updated Mirage to use a cosmic ray rate that matches what's been observed in flight, but I think the rate at the moment is ~reasonable. These should all be removed during the line-fitting step of the JWST calibration pipeline. What readpattern and number of groups are you using for your simulations? In general the NIRCam team recommends keeping individual integrations under about 1000 seconds in order to prevent too many pixels from being affected by cosmic rays. For the second issue, if you take one yaml file for each target, can you confirm that the catalog entries are correct, and also that the telescope pointing (ra, dec entries) are roughly correct? I'm assuming that your targets are not close enough to each other that the same background sources would be visible in both? |
|
Dear Bryan:
Attached are relevant files for two targets i.e, target-1(observation-1)
and target-4(observation-7): the files include yaml and point-source
catalogs.
*TARGET-1: name: NGC5408X-1* # Name of target
target_ra: 210.83179 # RA of the target, from APT file.
target_dec: -41.38297 # Dec of the target, from APT file.
observation_number: '001' # Observation Number
*Telescope pointing for Target-1:*
ra: 210.83184814468453 # RA of simulated pointing
dec: -41.38299869857642 # Dec of simulated pointing
*Target-4_name: HOL-IX-X1 # Name of target* target_ra: 149.47217 # RA of
the target, from APT file.
target_dec: 69.06336 # Dec of the target, from APT file.
observation_number: '007' # Observation Number
*Telescope pointing for Target-4:*
ra: 149.4722922890577 # RA of simulated pointing
dec: 69.06333141093295 # Dec of simulated pointing
Readout pattern same for both observations:
Readout = shallow2 (exposure <1000s)
groupe =10
integrations=6
I hope this helps.
cheers,
kalvir
…On Mon, Jan 23, 2023 at 2:12 PM Bryan Hilbert ***@***.***> wrote:
I haven't updated Mirage to use a cosmic ray rate that matches what's been
observed in flight, but I think the rate at the moment is ~reasonable.
These should all be removed during the line-fitting step of the JWST
calibration pipeline. What readpattern and number of groups are you using
for your simulations? In general the NIRCam team recommends keeping
individual integrations under about 1000 seconds in order to prevent too
many pixels from being affected by cosmic rays.
For the second issue, if you take one yaml file for each target, can you
confirm that the catalog entries are correct, and also that the telescope
pointing (ra, dec entries) are roughly correct? I'm assuming that your
targets are not close enough to each other that the same background sources
would be visible in both?
—
Reply to this email directly, view it on GitHub
<#823 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/A5CM5VUS47GKP3WYRF5LLV3WT3JWPANCNFSM6AAAAAATS3SM5I>
.
You are receiving this because you authored the thread.Message ID:
***@***.***>
|
|
That all looks reasonable. But you're saying the scenes for these two exposures look identical in terms of the astronomical source on the detector? Is the scene correct for one of the targets? |
|
Dear Bryan:
Good point -- will check that out.
kalvir
…On Mon, Jan 23, 2023 at 3:50 PM Bryan Hilbert ***@***.***> wrote:
That all looks reasonable. But you're saying the scenes for these two
exposures look identical in terms of the astronomical source on the
detector? Is the scene correct for one of the targets?
—
Reply to this email directly, view it on GitHub
<#823 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/A5CM5VRLCSIDBVEAVAQNUZ3WT3VHBANCNFSM6AAAAAATS3SM5I>
.
You are receiving this because you authored the thread.Message ID:
***@***.***>
|
c/o Mirage Team:
I've been trying to simulate a simple image for a NIRcam observation but without success. I've attached the script I'm running along with a couple of log files that list the errors that I run into. The log files refer to 2 cases:
a) a run with a single yaml testfile, and
b) a run with glob read of all yaml files
The main script has been constructed following the directions per miarge/web guidelines; the basic input is taken from the xml and pointing files initially exported from the APT file.
Both runs generate errors in the catalog_seed-image stage of the code.
For option (a) I get the following:
File "/opt/anaconda3/lib/python3.9/site-packages/numpy/core/fromnumeric.py", line 86, in _wrapreduction
return ufunc.reduce(obj, axis, dtype, out, **passkwargs)
ValueError: zero-size array to reduction operation maximum which has no identity
For option (b) I get the following:
File "/opt/anaconda3/lib/python3.9/site-packages/mirage/seed_image/catalog_seed_image.py", line 163, in make_seed
used_cats = determine_used_cats(self.params['Inst']['mode'], self.params['simSignals'])
KeyError: 'Inst'
Any help to straighten this out would be very much appreciated.
best,
kalvir dhuga
GW astro group
[email protected]
Attached files follow:
NIRcam-sims.pdf
[mirage_latest_single_test_f
mirage_latest_glob-read-all_yamlfiles.log
ile.log](https://github.com/spacetelescope/mirage/files/10358795/mirage_latest_single_test_file.log)
The text was updated successfully, but these errors were encountered: