Drizzle produces flux artifacts near edges of field

[stscijgbot-jp]

Issue JP-2111 was created on JIRA by David Law:

The drizzle algorithm appears to have a previously-undocumented behaviour that it can artificially increase the recovered source flux by a few percent near the edges of a field.  I noticed this first when developing a 3d variant of drizzle for use by the JWST IFU cube building software, but have since replicated the same issue using ordinary 2d drizzle as implemented in the JWST imaging pipeline.

In brief, when a point source gets close to the edge of the imaging field, part of the PSF starts to fall off the detector and the recovered flux from flux-calibrated _cal.fits files (logically) starts to decrease as well.  As the source centroid gets closer to, and eventually moves off the edge entirely, the flux continues to decrease until reaching zero.  See orange points in attached plot.

That much is expected behaviour, but using the drizzle-rectified _i2d.fits files the recovered source flux first increases as the PSF wings start to move off the field before decreasing as expected.  See blue points in attached plot.

I believe this is a natural consequence of the drizzle function weighting by the fractional overlap between input and output pixels.  Generally, this is a good thing.  However, if you are preferentially losing low-surface brightness samples the high-surface brightness samples near the PSF core are correspondingly upweighted in more output pixels.  I assume this would introduce a distortion to the drizzled PSF as well, although I haven't checked.

In the limit of a large field of view with many dithers, the impact of this is probably negligible in the region that most people would choose to use for science.  In the limit of a small field of view (for which the edge is a correspondingly much larger fraction of the total FOV) and low number of dithers though, this is quite noticeable.  Both are true for the JWST IFUs (field size 3x3 arcsec and 1-4 dithers), and hence this is something that we'll have to keep in mind in developing 3d drizzle variants.

I don't think I'm proposing any change to the ordinary 2d drizzle algorithm, but if someone else can confirm this behaviour for non-MIRI instruments (e.g., NIRCam or WFC3) it may be good to note it as a known artifact that can be seen under some conditions.

Linked test data is for a series of MIRI imaging simulations produced with mirisim that gradually scan a point source off the edge of the field of view and then compare the Image2 pipeline products.

[jdavies-st]

If this is an effect of the drizzle algorithm itself, it should be visible in HST data too, right?

Or is this specific to the JWST implementation that uses drizzle? (instead of drizzlepac used by HST)

[stscijgbot-jp]

Comment by Alicia Canipe on JIRA:

Matteo Correnti or Kevin Volk, have you seen any similar issues? 

[stscijgbot-jp]

Comment by Kevin Volk on JIRA:

I have not had a test case where this effect would be seen.  I would need to make a specific test for it to reproduce the result.

[stscijgbot-jp]

Comment by Matteo Correnti on JIRA:

Similar to what Kevin said, I haven't done a test case where this effect would be seen. I can try to make a specific test and report the results. 

Tagging Mattia Libralato since he is also working with some MIRI imaging simulation. 

[stscijgbot-jp]

Comment by David Law on JIRA:

Yes- if I'm right about the algorithm itself being to blame then it should be noticeable in HST data too.  Would be interesting to see if this is the case.  Most straight forward test case would be a single undithered exposure in which a point source scans gradually closer to the edge.

[stscijgbot-jp]

Comment by Anton Koekemoer on JIRA:

Just to add, it's not an effect that's expected (either in HST drizzlepac or in the JWST implementation of drizzle) - so to start with, I agree it would be good to see some quick simulation results for NIRCam, NIRISS and MIRI ( Matteo Correnti  Kevin Volk and Mattia Libralato  above) - eg stepping a star off the edge in increments of small fractions of a pixel, and show similar plots to those that David did for the IFU case.

[stscijgbot-jp]

Comment by David Law on JIRA:

Just a quick note- the example plot that I showed above is for MIRI imaging.

[stscijgbot-jp]

Comment by Anton Koekemoer on JIRA:

thanks David Law  - and yes, here's the multi-panel set of frames that David generated, each of which I think corresponds to the one of the points in his plot above (David could perhaps point you all to these MIRI images if you want to look at them further.

The ones that show the (unexpected) increase are ~20-35 in the sequence.

 

!miri-star-grid-dlaw.png|width=565,height=530!

[stscijgbot-jp]

Comment by Kevin Volk on JIRA:

An initial test for NIRISS with a star moved off the edge of the detector has shown signal oddities which are much worse than what David sees for MIRI.  I located stars from pixel 2037 to 2047 in steps of 0.2 pixels along the y direction and offset 50 pixels each in x and then ran a Mirage model for the scene (all the stars are of the same brightness).  Running this through the pipeline produced the signal estimates shown below.  For the raw or cal images the signal goes down smoothly as the star goes off the active part of the array.  But the i2d image shows a lot of scatter in the signal over the same set of points.  The scene has a background and where that dominates the two agree, but before that there are significant signal variations depending on how the star is positioned on a pixel.  It may be that drizzle is having issues with resampling the NIRISS pixels, which are just slightly rectangular.

The overall difference in signal is an issue that I have seen before (JP-1570).  Apparently it has not been fixed yet. 

This is for the F158M filter which is significantly under-sampled.  I need to do another test for F480M where the sampling is more or less Nyquist.

!signal_comparison_drizzle_normal.png!

[stscijgbot-jp]

Comment by David Law on JIRA:

Adding a quick note based on some iteration with Mattia Libralato ; in the MIRI case the offending pixels in the i2d frame lie in the column right along the edge of the drizzled image.  These are officially 'good', but have much lower WHT values (i.e., the number of input pixels contributing to a given output pixel) than the rest of the image.  Just having lower WHT is ok, but in this case the input pixels contributing to the final pixel values in this column are preferentially biased towards larger values than usual (as we've dropped the lower values further out on the psf wing off the detector), and this preferential biasing of the input produces the artificially higher output values when this column is included in an aperture sum.

I'd imagine this effect should show up anywhere that a strong gradient in the WHT map coincides with a strong gradient in the surface brightness.  Indeed, I remember just such a problem in processing the SDSS IFU data when dealing with regions near broken fibers or other 'dead' regions within the FOV.  There, the solution was to grow our DONOTUSE mask slightly to flag regions with abnormally low WHT.

[stscijgbot-jp]

Comment by Anton Koekemoer on JIRA:

thanks David Law  and Kevin Volk  for the updates, and also for the additional details about the MIRI weight values.

Another aspect of this is the choice of weighting - a popular option is inverse variance "ivm", but for this particular test it may complicate matters since the weight for each pixel depends on a variety of factors specific to that pixel. Would you mind please checking what type of weighting you used, and if it was "ivm", would you mind please running another set with weight type set to "exptime" instead (ie, effectively uniform weighting for all the valid pixels), for the same set of simulations (48 steps), and show the results here?

(it's quite conceivable that the results would be unchanged, but it would still be helpful to verify what the behaviour is with "exptime" weighting if that wasn't used in the initial test).

(Also let's confirm that pixfrac=1 and kernel="square", which I'm sure is the case so I'm really just stating that here for reference.)

[stscijgbot-jp]

Comment by David Law on JIRA:

Anton Koekemoer I can confirm that pixfrac=1 and kernel=square for the MIRI example.  Results are essentially indistinguishable using weight_type='exptime' as opposed to the default 'ivm'.