Project Roadmap

[ismael-mendoza]

Useful links:

• Project Proposal

Generative Model

Stage 0: Setting things up

• single image per band
• use galsim for now
• add survey parameters we should use (background, psf size, etc.) #7
• add support for saving forward model images and truth catalog to hdf5 #37
• check that we can generate ~100k obejcts in a reasonable amount of time #26
• ensure noise and sky level is implemented correctly in images #21
• add more realistic distributions for generating synthetic catalogs #25
• download cosmo DC2 catalog and use them to sample galaxies #22
• create footprints with single uncentered galaxies #23
• sample cutouts with only noise in them #24
• compare/validate with descwl-shear-sims #6
• add code to generate bulge+disk galaxies #2
• add code to generate a simple PSF model (atmospheric) #3
• add code to shear galaxies and add gaussian noise to them #4
• add code to draw galaxies in a scene based on a given "catalog" #9
• generate catalog where galaxies have uniform spacing in the scene (blending off) #5
• add code to sample galaxy parameters and locations in a scene #8
• discuss whether we need to optimize the code #10

Stage 1: Multiband / redshifts/ variable shear

• single image per band
• multiband
• redshift + SEDs
• variable shear
• galaxy locations poisson process of 2D density field
• feed angular power spectrum to galsim/colore -> 2D gaussian density map
• bin galaxies by true redshift
• lensing magnifications

Stage2: Blending / Impact of PSF

• blending effects
• noisy PSF estimate
• use correlation functions in PSF

Inference

Stage 0

Goals:

• connect BLISS with JIF/BFD and recover constant shear
• start defining what what outputs look like: probabilistic magnitudes + shear

Tasks:

• train BLISS on forward model images #11
• validate BLISS can recover counts, locations, ... on images from forward model #12
• validate BLISS uncertainties on images from forward model #13
• brainstorm how to connect BLISS with JIF (inputs/outputs) #14
• brainstorm how to connect BLISS with BFD (inputs/outputs) #15

Stage 1

• BFD is already multi-band (can potentially photo-z)
• check that we recover power spectrum (?)
• assume we know true redshift
• connect with RAIL (?)
  • catalogs: some aspect of photometry + position
  • how to connect posterior samples of photometry
  • tables_io

Stage 2

• recovering binning (tomo challenge)
• run on DC2

[EiffL]

So, here I propose a slight modification of our baseline plan.

Maybe we can use the https://github.com/LSSTDESC/descwl-shear-sims package to act as our reference implementation for the forward model in stage 0, and stage 0 is adding the code to implement a/several forward model that we will use for inference, validated against that package.

[ismael-mendoza]

Maybe we can use the https://github.com/LSSTDESC/descwl-shear-sims package to act as our reference implementation for the forward model in stage 0

Thanks @EiffL for the suggestion. To make sure I understand - are you suggesting that for stage 0 we use the functions in descwl-shear-sims directly? Or we extract the relevant pieces of code from there (perhaps modified for our needs)?

stage 0 is adding the code to implement a/several forward model that we will use for inference, validated against that package.

What exactly will be validate here ? Our inference methods of shear or the forward model ?

[ismael-mendoza]

I also assigned some other pixel-level people in the group to this issue. @rearmstr @mdschneider @aguinot @aboucaud feel free to chime in too :)

EDIT: Please let me know if I missed anyone

[EiffL]

yeah, my point is that the descwl-shear-sims is already built around GalSim to emulate the DM images. And it can go from simple images, to all sorts of complex artifacts:
https://github.com/LSSTDESC/descwl-shear-sims/blob/master/tutorials/tutorial-basic.md#simple-simulation

[EiffL]

so we can use that as a reference forward model (instead of For now, use galsim in our current plan). We can probably adapt code from that repo at a later stage also.

But yeah, then we implement a matching forward model that would be able to reproduce the same images, and be used in our inference frameworks.

[ismael-mendoza]

That sounds good to me, so for stage 0 we use desc-shear-sims directly for our simulations and as our forward model. We can already try to put together our inference procedures and run them on these sims (we perhaps just choose the flags to not have too many complex artifacts for now).

In later stages we can implement our own forward model that matches this code and do inference on that.

[EiffL]

yeah I mean, my thinking was in stage 0 to have the ref. model provided by shear-sims, but we need a separate forwrad model with which to do the inference. Could be from bliss, jif, etc...

[ismael-mendoza]

oh ok, yeah that makes sense now. We still need to build our own forward model whose images can be designed to match the ones in shear-sims

[ismael-mendoza]

This list is a bit big and overwhelming. I am closing this in favor of a more focused to-do list for stage0 which I'm about to open. In the future, we can refer back to this list once stage 0 is completed.