Define HLS layers for environmental justice story

[abarciauskas-bgse]

@alexandra mentioned that HLS data is more complex than our current model for layers since it has many band layers and there is no "default mosaic" that makes sense.

For the first release we should define a mosaic for the environmental justice story which is a mosaic produced for the spatial extent, temporal datetimes and band combination that makes sense for telling the story of the flooding (or other land cover changes?) during the Hurricanes Ida and Maria

Brian suggested the true color composite or the SWIR false color composite for FIRMS

From slack:

Brian - when you shared the sentinel scene for flooding during Hurricane Ida was that an image from an existing S30 band or was it some band combination

Brian: the one i showed in WV was a true color composite. I think Sean is also working on a SWIR false color composite for FIRMS that would be useful as well

Brian: a true color composite takes in data from bands 4,3, and 2 into a single layer (for both L30 and S30). The SWIR false color composite is a 3-band combination that is 12, 8A, 4 for S30 and 7,5,4 for L30

I was imagining we would just be ingesting S30 for the first version of the dashboard but this makes me think we might also need L30.

[sharkinsspatial]

@abarciauskas-bgse A note on the SWIR data. @freitagb had done some experimentation with various band stretches and provided some sample code for logarithmic band stretches. @vincentsarago helped me include a dataset dependency to apply an analogue of this stretch as a post processing function. You can see the code here NASA-IMPACT/eoAPI@8457367

[sharkinsspatial]

Though I believe @freitagb may have experimented with a different stretch approach for visualizing flood extents.

[abarciauskas-bgse]

I was reviewing developmentseed/eoAPI#22 (comment) about creating a new STAC collection to represent a mosaic built from another collection with presumably many items. @anayeaye you mentioned that HLS collection and items fall outside the current paradigm for the dashboard, I think because of the number of bands and requirements for a band formula for the front end display.

Do you think something like a "mosaic" collection for HLS could work? It might require some adaptation on the front end configuration, but I need to dig in more to how that is going to work to say exactly how.

Ultimately I think we need to display a mosaics of HLS true color composite and a SWIR false color for the place and time before, during and after Hurricanes Ida and Maria @freitagb do you agree?

cc @sharkinsspatial @vincentsarago

[anayeaye]

Aside: I'm linking an that includes a related UI discussion about how to manage datasets that require band combinations to generate meaningful map layers: delta-ui/issues/45.

[freitagb]

@abarciauskas-bgse - here are some helpful links for the band combinations for the Landsat and Sentinel components of the HLS products. Note that the band combinations outside of the RGB bands are different for Landsat vs. Sentinel-2.

Landsat 8/9 band combinations - https://www.l3harrisgeospatial.com/Learn/Blogs/Blog-Details/ArtMID/10198/ArticleID/15691/The-Many-Band-Combinations-of-Landsat-8

Sentinel-2A/2B band combinations - https://gisgeography.com/sentinel-2-bands-combinations/

The bands listed in the examples linked above correspond to the HLSL30 (Landsat) and HLSS30 (Sentinel-2) band number in the file name. The one we are using for FIRMS is the SWIR false color composite. It's confusing, but the SWIR FCC for Landsat 8 references bands 7,5,2 and sentinel-2 references 12,8A,4. We follow the Sentinel-2 definition which corresponds to bands 7,5,4 in Landsat-8. A band mapping between the two sensors can be found at the link here:

https://www.usgs.gov/centers/eros/science/usgs-eros-archive-sentinel-2-comparison-sentinel-2-and-landsat#overview

[sharkinsspatial]

@abarciauskas-bgse I included some related comments over here. I think it is may be a good idea to clarify the semantics/definition of a mosaic with respect to pg-stac and its internal implementation. Currently, mosaic registration uses a query hash mechanism to store the query body so that subsequent GET requests to the tile endpoint can use the hash of the query and avoid issues associated with serializing complex query bodies in the url parameters. The issue you linked has some good discussion about the pros and cons of potentially differentiating mosaic queries from normal ones.

After some discussions and recommendations from Vincent and David, I'm using the following approach for our HLS eoAPI implementation with the DOI. The first step for the client to use a mosaic tile layer will always be a new call to mosaic/register with the relevant query body (note, subsequent calls with the same query body will return the same hash key). The returned tilejson endpoint is used by the client application to configure the application's tile layer.

The big caveat (as mentioned in here) is that if you want to use an asset based expression with this mosaic tile endpoint, all of the resulting items will need to have a consistent asset structure. In my case, this means I can't have a mosaic query which mixes S30 and L30 collections since they have different asset structures and therefore need different expressions.

[abarciauskas-bgse]

Thanks @sharkinsspatial, that's helpful.

I think our implementation for HLS layers in the dashboard is still undecided, but I wanted to note that @freitagb shared this code https://gist.github.com/abarciauskas-bgse/ac7770219a5c6363f25c6ec20cd28c04 for generating visual indices which require data from both S30 and L30. We may want to inject those functions into our titiler, but that may warrant some new collection definition that can include items and assets from both L30 and S30.

[sharkinsspatial]

@abarciauskas-bgse I think there might be a misunderstanding of @freitagb 's code. The goal of HLS is for us to process input Landsat and Sentinel 2 data so that their bands appear as if they are identical and all coming from the same sensor. We maintain the original sensor's band index for the output products as this is what most users are accustomed to. You can read more here https://lpdaac.usgs.gov/data/get-started-data/collection-overview/missions/harmonized-landsat-sentinel-2-hls-overview/#introduction. So in the example indices provided by @freitagb the S30 key indicates the names of the S30 bands used for that index and the L30 key indicates the names of L30 bands used for that index. In this example S30 band 8A and L30 band 05 are nearly spectrally identical. We are never combining S30 and L30 products (in reality, there are only some orbital swaths where Landsat and Sentinel2 data are collected for the same location on the same day).

As a more concrete example of how this will works for our FIRMS project, they require a SWIR index. We use a custom post processing function (this is only necessary for better visual experience with the index) and then the resulting tiler requests look like this

For S30
https://tiler/mosaic/tiles/tileid/WGS1984Quad/12/4677/1330@1x?assets=B12&assets=B8A&assets=B04&post_process=swir

For L30
https://tiler/mosaic/tiles/tileid/WGS1984Quad/12/4677/1330@1x?assets=B07&assets=B05&assets=B04&post_process=swir

For a comprehensive listing of different indexes for both the Sentinel 2 and Landsat sensors I've found this one https://giscrack.com/list-of-spectral-indices-for-sentinel-and-landsat/ helpful.

So in the case of HLS we are always dealing with HLSS30 and HLSL30 as separate collections.

[abarciauskas-bgse]

I see thanks for the clarification @sharkinsspatial I updated my comment above to strikethrough the need for data from both collections.

[gadomski]

Closing as done on the board.