Search Statistics Across Assets in Multiple Collections Fails

[bransonf]

I have two collections: ["opera-loss", "nlcd"] that I want to mosaic assets from and run some statistics via Titiler.

I register a search via POST /searches/register

{
        "collections": ["opera-loss", "nlcd"],
        "filter": {
            "op": "OR",
            "args": [
                {
                    "op": "AND",
                    "args": [
                        {
                            "op": "eq",
                            "args": [{"property": "collection"}, "opera-loss"]
                        },
                        {"op": "eq", "args": [{"property": "datetime"}, "2024-11-11"]}
                    ]
                },
                {
                    "op": "AND",
                    "args": [
                        {"op": "eq", "args": [{"property": "collection"}, "nlcd"]},
                        {"op": "eq", "args": [{"property": "datetime"}, "2021-01-01"]}
                    ]
                }
            ]
        }
    }

And validate the assets are found via GET /searches/<SEARCHID>/-98.5795,39.8283/assets:

[
  {
    "id": "-230400_1843200__2024-11-11",
    "bbox": [
      -98.73314824304305,
      39.580039124827444,
      -97.8050728162776,
      40.28159660136621
    ],
    "assets": {
      "opera_loss": {
        "href": "s3://***/opera_weekly/-230400_1843200/2024-11-11/-230400_1843200__2024-11-11.tif",
        "type": "image/tiff; application=geotiff; profile=cloud-optimized",
        "roles": [
          "data"
        ],
        "title": "opera_loss",
        "proj:epsg": 5070,
        "description": "",
        "raster:bands": [
          {
            "nodata": 255,
            "sampling": "area",
            "data_type": "int8",
            "spatial_resolution": 30
          }
        ]
      }
    },
    "collection": "opera-loss"
  },
  {
    "id": "-230400_1843200__2021",
    "bbox": [
      -98.73314824304305,
      39.580039124827444,
      -97.8050728162776,
      40.28159660136621
    ],
    "assets": {
      "nlcd": {
        "href": "s3://***/cogs/-230400_1843200/2021/nlcd.tif",
        "type": "image/tiff; application=geotiff; profile=cloud-optimized",
        "roles": [
          "data",
          "labels",
          "labels-raster"
        ],
        "title": "NLCD Land Cover",
        "proj:epsg": 5070,
        "description": "",
        "raster:bands": [
          {
            "nodata": 0,
            "sampling": "area",
            "data_type": "uint8",
            "spatial_resolution": 30
          }
        ]
      },      
     "collection": "nlcd"
  }
]

But when attempting to run POST /searches/<SEARCHID>/statistics with params:

assets=[ "opera_loss", "nlcd" ]
asset_as_band=true
expression="opera_loss + nlcd"

I get the error:

{
  "detail": "Invalid band/asset name 'nlcd'"
}

Changing to expression=opera_loss I get:

{
  "detail": "Could not find any valid assets in 'opera_loss' expression. Assets are: ['nlcd']"
}

And expression=nlcd I get:

{
  "detail": "Could not find any valid assets in 'nlcd' expression. Assets are: ['opera_loss']"
}

Is this a misguided use of the /searches endpoint, or should this sort of query across different collection-assets be possible?

I can work around this by adding the appropriate assets from the nlcd collection to the opera-loss collection or creating a new collection with the assets from both and registering a single-collection search + statistics, but it's not as elegant as being able to do cross-collection mosaicing.

PgSTAC version 0.8.5
Titiler version 0.1.0

Thanks for any insights here!

[hrodmn]

Hi @bransonf 👋! Unfortunately it is not possible to use an expression that combines assets from items in different collections.

titiler-pgstac attempts to create an array out of every STAC item in the search results, and it fails when it finds an item that does not have all of the requested assets.

In the case you have presented half of the items are missing nlcd and other half are missing opera_loss, so the query is doomed ☠️.

This is a feature that I would like to support someday, though! Keep an eye on #189 for further discussion.

[vincentsarago]

You might be able to do this be it will involve customization of the backend methods

titiler-pgstac/titiler/pgstac/mosaic.py

Lines 322 to 357 in d212969

	def tile(
	self,
	tile_x: int,
	tile_y: int,
	tile_z: int,
	scan_limit: Optional[int] = None,
	items_limit: Optional[int] = None,
	time_limit: Optional[int] = None,
	exitwhenfull: Optional[bool] = None,
	skipcovered: Optional[bool] = None,
	**kwargs: Any,
	) -> Tuple[ImageData, List[str]]:
	"""Get Tile from multiple observation."""
	mosaic_assets = self.assets_for_tile(
	tile_x,
	tile_y,
	tile_z,
	scan_limit=scan_limit,
	items_limit=items_limit,
	time_limit=time_limit,
	exitwhenfull=exitwhenfull,
	skipcovered=skipcovered,
	)
	if not mosaic_assets:
	raise NoAssetFoundError(
	f"No assets found for tile {tile_z}-{tile_x}-{tile_y}"
	)
	def _reader(
	item: Dict[str, Any], x: int, y: int, z: int, **kwargs: Any
	) -> ImageData:
	with self.reader(item, tms=self.tms, **self.reader_options) as src_dst:
	return src_dst.tile(x, y, z, **kwargs)
	return mosaic_reader(mosaic_assets, _reader, tile_x, tile_y, tile_z, **kwargs)

The idea would be to avoid passing the expression to the reader method https://github.com/stac-utils/titiler-pgstac/blob/main/titiler/pgstac/mosaic.py#L351-L355 but use them to return image for assets and then apply the expression on the result, It's not guarantee that it will work but that's at least a track to explore

[vincentsarago]

The core of the problem is that titiler-pgstac assume that collections have the same assets and that indexes or expressions are set of asset level processing.

Maybe Cross collection visualization is something users need/want but for now we never had paying customers needed development for this feature 🙈