expression issues

[iferencik]

I'd like to point out some aspects related to parsing expressions.

I am trying to compute/select all pixels in a tile that have values larger/smaller than a certain threshold.
The range of my data is 0-1 and the type is float32

for the beginning I am displaying a tile using the viridis colormap

https://undp.livedata.link/hrea/tiles/16/39109/32807.png?url=https%3A%2F%2Fundpngddlsgeohubdev01.blob.core.windows.net%2Ftest%2FKenya_set_lightscore.vrt&bidx=1&rescale=0,1&colormap_name=viridis

this works well,

Now 'd like to filter the pixels with a value smaller than 0.5. In the above picture these are the bottom right ones. I remove the rescale and colormap_name parameters to make sure the values in the tile are not rescaled.

https://undp.livedata.link/hrea/tiles/16/39109/32807.png?url=https%3A%2F%2Fundpngddlsgeohubdev01.blob.core.windows.net%2Ftest%2FKenya_set_lightscore.vrt&bidx=1&expression=b1%3C0.5

which results in

Exception in ASGI application
Traceback (most recent call last):
  File "/home/ioan/.local/share/virtualenvs/titiler-ylkiCUEV/lib/python3.8/site-packages/uvicorn/protocols/http/h11_impl.py", line 377, in run_asgi
    result = await app(self.scope, self.receive, self.send)
  File "/home/ioan/.local/share/virtualenvs/titiler-ylkiCUEV/lib/python3.8/site-packages/uvicorn/middleware/proxy_headers.py", line 75, in __call__
    return await self.app(scope, receive, send)
  File "/home/ioan/.local/share/virtualenvs/titiler-ylkiCUEV/lib/python3.8/site-packages/fastapi/applications.py", line 212, in __call__
    await super().__call__(scope, receive, send)
  File "/home/ioan/.local/share/virtualenvs/titiler-ylkiCUEV/lib/python3.8/site-packages/starlette/applications.py", line 112, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/home/ioan/.local/share/virtualenvs/titiler-ylkiCUEV/lib/python3.8/site-packages/starlette/middleware/errors.py", line 181, in __call__
    raise exc
  File "/home/ioan/.local/share/virtualenvs/titiler-ylkiCUEV/lib/python3.8/site-packages/starlette/middleware/errors.py", line 159, in __call__
    await self.app(scope, receive, _send)
  File "/home/ioan/.local/share/virtualenvs/titiler-ylkiCUEV/lib/python3.8/site-packages/starlette/exceptions.py", line 82, in __call__
    raise exc
  File "/home/ioan/.local/share/virtualenvs/titiler-ylkiCUEV/lib/python3.8/site-packages/starlette/exceptions.py", line 71, in __call__
    await self.app(scope, receive, sender)
  File "/home/ioan/.local/share/virtualenvs/titiler-ylkiCUEV/lib/python3.8/site-packages/starlette/routing.py", line 656, in __call__
    await route.handle(scope, receive, send)
  File "/home/ioan/.local/share/virtualenvs/titiler-ylkiCUEV/lib/python3.8/site-packages/starlette/routing.py", line 259, in handle
    await self.app(scope, receive, send)
  File "/home/ioan/.local/share/virtualenvs/titiler-ylkiCUEV/lib/python3.8/site-packages/starlette/routing.py", line 61, in app
    response = await func(request)
  File "/home/ioan/.local/share/virtualenvs/titiler-ylkiCUEV/lib/python3.8/site-packages/fastapi/routing.py", line 226, in app
    raw_response = await run_endpoint_function(
  File "/home/ioan/.local/share/virtualenvs/titiler-ylkiCUEV/lib/python3.8/site-packages/fastapi/routing.py", line 161, in run_endpoint_function
    return await run_in_threadpool(dependant.call, **values)
  File "/home/ioan/.local/share/virtualenvs/titiler-ylkiCUEV/lib/python3.8/site-packages/starlette/concurrency.py", line 39, in run_in_threadpool
    return await anyio.to_thread.run_sync(func, *args)
  File "/home/ioan/.local/share/virtualenvs/titiler-ylkiCUEV/lib/python3.8/site-packages/anyio/to_thread.py", line 28, in run_sync
    return await get_asynclib().run_sync_in_worker_thread(func, *args, cancellable=cancellable,
  File "/home/ioan/.local/share/virtualenvs/titiler-ylkiCUEV/lib/python3.8/site-packages/anyio/_backends/_asyncio.py", line 818, in run_sync_in_worker_thread
    return await future
  File "/home/ioan/.local/share/virtualenvs/titiler-ylkiCUEV/lib/python3.8/site-packages/anyio/_backends/_asyncio.py", line 754, in run
    result = context.run(func, *args)
  File "/home/ioan/.local/share/virtualenvs/titiler-ylkiCUEV/lib/python3.8/site-packages/titiler/core/factory.py", line 497, in tile
    content = image.render(
  File "/home/ioan/.local/share/virtualenvs/titiler-ylkiCUEV/lib/python3.8/site-packages/rio_tiler/models.py", line 333, in render
    datatype_range = (dtype_ranges[str(data.dtype)],)
KeyError: 'bool'

The above expression produced a bool array and rio-tiler is not happy about it.

I tried something like this

where((b1==1) | (b1 > 0.5),1,0)

in numexpr and this produced an int32 array.
However rio-tiler could not parse this string

File "/home/ioan/.local/share/virtualenvs/titiler-ylkiCUEV/lib/python3.8/site-packages/rio_tiler/expression.py", line 52, in <listcomp>
    numexpr.evaluate(bloc.strip(), local_dict=dict(zip(bands, data)))
  File "/home/ioan/.local/share/virtualenvs/titiler-ylkiCUEV/lib/python3.8/site-packages/numexpr/necompiler.py", line 809, in evaluate
    _names_cache[expr_key] = getExprNames(ex, context)
  File "/home/ioan/.local/share/virtualenvs/titiler-ylkiCUEV/lib/python3.8/site-packages/numexpr/necompiler.py", line 696, in getExprNames
    ex = stringToExpression(text, {}, context)
  File "/home/ioan/.local/share/virtualenvs/titiler-ylkiCUEV/lib/python3.8/site-packages/numexpr/necompiler.py", line 274, in stringToExpression
    c = compile(s, '<expr>', 'eval', flags)
  File "<expr>", line 1
    where((b1==1) | (b1 > 0.5)

Any help is appreciated

[vincentsarago]

🤦

rio-tiler/rio_tiler/io/cogeo.py

Line 420 in e3e147d

blocks = expression.lower().split(",")

Your expression where((b1==1) | (b1 > 0.5),1,0) is a valid numexpr expression but rio-tiler will split it 🤦