A footprint problem with JPEG compressed 1 band + alpha TIFF

[jratike80]

I have created successfully thousands of image footprint polygons with gdal_footprint but it does not work for one image. All images have been created in the same way that is close to this.
gdalwarp -of COG -t_srs epsg:3067 -co compress=jpeg -co quality=90 -cutline temp.json -crop_to_cutline input.tif output.tif

The source image that makes trouble has 1 band and alpha while the others have 3 bands + alpha.

The problematic image is this: http://latuviitta.org/downloads/%2323491076!_2.tif
Original data are in this package (.jpg + mask TIFF which are georeferenced and combined with a vrt) http://latuviitta.org/downloads/1band.zip

The difference to successful footprint extraction seems to be that gdal_footprint finds extraordinary many polygon fragments from this image. The process is not at all ready when the status reaches "done" but gdal_footprint continues to generate very heavy CPU load. Probably that would last forever but I did not wait so long. I believe that the alpha band is also JPEG compressed because it has nearblack mice bite artefacts like in this image round the boundaries.

0...10GDALRasterPolygonEnumerator: Counted 345222 polygon fragments forming 41724 final polygons.
...20...30...40...50...60...70...80...90...100 - done.

Gdalinfo reports 2 bands from the image

Band 1 Block=512x512 Type=Byte, ColorInterp=Gray
  Overviews: 3085x3085, 1542x1542, 771x771, 385x385
  Mask Flags: PER_DATASET ALPHA
  Overviews of mask band: 3085x3085, 1542x1542, 771x771, 385x385
Band 2 Block=512x512 Type=Byte, ColorInterp=Alpha
  Overviews: 3085x3085, 1542x1542, 771x771, 385x385

From the 3 bands + alpha gdalinfo reports only 3 bands

Band 1 Block=512x512 Type=Byte, ColorInterp=Red
  Overviews: 2953x2953, 1476x1476, 738x738, 369x369
  Mask Flags: PER_DATASET
  Overviews of mask band: 2953x2953, 1476x1476, 738x738, 369x369
Band 2 Block=512x512 Type=Byte, ColorInterp=Green
  Overviews: 2953x2953, 1476x1476, 738x738, 369x369
  Mask Flags: PER_DATASET
  Overviews of mask band: 2953x2953, 1476x1476, 738x738, 369x369
Band 3 Block=512x512 Type=Byte, ColorInterp=Blue
  Overviews: 2953x2953, 1476x1476, 738x738, 369x369
  Mask Flags: PER_DATASET
  Overviews of mask band: 2953x2953, 1476x1476, 738x738, 369x369

Could there be some issue in converting alpha band into binary mask with single band data and JPEG compression?

[rouault]

hum it turns out that gdal_footprint on input datasets with an alpha band was totally broken (and will still be in 3.8.1). Fixed per #8839
A workaround is to explicitly specify "-b 2" (or "-b 4" for RGBA images) to explictly force the vectorization of the alpha band only
And in #8838, I've modified the COG driver such that for a grey+alpha input, it produces when JPEG compression is asked a JPEG-compressed grey + a DEFLATE-compressed 1-bit alpha

[rouault]

was totally broken

I'm exaggerating a bit. Let's say quite broken, because in some situations like #8792, the fix of b28a386 happens to work when the non-alpha bands and the alpha-bands are 0 and non-0 in the same pixels...