Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pyramid.fromLayerRDD fails to respect NODATA values with ResampleMethods.Max #3144

Closed
CloudNiner opened this issue Oct 30, 2019 · 2 comments · Fixed by #3155
Closed

Pyramid.fromLayerRDD fails to respect NODATA values with ResampleMethods.Max #3144

CloudNiner opened this issue Oct 30, 2019 · 2 comments · Fixed by #3155

Comments

@CloudNiner
Copy link
Contributor

I have a TileLayerRDD[SpatialKey] and am generating an image pyramid with the following code snippet:

def savePng(
    layer: TileLayerRDD[SpatialKey],
    colorMap: ColorMap,
    outputPath: String
  ): Unit = {
    implicit val sc: SparkContext = layer.sparkContext

    val (baseZoom, baseLayer) = layer.reproject(ZoomedLayoutScheme(WebMercator, 256))
    val pyramid = Pyramid.fromLayerRDD(
      baseLayer,
      Some(baseZoom),
      Some(0),
      resampleMethod = ResampleMethods.Max
    )

    pyramid.levels.foreach { case (zoom, tileRdd) =>
      val imageRdd: RDD[(SpatialKey, Array[Byte])] =
        tileRdd.mapValues(_.renderPng(colorMap).bytes)

      val keyToPath = { k: SpatialKey => s"${outputPath}/${zoom}/${k.col}/${k.row}.png" }
      if (outputPath.startsWith("s3")) {
        SaveToS3(imageRdd, keyToPath, { request =>
          request.toBuilder.acl(ObjectCannedACL.PUBLIC_READ).build()
        })
      } else {
        SaveToHadoop(imageRdd, keyToPath)
      }
    }
  }

When this layer is generated and displayed on a map, all zoom levels past the base zoom level replace nodata values with the value of the first color ramp:
https://tilejson.io/g/90e81c9fd2ed500609829d5df5af93dc/view

When I use the default ResampleMethod.Average this does not occur. It appears that MaxResample handles NoData values correctly

@CloudNiner
Copy link
Contributor Author

Here's two images at approximately the same location that demonstrate the issue (if the tilejson link ever disappears):

No data filled in as we zoom out

Screen Shot 2019-10-30 at 4 47 25 PM

Correct for basezoom

Screen Shot 2019-10-30 at 4 47 22 PM

@echeipesh
Copy link
Contributor

scala> val tile = ArrayTile.empty(DoubleConstantNoDataCellType, 4, 4)
tile: geotrellis.raster.MutableArrayTile = DoubleConstantNoDataArrayTile([D@708afe26,4,4)

scala> tile.asciiDrawDouble()
res13: String =
"    ND    ND    ND    ND
    ND    ND    ND    ND
    ND    ND    ND    ND
    ND    ND    ND    ND

"

scala> tile.resample(2, 2, ResampleMethods.Max).asciiDrawDouble()
res14: String =
" -2.147483648E9 -2.147483648E9
 -2.147483648E9 -2.147483648E9

"

Note, its important to use asciiDrawDouble because asciiDraw obscures the problem.

As @kervel pointed out on gitter:

    val doubleMax = indices.foldLeft(Double.MinValue) { case (currentMax, coords) =>
      val v = tile.getDouble(coords._1, coords._2)
      // Double.NaN would *always* be max
      if (isData(v)) math.max(currentMax, v) else currentMax
    }
    if (doubleMax == Double.MinValue) NODATA else doubleMax

here NODATA is the Int nodata... i think it should be doubleNODATA

echeipesh pushed a commit that referenced this issue Nov 16, 2019
…type. (#3155)

The current code set NODATA for nodata cells, but NODATA is only valid for Int cell types. Double.NaN or doubleNODATA should be used for double cells
See #3144

Signed-off-by: Frank Dekervel <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants