stretch() with histeq=TRUE crashes R with large (18gb) SpatRaster

[evhersh]

Hello,

Thanks for the great package! I'm processing some large Sentinel-2 imagery and I've run into an issue trying to use histogram equalization for visualization. I found it worked great for a smaller raster (5gb), but I am trying it again on a much larger one (18gb) and it has been crashing R outright (R session aborted - R encountered a fatal error). I'm guessing this has something to do with memory limitations, but I've also tried it on my HPC cluster (using 1 node with ~275gb of memory) and it failed there as well with this error:

s2_eq <- stretch(s2, histeq=TRUE, filename=here("Sentinel-2", "Output", "Median_2022_RSA_eq.tif"))
double free or corruption (out)
Aborted

I've also tried messing with the terraOptions which has helped me deal with memory issues in the past to no avail:

terraOptions(tempdir = tname, progress=10, memfrac=0.1, memmax=186, steps=20, verbose=TRUE)

Linear stretching using stretch(histeq=FALSE) does not crash R, but doesn't give me the desired effect (the image is still quite dark). plotRGB(stretch="hist") does work however. I'm using the latest development version of terra (1.7.49).

I am still a bit of a newbie so I do not know how to create example data of a file this large with this many bands, but I can provide the attributes of the SpatRaster object:

> s2 <- rast(here("Sentinel-2", "Output", "Median_2022_RSA.tif"))
> show(s2)
class       : SpatRaster 
dimensions  : 23288, 28279, 7  (nrow, ncol, nlyr)
resolution  : 10, 10  (x, y)
extent      : 453040, 735830, 6238970, 6471850  (xmin, xmax, ymin, ymax)
coord. ref. : WGS 84 / UTM zone 9N (EPSG:32609) 
source      : Median_2022_RSA.tif 
names       :   B02,   B03,   B04,      B05,        B06,        B07, ... 
min values  :   615,   896,   889,   976.25,   873.9375,   879.8125, ... 
max values  : 18672, 17936, 17504, 17184.06, 16990.4375, 16963.7812, ... 

Please let me know if you have any tips!

Thanks,

Evan

[rhijmans]

Thanks, that was implemented without considering memory limitations. Should work better now.

[evhersh]

Hello,

Thank you for the fix, it is working now on my large file. However, it appears that the layer names are lost after stretching:

library(terra)
x1 <- rast(xmin=-110, xmax=-60, ymin=40, ymax=70, res=1, vals=1, names="B01")
x2 <- rast(xmin=-110, xmax=-60, ymin=40, ymax=70, res=1, vals=2, names="B02")
x  <- c(x1, x2)
x
class       : SpatRaster 
dimensions  : 30, 50, 2  (nrow, ncol, nlyr)
resolution  : 1, 1  (x, y)
extent      : -110, -60, 40, 70  (xmin, xmax, ymin, ymax)
coord. ref. : lon/lat WGS 84 (CRS84) (OGC:CRS84) 
source(s)   : memory
names       : B01, B02 
min values  :   1,   2 
max values  :   1,   2 

y <- stretch(x, histeq=TRUE)
y
class       : SpatRaster 
dimensions  : 30, 50, 2  (nrow, ncol, nlyr)
resolution  : 1, 1  (x, y)
extent      : -110, -60, 40, 70  (xmin, xmax, ymin, ymax)
coord. ref. : lon/lat WGS 84 (CRS84) (OGC:CRS84) 
source(s)   : memory
names       : lyr.1, lyr.1 
min values  :     1,     1 
max values  :     1,     1 

[rhijmans]

Thanks, I now get

names(y)
#[1] "B01" "B02"

[evhersh]

Works now, thanks for the quick fix and all of your work on this awesome package!