jpeg2000 files are loaded not completely

[Alexandr-Lysikov]

I have problem with 2 jpeg2000 files.
In the 2.3, 2.4 and 3.0 versions with the extension gdal_ECW_JP2ECW.dll, both files are opened but file1.jp2 loaded completely, and the file2.jp2 not.
Without extension gdal_ECW_JP2ECW.dll both files are opened and on the contrary file1.jp2 loaded completely, and the file2.jp2 not.
screenshot

In 2.2.3 version both files are opened and loaded completely without extension gdal_ECW_JP2ECW.dll

I expected that both files will be loaded completely in new versions like in 2.2.3 version.
file1.jp2
file2.jp2

gdalinfo for files
file1_with_plugins.info
file2_with_plugins.info
file1_without_plugins.info
file2_without_plugins.info

Operating system Windows 10

[jratike80]

File1.jp2 is created with Kakadu 5.1
File2.jp2 id created with ECW JPEG 2000 3.3.0

Kakadu viever opens both images but says that file2 is not valid:

Profile violation detected (code-stream is technically illegal).

Gdal_translate does well with file1 but not with file2 with OpenJPEG driver.
Test with the OpenJPEG driver and GDAL 3.1dev gives these errors

gdal_translate -of GTiff -co tiled=yes -co compress=jpeg file2.jp2 file2.tif
Input file size is 5091, 6961
0ERROR 1: psImage->comps[0].data == nullptr
ERROR 1: psImage->comps[0].data == nullptr
ERROR 5: file2.jp2, band 1: Illegal nBlockXOff value (1464159776) in GDALRasterBand::TryGetLockedBlockRef()
ERROR 1: psImage->comps[0].data == nullptr
ERROR 5: file2.jp2, band 1: Illegal nBlockXOff value (1464159776) in GDALRasterBand::GetLockedBlockRef()
ERROR 5: file2.jp2, band 1: Illegal nBlockXOff value (-109024723) in GDALRasterBand::TryGetLockedBlockRef()
ERROR 5: file2.jp2, band 1: Illegal nBlockXOff value (-109024723) in GDALRasterBand::GetLockedBlockRef()
ERROR 1: psImage->comps[0].data == nullptr
ERROR 5: file2.jp2, band 1: Illegal nBlockXOff value (6029358) in GDALRasterBand::TryGetLockedBlockRef()
ERROR 5: file2.jp2, band 1: Illegal nBlockXOff value (6029358) in GDALRasterBand::GetLockedBlockRef()
ERROR 5: file2.jp2, band 1: Illegal nBlockXOff value (1311353664) in GDALRasterBand::TryGetLockedBlockRef()
ERROR 5: file2.jp2, band 1: Illegal nBlockXOff value (1311353664) in GDALRasterBand::GetLockedBlockRef()
ERROR 1: psImage->comps[0].data == nullptr
ERROR 1: file2.jp2, band 2: IReadBlock failed at X offset 0, Y offset 0: psImage->comps[0].data == nullptr

ECW JPEG 2000 SDK 3.x is known to be buggy and it would be better stop using it. Your plugin seems to be compiled with SDK 3.x. There was a reason to that version earlier days because when it works it is pretty fast, and it can also write JPEG 2000. Nowadays OpenJPEG can handle both cases so you can just start using that.

What remains is how to handle the technically illegal file. ECWJP2 driver that is using ECW SDK 5.3 can read and convert data, as well as JP2MrSID driver (tested with GDAL from OSGeo4W).

The real bug is probably in ECW SDK 3 but that will never get fixed and now you have images created with that version (stop creating more of such with your 3.x based GDAL plugin!). Next place to fix is OpenJPEG2000 library that could be made to accept illegal images if developers consider it worth the trouble. It is also possible that JPEG2000 has some other issue than the one with profile that was reported by Kakadu.

[antgraf]

@jratike80 I believe it is from Vermont Geodata Portal:
http://maps.vcgi.vermont.gov/opendata/tileselect_opendata.html?IndexLayerName=Index_NAIP_0_6M_CLRIR_2016&FolderURL=http://maps.vcgi.vermont.gov/gisdata/vcgi/imagery/NAIP/0_6M/CLRIR/2016/COMP/&cdownload=-1&InputLayerName=IMG_VCGI_NAIP2016_SP&InputFtype=imagery

[rouault]

This is a bug of openjpeg >= 2.2.0 and <= 2.3.0 (likely uclouvain/openjpeg#1120). I've verified that file2.jp2 is read correctly by the GDAL JP2OpenJPEG driver with openjpeg 2.3.1

[Alexandr-Lysikov]

Oh, thanks, everything works, for some reason the new version of openjp2.dll is not included in GDAL