Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GeoRaster: Added GENSTATS options, security fixes, and prevent failing when password is near expiration #9290

Merged
merged 37 commits into from
Feb 29, 2024
Merged

GeoRaster: Added GENSTATS options, security fixes, and prevent failing when password is near expiration #9290

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
37 commits
Select commit Hold shift + click to select a range
fd40934
GeoRaster: Added generate statistics option for GeoRaster driver
Alan5142 Jul 14, 2023
ebbfc7b
Merge pull request #1 from Alan5142/georaster-statistics
Alan5142 Jul 14, 2023
88b14b7
Georaster: Set Generate statistics res length to 5
Alan5142 Jul 17, 2023
7e54cb3
Georaster: Changed generate statistics to use BindName for certain pa…
Alan5142 Jul 17, 2023
46c5ac2
Georaster: Added docs for statistics options
Alan5142 Jul 18, 2023
2ee5282
GeoRaster: Prevent injection in INSERT and DESCRIPTION options
Alan5142 Jul 18, 2023
87ea7a3
Statistics and general use fix
coim32 Jul 18, 2023
a824e77
Merge pull request #2 from coim32/patch-1
Alan5142 Jul 18, 2023
6cb773e
GeoRaster: Fixed insert statement check for '--' comments
Alan5142 Jul 18, 2023
6a189bd
GeoRaster: fixed out of bounds when checking for invalid INSERT expre…
Alan5142 Jul 19, 2023
666a14a
GeoRaster: Added georaster missing options
Alan5142 Jul 19, 2023
0992659
GeoRaster: Fixed insert expression validation, now considers '' as es…
Alan5142 Jul 20, 2023
cb968c1
Merge branch 'OSGeo:master' into master
Alan5142 Jul 25, 2023
f61d933
fix: CreationOptionList XML now properly closed
Alan5142 Jul 31, 2023
dde317f
Merge branch 'georaster-statistics' into insert-injection-georaster-s…
Alan5142 Oct 20, 2023
b699da7
description of all tables on a schema
coim32 Oct 20, 2023
01f8b69
Merge branch 'insert-injection' into insert-injection-georaster-stati…
Alan5142 Oct 23, 2023
ac20c7f
changed genstatistics option name to genstats
Alan5142 Oct 24, 2023
1159a7e
Merge branch 'OSGeo:master' into master
Alan5142 Oct 24, 2023
ccd72f8
Merge branch 'insert-injection-georaster-statistics'
Alan5142 Oct 24, 2023
f547c8f
fixed GENSTATS_USEBIN field
Alan5142 Oct 24, 2023
df499cc
doc changes
Alan5142 Oct 25, 2023
7525bff
fixed GenStatsHistogram value
Alan5142 Oct 25, 2023
7835622
changed some CPLSPrintf parameters to use Bind instead
Alan5142 Oct 27, 2023
a129234
format and docs fixes
Alan5142 Oct 27, 2023
060755e
fixed georaster_dataset duplicated default value
Alan5142 Feb 7, 2024
7ddfaff
log warning when password is about to expire
Alan5142 Feb 12, 2024
f6f9bef
updated georaster GENSTATS docs
Alan5142 Feb 13, 2024
0551c5b
added password expired warning log workaround to display it only once
Alan5142 Feb 19, 2024
b9aa45f
added workaround for gdal warning message when using gdal_translate
Alan5142 Feb 21, 2024
4d5ea68
added a default error handler as a workaround
Alan5142 Feb 22, 2024
589c5a8
merge upstream
Alan5142 Feb 22, 2024
eed7b54
code format
Alan5142 Feb 23, 2024
28d6c8f
fixed XML metadata spacing
Alan5142 Feb 26, 2024
ab12d50
fixed connection string in autotest/gdrivers/georaster.py
Alan5142 Feb 28, 2024
f0fa59a
format python code
Alan5142 Feb 28, 2024
7a3e52d
fix xml tag in genstats
Alan5142 Feb 29, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 4 additions & 27 deletions frmts/georaster/georaster_dataset.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1267,12 +1267,7 @@ GDALDataset *GeoRasterDataset::Create(const char *pszFilename, int nXSize,
poGRD->poGeoRaster->nPyramidLevels = atoi(pszFetched);
}

pszFetched = CSLFetchNameValue(papszOptions, "GENSTATS");

if (pszFetched != nullptr)
{
poGRD->poGeoRaster->bGenStats = EQUAL(pszFetched, "TRUE");
}
poGRD->poGeoRaster->bGenStats = CPLFetchBool(papszOptions, "GENSTATS", FALSE);

bool bGenStatsOptionsUsed = false;

Expand Down Expand Up @@ -1306,13 +1301,7 @@ GDALDataset *GeoRasterDataset::Create(const char *pszFilename, int nXSize,
poGRD->poGeoRaster->bGenStatsUseSamplingWindow = true;
}

pszFetched = CSLFetchNameValue(papszOptions, "GENSTATS_HISTOGRAM");

if (pszFetched != nullptr)
{
bGenStatsOptionsUsed = true;
poGRD->poGeoRaster->bGenStatsHistogram = EQUAL(pszFetched, "TRUE");
}
poGRD->poGeoRaster->bGenStatsHistogram = CPLFetchBool(papszOptions, "GENSTATS_HISTOGRAM", FALSE);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hum I realize my suggestion to use CPLFetchBool might cause issues here as bGenStatsOptionsUsed = true will no longer be set.

But I also realize that autotest/gdrivers/georaster.py hasn't been updated. It could make sense for long-term maintenance to test that new functionality


pszFetched = CSLFetchNameValue(papszOptions, "GENSTATS_LAYERNUMBERS");

Expand All @@ -1322,13 +1311,7 @@ GDALDataset *GeoRasterDataset::Create(const char *pszFilename, int nXSize,
poGRD->poGeoRaster->sGenStatsLayerNumbers = pszFetched;
}

pszFetched = CSLFetchNameValue(papszOptions, "GENSTATS_USEBIN");

if (pszFetched != nullptr)
{
bGenStatsOptionsUsed = true;
poGRD->poGeoRaster->bGenStatsUseBin = EQUAL(pszFetched, "TRUE");
}
poGRD->poGeoRaster->bGenStatsUseBin = CPLFetchBool(papszOptions, "GENSTATS_USEBIN", FALSE);

pszFetched = CSLFetchNameValue(papszOptions, "GENSTATS_BINFUNCTION");

Expand All @@ -1347,13 +1330,7 @@ GDALDataset *GeoRasterDataset::Create(const char *pszFilename, int nXSize,
}
}

pszFetched = CSLFetchNameValue(papszOptions, "GENSTATS_NODATA");

if (pszFetched != nullptr)
{
bGenStatsOptionsUsed = true;
poGRD->poGeoRaster->bGenStatsNodata = EQUAL(pszFetched, "TRUE");
}
poGRD->poGeoRaster->bGenStatsNodata = CPLFetchBool(papszOptions, "GENSTATS_NODATA", FALSE);

if (bGenStatsOptionsUsed && !poGRD->poGeoRaster->bGenStats)
{
Expand Down
4 changes: 1 addition & 3 deletions frmts/georaster/georaster_wrapper.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -615,11 +615,9 @@ static bool ValidateInsertExpression(const CPLString &sInsertStatement)
static bool ValidateDescriptionExpression(const CPLString &sInsertStatement)
{
const char *rgpszInvalidChars[] = {";", "--", "/*", "*/", "//"};
const size_t nInvalidCharsSize = 5;

for (size_t nPos = 0; nPos < nInvalidCharsSize; ++nPos)
for (const char *pszInvalidChar: rgpszInvalidChars)
{
const char *pszInvalidChar = rgpszInvalidChars[nPos];
if (sInsertStatement.find(pszInvalidChar) != std::string::npos)
{
return false;
Expand Down
18 changes: 9 additions & 9 deletions frmts/georaster/georasterdrivercore.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -100,30 +100,30 @@ void GEORDriverSetCommonMetadata(GDALDriver *poDriver)
" </Option>"
" <Option name='GENPYRLEVELS' type='int' description='Number of "
"pyramid level to generate'/>"
" <Option name='GENSTATS' type='boolean'"
" <Option name='GENSTATS' type='boolean' "
"description='Generate statistics from the given rasters'"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suspect missing spaces cause XML validation issues:

Suggested change
"description='Generate statistics from the given rasters'"
"description='Generate statistics from the given rasters' "

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're right, I might missed those when merging with the latest main changes, I'm going to make a commit with those fixes.

"default='FALSE' />"
" <Option name='GENSTATS_SAMPLINGFACTOR' type='int'"
" <Option name='GENSTATS_SAMPLINGFACTOR' type='int' "
"description='Number of cells skipped in both row and column "
"dimensions when "
"the statistics are computed'"
"the statistics are computed' "
"default='1' />"
" <Option name='GENSTATS_SAMPLINGWINDOW' type='string'"
" <Option name='GENSTATS_SAMPLINGWINDOW' type='string' "
"description='Coordinates (4 numbers) of a rectangular "
"window to be used to sample the raster when generating statistics' />"
" <Option name='GENSTATS_HISTOGRAM' type='boolean'"
" <Option name='GENSTATS_HISTOGRAM' type='boolean' "
"description='Compute a histogram for the raster' default='FALSE' />"
" <Option name='GENSTATS_LAYERNUMBERS' type='string'"
" <Option name='GENSTATS_LAYERNUMBERS' type='string' "
"description='Layer numbers and/or ranges for which to compute "
"the statistics' />"
" <Option name='GENSTATS_USEBIN' type='boolean'"
" <Option name='GENSTATS_USEBIN' type='boolean' "
"description='Specifies if the statistics should use the bin function "
"provided by GENSTATS_USEBIN "
"to compute the statistics' default='TRUE' />"
" <Option name='GENSTATS_BINFUNCTION' type='string'"
" <Option name='GENSTATS_BINFUNCTION' type='string' "
"description='Array to specify the bin function (type, total number of "
"bins, first bin number, minimum, cell value, maximum cell value)' />"
" <Option name='GENSTATS_NODATA' type='boolean'"
" <Option name='GENSTATS_NODATA' type='boolean' "
"description='Whether or not to compare each cell values "
"with NODATA values defined in the metadata'"
" default='FALSE' />"
Expand Down
Loading