-
Notifications
You must be signed in to change notification settings - Fork 1.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Feature][PostgreSQL-jdbc] Supports GEOMETRY data type for PostgreSQL… #4673
[Feature][PostgreSQL-jdbc] Supports GEOMETRY data type for PostgreSQL… #4673
Conversation
Please reference https://github.com/apache/incubator-seatunnel/pull/4590/files and finished the document update. |
|
driver = "org.postgresql.Driver" | ||
user = "test" | ||
password = "test" | ||
query = "select * from spatial_data" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you test all supported field types?
A rigorous test can refer to the following information:
- The input and output of the test should include all supported data types.
- The input data should be deterministic and the values of each row and column can be verified in the output.
- In the output table, the values of each column in each row of data should be verified to ensure that there are no type conversion errors (precision loss, etc.) or missing values (a non null value in the input becomes null in the output) during the synchronization process.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think so, because currently pr supports geometric types, and if I wanted to test all types, I think I could open one and create a separate pr header such as Pg e2e to test it
@EricJoy2048
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
- When obtaining the GEOMETRY type directly through (AbstractJdbcRowConverter.toInternal), you can only get a string of numbers that are not understandable.
Therefore, it is necessary to introduce the PostGIS dependency, and then obtain the PGgeometry instance through the rs.getObject method and then call toString to get an understandable string.
<dependency>
<groupId>net.postgis</groupId>
<artifactId>postgis-jdbc</artifactId>
<version>2.5.0</version>
</dependency>
- Spatial data types not only include GEOMETRY but also GEOGRAPHY, which also needs to be supported.
Done |
@@ -0,0 +1,45 @@ | |||
# |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Remove this file
add geo field into this testcase
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Remove this file
add geo field into this testcase
Done
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
And the postgis dependency needs to be added to the seatunnel-dist pom.xml
switch (seaTunnelDataType.getSqlType()) { | ||
case STRING: | ||
if (metaDataColumnType.equals(PG_GEOMETRY)) { | ||
fields[fieldIndex] = ((PGgeometry) columnObj).getValue(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Suggest using the method rs.getObject(resultSetIndex).toString()
, because this way it won't throw an error when the postgis package is not imported (you will get a string of numbers), and you can also get the desired data when the postgis package is imported.
like
case STRING:
String columnTypeName = metaData.getColumnTypeName(resultSetIndex).toUpperCase(Locale.ROOT);
if (columnTypeName.equals("GEOGRAPHY") || columnTypeName.equals("GEOMETRY")) {
fields[fieldIndex] = rs.getObject(resultSetIndex) == null ? null : rs.getObject(resultSetIndex).toString();
break;
}
fields[fieldIndex] = rs.getString(resultSetIndex);
break;
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Rename JdbcPostgisIT.java to JdbcPostgresIT.java
@EricJoy2048 @ic4y help review thanks |
@EricJoy2048 @hailin0 Please check it out. Thank you |
Purpose of this pull request
issues:#4652
Check list
New License Guide
release-note
.