Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[#3002] fix(jdbc-mysql): Fix MySQL list tables bugs. #3229

Closed
wants to merge 5 commits into from

Conversation

yuqi1129
Copy link
Contributor

What changes were proposed in this pull request?

Set the connection database to the database passed by the method when getting the JDBC connection.

Why are the changes needed?

connection.getSchema() is null for MySQL and thus will cause the problem that list operation will get all tables in the MySQL instance NOT a specific database.

Fix: #3002

Does this PR introduce any user-facing change?

N/A.

How was this patch tested?

New UT was added.

@yuqi1129 yuqi1129 added branch-0.5 need backport Issues that need to backport to another branch labels Apr 30, 2024
@yuqi1129 yuqi1129 self-assigned this Apr 30, 2024
@jerryshao
Copy link
Contributor

Do we have a similar problem in PG and Doris?

@jerryshao jerryshao requested review from FANNG1 and mchades April 30, 2024 04:04
@@ -415,7 +416,8 @@ protected JdbcColumn getJdbcColumnFromTable(JdbcTable jdbcTable, String colName)
"Column %s does not exist in table %s", colName, jdbcTable.name()));
}

protected Connection getConnection(String catalog) throws SQLException {
@VisibleForTesting
public Connection getConnection(String catalog) throws SQLException {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you verify whether the problem still exists if getConnection using setSchema(xx) not setCatalog(xx)?

    Connection getConnection(String catalog) {
       Connection connection = dataSource.getConnection();
       connection.setCatalog(catalog);
    }

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

noticed that getTables using connection.getSchema() to retrieve tables, the total JDBC logic is muddledness :(

  protected ResultSet getTables(Connection connection) throws SQLException {
    final DatabaseMetaData metaData = connection.getMetaData();
    String databaseName = connection.getSchema();
    return metaData.getTables(databaseName, databaseName, null, JdbcConnectorUtils.getTableTypes());
  }

Copy link
Contributor Author

@yuqi1129 yuqi1129 Apr 30, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

noticed that getTables using connection.getSchema() to retrieve tables, the total JDBC logic is muddledness :(

  protected ResultSet getTables(Connection connection) throws SQLException {
    final DatabaseMetaData metaData = connection.getMetaData();
    String databaseName = connection.getSchema();
    return metaData.getTables(databaseName, databaseName, null, JdbcConnectorUtils.getTableTypes());
  }

I have removed the line String databaseName = connection.getSchema();.

Root cause of this problem: String databaseName = connection.getSchema(); is always null.
please review it again.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you verify whether the problem still exists if getConnection using setSchema(xx) not setCatalog(xx)?

    Connection getConnection(String catalog) {
       Connection connection = dataSource.getConnection();
       connection.setCatalog(catalog);
    }

It's not work to use setSchema(xx);

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you verify whether the problem still exists if getConnection using setSchema(xx) not setCatalog(xx)?

    Connection getConnection(String catalog) {
       Connection connection = dataSource.getConnection();
       connection.setCatalog(catalog);
    }

It's not work to use setSchema(xx);

when you test do you remove String databaseName = connection.getSchema();

Copy link
Contributor Author

@yuqi1129 yuqi1129 May 6, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you verify whether the problem still exists if getConnection using setSchema(xx) not setCatalog(xx)?

    Connection getConnection(String catalog) {
       Connection connection = dataSource.getConnection();
       connection.setCatalog(catalog);
    }

It's not work to use setSchema(xx);

when you test do you remove String databaseName = connection.getSchema();

No, the method:

  protected ResultSet getTables(Connection connection) throws SQLException {
    final DatabaseMetaData metaData = connection.getMetaData();
    String databaseName = connection.getSchema();

    // We need the database name here.
    return metaData.getTables(databaseName, databaseName, null, JdbcConnectorUtils.getTableTypes());
  }

The value of the schema name must be passed as a method parameter, otherwise we cannot remove it.

@yuqi1129
Copy link
Contributor Author

yuqi1129 commented May 1, 2024

Do we have a similar problem in PG and Doris?

Yes, I have checked, the changes in the PR is in the module jdbc-common and will fix a similar issue in all JDBC catalogs.

@jerryshao
Copy link
Contributor

@yuqi1129 can you please move forward this PR?

@yuqi1129
Copy link
Contributor Author

yuqi1129 commented May 9, 2024

@yuqi1129 can you please move forward this PR?

Yeah, A related PR has just been merged, I will rebase it soon.

@yuqi1129
Copy link
Contributor Author

yuqi1129 commented May 10, 2024

@danhuawang
Could you kindly verify if the performance test has returned to normal since today? It appears that this issue has been resolved by #3316.

@yuqi1129
Copy link
Contributor Author

@danhuawang confirmed that the problem has been resolved by #3316 and the PR will be temporarily closed.

@yuqi1129 yuqi1129 closed this May 10, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
need backport Issues that need to backport to another branch
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Bug report] Possible dead lock in tree lock
4 participants