Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] static JdbcConnectionPoolFactory occurs error when multi sources get Jdbc url #2449

Closed
2 tasks done
loserwang1024 opened this issue Aug 31, 2023 · 1 comment · Fixed by #2451
Closed
2 tasks done
Labels
bug Something isn't working

Comments

@loserwang1024
Copy link
Contributor

Search before asking

  • I searched in the issues and found nothing similar.

Flink version

1.17

Flink CDC version

2.4

Database and its version

no Database

Minimal reproduce step

Now JdbcConnectionPoolFactory is often use in JobMaster to reduce connection. In a Job, it's normal to set multiple source, For example, one source from mysql and another source from postgres, and then join them.
Howerver, now as a static variable, JdbcConnectionPoolFactory only have one instance in JobMaster. So only one implements of JdbcConnectionPoolFactory will make sense.The others database will use wrong JdbcConnectionPoolFactory to get jdbc url .
To verify this, only need a simple test:

  1. add getJdbcUrl to JdbcConnectionPools for test :
    @VisibleForTesting
    public String getJdbcUrl(JdbcSourceConfig sourceConfig) {
        return jdbcConnectionPoolFactory.getJdbcUrl(sourceConfig);
    }
  1. add a test unit:
public class JdbcConnectionPoolFactoryTest {

    @Test
    public void testMultiConnectionPoolFactory() {
        MockConnectionPoolFactory mockConnectionPoolFactory = new MockConnectionPoolFactory();
        MysqlPooledDataSourceFactory mysqlPooledDataSourceFactory = new MysqlPooledDataSourceFactory();
        JdbcConnectionPools mockInstance = JdbcConnectionPools.getInstance(mockConnectionPoolFactory);
        JdbcConnectionPools mysqlInstance = JdbcConnectionPools.getInstance(mysqlPooledDataSourceFactory);
        MySqlSourceConfig mySqlSourceConfig = new MySqlSourceConfig(
                StartupOptions.latest(),
                Arrays.asList("database"),
                Arrays.asList("table"),
                2,
                1,
                1.00,
                2.00,
                false,
                true,
                new Properties(),
                null,
                "com.mysql.cj.jdbc.Driver",
                "localhost",
                8080,
                "username",
                "password",
                20,
                "UTC",
                Duration.ofSeconds(10),
                2,
                3);
        Assert.assertEquals(mockInstance.getJdbcUrl(mySqlSourceConfig), mockConnectionPoolFactory.getJdbcUrl(mySqlSourceConfig));
        //todo: not equal here, because only one JDBCConnectionPoolFactory.
        Assert.assertNotEquals(mysqlInstance.getJdbcUrl(mySqlSourceConfig), mysqlPooledDataSourceFactory.getJdbcUrl(mySqlSourceConfig));
        Assert.assertEquals(mysqlInstance.getJdbcUrl(mySqlSourceConfig), mockConnectionPoolFactory.getJdbcUrl(mySqlSourceConfig));


    }

    static class MockConnectionPoolFactory extends JdbcConnectionPoolFactory{
        @Override
        public String getJdbcUrl(JdbcSourceConfig sourceConfig) {
            return "mock-url";
        }
    }
}

What did you expect to see?

Three assert return true.

What did you see instead?

The later two assert return false

Anything else?

No response

Are you willing to submit a PR?

  • I'm willing to submit a PR!
@loserwang1024 loserwang1024 added the bug Something isn't working label Aug 31, 2023
@loserwang1024 loserwang1024 changed the title [Bug] static JdbcConnectionPoolFactory occurs error when multi source get Jdbc url [Bug] static JdbcConnectionPoolFactory occurs error when multi sources get Jdbc url Aug 31, 2023
loserwang1024 added a commit to loserwang1024/flink-cdc-connectors that referenced this issue Sep 1, 2023
loserwang1024 added a commit to loserwang1024/flink-cdc-connectors that referenced this issue Sep 1, 2023
loserwang1024 added a commit to loserwang1024/flink-cdc-connectors that referenced this issue Sep 7, 2023
loserwang1024 added a commit to loserwang1024/flink-cdc-connectors that referenced this issue Sep 7, 2023
@leonardBang
Copy link
Contributor

👍🏻,pretty detailed analysis

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
2 participants