Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[INLONG-10703][Manager] Manager Add Oceanbase Support #10701

Merged
merged 8 commits into from
Jul 31, 2024

Conversation

xxsc0529
Copy link
Contributor

@xxsc0529 xxsc0529 commented Jul 23, 2024

Fixes #10703

Motivation

Added support for Oceanbase data sources to adapt to the Oceanbase ecosystem. close #10703

Modifications

It mainly adds oceanbase data source support and uses the oceanbase-client driver to connect.

It mainly adds oceanbase data source support and uses the oceanbase-client driver to connect. And write oceanbase's ExtractNode logic and LoadNode logic. The specific implementation logic is the same as mysql.Added support for oceanbase on the front-end page. Since the jdbc-inlong processing logic in the flink15 connection lacks the jdbc dialect, an error is reported. This is modified here.

Verifying this change

(Please pick either of the following options)

  • This change is a trivial rework/code cleanup without any test coverage.

  • This change is already covered by existing tests, such as:
    (please describe tests)
    Here's how the page test works
    image
    image
    image
    image

  • This change added tests and can be verified as follows:
    Please use the following test classes for testing org.apache.inlong.sort.parser.MySqlExtractNodeToOceanBaseLoadNodeTest
    (example:)

    • Added integration tests for end-to-end deployment with large payloads (10MB)
    • Extended integration test for recovery after broker failure

Documentation

  • Does this pull request introduce a new feature? (yes / no) yes
  • If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented) Data source types have been added to the README.md
  • If a feature is not applicable for documentation, explain why?
  • If a feature is not documented yet in this PR, please create a follow-up issue for adding the documentation

@xxsc0529 xxsc0529 changed the title [INLONG-10632][Manager] Manager Add Oceanbase Support [INLONG-10703][Manager] Manager Add Oceanbase Support Jul 23, 2024
EMsnap

This comment was marked as off-topic.

@xxsc0529
Copy link
Contributor Author

please add test result with real data from any source to oceanbase
What kind of test data is needed and where to put it

@dockerzhang
Copy link
Contributor

@xxsc0529 please fix the failed workflow and comments, thanks.

@xxsc0529
Copy link
Contributor Author

@xxsc0529 please fix the failed workflow and comments, thanks.

This codeQL doesn't work.,The same logic as mysql here.,I can't detect a lot of the original code here.,Do you have to solve this problem?,If you look at this diagram, the original code containing the DriverManager.getConnection logic did not pass codeQl
image

@dockerzhang
Copy link
Contributor

@xxsc0529 you could ignore the CodeQL fails, but it's better to click resolve if you fix the comments.

@xxsc0529
Copy link
Contributor Author

resolve

Resolved

@xxsc0529 xxsc0529 requested review from vernedeng and EMsnap July 31, 2024 03:18
@xxsc0529 xxsc0529 closed this Jul 31, 2024
@xxsc0529 xxsc0529 reopened this Jul 31, 2024
@aloyszhang aloyszhang merged commit bd4c3db into apache:master Jul 31, 2024
18 of 19 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Feature][Manager] Manager should support oceanbase
6 participants