Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[cdc-connector][db2] Db2 support incremental source #2870

Merged
merged 3 commits into from
Apr 24, 2024
Merged

[cdc-connector][db2] Db2 support incremental source #2870

merged 3 commits into from
Apr 24, 2024

Conversation

gong
Copy link
Contributor

@gong gong commented Dec 13, 2023 •
edited
Loading

Prepare a Pull Request

Motivation

Db2 support incremental source

Modifications

  • Add db2 incremental source code
  • Add doc
  • Add ITcase UT and e2e UT

@github-actions github-actions bot added docs Improvements or additions to documentation base e2e-tests db2-cdc-connector labels Dec 13, 2023
@gong
Copy link
Contributor Author

gong commented Dec 14, 2023

@leonardBang @GOODBOY008 PTAL

@gong
Copy link
Contributor Author

gong commented Dec 14, 2023 •
edited
Loading

@leonardBang test_1_ci_build core fail. because this PR #2845 cause error. I rollback this change.

@GOODBOY008 GOODBOY008 self-requested a review December 14, 2023 06:11
@gong gong force-pushed the feature-db2-paralle-read branch from 91b96f5 to 4eeb46e Compare December 14, 2023 10:15
@github-actions github-actions bot removed the runtime label Dec 14, 2023
@gong
Copy link
Contributor Author

gong commented Dec 15, 2023

random UT fail that is not relate with the db2

@gong gong force-pushed the feature-db2-paralle-read branch from 4eeb46e to e414129 Compare December 19, 2023 12:59
@gong gong force-pushed the feature-db2-paralle-read branch from e414129 to 44f8cc9 Compare December 28, 2023 02:47
@gong gong force-pushed the feature-db2-paralle-read branch from 44f8cc9 to 357a571 Compare January 5, 2024 02:33
@gong gong requested a review from lvyanquan January 5, 2024 02:34
lvyanquan
lvyanquan reviewed Jan 5, 2024
View reviewed changes
Copy link
Contributor

@lvyanquan lvyanquan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this great contribution, left some comments.

docs/content/connectors/db2-cdc.md Outdated Show resolved Hide resolved
...onnector-db2-cdc/src/main/java/com/ververica/cdc/connectors/db2/source/Db2SourceBuilder.java Outdated Show resolved Hide resolved
...-db2-cdc/src/main/java/com/ververica/cdc/connectors/db2/source/dialect/Db2ChunkSplitter.java Outdated Show resolved Hide resolved
...nector-db2-cdc/src/main/java/com/ververica/cdc/connectors/db2/source/utils/Db2TypeUtils.java Outdated Show resolved Hide resolved
@gong gong force-pushed the feature-db2-paralle-read branch from 357a571 to c728723 Compare January 6, 2024 14:51
@gong gong requested a review from lvyanquan January 6, 2024 14:54
lvyanquan
lvyanquan reviewed Jan 8, 2024
View reviewed changes
Copy link
Contributor

@lvyanquan lvyanquan left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall look good to me, left some comments.

...nnector-db2-cdc/src/test/java/com/ververica/cdc/connectors/db2/table/Db2ConnectorITCase.java Outdated Show resolved Hide resolved
...or-db2-cdc/src/main/java/com/ververica/cdc/connectors/db2/source/fetch/Db2ScanFetchTask.java Outdated Show resolved Hide resolved
...or-db2-cdc/src/main/java/com/ververica/cdc/connectors/db2/source/fetch/Db2ScanFetchTask.java Outdated Show resolved Hide resolved
docs/content/connectors/db2-cdc.md Outdated Show resolved Hide resolved
@gong gong requested a review from lvyanquan January 8, 2024 14:06
@lvyanquan
Copy link
Contributor

Hi @gong There is a failure in https://dev.azure.com/ververica/flink-cdc-connectors/_build/results?buildId=4654&view=logs&j=450e5386-ec4e-5c3c-8d86-ea427d28cb6f&t=b72b380c-c1b9-55cc-a168-123a50838bbf, could you please take a look?

@gong
Copy link
Contributor Author

gong commented Jan 9, 2024 •
edited
Loading

Hi @gong There is a failure in https://dev.azure.com/ververica/flink-cdc-connectors/_build/results?buildId=4654&view=logs&j=450e5386-ec4e-5c3c-8d86-ea427d28cb6f&t=b72b380c-c1b9-55cc-a168-123a50838bbf, could you please take a look?

@lvyanquan ok

lvyanquan
lvyanquan approved these changes Jan 10, 2024
View reviewed changes
Copy link
Contributor

@lvyanquan lvyanquan left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.
There is a mix of binlog redolog and wal in the code, it is best to unify them.

@gong
Copy link
Contributor Author

gong commented Jan 10, 2024 •
edited
Loading

LGTM. There is a mix of binlog redolog and wal in the code, it is best to unify them.

@lvyanquan ok

@gong
Copy link
Contributor Author

gong commented Jan 10, 2024

I will rebase code after #2978 is merged.

@gong gong force-pushed the feature-db2-paralle-read branch from 5d1cdd4 to f73a545 Compare January 10, 2024 14:45
@GOODBOY008 GOODBOY008 mentioned this pull request Jan 10, 2024
Closed
2 tasks
@gong gong force-pushed the feature-db2-paralle-read branch from f73a545 to 617be76 Compare January 14, 2024 04:50
@gong gong force-pushed the feature-db2-paralle-read branch from d28278c to a489e46 Compare January 19, 2024 06:59
@gong
Copy link
Contributor Author

gong commented Jan 19, 2024

@lvyanquan CI error is not related with the PR. UT can pass in local.

@gong gong force-pushed the feature-db2-paralle-read branch from a489e46 to db04a47 Compare March 10, 2024 16:38
@github-actions github-actions bot removed the base label Mar 10, 2024
@gong gong force-pushed the feature-db2-paralle-read branch 2 times, most recently from fba9516 to 93085e2 Compare March 18, 2024 04:09
@gong gong force-pushed the feature-db2-paralle-read branch from 93085e2 to 5ed9cf3 Compare March 27, 2024 12:56
@leonardBang
Copy link
Contributor

Thanks @gong for the great work, I'd like to have a final review base on @lvyanquan 's review work.

@leonardBang leonardBang requested review from leonardBang and removed request for GOODBOY008 April 3, 2024 09:52
leonardBang
leonardBang approved these changes Apr 23, 2024
View reviewed changes
Copy link
Contributor

@leonardBang leonardBang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @gong for the contribution and @lvyanquan for the review, generally looks good to me, I append a minor commit to improve the code style.

leonardBang pushed a commit to gong/flink-cdc-connectors that referenced this pull request Apr 23, 2024
[minor][cdc-connector][db2] Fix typos and improve the code style
6abf4d4 
This closes apache#2870
@leonardBang leonardBang force-pushed the feature-db2-paralle-read branch from 80add56 to 6abf4d4 Compare April 23, 2024 11:59
@github-actions github-actions bot added the base label Apr 23, 2024
leonardBang pushed a commit to gong/flink-cdc-connectors that referenced this pull request Apr 23, 2024
[minor][cdc-connector][db2] Fix typos and improve the code style
1c8209c 
This closes apache#2870
@leonardBang leonardBang force-pushed the feature-db2-paralle-read branch from 6abf4d4 to 1c8209c Compare April 23, 2024 14:11
@leonardBang leonardBang force-pushed the feature-db2-paralle-read branch from 1c8209c to 2cfb89d Compare April 24, 2024 11:54
@leonardBang
Copy link
Contributor

CI passed, merging...

@leonardBang leonardBang merged commit ddf557c into apache:master Apr 24, 2024
16 checks passed
wuzhenhua01 pushed a commit to wuzhenhua01/flink-cdc-connectors that referenced this pull request Aug 4, 2024
@wuzhenhua01
[tests][cdc-connector][db2] Improve the DB2 tests
250ed0c 
This closes apache#2870
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@lvyanquan lvyanquan lvyanquan approved these changes

@leonardBang leonardBang leonardBang approved these changes

Assignees
No one assigned
Labels
base db2-cdc-connector docs Improvements or additions to documentation e2e-tests mysql-cdc-connector
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[cdc-connector][db2] Db2 support incremental source
3 participants
@gong @lvyanquan @leonardBang