Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

table_pipeline: Fix blocking entire pipeline when DDL puller is lagging behind #2078

Merged

Conversation

liuzix
Copy link
Contributor

@liuzix liuzix commented Jun 17, 2021

What problem does this PR solve?

  • If DDL puller is behind, mounter could block the entire tablePipeline, causing deadlocks together with TiKV concurrency control or other components.

What is changed and how it works?

  • Events waiting to be mounted are buffered in a Deque, which has no explicit size limit (the flowController ensures there will be no OOM).

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (WIP)

Side effects

  • Possible performance regression
  • Increased code complexity

Related changes

  • Need to cherry-pick to the release branch

Release note

  • No release note

@ti-chi-bot ti-chi-bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jun 17, 2021
@ti-chi-bot ti-chi-bot requested review from zier-one and lonng June 17, 2021 04:39
@liuzix
Copy link
Contributor Author

liuzix commented Jun 17, 2021

/run-integration-tests

@ti-chi-bot ti-chi-bot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Jun 17, 2021
@liuzix
Copy link
Contributor Author

liuzix commented Jun 17, 2021

/run-integration-tests

@codecov-commenter
Copy link

codecov-commenter commented Jun 17, 2021

Codecov Report

Merging #2078 (4406554) into master (0138edc) will increase coverage by 16.2256%.
The diff coverage is 60.8783%.

@@                Coverage Diff                @@
##             master      #2078         +/-   ##
=================================================
+ Coverage   37.8006%   54.0262%   +16.2256%     
=================================================
  Files           110        164         +54     
  Lines         11394      17560       +6166     
=================================================
+ Hits           4307       9487       +5180     
- Misses         6662       7064        +402     
- Partials        425       1009        +584     

@liuzix
Copy link
Contributor Author

liuzix commented Jun 17, 2021

/run-kafka-integration-tests

@liuzix
Copy link
Contributor Author

liuzix commented Jun 17, 2021

/run-kafka-integration-test

@liuzix liuzix marked this pull request as ready for review June 17, 2021 05:24
@ti-chi-bot ti-chi-bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jun 17, 2021
@liuzix liuzix added status/ptal Could you please take a look? type/bugfix This PR fixes a bug. severity/critical component/replica-model Replication model component. needs-cherry-pick-release-5.1 Should cherry pick this PR to release-5.1 branch. needs-cherry-pick-release-5.0 Should cherry pick this PR to release-5.0 branch. needs-cherry-pick-release-4.0 Should cherry pick this PR to release-4.0 branch. labels Jun 17, 2021
@liuzix
Copy link
Contributor Author

liuzix commented Jun 17, 2021

/run-all-tests

@liuzix liuzix requested review from amyangfei and overvenus June 17, 2021 05:45
@amyangfei amyangfei added the release-blocker This issue blocks a release. Please solve it ASAP. label Jun 17, 2021
@liuzix
Copy link
Contributor Author

liuzix commented Jun 17, 2021

/run-integration-tests

liuzix added 2 commits June 18, 2021 11:33

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
@liuzix
Copy link
Contributor Author

liuzix commented Jun 18, 2021

/run-all-tests

@ti-chi-bot
Copy link
Member

[REVIEW NOTIFICATION]

This pull request has been approved by:

  • amyangfei
  • leoppro

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

@ti-chi-bot ti-chi-bot added status/LGT2 Indicates that a PR has LGTM 2. and removed status/LGT1 Indicates that a PR has LGTM 1. labels Jun 18, 2021
@amyangfei
Copy link
Contributor

/merge

@ti-chi-bot
Copy link
Member

This pull request has been accepted and is ready to merge.

Commit hash: 4406554

@ti-chi-bot ti-chi-bot added the status/can-merge Indicates a PR has been approved by a committer. label Jun 18, 2021
@amyangfei amyangfei added this to the v5.1.0 milestone Jun 18, 2021
@ti-chi-bot ti-chi-bot merged commit 9eabc17 into pingcap:master Jun 18, 2021
ti-chi-bot pushed a commit to ti-chi-bot/tiflow that referenced this pull request Jun 18, 2021
@ti-chi-bot
Copy link
Member

In response to a cherrypick label: new pull request created: #2091.

ti-chi-bot pushed a commit to ti-chi-bot/tiflow that referenced this pull request Jun 18, 2021
@ti-chi-bot
Copy link
Member

In response to a cherrypick label: new pull request created: #2092.

@ti-chi-bot
Copy link
Member

In response to a cherrypick label: new pull request created: #2093.

ti-chi-bot added a commit that referenced this pull request Jun 18, 2021

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
…ng behind (#2078) (#2093)
liuzix added a commit to liuzix/ticdc that referenced this pull request Jun 24, 2021
liuzix added a commit to ti-chi-bot/tiflow that referenced this pull request Aug 11, 2021
liuzix added a commit to ti-chi-bot/tiflow that referenced this pull request Aug 11, 2021
ti-chi-bot added a commit that referenced this pull request Aug 11, 2021

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
…ng behind (#2078) (#2091)
liuzix added a commit that referenced this pull request Sep 2, 2021
overvenus added a commit to overvenus/ticdc that referenced this pull request Sep 7, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component/replica-model Replication model component. needs-cherry-pick-release-4.0 Should cherry pick this PR to release-4.0 branch. needs-cherry-pick-release-5.0 Should cherry pick this PR to release-5.0 branch. needs-cherry-pick-release-5.1 Should cherry pick this PR to release-5.1 branch. release-blocker This issue blocks a release. Please solve it ASAP. severity/critical size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. status/can-merge Indicates a PR has been approved by a committer. status/LGT2 Indicates that a PR has LGTM 2. status/ptal Could you please take a look? type/bugfix This PR fixes a bug.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants