Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

sink(ticdc): support TTL table for cdc sink #7998

Merged
merged 21 commits into from
Feb 20, 2023
Merged

sink(ticdc): support TTL table for cdc sink #7998

merged 21 commits into from
Feb 20, 2023

Conversation

lcwangchao
Copy link
Contributor

@lcwangchao lcwangchao commented Dec 29, 2022 •
edited
Loading

What problem does this PR solve?

Issue Number: close #7997

What is changed and how it works?

The DDL statements with TTL options will be changed when syncing to donwstream

  • CREATE TABLE ... TTL=t + INTERVAL 1 DAY => CREATE TABLE ... TTL=t + INTERVAL 1 DAY TTL_ENABLE='OFF'
  • ALTER TABLE t TTL=t + INTERVAL 1 DAY => ALTER TABLE t TTL=t + INTERVAL 1 DAY TTL_ENABLE='OFF'
  • ALTER TABLE t TTL_ENABLE='ON' => ALTER TABLE t

TTL is a new experimental feature since v6.5.0, see: https://docs.pingcap.com/tidb/stable/time-to-live

We can add TTL option to a table to identify the rows in the table will be deleted when they are expired.

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
  • No code

Questions

Will it cause performance regression or break compatibility?
Do you need to update user documentation, design documentation or monitoring documentation?

Release note 

When syncing DDL with TTL options to downstream, the `TTL_ENABLE` will force be set to `OFF`

@ti-chi-bot
Copy link
Member

ti-chi-bot commented Dec 29, 2022 •
edited
Loading

[REVIEW NOTIFICATION]

This pull request has been approved by:

  • asddongmen
  • nongfushanquan

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

@ti-chi-bot ti-chi-bot added first-time-contributor Indicates that the PR was contributed by an external member and is a first-time contributor. release-note Denotes a PR that will be considered when it comes time to generate release notes. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Dec 29, 2022
@lcwangchao
Copy link
Contributor Author

/run-verify

@lcwangchao
Copy link
Contributor Author

/run-verify

nongfushanquan
nongfushanquan reviewed Dec 30, 2022
View reviewed changes
Copy link
Contributor

@nongfushanquan nongfushanquan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could you add more background information about the ttl table feature?

@ti-chi-bot ti-chi-bot added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Feb 8, 2023
@ti-chi-bot ti-chi-bot added the status/LGT1 Indicates that a PR has LGTM 1. label Feb 15, 2023
@ti-chi-bot ti-chi-bot added status/LGT2 Indicates that a PR has LGTM 2. and removed status/LGT1 Indicates that a PR has LGTM 1. labels Feb 16, 2023
@lcwangchao
Copy link
Contributor Author

/merge

@ti-chi-bot
Copy link
Member

@lcwangchao: /merge is only allowed for the committers, you can assign this pull request to the committer in list by filling /assign @committer in the comment to help merge this pull request.

In response to this:

/merge

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

@nongfushanquan
Copy link
Contributor

/run-all-tests

@lcwangchao
Copy link
Contributor Author

/run-all-tests

1 similar comment
@lcwangchao
Copy link
Contributor Author

/run-all-tests

@lcwangchao
Copy link
Contributor Author

/run-all-tests

@lcwangchao
Copy link
Contributor Author

/run-all-tests

@lcwangchao
Copy link
Contributor Author

/run-all-tests

@lcwangchao
Copy link
Contributor Author

/run-all-tests

1 similar comment
@lcwangchao
Copy link
Contributor Author

/run-all-tests

@lcwangchao
Copy link
Contributor Author

/run-integration-test

@lcwangchao
Copy link
Contributor Author

/test integration-test

@lcwangchao
Copy link
Contributor Author

/merge

@ti-chi-bot
Copy link
Member

@lcwangchao: /merge is only allowed for the committers, you can assign this pull request to the committer in list by filling /assign @committer in the comment to help merge this pull request.

In response to this:

/merge

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

@asddongmen
Copy link
Contributor

/merge

@ti-chi-bot
Copy link
Member

This pull request has been accepted and is ready to merge.

Commit hash: afbcd7c

@ti-chi-bot ti-chi-bot added the status/can-merge Indicates a PR has been approved by a committer. label Feb 20, 2023
@ti-chi-bot
Copy link
Member

@lcwangchao: Your PR was out of date, I have automatically updated it for you.

At the same time I will also trigger all tests for you:

/run-all-tests

trigger some heavy tests which will not run always when PR updated.

If the CI test fails, you just re-trigger the test that failed and the bot will merge the PR for you after the CI passes.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

@lcwangchao
Copy link
Contributor Author

/test kafka-integration-test

1 similar comment
@lcwangchao
Copy link
Contributor Author

/test kafka-integration-test

@lcwangchao
Copy link
Contributor Author

/run-all-tests

@lcwangchao
Copy link
Contributor Author

/test engine-integration-test

@lcwangchao
Copy link
Contributor Author

/retest

@lcwangchao
Copy link
Contributor Author

/run-engine-integration-test

@ti-chi-bot ti-chi-bot merged commit 3d5cbb4 into pingcap:master Feb 20, 2023
@lcwangchao lcwangchao deleted the ttl_support branch February 20, 2023 10:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@nongfushanquan nongfushanquan nongfushanquan approved these changes

@asddongmen asddongmen asddongmen approved these changes

Assignees
No one assigned
Labels
first-time-contributor Indicates that the PR was contributed by an external member and is a first-time contributor. release-note Denotes a PR that will be considered when it comes time to generate release notes. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. status/can-merge Indicates a PR has been approved by a committer. status/LGT2 Indicates that a PR has LGTM 2.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

CDC support TTL table
4 participants
@lcwangchao @ti-chi-bot @nongfushanquan @asddongmen