Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

DDL ignored by ddl job finishedTs less than puller resolvedTs #11714

Closed
sdojjy opened this issue Nov 5, 2024 · 0 comments · Fixed by #11733
Closed

DDL ignored by ddl job finishedTs less than puller resolvedTs #11714

sdojjy opened this issue Nov 5, 2024 · 0 comments · Fixed by #11733
Labels
affects-5.4 This bug affects the 5.4.x(LTS) versions. affects-6.1 This bug affects the 6.1.x(LTS) versions. affects-6.5 This bug affects the 6.5.x(LTS) versions. affects-7.1 This bug affects the 7.1.x(LTS) versions. affects-7.5 This bug affects the 7.5.x(LTS) versions. affects-8.1 This bug affects the 8.1.x(LTS) versions. affects-8.5 This bug affects the 8.5.x(LTS) versions. area/ticdc Issues or PRs related to TiCDC. found/gs severity/major type/bug The issue is confirmed as a bug.

Comments

@sdojjy
Copy link
Member

sdojjy commented Nov 5, 2024 •
edited
Loading

assume there are two ddl jobs
Job 60 : ALTER TABLE a ADD COLUMN x, y, z， it's commit ts is 400
Job 62: ALTER TABLE b ADD COLUMN y it's commit ts is 300

TiCDC sorts these jobs and run Job 62 first, then Job 60
DDL Puller processed Job 62 and set p.schemaVersion to 62,

tiflow/cdc/puller/ddl_puller.go

Line 483 in d728d02

p.schemaVersion = job.BinlogInfo.SchemaVersion

Job 60 will be ignored due to b.BinlogInfo.SchemaVersion <= p.schemaVersion 

ddl job finishedTs less than puller resolvedTs,discard the ddl job

tiflow/cdc/puller/ddl_puller.go

Line 321 in d728d02

job.BinlogInfo.SchemaVersion <= p.schemaVersion {

@sdojjy sdojjy added type/bug The issue is confirmed as a bug. area/ticdc Issues or PRs related to TiCDC. severity/major labels Nov 5, 2024
@sdojjy sdojjy added affects-6.5 This bug affects the 6.5.x(LTS) versions. affects-7.1 This bug affects the 7.1.x(LTS) versions. affects-7.5 This bug affects the 7.5.x(LTS) versions. affects-8.1 This bug affects the 8.1.x(LTS) versions. affects-8.5 This bug affects the 8.5.x(LTS) versions. and removed may-affects-5.4 may-affects-6.1 may-affects-6.5 may-affects-7.1 may-affects-7.5 may-affects-8.1 may-affects-8.5 labels Nov 5, 2024
@wlwilliamx wlwilliamx mentioned this issue Nov 8, 2024
Merged
@ti-chi-bot ti-chi-bot bot closed this as completed in b38183b Nov 15, 2024
This was referenced Nov 15, 2024
Merged
Merged
Merged
Merged
@ti-chi-bot ti-chi-bot mentioned this issue Nov 15, 2024
Merged
ti-chi-bot bot pushed a commit that referenced this issue Nov 18, 2024
@ti-chi-bot
ddl_puller.go(ticdc): fix DDLs are ignored when schema versions are o…
8ccf02c 
…ut of order (#11733) (#11755)

close #11714
ti-chi-bot bot pushed a commit that referenced this issue Nov 19, 2024
@ti-chi-bot
ddl_puller.go(ticdc): fix DDLs are ignored when schema versions are o…
3ef3c42 
…ut of order (#11733) (#11757)

close #11714
@ti-chi-bot ti-chi-bot bot added affects-6.1 This bug affects the 6.1.x(LTS) versions. affects-5.4 This bug affects the 5.4.x(LTS) versions. labels Nov 19, 2024
mjonss pushed a commit to mjonss/tiflow that referenced this issue Nov 25, 2024
@wlwilliamx @mjonss
ddl_puller.go(ticdc): fix DDLs are ignored when schema versions are o…
23b538c 
…ut of order (pingcap#11733)

close pingcap#11714
@hongyunyan hongyunyan mentioned this issue Nov 26, 2024
Merged
16 tasks
ti-chi-bot bot pushed a commit that referenced this issue Dec 3, 2024
@ti-chi-bot
ddl_puller.go(ticdc): fix DDLs are ignored when schema versions are o…
31845e3 
…ut of order (#11733) (#11758)

close #11714
ti-chi-bot bot pushed a commit that referenced this issue Dec 3, 2024
@ti-chi-bot
ddl_puller.go(ticdc): fix DDLs are ignored when schema versions are o…
46a22f7 
…ut of order (#11733) (#11756)

close #11714
ti-chi-bot bot pushed a commit that referenced this issue Dec 9, 2024
@ti-chi-bot
ddl_puller.go(ticdc): fix DDLs are ignored when schema versions are o…
b663704 
…ut of order (#11733) (#11759)

close #11714
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
affects-5.4 This bug affects the 5.4.x(LTS) versions. affects-6.1 This bug affects the 6.1.x(LTS) versions. affects-6.5 This bug affects the 6.5.x(LTS) versions. affects-7.1 This bug affects the 7.1.x(LTS) versions. affects-7.5 This bug affects the 7.5.x(LTS) versions. affects-8.1 This bug affects the 8.1.x(LTS) versions. affects-8.5 This bug affects the 8.5.x(LTS) versions. area/ticdc Issues or PRs related to TiCDC. found/gs severity/major type/bug The issue is confirmed as a bug.
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

ddl_puller.go(ticdc): fix DDLs are ignored when schema versions are out of order wlwilliamx/tiflow
2 participants
@kennytm @sdojjy