Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Patch DF to circumvent an analyzer rule #586

Merged
merged 1 commit into from
Aug 2, 2024
Merged

Conversation

gruuya
Copy link
Contributor

@gruuya gruuya commented Aug 2, 2024

Also disable all analyzer and optimizer rules, to avoid the stack overflow issue when applying a large number of sync calls during flushing.

Finally, make DataSyncConfig::max_syncs_per_url default to 100 now; it also works for 200 too but that incurs a performance penalty, so we should strive to get inter-sync dataframe squashing as a follow-up.

@gruuya gruuya requested a review from mildbyte August 2, 2024 08:23
@gruuya gruuya merged commit ccacfe0 into main Aug 2, 2024
1 check passed
@gruuya gruuya deleted the increase-sync-len-threshold branch August 2, 2024 09:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants