Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Configuration Data model for Incremental in AirbyteCatalog #988

Closed
cgardens opened this issue Nov 16, 2020 · 1 comment
Closed

Configuration Data model for Incremental in AirbyteCatalog #988

cgardens opened this issue Nov 16, 2020 · 1 comment
Assignees
Labels
type/enhancement New feature or request
Milestone

Comments

@cgardens
Copy link
Contributor

cgardens commented Nov 16, 2020

Child of #987

Tell us about the problem you're trying to solve

  • Add necessary fields for incremental to the AirbyteCatalog.

Impl

  • Separate facts about the data versus configuration applied to the data.
    • e.g. table name is a fact. whether it can be synced as full refresh, incremental, or both is a fact. whether this table is going to be synced as full refresh or incremental is a configuration.
  • looks something like this? { streams: [ { …, sync_mode: (incremental or full_refresh), comparable: updated_at } ]
This was referenced Nov 16, 2020
@cgardens cgardens self-assigned this Nov 16, 2020
@cgardens cgardens added this to the v0.6.0 milestone Nov 16, 2020
@cgardens
Copy link
Contributor Author

closed by #998

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type/enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant