You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
There are some situations where the flux check task in a FlowETL dag is unnecessary (e.g. a two-stage ingestion process where data from a multiple-day file are ingested into a staging table, and then daily dag runs extract from that staging table - a flux check is needed in the first stage, but the second-stage daily runs do not need to check for flux again). Since the table flux check can be quite slow, create_dag should provide an option to disable this check.
The text was updated successfully, but these errors were encountered:
There are some situations where the flux check task in a FlowETL dag is unnecessary (e.g. a two-stage ingestion process where data from a multiple-day file are ingested into a staging table, and then daily dag runs extract from that staging table - a flux check is needed in the first stage, but the second-stage daily runs do not need to check for flux again). Since the table flux check can be quite slow,
create_dag
should provide an option to disable this check.The text was updated successfully, but these errors were encountered: