New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support 1 table per input file use cases #436

Open

kemp-google opened this issue Jan 29, 2019 · 1 comment

Labels

kemp-google commented Jan 29, 2019

From the discussion group:

For example if I have
1.vcf
2.vcf
3.vcf

Now, I upload each file dependability to have
1 table
2 table
3 table

and if I used
INPUT_PATTERN=gs://PROJECT/*.vcf
the output would be one merged data file

kemp-google added the enhancement label

Contributor

allieychen commented May 10, 2019

How about

Add one flag --one_table_per_file for this feature.
Separate dataflow job for each file.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment