send dags for backstage #1509
send dags for backstage #1509
Conversation
|
Just prior to submitting this PR, BSLW confirmed that the weekly will be ftp like before, but the initial files can be uploaded through their online portal. I logged into it and it seemed like we would not be able to submit the full dump files via http (like we do with POD, which has an API but is really just an upload form submission). I'm not sure how many files there will be for the initial file since it's just since cataloging in FOLIO. I'm not sure how feasible it will be to transmit them through their online portal. @ahafele do you know if ShareVDE's initial files will be transmitted via ftp? I wonder if I should remove the changes I made to the send_all_records DAG or stash it away until we have a vendor that wants a full dump via ftp. |
|
I believe Sharevde full dump will be ftp but maybe @dlrueda can confirm from conversations with them? Re: BSLW initial files - they should be relatively small in comparison to a full dump - max 500k |
|
Connection details for SVDE from Symphony in this GDrive doc |
Closes #1440
Awaiting confirmation from Backstage how we will be sending the initial and weekly files. This PR is based on what we did before (ftp).