Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use GCS REST API in place of Firebase Storage REST API. #2343

Merged
merged 3 commits into from
Jun 8, 2020

Conversation

taeold
Copy link
Contributor

@taeold taeold commented Jun 8, 2020

Description

Firebase Storage REST API has a small issue now where file uploads for payloads >100KB reliably fails for some storage buckets.

Here, we change the implementation of src/gcp/storage.js to call GCS REST API to upload payloads (it doesn't fail for payloads >100KB).

Given that storage.js is placed undergcp/ directory, I think the transition to use GCP API vs Firebase API makes sense.

Scenarios Tested

Try installing an extension (whose total archive size would exceed 100KB) using its local source:

git clone [email protected]:firebase/extensions.git && cd extensions
firebase ext:install ./firestore-counter --project={project-id} --debug

Without the proposed change, the command fails with a 503 from the Firebase Storage API

[debug] [2020-05-29T18:18:17.476Z] >>> HTTP REQUEST POST https://firebasestorage.googleapis.com/v0/b/firebase-ext-eap-uploads/o/firebase-archive-10246n4tFTiaFcPmm.zip  
 <request body omitted>
[debug] [2020-05-29T18:18:19.211Z] <<< HTTP RESPONSE 503 {"x-guploader-uploadid":"AAANsUmdotV4qnjGKD7lViB9G4hQh9im1UXUmvMEpcpEGv2NKw6SEQc-sf_bBrzq1V6FN6RAEmUaeF0MKSE6tl5cLfAmI4XndQ","content-length":"0","date":"Fri, 29 May 2020 18:18:19 GMT","server":"UploadServer","content-type":"text/html; charset=UTF-8","alt-svc":"h3-27=\":443\"; ma=2592000,h3-25=\":443\"; ma=2592000,h3-T050=\":443\"; ma=2592000,h3-Q050=\":443\"; ma=2592000,h3-Q049=\":443\"; ma=2592000,h3-Q048=\":443\"; ma=2592000,h3-Q046=\":443\"; ma=2592000,h3-Q043=\":443\"; ma=2592000,quic=\":443\"; ma=2592000; v=\"46,43\""}

Firebase Storage REST API has a small issue now where file uploads for payloads >100KB reliably fails for some storage buckets. We change the implementation of src/gcp/storage.js to communicate directly with the GCS REST API which doesn't have this issue.

Aside - given that storage.js is placed inside the gcp/ directory, it's probably more correct to use GCS in the first place.
@googlebot googlebot added the cla: yes Manual indication that this has passed CLA. label Jun 8, 2020
@taeold taeold marked this pull request as ready for review June 8, 2020 16:23
@taeold taeold requested review from joehan and laurenzlong June 8, 2020 18:00
Copy link
Contributor

@joehan joehan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code looks good to me, just 1 question

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cla: yes Manual indication that this has passed CLA.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants