Free hosting with backblaze b2 and cloudflare CDN

[johtso]

So I was looking around for alternatives to github pages.. and managed to find a working free setup with backblaze b2 and cloudflare.

Backblaze has no bandwidth limits when routed through cloudflare.

Backblaze also has a 10GB free storage limit compared with github's 1GB limit. Definitely useful if working with a large dataset and lots of indexes!

One thing that tripped me up was that you need to change cloudflare's default SSL setting from "flexible" to "full" otherwise you just get timeouts as it tries to make requests to backblaze without ssl.

https://help.backblaze.com/hc/en-us/articles/217666928-Using-Backblaze-B2-with-the-Cloudflare-CDN

Thought this might be of use to others!

[phiresky]

That's great!

The "flexible" configuration is an issue I've encountered multiple times with cloudflare, I don't understand why they still have that as the default.

One thing to consider here is how exactly cloudflare caches the requests because that might impact the user experience and mean different chunking configurations would be better for sql.js-httpvfs. I can think of three ways they could do it:

1. They forward the exact range request a user sends to the backing storage. If so, do they cache the data at all or is the request always forwarded?
2. They download the whole file first and then serve the range request. This would imply there's cold starts where the first request within one region (per cloudflare datacenter) would be slow, then the next ones would be fast.
3. They download a larger fixed chunk of the file (e.g. 1MB), cache that chunk, then serve the request from that chunk. This is basically how IPFS does it, but I guess it's fairly unlikely here.

Have you noticed any cold starts?

[johtso]

So I'm not 100% sure what Cloudflare is doing, I did a bit of experimentation.

With cache level "standard" in the page rule, nothing is cached for the range requests, I think just because it's not one of the file extensions that they cache.

When you set the cache level to "cache everything" then things start to happen.

Performance with caching disabled using one big file (all requests going through to b2) looks like this:

Not great, but just about acceptable if you keep the number of requests low with indexes. Can't remember exactly how this compares with github pages, I think github was a bit faster.

Trying cache everything with one big 350MB database, you get some super sluggish requests when the cache is cold and cloudflare are presumably making you wait for them to download the whole file:

But after that it actually seems to perform really well!

When I tried the same with a larger 1.2GB database I just ended up with requests that seem to stall forever. Possibly with a bit more patience it would eventually get into the cache?

Requests with cache everything and 10MB chunks with cold CDN look like this:

Obviously with lots of chunks is harder to get them all cached at all the nodes..

I see mention of a 512MB max file size for caching.. not sure if that's true. I'm not sure what the optimal chunk size would be.. something quite large so that it's easy to get everything into the CDN, but not so large that if there's a cache miss the user is waiting for 15 seconds. Maybe 50-100MB chunks or something, or just one big file if the dataset is small enough.

All in all caching seems to be looking quite promising!

[johtso]

Oh dear I spoke too soon..

Randomly deciding to return the whole file isn't good!

Request URL: https://data.infinitenovelty.com/file/iabi-data/db.2.sqlite
Request Method: GET
Status Code: 200 
Remote Address: 104.21.88.215:443
Referrer Policy: strict-origin-when-cross-origin

Response:

accept-ranges: bytes
access-control-allow-credentials: true
access-control-allow-origin: https://infinitenovelty.com
age: 2720
alt-svc: h3=":443"; ma=86400, h3-29=":443"; ma=86400
cache-control: max-age=14400
cf-cache-status: HIT
cf-ray: 6fae120f8fac7447-LHR
content-length: 348270592
content-type: application/octet-stream
date: Tue, 12 Apr 2022 18:36:58 GMT
expect-ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
last-modified: Tue, 12 Apr 2022 17:51:38 GMT
nel: {"success_fraction":0,"report_to":"cf-nel","max_age":604800}
report-to: {"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v3?s=nJHvdB%2BoGgIc92OLvM6nquDYwtefQWJVlL7scHA77OHNyAx7TuAWtgf2qnMM%2F%2B40dYgrJszz7Lg6ZEiqgPrWR3%2BNosQb7%2FsyZobVBHRR7yxJVPuKnJqm0UzxPB2Xbg0KA1xsd10wUSMtiww%3D"}],"group":"cf-nel","max_age":604800}
server: cloudflare
vary: origin, access-control-request-method, access-control-request-headers, Accept-Encoding
x-bz-content-sha1: none
x-bz-file-id: 4_zc817b03135a21b918f04091c_f202a35e0ad12a570_d20220412_m031118_c004_v0402000_t0024_u01649733078130
x-bz-file-name: db.2.sqlite
x-bz-info-src_last_modified_millis: 1649732923585
x-bz-upload-timestamp: 1649733078130

Request:

DNT: 1
Range: bytes=10825728-10891263
Referer: https://infinitenovelty.com/
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/100.0.4896.75 Safari/537.36

[johtso]

I'm wondering if it would be good to be able to reject any response that was 200 instead of 206.. or a content length that looked worrying, rather than just the download limit? Then the request could be retried.

[phiresky]

thanks for all the info! that 200 seems like a really weird bug on the side of cloudflare? :o

[johtso]

Yeah.. almost seems like one of the CDN nodes misbehaving or behaving differently. I'm pretty sure I remember reading something about one of cloudflares products where they say they respond with 200 to range requests, but it only seems to happen sporadically.

[johtso]

Another free option to add to the pile is g-core labs. I actually have my files going g-core labs > cloudflare > backblaze, so that I still benefit from the free backblaze -> cloudflare data.

g-core labs also let you prefetch things which is quite nice, and they have a CDN setting that does slightly more intelligent caching of range requests where it only downloads a 20MB chunk around the data you request (or something along those lines).

I raised a ticket with Cloudflare about that strange CDN behaviour, but I actually haven't experienced it again recently.