Host model weights on Huggingface

[laurens-gs]

Big organizations put up firewalls that make it impossible to download model weights. It is unlikely that large organizations have the capacity to whitelist every single source for model weights for every derivative package. However a blanket-approval is often granted for huggingface (via proxies that do malware scanning and license checks).

So my ask is; would it be possible to use huggingface as a host? It doesn't need to be the primary host. But for poor folks behind corporate firewalls having a secondary host on huggingface might just save them tons of bureaucracy. It will definitely help with adoption, as otherwise I'd be forced to stick with (sentence-)transformers.

[NirantK]

Thanks for the detailed note @laurens-gs, will look into this!

Will treat this as a new feature since this involves maintaining backward compatibility for existing users who might be using Google Cloud downloads right now.

[earonesty]

technically its pretty easy to just download the model weights and re-upload them to hugging face, license permitting, right? is there anything stopping me from doing that now?

[NirantK]

Yeah, nothing stops you from doing so! If you do decide to do so, please let us know where you push them, will take it forward from there.

[generall]

related issue: #81

[NirantK]

Fixed with #94