Fix downloading models from Hub

[osanseviero]

The current approach has some issues when downloading repos from the Hub. The current approach skips downloading if a local directory is already present. This means that local files can easily get outdated if there are changes in the repo in the Hub. Users only get the latest model if they delete the directory from the cache. Additionally, if one file is missing (such as modules.json, it doesn't get automatically downloaded).

With this new approach, we always call snapshot_download and directly save in the expected directory instead of renaming the directory afterwards. It has this behavior:

• If the file already exists in the cache, it won't get re-downloaded.
• If a file is missing locally, it gets downloaded.
• If a file is updated in the Hub, it also gets downloaded.

This ensures that the local files are updated but maintaining the benefits of client-side caching.

[osanseviero]

Pinging @nreimers since this bug will likely hit again in some time.

[nreimers]

Hi @osanseviero
Sorry they I have not yet reacted. Was quite busy in the last days to finalize the release of the new models.

Don't worry, I did not forgot this PR. Will try to have a look next week.

[nreimers]

Thanks, the fix works well :)