hf-hub
hf-hub copied to clipboard
feat/reuse blobs
Current behavior
When the file/symlink is not found in snapshots the blob is downloaded regardless of whether or not it is already present.
New behavior
When the file/symlink is not found in snapshots the etag is used to check for the corresponding blob. It is used if already present.
Potential Issues
- The python library provides users with the option to force the download of the file. If this request is accepted we should likely do the same. I am however unaware of how such options should be introduced in this repository
- Probably not really an issue, existed before this change was introduced: If symlinking generally works on Windows but fails once, we move the file and end up with invalid symlinks (if I am not mistaken?, not 100% sure because it is Windows, but from what I understand of CreateSymbolicLinkW and std::os::windows::fs::symlink_file).