wget2 icon indicating copy to clipboard operation
wget2 copied to clipboard

Wishlist/Enhancement - Add support for new option to only update existing file if etag has changed

Open nutterthanos opened this issue 1 year ago • 4 comments

This could help when an file's content has not changed but the timestamp has since changed and as such it downloads a existing file unnecessarily but how could this be implemented if such thing came to frutation?

What i thought was we would have to store the etags for each file if we have the option turned on in some form of database or something since if we don't store it if its supported by the server then again we are unnecessarily downloading a file thats its timestamp has somehow changed but not the actual files content

nutterthanos avatar Dec 09 '23 23:12 nutterthanos

It's a good idea. Instead of maintaining a database, we could store the ETAG with the "--xattr" feature. These are extended attributes supported by the common Linux file systems. I know that we support FreeBSD extended attributes as well.

Currently we store user.xdg.origin.url, user.xdg.referrer.url, user.mime_type and user.charset.

Adding the ETAG here and making use of it is at least a start. WDYT?

rockdaboot avatar Dec 15 '23 19:12 rockdaboot

It's a good idea. Instead of maintaining a database, we could store the ETAG with the "--xattr" feature. These are extended attributes supported by the common Linux file systems. I know that we support FreeBSD extended attributes as well.

Currently we store user.xdg.origin.url, user.xdg.referrer.url, user.mime_type and user.charset.

Adding the ETAG here and making use of it is at least a start. WDYT?

Yes I agree but what would be used on/with Windows and how does xattr store it?

nutterthanos avatar Dec 23 '23 03:12 nutterthanos

Also some etag implementations are terrible as well

nutterthanos avatar Dec 26 '23 08:12 nutterthanos