wget2
wget2 copied to clipboard
Wishlist/Enhancement - Add support for new option to only update existing file if etag has changed
This could help when an file's content has not changed but the timestamp has since changed and as such it downloads a existing file unnecessarily but how could this be implemented if such thing came to frutation?
What i thought was we would have to store the etags for each file if we have the option turned on in some form of database or something since if we don't store it if its supported by the server then again we are unnecessarily downloading a file thats its timestamp has somehow changed but not the actual files content
It's a good idea. Instead of maintaining a database, we could store the ETAG with the "--xattr" feature. These are extended attributes supported by the common Linux file systems. I know that we support FreeBSD extended attributes as well.
Currently we store user.xdg.origin.url
, user.xdg.referrer.url
, user.mime_type
and user.charset
.
Adding the ETAG here and making use of it is at least a start. WDYT?
It's a good idea. Instead of maintaining a database, we could store the ETAG with the "--xattr" feature. These are extended attributes supported by the common Linux file systems. I know that we support FreeBSD extended attributes as well.
Currently we store
user.xdg.origin.url
,user.xdg.referrer.url
,user.mime_type
anduser.charset
.Adding the ETAG here and making use of it is at least a start. WDYT?
Yes I agree but what would be used on/with Windows and how does xattr store it?
Also some etag implementations are terrible as well