diskv
diskv copied to clipboard
peterbourgon
Suggest: add key expiration
write key with expiration. while the expire time arrives, delete the key file.
This would require an active component in diskv that doesn't currently exist.
In my opinion, there already are enough Go built-ins for this functionality. In rssfs, I solved it by calling a timer after each .Write().
@cye1024 if what you have in mind is a garbage collector, I guess you could always delete the files on a separate goroutine (not tested!) and let diskv give an error if a certain key has no file on disk — and use Import to get it again.
An alternative would be to use keys that embed the expire time in it, and do a check before Getting the item from disk: if the current time is bigger than the expire time on the key, well, discard the key and remove the key file.
@dertuxmalwieder There are a lot of problems with this approach: it leaks goroutines, it's not cancelable, there's no way to know if it will or has already executed, and so on.
@peterbourgon Additionally, that does not work for things like a CLI, where there will not be one consistent execution of the binary. The storage of the TTL really ought to happen alongside the value being stored. And even for long-lived executions, you lose all your TTL functionality if you ever need to redeploy your application or restart the service.
One consistent execution of diskv for a given directory is a core requirement.
I've emulated TTL by using an additional layer of in-memory caching. Usually, these caches have a user-defined function running on eviction, so by using that, you can remove objects from diskv.
I use github.com/patrickmn/go-cache with OnEvict method.
Implementation details are at github.com/coinpaprika/echo-http-cache/blob/master/adapter/disk/disk.go.
One note: we don't need the cache to survive rollouts, so there is no need for embedded TTLs.
@dertuxmalwieder 's workaround could be implemented more efficiently using time.AfterFunc.
However that simple solution doesn't handle the case where a key is refreshed (the key will be deleted after the expiration of the first timer for that key, and not kept even if the value has been refreshed before expiration).
