Refreshing page after updating content results in stale data

[georgeaf99]

It looks like updated content is not recognized, resulting in stale data being loaded, even if the server is serving new content.

Deleting locally stored content seems to fix the issue.

[xuset]

Are you sure your updating the service worker? Refreshing the page won't update it, you'll need to do it from the dev tools on chrome or about:serviceworkers on firefox

Edit: Also which content? the planktos scripts or the website's content?

[georgeaf99]

If a real website is being served from planktos and an update is made, when the user refreshes they should receive the changes, right? For development, I agree that updating the service workers on refresh works.

[xuset]

oh I see, yeah that's right. Those kind of changes will only be reflected if the torrent is updated after the file is changed. Auto checking the .torrent file to see if it has changed hasn't been done though. As of right now, it's only updated when the service worker is installed, but it is possible to manually check for an updated .torrent without re-installing the service worker.

[georgeaf99]

Then it seems like a good idea to check for an updated .torrent file on every page load

[xuset]

I'm thinking every page load may be needlessly excessive.. I don't know though

[xuset]

After thinking about it more:

• The request to fetch the .torrent file should probably not block page loads. Blocking can potentially cause overly long page loads. This does mean that if an updated .torrent file is found, the changes won't be reflected until the next page load
• Checking every page load while not blocking does limit the benefits of checking every page load
• Don't want to spam the server with excessive .torrent requests
• If the server uses caching, then the overhead for .torrent requests are reduced a lot, but not all servers do this (most do... maybe?)
• Right now, there is just on .torrent file, but the plan is to be able to split files into multiple .torrent files, for various reasons. Every check would require fetching every .torrent file. This is a future problem but should be kept in mind.

No solution really stands out as a winner though.. Some ideas:

• Check every page load and block the page load until the .torrent file is retrieved
• Check every page load in a non-blocking way
• Some timed check; check every x minutes.
• Don't check at all (aside from sw install), let the website owner decide on when to update

[xuset]

Not auto updating when the website's torrent is recreated is problematic. A non-blocking request to the manifest.json/root.torrent on every page seems like a good way to fix this.

[georgeaf99]

To Summarize our Conversation Yesterday:

After every page load Planktos will check for an updated torrent in a non-blocking way, updating the torrent if changes have been found. This means that any network requests coming after the torrent has been updated will load assets from the updated torrent.

This approach may be problematic if the user makes network requests after updating the torrent and before refreshing the page again, resulting in parts of the page being loaded from the old torrent and parts from the new torrent. This can be solved by remembering which torrent was used for the current page load, downloading the new torrent without overwriting the old one, and swapping the torrents on the next page load. This means that the new torrent will only be visible to the user on the second refresh. This will eventually be the default way to update the torrent, but, for now, we are going to implement the naive approach first.

Additionally, since both of these methods result in loading stale data, it would be good to offer the blocking option as well. This could be manually specified in some configuration.

[georgeaf99]

@xuset Can you tackle the initial implementation for this? I think it should be accomplished before we get our first round of feedback.

[xuset]

To implement the blocking auto-update: the service worker should immediately call update() when it is initialized and fetch requests should wait for the update promise to fulfill before sending the requests to fetch()