aclist
[FEAT] Automating the cached magazines (loadMags)
Browser
Chromium-based
Script manager
Tampermonkey
KES version
4.3.0-beta.8
Describe the feature
I'm making this issue to share some ideas I've had to improve on the magazine loading functionality I moved to the safeGM file. The shared idea here is to reduce the need for users to turn off and on one of the affected mods every time they subscribe to a magazine and want to see that reflected in MES mods.
- Hook into subscribing/unsubscribing to automatically change the cached subscriptions as the user takes actions that change their actual subscriptions. This would mean they only need to turn a mod off and on again when switching between devices.
- Refresh the cache every session or day. This would mean that mods using this functionality take a bit longer to load when opening Mbin the first time in a session or day, but it would also simplify switching from one device to another.
Together, I think these two would cover 90% of cases in which one would have to turn a mod using this feature off and on again.
Alternatively, a button to reload the magazines would be good to have somewhere. As it is currently, requiring users to turn their mods off and on again, it feels very awkward.
What do you think about this topic @aclist?
I agree that the current approach is awkward.
Hook into subscribing/unsubscribing to automatically change the cached subscriptions as the user takes actions that change their actual subscriptions. This would mean they only need to turn a mod off and on again when switching between devices.
That seems reasonable. It does seem like a flaw that the subscriptions drift out of synch.
Refresh the cache every session or day. This would mean that mods using this functionality take a bit longer to load when opening Mbin the first time in a session or day, but it would also simplify switching from one device to another.
What is the main use case, here? If I understood correctly, you're talking about exporting MES settings from one device and then importing them into a new device, so the mods in question are already enabled, but the cache is missing. Is that correct?
Maybe it should be the responsibility of the import logic to fetch the cache first, rather than this per-session logic?
Maybe it would be possible to use the Reset button element, already provided in the MES settings menu, to tell the mod to reset and update the cache? (Talking about installing MES on a new device)
What is the main use case, here? If I understood correctly, you're talking about exporting MES settings from one device and then importing them into a new device, so the mods in question are already enabled, but the cache is missing. Is that correct?
Not really, no. With "switching from one device to another" I meant that if you're using Mbin on both PC and mobile. The first point only helps when making changes to subscriptions locally on the same device, but if you're using Mbin on another device too, there's no way for MES to automatically catch on to changes made there. That's why my second idea was to limit the duration of the cache. So that while the subscriptions may go out of sync, they do resync eventually automatically.
Ah, by switching, you didn't mean migrating from one device to another, but actively switching between them, so the activity on both is not being synched.
It seems intuitive enough to add a sync button to the omnibar modal menu, but this doesn't really resolve the issue with thread checkmarks, since that's something that happens passively when you browse the thread index. And you're not liable to remember that you need to sync your subs when just browsing the index.
Obviously, the simplest solution is just fetching subscriptions every time, but the extra load time is not going to feel good.
Refresh the cache every session or day. This would mean that mods using this functionality take a bit longer to load when opening Mbin the first time in a session or day, but it would also simplify switching from one device to another.
What if you have mbin open on two discrete browsers at the same time, both with open sessions that you don't close, and you stop browsing on the first browser for an hour and start subscribing to stuff on the second browser? What's the natural expectation for when these subs will appear on the first browser? Wouldn't you have to wait at least a day, or refresh the session? Can this be considered expected behavior?
I don't know, maybe there needs to be a universal sync button that is part of an overlay?
Taking a more monolithic approach: maybe when reloading the page, as a general jumping-off point, MES always fetches the user's metadata so that it's accessible to every mod? (Subscriptions and subscription dates, reputation, notifications, follows, etc.) It adds to the load time, but it's a one-and-done for every mod.
Just thinking aloud here.
What if you have mbin open on two discrete browsers at the same time, both with open sessions that you don't close, and you stop browsing on the first browser for an hour and start subscribing to stuff on the second browser? What's the natural expectation for when these subs will appear on the first browser? Wouldn't you have to wait at least a day, or refresh the session? Can this be considered expected behavior?
Well, we have to draw the line at some point. The only real way to automate it fully is to fetch every time, but I've tried that and the load times aren't good with my account at least. People who are subscribed to less magazines probably have it easier there though. The ideas I've come up with were just quick ones I got while moving the function, not fully thought through. But the point behind the resetting once a day was to be a middleground between fetching every time and having it be fully manual. Not an actual full solution to the problem. It's meant to automate some of it, but without getting to the point where it hurts the user experience in other areas (too much waiting for things to load).
I don't know, maybe there needs to be a universal sync button that is part of an overlay?
That sounds like a good idea.
I don't like the idea of just loading everything every time the page is reloaded, because for me personally the magazine fetching takes quite a bit of time alone (since I'm subscribed to a lot of them). Then you add the follows and other stuff... MES would probably become unusable to me if it fetched both the magazines and follows every time I navigate somewhere (Mbin doesn't have kbin's system of just replacing the html tag). But a button to sync everything on demand, I think that's a good idea. Provided it's not too hidden to the user. I think it would be ideal if mods didn't have to instruct users how to reload the subscribed magazines, but I'm not sure if we can get to that point.
One idea I've actually just had was to go back to the previous system of the fetch logic being in a mod, but to have it be a separate one from both omnibar and checkmarks. This mod would simply expand the already existing magazine list in the sidebar to contain all subscribed magazines without the current upper limit, as well as a button to reload the list. And then the omnibar and checkmarks mods could just read from that list instead of the profile.
Again, just a quick thought that came to mind while responding here. There would be its own problems with this, like how there's a setting to remove this list from the sidebar, which would have to be worked around.
But it would place a sync button in an intuitive location (for this use case, not a universal sync button) and also allows us to possibly add settings for the magazine fetching functionality itself, like my suggestion of loading everything once a day, if we feel the need to. Right now, we either have to make choices for the user, or add the setting to every mod that uses the fetching function.
I'm thinking.
Provided it's set to enabled, does the subscriptions panel contain the entire subscriptions list, even for a huge number of subscriptions?
If we want to wash our hands of all of this, you could just require having the subscriptions list enabled at the mbin level in order to fetch subs. If they don't want to see the list at that point, they can enable the mod to hide sidebar elements.
Perhaps the cowardly way out, but just throwing it on the table.
In terms of a sync button, having functionality similar to those "click here if you want to open a chatbot" things you see on consumer-facing sites that is always on the top layer might do the job. (I realize the irony of copying something so annoying, just providing a frame of reference here.)
I think it would be ideal if mods didn't have to instruct users how to reload the subscribed magazines, but I'm not sure if we can get to that point.
There's potentially something that can be recycled here: that alert area that MES prints to the top of the screen if the MES resources could not be loaded. This could be made into a generic notifier area that mods could hook into to print a generic warning on the screen.
If it's been >N hours/days since the cache was refreshed, you could pop this notifier and warn the user they need to update. Rather than burying these admonitions in the mod's settings menu, maybe active notifiers would be a good compromise?
Can add a close button to the notifier area so that the alert can be closed. You'd need some queueing system so that multiple mods can add notifications, though.
Maybe MES: finished with 2 errors, 3 notifications and you click a down caret to open the drawer and see the notifications, sort of like an error log.
Provided it's set to enabled, does the subscriptions panel contain the entire subscriptions list, even for a huge number of subscriptions?
It doesn't, otherwise I would have changed the logic to just read from that when I reimplemented the fetch logic in safeGM.
There's an upper limit on subscriptions shown there, after which there's a button to "show more" which simply loads the profile's page that we're already requesting.
This is the same even when it's moved to its own sidebar.
That's why I mentioned above that such a new mod, besides offering the button, would expand the list.
This mod would simply expand the already existing magazine list in the sidebar to contain all subscribed magazines without the current upper limit
That said, we could absolutely use this list to shorten the load time for people with less subscriptions. The upper limit seems to be 50 subscriptions, which is already a lot. I'm at like 66, which isn't much more. Most people are probably below.
Got it now. Sorry, it was the first time I'm seeing this subscription panel, so didn't know that expanding the list meant fetching another page.
I just loaded up a bunch of subscriptions and gave it a try. It seems to redirect right to the subs page. Kind of a weird design choice, but whatever.
So putting it all together:
As a shortcut, we could use this list as a starter when building the cache for the first time. If it doesn't have a "Show more" button and/or the number of subscriptions is below the threshold, we can abort right away and use that as the full list.
That is, provided they have this sidebar enabled. And if the list is too long, then we have to fetch from the full subscriptions page. So sidebar enabled + subs <= 50 is the best case scenario from a time complexity perspective.
Otherwise, it might take awhile, but it should be a one-time thing, and then you can catch (un)subscriptions when they happen, one at a time, and keep adding to the cache.
I'm just talking about the single device scenario now. For cross-device sync, still seems like you need a button or a passive sync.
I guess it takes a long time for you because it has to manually walk through multiple pages. Maybe there is a direct API endpoint that would return the full list of subscriptions in one blow?
I guess it takes a long time for you because it has to manually walk through multiple pages. Maybe there is a direct API endpoint that would return the full list of subscriptions in one blow?
Their ActivityPub implementation sadly doesn't seem to support subscriptions as an endpoint, even when the user has set them to public. Unlike some other fediverse platforms, the /following endpoint seems to only return followed users, not magazines.
I took a quick glance at the API documentation (the API meant for mobile apps), but got a 500 error when trying the magazine fetch endpoint on the instance I'm on. I didn't do the whole oauth stuff, so maybe that's why. Though the error should be different then.
I took a look through the (sparse) API docs as well and did not come away with any actionable info.
But a button to sync everything on demand, I think that's a good idea. Provided it's not too hidden to the user. I think it would be ideal if mods didn't have to instruct users how to reload the subscribed magazines, but I'm not sure if we can get to that point.
In conjunction with https://github.com/aclist/kbin-kes/pull/335, it doesn't seem out of the question that we might want to cache other sorts of information in the future besides subscriptions.
Thinking about this some more, maybe there will be a time where it makes sense to have a standalone page/menu in MES that is devoted to managing/updating all of your cached data across mods? Similar to how TamperMonkey provides a panel letting you flush your externals.
So you could have a table or grid with the name of the cache, what mods it pertain to, the date it was last synched (important for showing how stale the data is), and a button to delete/update it.
We already have a kind of barebones area where you can reset your MES settings, so maybe expanding on that idea and giving the user some control over what stuff is getting saved in the GM cache and/or in localStorage.