iCloud sync long on initial sync on new device

[brentsimmons]

I think we may have to stop syncing the contents of unread articles (but continue to sync contents for starred articles).

The downsides to this are that 1) you might have some content on Device A that never appears on Device B, and 2) unread counts won’t match in the case where that content is unread.

I think that’s a reasonable trade-off for the long syncs.

Am I missing any downsides? Or is there some other solution?

[vincode-io]

I think iCloud sync as it is works pretty well except for the initial sync and if you haven't synced for a while on a given device. I personally use it for over 500 feeds and don't have any problems unless I am setting up a new device.

Maybe before we change it, we try to cut down on the support issues by better educating the user about iCloud syncing limitations?

What if we put up an Alert when a user creates an iCloud account and any time iCloud syncing hasn't completed 3 times in a row?

The Alert would have a title of "iCloud Syncing Times" that has an info message that says something like, "Syncing iCloud accounts can take a very long time due to iCloud service constraints. Learn more." The "Learn More" verbaige could be linked to a help page that explains in detail the issue with long sync times and provide solutions (like disabling auto lock on iOS to let it sync for an extended period of time). We could also suggest that for users with large numbers of feeds, that they use another sync service that NetNewsWire supports.

[vincode-io]

I'm not sure how we could make it work to remove content at this time and still sync with versions of NetNewsWire that still require content.

For older versions of NetNewsWire we would receive article from newer versions and insert them into the database, but without the contentHTML or the contentText field. I can't find anything in the fetch queries or anywhere else we would filter for this. I think what would happen is that older versions of NetNewsWire would have articles show up as unread and appear in the timeline without anything to render. Newer versions would still download content and be slower.

I'm not sure how to resolve this yet.

I think that the new way of syncing without content may wipe out content on older versions in some situations. For example if an article was received on a newer version of NetNewsWire, it would show up as an update and push an article record that doesn't have content to iCloud which would be picked up by the older version and update its local database. Since the dateModifiedfield is on the same iCloud record as the content ones, I'm not sure how I could stop this from happening.

[brentsimmons]

I’m starting to think we might need to do it as a separate thing — call it iCloud2 or “Light iCloud” or something, and maybe give people a way to migrate from iCloud to iCloud2, and eventually phase out the old iCloud.

Don’t know yet. Needs thought.

[vincode-io]

I would hate to see us reduce the capabilities of iCloud. I honestly think it works well for almost all our users. I know it works well for me and I have a lot of feeds.

I believe that this problem can be solved by correctly setting user expectations and letting the users know that if the free iCloud sync doesn't work for them that we have other syncing options available.

[vincode-io]

@brentsimmons One thing I would like to do is that while we are doing the incremental fetch of data, is to switch the progress indicator to be indeterminate, until we get to the point that we can begin refreshing the feeds. Then it would begin progressing again like normal.

The other thing I would like to do is not use the "x of x" label on the Mac status bar when iCloud is one of the accounts. In fact, I think that should be the case for any syncing account, since those numbers are largely worthless for any account type except the local account.

[brentsimmons]

Makes sense to me. Let’s make it indeterminate when we don’t know how many actions it will take.

And let’s use “x of x” only when the only accounts are local accounts (agreed that those numbers are otherwise meaningless). That label should says something like “Refreshing three accounts.” When there’s just one account left in the refresh session, it should say “Refreshing [accountName]”

[vincode-io]

I was planning on only using the indeterminate progress when pulling data from iCloud and then switch to a determinate progress bar for the rest of the sync. Only the iCloud account would have this behavior.

The label change makes sense to me for all accounts.

[vincode-io]

I've implemented incremental fetches of data and changed the progress indicator to work as specified.

I think the only thing we can do more with this is to remove the iCloud history by deleting the Articles zone and recreating it. It might be a little complicated for the user.

If a user is trying to create an iCloud account on a device, we could query the Account zone. This has information about how old the Articles zone is and how many feeds the user has. If the user has a large number of feeds and hasn't recreated the Articles zone recently, we could prompt them to prune the Articles zone using their other device.

On the other device the user could run a process that deletes the Articles zone and recreates it. The other device would then upload any necessary articles and article statuses.

The user could then set up the other device. This would go much more quickly since it is a straight download without all the Article zone history.

There are some implementation details to work out, but I'd like know if this is an acceptable user experience before spending more time on the minutia. If we want to go ahead with this, I'll write up another issue to track it.