Large attachments - crashing difficult recovery

[andymarden]

I put a 100mb audio file in a note. Ok - your area doing to way "what possessed you?" but...

The vault in question is synced between 2 people and 2 devices each via couchdb.

After a while, I sat the more on other devices but now I get the app crashing when it tries the sync.

It's an unhandled exception, so it should not really occur - it should be handled.

So - recommendations:

1. First thing to do is to add a max note size option to the config. Default that to something comfortable (20mb) but let people change it with a warning. The sync process would silently/visible log a warning that it has not synced a note because of this (another option). Dead easy to implement and would allow me get out of a problem situation early or prevent it occurring on the first place.

2. Speaking of problems, I can't get rid of this now. I have stopped the syncing and file change detection (I can do that if I and quick, before it crashes) and then feels the note and verify & repair. I thought that would get rid of it but no, whatever I try, up it pops again and crashes the app. I think I will have to delete the contents of the couchdb database. That has to work right?

3. And if it does encounter a problem, it needs to tell the error and handle it.

4. What's quite important in this plugin is that all devices have the same settings. You have the setup URL but that is clunky and things can get out of sync. I don't know what you don't just store a special note in the vault which is the config for the plugin (you could have things as common or per device in the as appropriate. Then you don't need that cumbersome URL initially and try to remember to have the required settings on an ongoing basis. A new device just sets up a new Vault, you send them the md file for this special plugin which you put on the vault and, voila, the plugin picks up the config. Job done and the sync will sync the settings themselves. What could be simpler than using the datastore you have to store the config for a plugin? You could allow a used to be notified the config but had changed and all whether they want to import changes now, or wait (still they can check the more first perhaps).

What do you think?

[vrtmrz]

Thanks to your report, I have also found this issue. Especially in iOS devices, larger file operations seem to fail in some cases. The final and ideal fix is handling larger files smoothly. However, the workaround that you have proposed might be helpful for us.

1. It sounds easier but possibly difficult. It is very smart behaviour if the file is originally over 20MB. However, if it got over 20MB gradually, and decreased in different devices again, it might be very unstable. (And unfortunately, Self-hosted LiveSync can have different thresholds in each device). We have to think about it more in deep, e.g., automatically generates dot-ignore files and forcely synchronises them.
   So, let me think about it a bit more.

2. As you mentioned, we could not operate the UI in some cases. We can reset all databases by following the instructions. (It should be performed at the most reliable vault).

   • Stop the Obsidian.
   • Delete the file which causes the trouble.
   • Place redflag2.md in the root directory of the vault.
   • Lauhcn Obsidian.

By this operation, the local database and the remote database will be erased and rebuilt by local files. If the remote files are already fine, use redflag3.md in the same method for retrieving the files again.

4. I think that it is a very essential topic. In the original design, I thought each device should use a different configuration in some cases. However, as you pointed out, Today, it not be required always. Of course, especially in rescuing misconfiguration, we have to prepare the local configuration and have the chance to decide accept or reject remote changes. Of course, we should not store the passphrase, password and credentials of the database. And some device-specific things also still be. Therefore, we probably still need the configuration URI, but it would be shortened. Thank you for the excellent proposal! I will address it later.

[vrtmrz]

@andymarden Unfortunately, we seem unable to synchronise the large file on the mobile platform for now. It depends on the limitations of Obsidian. Calling the API of Obsidian with a large payload causes a small UI freeze. Then, Obsidian detects that and restarts.

Unfortunately further, also we do seem not to have a workaround. (If there were binary append API, we could use it).

However, we have to handle other things for the daily work. Therefore, as per your proposal, I made a new feature that filters files by their size! Thank you for the great idea! There is only one additional specification, which is to use only the size of what has changed.

And, possibly some people want to apply this feature only on mobile devices. So, I kept the configuration in the Self-hosted LiveSync. However, if configured Share settings via markdown, it would be written into the setting markdown and it could be synchronised. If the new setting markdown has been synchronised (or changed), and the filename is matched to the device configuration, we will get a notification to use or ignore it. We have to make a decision, but it will be performed as you have proposed! I appreciate again for your very idea.

So, v0.22.2 has been released. Would you please try this version?