Corruption during sync process breaks plugin files locally and in remote DB

[izosozi]

EDIT: solved (partially)

It turns out that there was a failure to decrypt some of the .obsidian files. I will leave more details in a comment.

Abstract

Some (but not all) plugins are failing to load on start. The plugin folders appear intact in the filesystem, but not within the app.

I'm not 100% certain that LiveSync did cause this issue; It's very possible that this was totally my own error or another plugin caused it. However, it happened at the same time that I was messing with various LiveSync settings, and I hadn't made any real changes to other plugins or Obsidian in the several hours before the issue happened.

Expected behavior

• Plugins visible in the Community Plugins interface match the plugins in .obsidian/plugins
• Installed plugins don't disappear without a reason

Actually happened

• Over half of the community plugins are not visible in the Obsidian Settings sidebar or the Community Plugins settings pane
• Plugin functionality is not loading for these plugins
• Some plugins remain functional, unclear why

Reproducing procedure

I have no idea what caused it, but I'll provide environment details below. The issue is persistent.

Report materials

Went over the length limit...

report-data.txt

Screenshots

Server config check:

Other information, insights and intuition.

Obsidian plugins

In GUI

The plugins shown in the Community Plugins GUI in Obsidian:

In JSON

The plugins listed in .obsidian/community-plugins.json:

[
  "github-embeds",
  "obsidian-excalidraw-plugin",
  "obsidian-minimal-settings",
  "obsidian-style-settings",
  "obsidian-kanban",
  "obsidian-tasks-plugin",
  "dataview",
  "execute-code",
  "templater-obsidian",
  "recent-files-obsidian",
  "obsidian-mind-map",
  "email-block-plugin",
  "obsidian-outliner",
  "omnisearch",
  "obsidian-linter",
  "nldates-obsidian",
  "better-word-count",
  "highlightr-plugin",
  "table-editor-obsidian",
  "obsidian-vimrc-support",
  "vim-yank-highlight",
  "obsidian-relative-line-numbers",
  "vim-toggle",
  "colored-tags-wrangler",
  "obsidian-icon-folder",
  "media-extended",
  "obsidian-link-embed",
  "dataview-serializer",
  "obsidian-advanced-uri",
  "meld-encrypt",
  "obsidian-auto-link-title",
  "links",
  "obsidian-local-images",
  "wikipedia-search",
  "obsidian-dangling-links",
  "quick-preview",
  "boost-link-suggestions",
  "obsidian-full-calendar",
  "calendar",
  "obsidian-text-format",
  "obsidian-regex-replace",
  "copy-as-html",
  "find-and-replace-in-selection",
  "remember-cursor-position",
  "obsidian-cursor-location-plugin",
  "advanced-cursors",
  "smart-connections-visualizer",
  "homepage",
  "advanced-canvas",
  "canvas-mindmap",
  "obsidian-raindrop-highlights",
  "janitor",
  "random-numbers-generator",
  "obsidian-dice-roller",
  "improved-random-note",
  "share-note",
  "obsidian-livesync"
]

In local filesystem

The content of .obsidian/plugins/:

.
|-- advanced-canvas
|   |-- main.js
|   |-- manifest.json
|   `-- styles.css
|-- advanced-cursors
|   |-- main.js
|   |-- manifest.json
|   `-- styles.css
|-- better-word-count
|   |-- main.js
|   |-- manifest.json
|   `-- styles.css
|-- boost-link-suggestions
|   |-- data.json
|   |-- main.js
|   |-- manifest.json
|   `-- styles.css
|-- calendar
|   |-- data.json
|   |-- main.js
|   `-- manifest.json
|-- canvas-mindmap
|   |-- data.json
|   |-- main.js
|   |-- manifest.json
|   `-- styles.css
|-- colored-tags-wrangler
|   |-- data.json
|   |-- main.js
|   |-- manifest.json
|   `-- styles.css
|-- convert-url-to-iframe
|   |-- main.js
|   |-- manifest.json
|   `-- styles.css
|-- copy-as-html
|   |-- main.js
|   `-- manifest.json
|-- dataview
|   |-- data.json
|   |-- main.js
|   |-- manifest.json
|   `-- styles.css
|-- dataview-serializer
|   |-- data.json
|   |-- main.js
|   |-- manifest.json
|   `-- styles.css
|-- email-block-plugin
|   |-- main.js
|   |-- manifest.json
|   `-- styles.css
|-- execute-code
|   |-- data.json
|   |-- main.js
|   |-- manifest.json
|   `-- styles.css
|-- find-and-replace-in-selection
|   |-- main.js
|   `-- manifest.json
|-- github-embeds
|   |-- data.json
|   |-- main.js
|   |-- manifest.json
|   `-- styles.css
|-- heading-level-indent
|   |-- data.json
|   |-- main.js
|   `-- manifest.json
|-- highlightr-plugin
|   |-- main.js
|   |-- manifest.json
|   `-- styles.css
|-- homepage
|   |-- data.json
|   |-- main.js
|   |-- manifest.json
|   `-- styles.css
|-- improved-random-note
|   |-- data.json
|   |-- main.js
|   `-- manifest.json
|-- ink
|   |-- data.json
|   |-- main.js
|   |-- manifest.json
|   `-- styles.css
|-- janitor
|   |-- main.js
|   |-- manifest.json
|   `-- styles.css
|-- link-exploder
|   |-- main.js
|   `-- manifest.json
|-- link-favicon
|   |-- data.json
|   |-- main.js
|   |-- manifest.json
|   `-- styles.css
|-- links
|   |-- data.json
|   |-- main.js
|   |-- manifest.json
|   `-- styles.css
|-- media-extended
|   |-- data.json
|   |-- main.js
|   |-- manifest.json
|   `-- styles.css
|-- meld-encrypt
|   |-- main.js
|   |-- manifest.json
|   `-- styles.css
|-- nldates-obsidian
|   |-- main.js
|   `-- manifest.json
|-- numerals
|   |-- main.js
|   |-- manifest.json
|   `-- styles.css
|-- obsidian-advanced-uri
|   |-- main.js
|   `-- manifest.json
|-- obsidian-auto-link-title
|   |-- main.js
|   |-- manifest.json
|   `-- styles.css
|-- obsidian-cursor-location-plugin
|   |-- main.js
|   `-- manifest.json
|-- obsidian-dangling-links
|   |-- main.js
|   |-- manifest.json
|   `-- styles.css
|-- obsidian-dice-roller
|   |-- data.json
|   |-- main.js
|   |-- manifest.json
|   `-- styles.css
|-- obsidian-excalidraw-plugin
|   |-- data.json
|   |-- main.js
|   |-- manifest.json
|   `-- styles.css
|-- obsidian-full-calendar
|   |-- data.json
|   |-- main.js
|   |-- manifest.json
|   `-- styles.css
|-- obsidian-git
|   |-- data.json
|   |-- main.js
|   |-- manifest.json
|   `-- styles.css
|-- obsidian-icon-folder
|   |-- data.json
|   |-- main.js
|   |-- manifest.json
|   `-- styles.css
|-- obsidian-importer
|   |-- main.js
|   |-- manifest.json
|   `-- styles.css
|-- obsidian-kanban
|   |-- data.json
|   |-- main.js
|   |-- manifest.json
|   `-- styles.css
|-- obsidian-link-embed
|   |-- main.js
|   |-- manifest.json
|   `-- styles.css
|-- obsidian-linter
|   |-- data.json
|   |-- main.js
|   |-- manifest.json
|   `-- styles.css
|-- obsidian-livesync
|   |-- data.json
|   |-- main.js
|   |-- manifest.json
|   `-- styles.css
|-- obsidian-local-images
|   |-- data.json
|   |-- main.js
|   |-- manifest.json
|   `-- styles.css
|-- obsidian-math-plus
|   |-- data.json
|   |-- main.js
|   |-- manifest.json
|   `-- styles.css
|-- obsidian-mind-map
|   |-- main.js
|   `-- manifest.json
|-- obsidian-minimal-settings
|   |-- data.json
|   |-- main.js
|   `-- manifest.json
|-- obsidian-outliner
|   |-- main.js
|   |-- manifest.json
|   `-- styles.css
|-- obsidian-raindrop-highlights
|   |-- data.json
|   |-- main.js
|   |-- manifest.json
|   `-- styles.css
|-- obsidian-regex-replace
|   |-- main.js
|   |-- manifest.json
|   `-- styles.css
|-- obsidian-relative-line-numbers
|   |-- main.js
|   |-- manifest.json
|   `-- styles.css
|-- obsidian-style-settings
|   |-- data.json
|   |-- main.js
|   |-- manifest.json
|   `-- styles.css
|-- obsidian-tasks-plugin
|   |-- data.json
|   |-- main.js
|   |-- manifest.json
|   `-- styles.css
|-- obsidian-text-format
|   |-- data.json
|   |-- main.js
|   |-- manifest.json
|   `-- styles.css
|-- obsidian-vimrc-support
|   |-- main.js
|   `-- manifest.json
|-- omnisearch
|   |-- data.json
|   |-- main.js
|   |-- manifest.json
|   `-- styles.css
|-- quick-preview
|   |-- data.json
|   |-- main.js
|   |-- manifest.json
|   `-- styles.css
|-- random-numbers-generator
|   |-- main.js
|   `-- manifest.json
|-- recent-files-obsidian
|   |-- data.json
|   |-- main.js
|   |-- manifest.json
|   `-- styles.css
|-- remember-cursor-position
|   |-- cursor-positions.json
|   |-- main.js
|   `-- manifest.json
|-- share-note
|   |-- data.json
|   |-- main.js
|   |-- manifest.json
|   `-- styles.css
|-- smart-connections
|   |-- data.json
|   |-- main.js
|   |-- manifest.json
|   `-- styles.css
|-- smart-connections-visualizer
|   |-- main.js
|   |-- manifest.json
|   `-- styles.css
|-- solve
|   |-- data.json
|   |-- main.js
|   |-- manifest.json
|   `-- styles.css
|-- system3-relay
|   |-- data.json
|   |-- main.js
|   |-- manifest.json
|   `-- styles.css
|-- table-editor-obsidian
|   |-- data.json
|   |-- main.js
|   |-- manifest.json
|   `-- styles.css
|-- tag-wrangler
|   |-- main.js
|   |-- manifest.json
|   `-- styles.css
|-- templater-obsidian
|   |-- data.json
|   |-- main.js
|   |-- manifest.json
|   `-- styles.css
|-- vim-toggle
|   |-- main.js
|   `-- manifest.json
|-- vim-yank-highlight
|   |-- main.js
|   |-- manifest.json
|   `-- styles.css
`-- wikipedia-search
    |-- data.json
    |-- main.js
    `-- manifest.json

Potential causes

• Multiple rebuilds, merges, and fetches while I was troubleshooting other issues
• Quitting Obsidian during data operations
• Briefly enabling Customization Sync for LiveSync's own plugin data files
• Toggling options before I understood what they did
• A couple days ago I accidentally put self-referencing symlinks in the .obsidian folder, and LiveSync hit a recursion error trying to parse them. I removed these and LiveSync seemed to recover fine.
• Laptop and phone both running LiveSync simultaneously
• General recklessness

Attempts to fix

• LiveSync > rebuild local db and remote db from local files
• LiveSync > fetch from remote db
• Disabling LiveSync and relaunching obsidian
• Disabling LiveSync and duplicating the vault locally
• Uninstalling LiveSync and relaunching
• Uninstalling LiveSync and duplicating the vault locally
• Reinstalling Obsidian and opening both vaults
• Created a vault with only the .obsidian directory and no other files
• Verified file permissions for all files in .obsidian/

While LiveSync seems to have caused the problem initially, the problem seems persistent in the vault contents, probably

Other observations

• Multiple syntax errors in the Obsidian console, these might each correspond to an individual failure to load a plugin?
• An extraneous [] in syncIgnoreRegEx: \.DS_Store|[]|^raindrop\/ in livesync-config
• The size of the remote went from ~600MB to ~150MB around the same time, but I can't say for sure if this is related, since I was implementing ignore patterns which might have removed files separately from this issue.

The end

I do have backups and nothing was lost in this case thankfully!

Thanks so much for your hard work on this plugin; It's certainly the best open source option for Obsidian syncing I've seen, and I'd be interested in contributing once I get it all figured out.

EDIT: Solved partially

See comment below