obsidian-livesync icon indicating copy to clipboard operation
obsidian-livesync copied to clipboard
verified publisher icon vrtmrz

[0.23.17] In latest version, any note update will trigger resynchronisation of all attachments in all devices

Open flyingnobita opened this issue 1 year ago β€’ 4 comments

Hi! Thanks for your great work! Just noticed something strange in the latest version that I like to report and wondering if you can help me with. Thanks in advance!

Abstract

I just updated to the newest version of 0.23.17. Syntonisation was working fine in previous version.

Whenever a change to a note is made, all attachments (mostly pictures and pdfs) are being created as "new note", re-synced, marked as conflict, and finally merged automatically as no change was detected.

Expected behaviour

  • Synchronisation of changed files only (as it was in the previously version)

Actually happened

  1. make any change in a note in one of my device
  2. For only attachments (where they are all stored in /zz_attachments/), the following message would appear for (seemingly) all files in /zz_atachments/:
7/15/2024, 11:32:18 PM->Processing: zz_attachments/05f18db3dcfc4352f02f50651aee725f.gif: (new note)...
7/15/2024, 11:32:18 PM->Processing: zz_attachments/05f18db3dcfc4352f02f50651aee725f.gif: Conflicted revision has been deleted, but there were more conflicts...
7/15/2024, 11:32:18 PM->Processing: zz_attachments/05f18db3dcfc4352f02f50651aee725f.gif: Done
  • This process takes ~5-10 mins where new lines are created in the logs non-stop (a sign to me that suggests the same thing was probably happening to all attachment files)
  • It occurs in all my other devices (~5) even though no changes were made there (so just syncing changes from my first device)
  • At the end, all files synchronised and no files are incorrectly deleted (fortunately!)
  • When this happens on my iPhone, sometimes Obsidian would restart during the above and continue after restart finishes
  • If I make a change in another device, the same strange behaviour above would happen to the rest of the devices, suggesting that it is not a platform or device issue

Reproducing procedure

  1. Update to version 0.23.17
  2. make change in device 1
  3. Start synchronization on other devices

Report from the LiveSync

If the above isn't sufficient, I can follow up with more information, just let me know.

flyingnobita avatar Jul 15 '24 15:07 flyingnobita

+1, I have the same issue.

ZHmao avatar Jul 16 '24 15:07 ZHmao

Thank you for your detailed information! However, I regret to say that I still cannot reproduce it.

Would you share with me a list of your other plug-ins (This can be seen by Show debug info from the command palette), And a configuration report of Self-hosted LiveSync (This can be dumped from the Make report button on the Hatch pane on the setting dialogue), please?

And, if nothing has changed, it should not be saved. Possibly it looks like the Conflict has been successfully resolved, but in fact, it has not. Does Pick a file to resolve conflict show the files? If so, rebuilding the database may solve the problem for now.

vrtmrz avatar Aug 02 '24 09:08 vrtmrz

Thank you for your reply.

I just tried rebuilding the database and then fetching it from other devices, but it didn't help and the same behaviour I reported occurred unfortunately.

Does Pick a file to resolve conflict show the files?

No files are shown.

Other Plugins 
Plugins installed: 59
	Plugins enabled: 57
		1: Git v2.25.0
		2: Self-hosted LiveSync v0.23.20
		3: Auto Link Title v1.5.4
		4: Better Word Count v0.10.1
		5: Link Favicons v1.8.4
		6: Plugin Update Tracker v1.5.2
		7: Recent Files v1.4.1
		8: Vault Statistics v0.1.3
		9: Style Settings v1.0.8
		10: Settings Search v1.3.10
		11: Find orphaned files and broken links v1.10.0
		12: Trash Explorer v1.2.2
		13: Dataview v0.5.67
		14: Natural Language Dates v0.6.2
		15: Advanced Tables v0.21.0
		16: Optimize Canvas Connections v1.0.0
		17: Supercharged Links v0.12.1
		18: Tag Wrangler v0.6.1
		19: TagFolder v0.18.7
		20: Safe Filename Linter v1.1.4
		21: Templater v2.4.1
		22: Note Refactor v1.8.2
		23: Tab Switcher v1.4.0
		24: Emoji Toolbar v0.4.1
		25: Footnote Shortcut v0.1.3
		26: Hotkeys for specific files v1.4.1
		27: Smarter Markdown Hotkeys v1.16.4
		28: Todoist Sync v1.13.0
		29: Tasks v7.6.1
		30: Calendar v1.5.10
		31: Periodic Notes v0.0.17
		32: QuickAdd v1.11.0
		33: Hider v1.4.0
		34: Keyboard Analyzer v0.1.3
		35: Task Marker v0.5.1
		36: Workspaces Plus v0.3.3
		37: Excalidraw v2.2.13
		38: Image Converter v1.1.7
		39: File Diff v1.1.1
		40: Quiet Outline v0.3.36
		41: Heading Shifter v1.5.1
		42: Editor Syntax Highlight v0.1.3
		43: Commander v0.5.1
		44: Number Headings v1.16.0
		45: Linter v1.25.0
		46: Projects v1.17.4
		47: ProgressBar v0.6.0
		48: Homepage v4.0.3
		49: Outliner v4.8.1
		50: Home tab v1.2.1
		51: Image Toolkit v1.4.1
		52: Minimal Theme Settings v8.0.2
		53: Zotero Integration v3.1.8
		54: Mononote v1.2.2
		55: Lapel v0.1.5
		56: Omnisearch v1.24.0
		57: Readwise Official v2.0.1
LiveSync Report 
---- Obsidian info ----
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) obsidian/1.4.13 Chrome/114.0.5735.289 Electron/25.8.1 Safari/537.36
---- remote config ----
cluster:
  n: "1"
cors:
  credentials: "true"
  headers: accept, authorization, content-type, origin, referer
  max_age: "3600"
  methods: GET, PUT, POST, HEAD, DELETE
  origins: app://obsidian.md,capacitor://localhost,http://localhost
chttpd:
  bind_address: 0.0.0.0
  enable_cors: "true"
  max_http_request_size: "4294967296"
  port: "5984"
  require_valid_user: "true"
admins: 𝑅𝐸𝐷𝐴𝐢𝑇𝐸𝐷
vendor:
  name: The Apache Software Foundation
feature_flags:
  partitioned||*: "true"
chttpd_auth:
  authentication_redirect: /_utils/session.html
  hash_algorithms: sha256, sha
  require_valid_user: "true"
  secret: b3e793bd47164192d8c353ecaeb12e5d
indexers:
  couch_mrview: "true"
prometheus:
  additional_port: "false"
  bind_address: 127.0.0.1
  port: "17986"
httpd:
  WWW-Authenticate: Basic realm="couchdb"
  bind_address: 127.0.0.1
  enable_cors: "true"
  port: "5986"
smoosh:
  state_dir: ./data
couch_httpd_auth:
  authentication_db: 𝑅𝐸𝐷𝐴𝐢𝑇𝐸𝐷
  secret: 𝑅𝐸𝐷𝐴𝐢𝑇𝐸𝐷
  authentication_redirect: 𝑅𝐸𝐷𝐴𝐢𝑇𝐸𝐷
couchdb_engines:
  couch: couch_bt_engine
couchdb:
  database_dir: ./data
  max_document_size: "50000000"
  single_node: "true"
  uuid: 𝑅𝐸𝐷𝐴𝐢𝑇𝐸𝐷
  view_index_dir: ./data

---- Plug-in config ---
version:0.23.20
remoteType: ""
useCustomRequestHandler: false
couchDB_URI: self-hosted(HTTP)
couchDB_USER: 𝑅𝐸𝐷𝐴𝐢𝑇𝐸𝐷
couchDB_PASSWORD: 𝑅𝐸𝐷𝐴𝐢𝑇𝐸𝐷
couchDB_DBNAME: 𝑅𝐸𝐷𝐴𝐢𝑇𝐸𝐷
liveSync: true
syncOnSave: false
syncOnStart: false
savingDelay: 200
lessInformationInLog: true
gcDelay: 0
versionUpFlash: ""
minimumChunkSize: 20
longLineThreshold: 250
showVerboseLog: false
suspendFileWatching: false
trashInsteadDelete: true
periodicReplication: false
periodicReplicationInterval: 60
syncOnFileOpen: false
encrypt: true
passphrase: 𝑅𝐸𝐷𝐴𝐢𝑇𝐸𝐷
usePathObfuscation: false
doNotDeleteFolder: false
resolveConflictsByNewerFile: false
batchSave: false
batchSaveMinimumDelay: 5
batchSaveMaximumDelay: 60
deviceAndVaultName: MBP
usePluginSettings: false
showOwnPlugins: false
showStatusOnEditor: false
showStatusOnStatusbar: true
showOnlyIconsOnEditor: false
usePluginSync: false
autoSweepPlugins: false
autoSweepPluginsPeriodic: false
notifyPluginOrSettingUpdated: false
checkIntegrityOnSave: false
batch_size: 50
batches_limit: 40
useHistory: true
disableRequestURI: true
skipOlderFilesOnSync: true
checkConflictOnlyOnOpen: false
showMergeDialogOnlyOnActive: false
syncInternalFiles: false
syncInternalFilesBeforeReplication: false
syncInternalFilesIgnorePatterns: \/node_modules\/, \/\.git\/, \/obsidian-livesync\/
syncInternalFilesInterval: 60
additionalSuffixOfDatabaseName: ""
ignoreVersionCheck: false
lastReadUpdates: 23
deleteMetadataOfDeletedFiles: false
syncIgnoreRegEx: ""
syncOnlyRegEx: ""
customChunkSize: 0
readChunksOnline: true
watchInternalFileChanges: true
automaticallyDeleteMetadataOfDeletedFiles: 0
disableMarkdownAutoMerge: false
writeDocumentsIfConflicted: false
useDynamicIterationCount: false
syncAfterMerge: false
configPassphraseStore: ""
encryptedPassphrase: 𝑅𝐸𝐷𝐴𝐢𝑇𝐸𝐷
encryptedCouchDBConnection: 𝑅𝐸𝐷𝐴𝐢𝑇𝐸𝐷
permitEmptyPassphrase: false
useIndexedDBAdapter: true
useTimeouts: false
writeLogToTheFile: false
doNotPaceReplication: false
hashCacheMaxCount: 300
hashCacheMaxAmount: 50
concurrencyOfReadChunksOnline: 100
minimumIntervalOfReadChunksOnline: 333
hashAlg: xxhash64
suspendParseReplicationResult: false
doNotSuspendOnFetching: false
useIgnoreFiles: false
ignoreFiles: .gitignore
syncOnEditorSave: false
pluginSyncExtendedSetting: {}
syncMaxSizeInMB: 50
settingSyncFile: ""
writeCredentialsForSettingSync: false
notifyAllSettingSyncFile: false
isConfigured: true
settingVersion: 0
enableCompression: false
accessKey: 𝑅𝐸𝐷𝐴𝐢𝑇𝐸𝐷
bucket: 𝑅𝐸𝐷𝐴𝐢𝑇𝐸𝐷(0 letters)
endpoint: Not configured or AWS
region: 𝑅𝐸𝐷𝐴𝐢𝑇𝐸𝐷(4 letters)
secretKey: 𝑅𝐸𝐷𝐴𝐢𝑇𝐸𝐷
useEden: false
maxChunksInEden: 10
maxTotalLengthInEden: 1024
maxAgeInEden: 10
disableCheckingConfigMismatch: false
displayLanguage: ""
enableChunkSplitterV2: false
disableWorkerForGeneratingChunks: false
processSmallFilesInUIThread: false
notifyThresholdOfRemoteStorageSize: 0
usePluginSyncV2: false
usePluginEtc: false
useV1: true
configPassphrase: ""
preset: ""
syncMode: LIVESYNC
dummy: 0

I hope this helps, thank you!

flyingnobita avatar Aug 02 '24 13:08 flyingnobita

+1 Here too.

I've rebuilt my remote and all my local databases and the issue persists. I've also removed any orphan attachments just in case. I haven't had a chance to dig into things and see what's going on yet, but I can confirm the issue. I'll see if I can produce a Minimal Reproducible Example once I've got some free time.

Laharah avatar Aug 10 '24 21:08 Laharah