obsidian-livesync icon indicating copy to clipboard operation
obsidian-livesync copied to clipboard
verified publisher icon vrtmrz

Replication errors between Mac / iPad / iPhone: TypeError, AbortError

Open kuyper opened this issue 1 year ago β€’ 5 comments

Abstract

Livesync throws TypeError and AbortError when trying to sync between Mac / iPad / iPhone.

Expected behaviour

  • Synchronisation ends with the message Replication completed
  • Everything synchronised

Actually happened

  • Initial synch of Mac with CouchDB works.
  • After adding synch to iPad, the following happens on the iPad:
HTTP:Post (xxx) to:/_bulk_docs -> failed
TypeError: Load failed
  • I also see
HTTP:GET to:/_changes?style=all_docs&feed=.... -> false
AbortError:Fetch is aborted

Reproducing procedure

  1. Start with a fresh database in CouchDB
  2. Setup a new vault on Mac
  3. Enable community plugins and install livesync
  4. Configure livesync to use the fresh database
  5. Keep hidden file sync disabled
  6. Setup new vault on iPad, install livesync
  7. Configure livesync to use the database that has just been created
  8. Keep hidden files disabled
  9. Wait until first sync is done (which sometimes takes a while)
  10. Enable customisation sync

--> Error start appearing in the logs

My feeling is that sync is generally unstable with this version. I'm using 0.21 with Mac / iPhone / iPad with three different vaults in a "production" setting and it works well. But with 0.22 I haven't been able to get it to run.

Report materials

If the information is not available, do not hesitate to report it as it is. You can also of course omit it if you think this is indeed unnecessary. If it is necessary, I will ask you.

Report from the LiveSync

For more information, please refer to Making the report.

Report from hatch 

---- Obsidian info ----
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko)
---- remote config ----
cors:
  credentials: "true"
  headers: accept, authorization, content-type, origin, referer
  max_age: "3600"
  methods: GET,PUT,POST,HEAD,DELETE
  origins: app://obsidian.md,capacitor://localhost,http://localhost
chttpd:
  bind_address: any
  enable_cors: "true"
  max_http_request_size: "4294967296"
  port: "5984"
  require_valid_user: "true"
admins: "𝑅𝐸𝐷𝐴𝐢𝑇𝐸𝐷"
vendor:
  name: The Apache Software Foundation
feature_flags:
  partitioned||*: "true"
chttpd_auth:
  authentication_redirect: /_utils/session.html
  hash_algorithms: sha256, sha
  require_valid_user: "true"
indexers:
  couch_mrview: "true"
prometheus:
  additional_port: "false"
  bind_address: 127.0.0.1
  port: "17986"
httpd:
  WWW-Authenticate: Basic realm="couchdb"
  bind_address: 0.0.0.0
  enable_cors: "true"
  port: "5986"
smoosh:
  state_dir: ./data
couch_httpd_auth:
  authentication_db: "𝑅𝐸𝐷𝐴𝐢𝑇𝐸𝐷"
  secret: "𝑅𝐸𝐷𝐴𝐢𝑇𝐸𝐷"
  authentication_redirect: "𝑅𝐸𝐷𝐴𝐢𝑇𝐸𝐷"
couchdb_engines:
  couch: couch_bt_engine
couchdb:
  database_dir: ./data
  max_document_size: "50000000"
  single_node: "true"
  uuid: "𝑅𝐸𝐷𝐴𝐢𝑇𝐸𝐷"
  view_index_dir: ./data

---- Plug-in config ---
version:0.22.7
couchDB_URI: self-hosted
couchDB_USER: "𝑅𝐸𝐷𝐴𝐢𝑇𝐸𝐷"
couchDB_PASSWORD: "𝑅𝐸𝐷𝐴𝐢𝑇𝐸𝐷"
couchDB_DBNAME: "𝑅𝐸𝐷𝐴𝐢𝑇𝐸𝐷"
liveSync: true
syncOnSave: false
syncOnStart: false
savingDelay: 200
lessInformationInLog: false
gcDelay: 0
versionUpFlash: ""
minimumChunkSize: 20
longLineThreshold: 250
showVerboseLog: true
suspendFileWatching: false
trashInsteadDelete: true
periodicReplication: false
periodicReplicationInterval: 60
syncOnFileOpen: false
encrypt: false
passphrase: "𝑅𝐸𝐷𝐴𝐢𝑇𝐸𝐷"
usePathObfuscation: false
doNotDeleteFolder: false
resolveConflictsByNewerFile: false
batchSave: false
deviceAndVaultName: ""
usePluginSettings: false
showOwnPlugins: false
showStatusOnEditor: true
showOnlyIconsOnEditor: false
usePluginSync: true
autoSweepPlugins: false
autoSweepPluginsPeriodic: false
notifyPluginOrSettingUpdated: false
checkIntegrityOnSave: false
batch_size: 50
batches_limit: 40
useHistory: true
disableRequestURI: true
skipOlderFilesOnSync: true
checkConflictOnlyOnOpen: false
showMergeDialogOnlyOnActive: false
syncInternalFiles: false
syncInternalFilesBeforeReplication: false
syncInternalFilesIgnorePatterns: \/node_modules\/, \/\.git\/, \/obsidian-livesync\/
syncInternalFilesInterval: 60
additionalSuffixOfDatabaseName: documents/test3
ignoreVersionCheck: false
lastReadUpdates: 0
deleteMetadataOfDeletedFiles: false
syncIgnoreRegEx: ""
syncOnlyRegEx: ""
customChunkSize: 50
readChunksOnline: true
watchInternalFileChanges: true
automaticallyDeleteMetadataOfDeletedFiles: 0
disableMarkdownAutoMerge: false
writeDocumentsIfConflicted: false
useDynamicIterationCount: false
syncAfterMerge: false
configPassphraseStore: ""
encryptedPassphrase: "𝑅𝐸𝐷𝐴𝐢𝑇𝐸𝐷"
encryptedCouchDBConnection: "𝑅𝐸𝐷𝐴𝐢𝑇𝐸𝐷"
permitEmptyPassphrase: false
useIndexedDBAdapter: true
useTimeouts: false
writeLogToTheFile: false
doNotPaceReplication: false
hashCacheMaxCount: 300
hashCacheMaxAmount: 50
concurrencyOfReadChunksOnline: 100
minimumIntervalOfReadChunksOnline: 333
hashAlg: xxhash64
suspendParseReplicationResult: false
doNotSuspendOnFetching: false
useIgnoreFiles: false
ignoreFiles: .gitignore
syncOnEditorSave: false
pluginSyncExtendedSetting: {}
syncMaxSizeInMB: 50
settingSyncFile: ""
writeCredentialsForSettingSync: false
notifyAllSettingSyncFile: false
isConfigured: true
settingVersion: 0

kuyper avatar Feb 23 '24 11:02 kuyper

Thank you for reporting the issue! And sorry for being delayed!

I have created the same configuration environment. However, unfortunately, I could not reproduce it.

If your CouchDB is hosted on HTTP instead of HTTPS, it might be the same problem as #385. It will be happy to allow me to report the latest news on that thread. If that is hosted on HTTPS, please let me know.

vrtmrz avatar Mar 01 '24 01:03 vrtmrz

My couchDB is hosted behind a reverse proxy that offers https access, has an official DNS entry and also valid letsencrypt certificates. I use nginx as reverse proxy.

kuyper avatar Mar 24 '24 12:03 kuyper

I've started from scratch with the newest version, a new vault, new DB and I get this

24.3.2024, 13:42:32->Replication activated
24.3.2024, 13:42:37->STORAGE --> DB:ix:desktop/CONFIG/workspace.json.md: (config) Done
24.3.2024, 13:42:39->HTTP:POST (1174270) to:/_bulk_docs -> failed
24.3.2024, 13:42:39->TypeError:Failed to fetch
24.3.2024, 13:42:39->Replication paused

kuyper avatar Mar 24 '24 12:03 kuyper

Thank you for the additional information! Your configuration which we can check is all ok. There may be some trouble with some network configurations.

Would you mind if I ask you to check the DevTools log of your iOS devices via your Mac? You can connect and check it as this document

vrtmrz avatar Apr 08 '24 02:04 vrtmrz

I have the same problem on Ubuntu. Tried to re-fetch, rebuild. Nothing helps. What log can I give? couch_db.log

innerself avatar Apr 13 '24 15:04 innerself

I'm also hitting this issue for my Android phone (my config is in #408 ). My log is very similar:

21/04/2024, 23:01:59->OneShot Sync begin... (sync)
21/04/2024, 23:01:59->Looking for the point last synchronized point.
21/04/2024, 23:02:00->Replication activated
21/04/2024, 23:02:00->HTTP:POST (23920717) to:/_bulk_docs -> failed
21/04/2024, 23:02:00->TypeError:Failed to fetch
21/04/2024, 23:02:00->Replication error
21/04/2024, 23:02:00->Error:Failed to fetch
21/04/2024, 23:02:00->Replication stopped.
21/04/2024, 23:02:00->Retry with lower batch size:27/27
21/04/2024, 23:02:00->OneShot Sync begin... (sync)
21/04/2024, 23:02:00->Looking for the point last synchronized point.
21/04/2024, 23:02:00->Replication activated
21/04/2024, 23:02:01->HTTP:POST (1631150) to:/_bulk_docs -> failed
21/04/2024, 23:02:01->TypeError:Failed to fetch
21/04/2024, 23:02:01->Replication error
21/04/2024, 23:02:01->Error:Failed to fetch
21/04/2024, 23:02:01->Replication stopped.
21/04/2024, 23:02:01->Retry with lower batch size:16/16
21/04/2024, 23:02:01->OneShot Sync begin... (sync)
21/04/2024, 23:02:01->Looking for the point last synchronized point.
21/04/2024, 23:02:01->Replication activated
21/04/2024, 23:02:01->HTTP:POST (1527781) to:/_bulk_docs -> failed
21/04/2024, 23:02:01->TypeError:Failed to fetch
21/04/2024, 23:02:01->Replication error
21/04/2024, 23:02:01->Error:Failed to fetch
21/04/2024, 23:02:01->Replication stopped.
21/04/2024, 23:02:01->Retry with lower batch size:10/10
21/04/2024, 23:02:01->OneShot Sync begin... (sync)
21/04/2024, 23:02:01->Looking for the point last synchronized point.
21/04/2024, 23:02:01->Replication activated
21/04/2024, 23:02:01->HTTP:POST (1518925) to:/_bulk_docs -> failed
21/04/2024, 23:02:01->TypeError:Failed to fetch
21/04/2024, 23:02:01->Replication error
21/04/2024, 23:02:01->Error:Failed to fetch
21/04/2024, 23:02:01->Replication stopped.
21/04/2024, 23:02:01->Retry with lower batch size:7/7
21/04/2024, 23:02:01->OneShot Sync begin... (sync)
21/04/2024, 23:02:01->Looking for the point last synchronized point.
21/04/2024, 23:02:01->Replication activated
21/04/2024, 23:02:02->HTTP:POST (1514053) to:/_bulk_docs -> failed
21/04/2024, 23:02:02->TypeError:Failed to fetch
21/04/2024, 23:02:02->Replication error
21/04/2024, 23:02:02->Error:Failed to fetch
21/04/2024, 23:02:02->Replication stopped.
21/04/2024, 23:02:02->Retry with lower batch size:6/6
21/04/2024, 23:02:02->OneShot Sync begin... (sync)
21/04/2024, 23:02:02->Looking for the point last synchronized point.
21/04/2024, 23:02:02->Replication activated
21/04/2024, 23:02:02->HTTP:POST (1462233) to:/_bulk_docs -> failed
21/04/2024, 23:02:02->TypeError:Failed to fetch
21/04/2024, 23:02:02->HTTP:POST (36) to:/_changes?style=all_docs&filter=_selector&since=48-g1AAAACLeJzLYWBgYMpgTmHgzcvPy09JdcjLz8gvLskBCScyJNX___8_K4M5USwXKMCeZJ5iaZJojq4Yh_Y8FiDJ0ACk_kNNkQKbYpRmaZlmlIauJwsAY1Eq9A&limit=6 -> failed
21/04/2024, 23:02:02->AbortError:The user aborted a request.
21/04/2024, 23:02:02->Replication error
21/04/2024, 23:02:02->Error:Failed to fetch
21/04/2024, 23:02:02->Replication stopped.
21/04/2024, 23:02:02->We can't replicate more lower value.

M-Davies avatar Apr 21 '24 22:04 M-Davies

Found the answer for my issue #344

M-Davies avatar Apr 21 '24 22:04 M-Davies

Sorry for the absence! I am relieved to hear that the problems have been resolved. (If I have misunderstood something, please feel free to reopen the issue).

vrtmrz avatar May 27 '24 10:05 vrtmrz