obsidian-livesync icon indicating copy to clipboard operation
obsidian-livesync copied to clipboard

Possible memory leak: MaxListenersExceededWarning

Open dkbarn opened this issue 6 months ago β€’ 3 comments

Abstract

I have liveSync mode enabled. When I begin typing in a note and continue typing, memory usage steadily rises and never falls. In one test, I was typing for about 5 minutes and watched the memory usage of Obsidian, as follows:

start: 630 MB 1 min: 653 MB 2 min: 683 MB 3 min: 933 MB 4 min: 1,026 MB 5 min: 1,080 MB

During this period, I observed the developer console log which at one point output this warning message:

plugin:obsidian-livesync:6 MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 destroyed listeners added. Use emitter.setMaxListeners() to increase limit
    at _addListener (plugin:obsidian-livesync:6:761188)
    at klass.addListener (plugin:obsidian-livesync:6:766298)
    at klass.once4 [as once] (plugin:obsidian-livesync:6:766610)
    at new Changes2 (plugin:obsidian-livesync:18:1140846)
    at klass.changes (plugin:obsidian-livesync:18:1156599)
    at Changes.eventFunction (plugin:obsidian-livesync:18:1136206)
    at Changes.emit2 [as emit] (plugin:obsidian-livesync:6:766171)
    at Changes.notify (plugin:obsidian-livesync:18:1136861)
    at changes2 (plugin:obsidian-livesync:6:379118)
    at klass.eval (plugin:obsidian-livesync:6:387953)

Expected behaviour

  • Sustained typing when in liveSync mode does not leak memory and there are no MaxListenersExceededWarnings coming from the obsidian-livesync plugin.

Actually happened

  • Memory steadily rises during sustained periods of typing in a single note.

Reproducing procedure

  1. Configure LiveSync as in the attached material.
  2. Open a note and begin typing, with the developer console log visible.
  3. Continue typing for several minutes, observe the log and observe memory usage of the obsidian process.
  4. Look for MaxListenersExceededWarnings and confirm that memory usage steadily climbs and climbs.

Report materials

Report from the LiveSync

Report from hatch
# ---- Obsidian info ----
navigator: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) obsidian/1.8.9 Chrome/132.0.6834.210 Electron/34.3.0 Safari/537.36
fileSystem: insensitive

---
# ---- remote config ----
cluster:
  n: "1"
cors:
  credentials: "true"
  origins: app://obsidian.md,capacitor://localhost,http://localhost
chttpd:
  bind_address: 0.0.0.0
  enable_cors: "true"
  max_http_request_size: "4294967296"
  port: "5984"
  require_valid_user: "true"
admins: 𝑅𝐸𝐷𝐴𝐢𝑇𝐸𝐷
vendor:
  name: The Apache Software Foundation
nouveau:
  url: http://127.0.0.1:5987
feature_flags:
  partitioned||*: "true"
chttpd_auth:
  hash_algorithms: sha256, sha
  require_valid_user: "true"
  secret: 𝑅𝐸𝐷𝐴𝐢𝑇𝐸𝐷
log:
  file: /var/log/couchdb/couchdb.log
  writer: file
indexers:
  couch_mrview: "true"
prometheus:
  additional_port: "false"
  bind_address: 127.0.0.1
  port: "17986"
httpd:
  WWW-Authenticate: Basic realm="couchdb"
  bind_address: 127.0.0.1
  enable_cors: "true"
  port: "5986"
smoosh:
  state_dir: ./data
couch_httpd_auth:
  authentication_db: 𝑅𝐸𝐷𝐴𝐢𝑇𝐸𝐷
  secret: 𝑅𝐸𝐷𝐴𝐢𝑇𝐸𝐷
  authentication_redirect: 𝑅𝐸𝐷𝐴𝐢𝑇𝐸𝐷
couchdb_engines:
  couch: couch_bt_engine
couchdb:
  database_dir: ./data
  max_document_size: "50000000"
  uuid: 𝑅𝐸𝐷𝐴𝐢𝑇𝐸𝐷
  view_index_dir: ./data

---
# ---- Plug-in config ----
version: 0.25.23
remoteType: ""
useCustomRequestHandler: false
couchDB_URI: self-hosted(HTTPS)
couchDB_USER: 𝑅𝐸𝐷𝐴𝐢𝑇𝐸𝐷
couchDB_PASSWORD: 𝑅𝐸𝐷𝐴𝐢𝑇𝐸𝐷
couchDB_DBNAME: 𝑅𝐸𝐷𝐴𝐢𝑇𝐸𝐷
liveSync: true
syncOnSave: true
syncOnStart: true
savingDelay: 200
lessInformationInLog: false
gcDelay: 0
versionUpFlash: ""
minimumChunkSize: 20
longLineThreshold: 250
showVerboseLog: true
suspendFileWatching: false
trashInsteadDelete: true
periodicReplication: false
periodicReplicationInterval: 10
syncOnFileOpen: true
encrypt: true
passphrase: 𝑅𝐸𝐷𝐴𝐢𝑇𝐸𝐷
usePathObfuscation: true
doNotDeleteFolder: true
resolveConflictsByNewerFile: false
batchSave: true
batchSaveMinimumDelay: 2
batchSaveMaximumDelay: 60
deviceAndVaultName: ""
usePluginSettings: false
showOwnPlugins: false
showStatusOnEditor: true
showStatusOnStatusbar: true
showOnlyIconsOnEditor: false
hideFileWarningNotice: false
usePluginSync: false
autoSweepPlugins: false
autoSweepPluginsPeriodic: false
notifyPluginOrSettingUpdated: false
checkIntegrityOnSave: false
batch_size: 50
batches_limit: 50
useHistory: true
disableRequestURI: true
skipOlderFilesOnSync: true
checkConflictOnlyOnOpen: false
showMergeDialogOnlyOnActive: false
syncInternalFiles: true
syncInternalFilesBeforeReplication: true
syncInternalFilesIgnorePatterns: \/node_modules\/, \/\.git\/, \/obsidian-livesync\/, ^\.git\/,\/workspace$ ,\/workspace.json$,\/workspace-mobile.json$
syncInternalFilesTargetPatterns: ""
syncInternalFilesInterval: 60
additionalSuffixOfDatabaseName: f5bacf13bc83235b
ignoreVersionCheck: false
lastReadUpdates: 0
deleteMetadataOfDeletedFiles: false
syncIgnoreRegEx: ""
syncOnlyRegEx: ""
customChunkSize: 60
readChunksOnline: true
watchInternalFileChanges: true
automaticallyDeleteMetadataOfDeletedFiles: 0
disableMarkdownAutoMerge: false
writeDocumentsIfConflicted: false
useDynamicIterationCount: false
syncAfterMerge: true
configPassphraseStore: ""
encryptedPassphrase: 𝑅𝐸𝐷𝐴𝐢𝑇𝐸𝐷
encryptedCouchDBConnection: 𝑅𝐸𝐷𝐴𝐢𝑇𝐸𝐷
permitEmptyPassphrase: false
useIndexedDBAdapter: true
useTimeouts: false
writeLogToTheFile: false
doNotPaceReplication: false
hashCacheMaxCount: 300
hashCacheMaxAmount: 50
concurrencyOfReadChunksOnline: 100
minimumIntervalOfReadChunksOnline: 100
hashAlg: xxhash64
suspendParseReplicationResult: false
doNotSuspendOnFetching: false
useIgnoreFiles: false
ignoreFiles: .gitignore
syncOnEditorSave: true
pluginSyncExtendedSetting: {}
syncMaxSizeInMB: 50
settingSyncFile: ""
writeCredentialsForSettingSync: false
notifyAllSettingSyncFile: false
isConfigured: true
settingVersion: 10
enableCompression: false
accessKey: 𝑅𝐸𝐷𝐴𝐢𝑇𝐸𝐷
bucket: 𝑅𝐸𝐷𝐴𝐢𝑇𝐸𝐷(0 letters)
endpoint: Not configured or AWS
region: 𝑅𝐸𝐷𝐴𝐢𝑇𝐸𝐷(4 letters)
secretKey: 𝑅𝐸𝐷𝐴𝐢𝑇𝐸𝐷
useEden: false
maxChunksInEden: 10
maxTotalLengthInEden: 1024
maxAgeInEden: 10
disableCheckingConfigMismatch: false
displayLanguage: def
enableChunkSplitterV2: false
disableWorkerForGeneratingChunks: false
processSmallFilesInUIThread: false
notifyThresholdOfRemoteStorageSize: 800
usePluginSyncV2: true
usePluginEtc: false
handleFilenameCaseSensitive: false
doNotUseFixedRevisionForChunks: true
showLongerLogInsideEditor: false
sendChunksBulk: false
sendChunksBulkMaxSize: 1
useSegmenter: false
useAdvancedMode: true
usePowerUserMode: true
useEdgeCaseMode: false
enableDebugTools: false
suppressNotifyHiddenFilesChange: false
syncMinimumInterval: 2000
P2P_Enabled: false
P2P_AutoAccepting: 0
P2P_AppID: 𝑅𝐸𝐷𝐴𝐢𝑇𝐸𝐷(20 letters)
P2P_roomID: 𝑅𝐸𝐷𝐴𝐢𝑇𝐸𝐷(0 letters)
P2P_passphrase: 𝑅𝐸𝐷𝐴𝐢𝑇𝐸𝐷(0 letters)
P2P_relays: 𝑅𝐸𝐷𝐴𝐢𝑇𝐸𝐷(27 letters)
P2P_AutoBroadcast: false
P2P_AutoStart: false
P2P_AutoSyncPeers: ""
P2P_AutoWatchPeers: ""
P2P_SyncOnReplication: ""
P2P_RebuildFrom: ""
P2P_AutoAcceptingPeers: ""
P2P_AutoDenyingPeers: ""
P2P_IsHeadless: false
doctorProcessedVersion: 0.25.0
bucketCustomHeaders: 𝑅𝐸𝐷𝐴𝐢𝑇𝐸𝐷(0 letters)
couchDB_CustomHeaders: 𝑅𝐸𝐷𝐴𝐢𝑇𝐸𝐷(0 letters)
useJWT: false
jwtAlgorithm: ""
jwtKey: 𝑅𝐸𝐷𝐴𝐢𝑇𝐸𝐷(0 letters)
jwtKid: 𝑅𝐸𝐷𝐴𝐢𝑇𝐸𝐷(0 letters)
jwtSub: 𝑅𝐸𝐷𝐴𝐢𝑇𝐸𝐷(0 letters)
jwtExpDuration: 5
useRequestAPI: false
bucketPrefix: ""
chunkSplitterVersion: v3-rabin-karp
E2EEAlgorithm: v2
processSizeMismatchedFiles: false
forcePathStyle: true

dkbarn avatar Nov 07 '25 04:11 dkbarn

Thank you for reporting this issue! Fortunately, I have reproduced this!

However, it seems it has occurred on a deeper layer of PouchDB. ondestroy is not used on Self-hosted LiveSync, and it seems failed to unregister the handler during disconnection on aborting (and Self-hosted LiveSync abort and disconnects when it gets into the background). I will address it! I will look into whether a workaround is possible for now, but may need some send a patch to the upstream.

vrtmrz avatar Nov 08 '25 02:11 vrtmrz

Progress note: I found that this is caused by the polyfill of EventEmitter and a usage problem (still unclear) on PouchDB. And, also, I found that this can avoid this by enabling Patches->Compatibility -> (Obsolete) Use an old adapter for compatibility. In other words, it seems to be owed to the PouchDB's adapter (and we need a bit longer). However, switching this requires rebuilding the local database for now.

I think still some advantages in IDB (an old adapter), also on other sides. Therefore, I am going to implement a feature that switches safely, and do not check this configuration in the doctor. Please wait for a while!

vrtmrz avatar Nov 11 '25 10:11 vrtmrz

In v0.25.27, idb (an older adapter) has reverted to being the default. It is also no longer detected by the Doctor. Furthermore, with this release, changing the adapter no longer requires a rebuild (though a minor migration is necessary). Would you mind if I ask you to check the behaviour, please?

vrtmrz avatar Nov 13 '25 06:11 vrtmrz

Apologies for not responding sooner. I have just had a chance to test against the latest version of livesync (v0.25.36). I did a 5 minute timed test of typing into a note continuously, monitoring the output of the developer console and the memory usage of the Obsidian process in Windows Task Manager.

The results look good! I never encountered the MaxListenersExceededWarning in the developer console, and the memory usage did not steadily climb out of control, it stayed fairly constant. So I think it's fair to say that your latest changes have resolved this issue, and we can close this one out. Thank you for the fix!

dkbarn avatar Jan 03 '26 19:01 dkbarn