Make config and data files more friendly to git sync
Hello! I use obsidian-git to sync my repository among a few computers & mobile phones on daily basis. Everytime there is 3 files changed, which is from make.md, even I didn't operated on make.md features this time. These 3 files are:
Context/spacesLastSync.logOnly a timestamp in it..obsidian/plugins/make-md/data.jsonIf I don't operate on make.md features, there is only 1 line changed, which "pollutes" whole config file:"spacesAutoBackupLast": 1689644873935,The file size is 3kb+..obsidian/plugins/make-md/Spaces.mdbThis is a data file and I have no MS Access so I don't know what's in it. The size is 420kb+.
Since I am still in trial stage and haven't fully used every function of make.md, there may be more such files I haven't activated yet.
Above files make me concern, since IMO they change not on behalf of user(me), but on behalf of internal mechanics of make.md. This is not friendly to git mechanics. Git preserves every bit of history change, and it accumulates among time. So in above files:
- File 1 and 2 is small but every time I commit they are committed too, and the timestamps seemingly meaningless but I cannot be sure. However currently I manually discard timestamp changes when committing my repository.
- File 3 is data file and doesn't play well with git, and the size is considerably big considering I commit repository 1~3 times a day, and every time it accumulates 420kb to git repository, which is roughly 300+MB per year. And my repository has run for 2 years and the content size is only 204M, with a lot of picture attachments. So that's a big burden to the repository.
- I see issue #100 in which you explained the context.mdb files cannot be moved currently, and I haven't met them, so the size accumulation problem may be worse.
So, please consider, if it is possible & worthwile to change behavior. My ideas regarding above issue(Possibly impratical):
- If it is not absolutely necessary, don't record timestamps in repository; If it is necessary, record them in a single file which only contains timestamp and no other configures (against file 2). Timestamp is implementation data which constantly changes, it is best not to mix it with relatively static files such as configures and plugin code.
- Put data files such as spaces.mdb out of Obsidian repository I don't know how make.md works inside, but regarding to the design principles, IMO it is good practice to use a text format such json or yaml to store user configure information about space layout, and put cached or generated files in local temp place, outside of repository. Or let us know that these files can be localized and be ignored from git repository. Because in my (and some others I presume) case the repository needs to be synchronized.
Anyway make.md is a great piece of software that pushes powerness of Obsidian to a new level. Thank you very much!
Thank you @WilliamStone for raising this issue between this plugin and vault synchronization with Git.
I do the same and the Context/spacesLastSync.log in particular causes persistent merge conflicts when rebasing a master branch between repository clones.
I ended up adding that file to the .gitignore list since it doesn't seem critical to this plugin.
Would love to see a more elegant solution where this plugin does not leave temporary files in the vault.
The spacesLastSync log is only needed for real-time sync like Obsidian or Dropbox. If you use Git, you can ignore it. It's required because of how real-time sync works to avoid overwriting the database on new device initialization
Addressed in 0.8
I like Make.md and its features. For Git I would prefer text-only configuration files. JSON is ok. I wonder why you are using 3 different ways for storing information. JSON, SQLite (stored as MDB which looks like MS Access but it isn't) and MDC (in fact ZIP archives with MDB SQLite files in it).
As long as I am working alone on my vault, syncing would be easy, because I simply take the modification date of the files or because of I am the only user of my Git Repo (which primarily serves as my vault changelog system). Merging changes from another user would not be possible with these binary files.