data.json growing continuously

[CovetingEpiphany2152]

I noticed that data.json has a notesToDates field that is being updated everyday. If this goes on it's size would turn humungous in the long run:

"notesToDates":[{"notePath":"resource/note/daily/2021-05-31.md","date":"Mon May 31 2021"},{"notePath":"resource/note/daily/2021-06-01.md","date":"Tue Jun 01 2021"},{"notePath":"resource/note/daily/2021-06-02.md","date":"Wed Jun 02 2021"},{"notePath":"resource/note/daily/2021-06-03.md","date":"Thu Jun 03 2021"},{"notePath":"resource/note/daily/2021-06-04.md","date":"Fri Jun 04 2021"},{"notePath":"resource/note/daily/2021-06-05.md","date":"Sat Jun 05 2021"},{"notePath":"resource/note/daily/2021-06-06.md","date":"Sun Jun 06 2021"},{"notePath":"resource/note/daily/2021-06-07.md","date":"Mon Jun 07 2021"},{"notePath":"resource/note/daily/2021-06-08.md","date":"Tue Jun 08 2021"},{"notePath":"resource/note/daily/2021-06-09.md","date":"Wed Jun 09 2021"},{"notePath":"resource/note/daily/2021-06-10.md","date":"Thu Jun 10 2021"}]

Not sure if it matters, I'm using the plugin in command mode.

Suggestion:

• in file mode, automatically search the given folder for files, and do not keep a notesToDates field
• in command mode, only keep noteForToday, and only store the path to today's note

[ben-xD]

Hey @weeyizhi, what do you mean by humongous? When asking for an optimisation, what's really important is measurements. After 1 month, how big is your file? Just roughly looking at the text you provided, it looks like it's adding a few bytes every day, and I don't expect the file to be even 1MB after 1 year of use, though I haven't measured it myself.

[CovetingEpiphany2152]

Hi @ben-xD , thanks for pointing that out. You are right, even if the path to files and file names are very long, it shouldn't take more than one or two MBs in a year. I agree that my usage of humongous is ambiguous and rather incorrect.

I think that for text files, anything that gets close to 10MB is large. That is of course a personal opinion. However, I think my initial point still stands, this file will continuously get larger and harder to read over time. And I don't know of any good reason for it to behave this way.

As far as I know, there is no usage of previous dates in the plugin. The only time I think it could be of use is if time goes backwards (by editing the device's time or moving timezones). Perhaps that is good enough a reason to retain the current behavior, but I'd like to keep this open until the dev responds (which likely wouldn't happen considering there has been no activity here for quite a while).

[ben-xD]

Also, the existing design would not be a good fit for git, preventing people from storing their ObsidianVault in git. Mainly because the new file are stored every time a new change is made, the git repo grows exponentially over time. (Git doesn't store diffs, but original files.)

[lynchjames]

Thanks for asking about this and for the suggestions. The data.json file was used originally to keep a reliable mapping between dates and note paths that would not rely on the core Daily notes plugin and would allow the user to switch between the File and Command modes. There was the potential for the data to be used to allow the user to view the timeline for any day planner note that had been created previously but I decided against that feature. Without something like that you are correct that there is no good reason to retain the note -> date mappings for dates in the past.

I also agree that it is not a sustainable solution to use a file that will continue to grow over time, particularly when that file is also being used by the plugin to load its settings.

I will make a change to remove entries from previous notesToDates entries leaving only one entry at a time.

[ivan-lednev]

Should be fixed in 0.7.0.