Refactor: Projects & Drafts 2.0

[kevboh]

trafficstars

After using Longform for a few months, seeing some issues and suggestions roll in, and toying with options, I’ve more or less settled on a set of changes to how Longform projects are structured that should address most project/organizer feature requests and set up Longform to be more flexible for future changes.

The crux of the refactor is that Index files will live at the "draft" level, and Longform drafts will effectively become projects themselves. This results in several immediate benefits:

1. Users are no longer locked into the Project > Draft hierarchy. For example, if someone wanted to only have a single “draft” in their project and didn’t want a needlessly nested Drafts/Draft 1 folder structure, they could simply create the project and have a single Index in it. Or if someone wanted to organize drafts in some other way, they could.
2. Operations that happen at the manuscript level (in 1.0, the Draft level) become far easier. Because Index files can act as a lock on the manuscipt, sync issues are reduced. Per-manuscript word counts become easier. Cloning and deleting drafts are more straightforward.
3. Longform itself becomes more flexible. I like writing short stories in Longform because it feels like the place I write (mostly because of the custom CSS in projects!), but a short story if often a single scene, which means I have a deeply nested set of folders that contain a single file. Better to cut out at least one folder—and maybe we even get to a point where the frontmatter on a file can act as the index, and a short story can live by itself as a lone file somewhere.
4. By moving Index files to the manuscript level, the internal discovery and tracking of Longform projects becomes easier. Currently, Longform projects are imperatively tracked: by "marking" a folder as a project, that project is added to an internal list stored in Obsidian plugin storage. This is brittle. Instead, Longform can treat any folder that contains a file named Index.md with certain frontmatter as a project. If you manually create that folder and file, it appears in Longform. If you delete it, it disappears. This eliminates a class of bugs often filed and related to sync, multiple device use, etc.

There is a lot more here, and I want to finish the compile refactor first, but this is the next step of making Longform more flexible and reliable in future.

This issue tracks a number of others, and closing this should close: #7, #9, #11, #12, #16, #21, #29, #32, #33, #34, #63, #65, #67, #70

[chrisgrieser]

First of all, thank you so much for this wonderful plugin – I think I am one of the power users of the plugin, having tailored my theme specifically to it, and having build a word count dashboard for it.

While at first I wasn't too sure about the idea of index files, by now I very much like the idea, and the improvement you are proposing. Storing the data in an index yaml allows dataview to easily read the data, which I use e.g. make the notes in the word count dashboard automatically follow the order defined in the longform plugin:

Also, another thing that I was able to implement with some (in part pretty hacky) css is to do subsections and section numbering:

So I know this post hasn't really any big message other than showing you what people have done based on the plugin, but maybe you can get some inspiration for further ideas. Storing word or character counts in the yaml data of the index file could, for example, be useful for other people attempting to build word count tables, as my implementation is pretty hard to customize for non-coders.

[pdworkman]

I kept a few notes when writing my last book in Longform, and have some wishes/feature suggestions for the future.

• ability to nest scenes within chapters (ie more than one level)
• ability to sort scenes numerically or alphabetically if I am starting with a manuscript that didn't begin in longform. I did figure out how to kludge this by opening the index file and sorting the list of files alphabetically. Much faster than trying to drag 50 scenes into the right order.
• Autonumbering would be great if I am switching between editing in longform/obsidian and with another text editor on another device (ie. I can sort files alphabetically in the file explorer on any platform, so if they are numbered they will show up in the right order.)
• In moving projects over from Notebooks app to Obsidian, I at first ran into problems with longform because Notebooks app creates a plist file for each scene, and longform didn't know what to do with these files. Once I learned to delete the plist files before designating a longform project, I was fine
• my writing directories are in Dropbox, and often when switching between computers I would end up with conflicted copies of the index file. I try to remember to close Obsidian before switching computers to help prevent this.
• creating a new draft as a copy of the previous draft is tricky. I've only managed to do it once. The rest of the time I had to create an empty draft and then copy the files over in Finder. And then resort the index file alphabetically once created.

Longform is a great little plugin. Really enjoying using it. Looking forward to future releases.

You can see more on my use of Longform in my blog post, if you haven't already: https://pdworkman.com/writing-a-novel-in-markdown/

[thecookiemomma]

I am using Longform and also using Zootelkeeper, and wondering if the two could play together nicely or ignore each other. I love Longform, and am excited to see these improvements. Thank you so much for all your hard work.

[kevboh]

@chrisgrieser woww—any chance you could send me a sample vault so I can see how you set that stuff up? Worth looking at for the inspiration if nothing else.

@pdworkman lots of great suggestions/bug reports in there. I suspect many of them are made more solvable by this change, and I'll certainly keep an eye on the rest. The conflicted Index thing is annoying—having no control over sync is tough.

@thecookiemomma great note! I'll look into it.

Thank you all for chiming in with such helpful comments, I really appreciate it.

[chrisgrieser]

@kevboh Tbh, there is not much to it content wise – you can take at look how both my theme and the wordcount dashboard work with the longform plugin with any longform project you already have. That's the beauty of it. :)

My theme has tons of styling for the longform plugin, if you search for < Longform in the .css file you'll find the css behind it.

The word count dashboard is basically ~300 lines of dataviewjs that takes a bunch of notes, counts words, characters etc. and then summarizes them in a big table. I tinkered around quite a bit until I realized that dataview has basically YAML-parsing built-in, so that ordering the table rows by their order in the index file is pretty much a one liner.

One thing I came up for with both is the use a preceding underscore _ to mark a section as a subsection. The recognizes that via data attribute and does some hacky stuff to indent & "hide" the underscore in the sidebar, and with dataviewjs it wasn't too much of a big deal to change output the rows of subsections a bit.

As you can see by the amount of work I put into that, the missing subsection feature is probably the biggest pain point I had for the longform plugin. Especially since I am going to write my PhD Thesis in Obsidian, having subsections (and subsubsections, etc.) would be immensely useful.

[pdworkman]

I am now using multiple workflows. :-)

I have one workflow to back up my draft into a single md file, which keeps my metadata in html comments (in case of some kind of data loss or overwrite of a scene)

and one workflow for formatting a manuscript, which removes the metadata comments

I thought you would appreciate knowing that your work is being put to good use.

[blackpeasy]

Autonumbering would be great if I am switching between editing in longform/obsidian and with another text editor on another device (ie. I can sort files alphabetically in the file explorer on any platform, so if they are numbered they will show up in the right order.)

Second this. Either adding the scene number as a prefix to the file name, or auto-updating a sceneNumber: n in the scene's YAML frontmatter when they are moved around in the Longform pane, would help tremendously for dataview queries as well.

Imagine having a Character note with an auto-sorting dataview table of all scenes this character appears in always in the same order as in the Longform pane. This would really be a game-changing tool to get an overview of the plotlines and the story.

[kevboh]

I've updated this issue to record the issues it tracks. You can now beta-test it using BRAT and entering kevboh/longform to install a beta of 2.0.

[yogeshwersharma]

Thank you Kevin. This is wonderful.

I see two possible issues. Maybe I am doing something wrong, or they are real issues.

1. The format of the index file seems off. I lost my earlier test project in migration, but that is not a big deal. Here is what I see.

scenes:
    - Test scene
    - - A section
      - - A subsection
    - Second chapter

Maybe those addition '-' at the beginning of indented scenes are creating problem?

But things look good in the new project I just created.

2. Vim bindings don't seem to work with this version. When I press 'i' to go insert mode, it momentarily goes to insert mode by then immediately goes back to normal mode. I am not able to insert any characters.

I had to disable vim mode in order to write anything.

Would love to hear your thoughts about these two things.

Thank you for putting all the work into this.

[kevboh]

Thank you!

1. What exactly is wrong with your project? The yaml you shared is valid. What do you mean by you “lost” the test project?

2. I haven’t tested vim bindings at all, but can. I don’t see why it would affect anything. Were you editing a scene or an index file?

On Jul 16, 2022, at 2:18 PM, yogeshwersharma @.***> wrote:

 Thank you Kevin. This is wonderful.

I see two possible issues. Maybe I am doing something wrong, or they are real issues.

The format of the index file seems off. I lost my earlier test project in migration, but that is not a big deal. Here is what I see. scenes: - Test scene - - A section - - A subsection - Second chapter Maybe those addition '-' at the beginning of indented scenes are creating problem?

But things look good in the new project I just created.

Vim bindings don't seem to work with this version. When I press 'i' to go insert mode, it momentarily goes to insert mode by then immediately goes back to normal mode. I am not able to insert any characters. I had to disable vim mode in order to write anything.

Would love to hear your thoughts about these two things.

Thank you for putting all the work into this.

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.

[kevboh]

Hi all! I just recreated the 2.0.0 beta to include the following new features:

• word counts + a bunch of word count settings + writing session tracking
• project creation menu item + modal
• draft creation modal with optional scene copying
• an API (app.plugins.plugins.longform.api) to help with converting to and from Longform's scene YAML
• fixes

To install it you'll need to remove and re-add kevboh/longform in BRAT, since I reused the same version number.

At this point 2.0.0 is more or less feature-complete. I plan on fixing bugs as I find them during my own testing period—please report bugs you find here or as new issues. Thank you!

[pdworkman]

Okay, I finally got around to duplicating one of my series vaults so that I can play around with the new longform plugin.

I created a new project and gave it a name, then decided it was not the way I wanted it named and renamed it. I created a new draft (or maybe 2 new drafts) and everything got jammed up. I couldn't make a selection from the project list and couldn't select which draft I wanted to be on.

Closed the vault and reopened it, and my drafts were showing up properly and everything moving smoothly.

---

I'm sure I saw instructions somewhere on how to make an old longform project into a new longform project, but I've looked everywhere and can't find them.

Pam

P.D. Workman

Award-winning and USA Today Bestselling Author Find me at pdworkman.com http://pdworkman.com/on-the-slab-pie Hot new release! On the Slab Pie A Culinary and Cat Cozy Mystery https://www.facebook.com/facebook.com/pdworkmanauthor https://www.linkedin.com/linkedin.com/in/pdworkman https://www.twitter.com/twitter.com/pdworkmanauthor

Join my e-mail list https://pdworkman.com/signup to get a free ebook and to be notified when my books are published.

On Sun, Jul 24, 2022 at 6:30 AM Kevin Barrett @.***> wrote:

Hi all! I just recreated the 2.0.0 beta to include the following new features:

• word counts + a bunch of word count settings + writing session tracking
• project creation menu item + modal
• draft creation modal with optional scene copying
• fixes

To install it you'll need to remove and re-add kevboh/longform in BRAT, since I reused the same version number.

At this point 2.0.0 is more or less feature-complete. I plan on fixing bugs as I find them during my own testing period—please report bugs you find here or as new issues. Thank you!

— Reply to this email directly, view it on GitHub https://github.com/kevboh/longform/issues/35#issuecomment-1193308718, or unsubscribe https://github.com/notifications/unsubscribe-auth/AV42KG4S3X3NLMVSAXATJBDVVUZMNANCNFSM5K576QOA . You are receiving this because you were mentioned.Message ID: @.***>

[pdworkman]

One feature request after playing with compiling and converting to my final manuscript format a bit.

I would like to be able to prepend title for the top level (for me, chapter) scenes, but not to prepend the title for the indented scenes.

But I do have a workaround if that is not a feature you can add. It is not a deal breaker.

Pam

P.D. Workman

Award-winning and USA Today Bestselling Author Find me at pdworkman.com http://pdworkman.com/on-the-slab-pie Hot new release! On the Slab Pie A Culinary and Cat Cozy Mystery https://www.facebook.com/facebook.com/pdworkmanauthor https://www.linkedin.com/linkedin.com/in/pdworkman https://www.twitter.com/twitter.com/pdworkmanauthor

Join my e-mail list https://pdworkman.com/signup to get a free ebook and to be notified when my books are published.

On Sun, Jul 24, 2022 at 10:27 PM pdw @.***> wrote:

Okay, I finally got around to duplicating one of my series vaults so that I can play around with the new longform plugin.

I created a new project and gave it a name, then decided it was not the way I wanted it named and renamed it. I created a new draft (or maybe 2 new drafts) and everything got jammed up. I couldn't make a selection from the project list and couldn't select which draft I wanted to be on.

Closed the vault and reopened it, and my drafts were showing up properly and everything moving smoothly.

---

I'm sure I saw instructions somewhere on how to make an old longform project into a new longform project, but I've looked everywhere and can't find them.

Pam

P.D. Workman

Award-winning and USA Today Bestselling Author Find me at pdworkman.com http://pdworkman.com/on-the-slab-pie Hot new release! On the Slab Pie A Culinary and Cat Cozy Mystery https://www.facebook.com/facebook.com/pdworkmanauthor https://www.linkedin.com/linkedin.com/in/pdworkman https://www.twitter.com/twitter.com/pdworkmanauthor

Join my e-mail list https://pdworkman.com/signup to get a free ebook and to be notified when my books are published.

On Sun, Jul 24, 2022 at 6:30 AM Kevin Barrett @.***> wrote:

Hi all! I just recreated the 2.0.0 beta to include the following new features:

• word counts + a bunch of word count settings + writing session tracking
• project creation menu item + modal
• draft creation modal with optional scene copying
• fixes

To install it you'll need to remove and re-add kevboh/longform in BRAT, since I reused the same version number.

At this point 2.0.0 is more or less feature-complete. I plan on fixing bugs as I find them during my own testing period—please report bugs you find here or as new issues. Thank you!

— Reply to this email directly, view it on GitHub https://github.com/kevboh/longform/issues/35#issuecomment-1193308718, or unsubscribe https://github.com/notifications/unsubscribe-auth/AV42KG4S3X3NLMVSAXATJBDVVUZMNANCNFSM5K576QOA . You are receiving this because you were mentioned.Message ID: @.***>

[kevboh]

All of the subtickets this issue tracks have been closed. I have a bit more polishing to do, then will release 2.0 to production.